Rankings
Engine
Magnitude
Magnitude · Apache-2.0, open source · Homepage
Top Engines
- Desktop
- Not ranked
- Mobile
- Not measured
details
Approximate comparison: The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 82, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 valid | Counted | 444.5range437.1–446.0 | 26.6range26.2–26.8 | 9,627range9,540–9,771 | 1.56range1.55–1.56 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 88, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 valid | Counted | 330.1range327.3–330.9 | 25.5range25.3–25.6 | 10,049range9,989–10,109 | 1.90range1.89–1.90 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 valid | Counted | 199.6range194.8–200.4 | 28.7range19.6–31.0 | 8,926range8,265–13,076 | 2.95range2.94–2.96 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 82, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 valid | Counted | 911.9range904.9–915.9 | 18.7range18.4–25.0 | 13,679range10,229–13,922 | 1.05range1.05–1.06 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 93, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 valid | Counted | 764.7range759.1–768.9 | 18.0range17.8–19.1 | 14,188range13,437–14,349 | 1.04range1.03–1.04 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 valid | Counted | 524.1range512.1–526.0 | 19.1range18.9–19.5 | 13,435range13,137–13,547 | 1.35range1.33–1.35 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q2_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 85, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 valid | Counted | 450.2range441.1–466.3 | 109range96.1–120 | 18,730range17,068–21,319 | 1.56range1.56–1.56 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 89, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 valid | Counted | 329.6range328.4–330.9 | 96.3range96.1–96.5 | 21,262range21,223–21,308 | 1.90range1.90–1.90 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 valid | Counted | 199.6range197.5–200.8 | 113range113–119 | 18,058range17,233–18,113 | 2.96range2.96–2.96 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 93, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 valid | Counted | 884.9range879.3–906.6 | 46.5range46.3–49.1 | 44,058range41,750–44,266 | 1.05range1.05–1.06 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 valid | Counted | 747.9range731.2–756.8 | 44.4range44.3–44.5 | 46,119range46,034–46,275 | 1.04range1.04–1.05 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 valid | Counted | 510.1range491.6–519.0 | 48.9range48.7–49.0 | 41,885range41,783–42,028 | 1.35range1.34–1.35 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q2_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |