Engine
Ollama
Ollama · MIT, open source · Homepage
Top Engines
- Desktop
- Rank 2, 93 out of 100, range 91 to 94, approximate comparison
- Mobile
- Not measured
details
Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 84, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 454.6range452.3–458.8 | 30.6range23.9–33.6 | 8,803range7,994–11,423 | 0.93range0.93–0.95 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 326.2range294.8–332.2 | 30.5range24.1–32.3 | 8,827range8,336–11,302 | 1.41range1.41–1.43 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 199.6range199.4–200.3 | 33.8range33.0–35.1 | 7,900range7,605–8,085 | 2.43range2.43–2.45 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 155.1range155.0–155.2 | 52.7range52.1–52.9 | 4,995range4,957–5,051 | 2.96range2.96–2.97 |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 91.7range91.4–92.2 | 57.0range56.4–57.1 | 4,596range4,586–4,638 | 5.31range5.31–5.33 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 67, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 742.8range734.2–745.7 | 21.4range21.3–21.6 | 12,795range12,754–12,870 | 0.46range0.46–0.47 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 77, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 630.3range624.6–645.1 | 21.4range21.1–21.5 | 12,907range12,819–13,117 | 0.60range0.60–0.61 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 84, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 479.5range475.4–483.2 | 22.3range22.1–22.4 | 12,319range12,186–12,385 | 0.91range0.91–0.92 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 78, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 26.0range25.7–26.0 | 394range393–423 | 652.0range608.6–654.4 | 18.90range18.89–19.08 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 19.0range18.9–19.0 | 392range391–452 | 655.7range569.1–657.0 | 27.46range27.45–27.64 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q2_0 · 6/6 valid | Counted | 45.1range45.0–45.2 | 375range374–385 | 685.8range667.3–687.0 | 7.94range7.94–8.12 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 84, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 442.1range439.9–445.3 | 139range138–141 | 15,035range14,809–15,199 | 0.95range0.95–0.95 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 321.4range321.1–322.4 | 133range132–135 | 15,732range15,520–15,885 | 1.43range1.43–1.43 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 197.6range191.3–198.4 | 136range135–138 | 15,385range15,184–15,533 | 2.45range2.45–2.45 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 153.6range153.3–153.9 | 283range283–284 | 7,306range7,275–7,317 | 2.97range2.97–2.97 |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 91.1range90.0–91.5 | 289range287–290 | 7,147range7,127–7,196 | 5.33range5.33–5.33 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 74, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 700.6range693.0–726.6 | 64.1range54.1–64.4 | 33,581range33,462–39,979 | 0.47range0.47–0.47 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 77, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 599.8range586.3–603.9 | 61.5range61.5–61.7 | 34,935range34,835–35,108 | 0.61range0.61–0.61 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 88, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 valid | Counted | 469.0range429.0–473.7 | 63.4range63.2–63.5 | 33,834range33,703–33,880 | 0.92range0.91–0.92 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 78, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Counted | 25.9range25.8–25.9 | 2,823range2,749–3,328 | 727.0range616.2–746.3 | 19.07range19.07–19.08 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 valid | Counted | 18.9range18.7–18.9 | 2,748range2,663–3,237 | 746.6range633.6–770.4 | 27.63range27.62–27.64 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q2_0 · 6/6 valid | Counted | 44.6range44.2–44.6 | 2,672range2,572–2,941 | 767.8range697.4–798.1 | 8.12range8.12–8.12 |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Check failed | 228.0range226.9–229.3 | 51.9range51.3–54.9 | 5,074range4,798–5,132 | 1.84range1.84–1.86 |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 valid | Check failed | 224.1range223.7–225.4 | 297range296–298 | 6,966range6,939–6,972 | 1.86range1.86–1.86 |