Rankings

Engine

Ollama

Ollama · MIT, open source · Homepage

Top Engines

Desktop
Rank 2, 93 out of 100, range 91 to 94, approximate comparison Speed score · 4 of 4 counted comparisons
Mobile
Not measured
details

Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.

How rankings work

Supporting experiments

Ollama: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 84, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 454.6range452.3–458.830.6range23.9–33.68,803range7,994–11,4230.93range0.93–0.95
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 326.2range294.8–332.230.5range24.1–32.38,827range8,336–11,3021.41range1.41–1.43
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 199.6range199.4–200.333.8range33.0–35.17,900range7,605–8,0852.43range2.43–2.45
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 155.1range155.0–155.252.7range52.1–52.94,995range4,957–5,0512.96range2.96–2.97
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 91.7range91.4–92.257.0range56.4–57.14,596range4,586–4,6385.31range5.31–5.33
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 67, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 742.8range734.2–745.721.4range21.3–21.612,795range12,754–12,8700.46range0.46–0.47
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 77, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 630.3range624.6–645.121.4range21.1–21.512,907range12,819–13,1170.60range0.60–0.61
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 84, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 479.5range475.4–483.222.3range22.1–22.412,319range12,186–12,3850.91range0.91–0.92
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 78, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 26.0range25.7–26.0394range393–423652.0range608.6–654.418.90range18.89–19.08
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 19.0range18.9–19.0392range391–452655.7range569.1–657.027.46range27.45–27.64
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q2_0 · 6/6 validCounted 45.1range45.0–45.2375range374–385685.8range667.3–687.07.94range7.94–8.12
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 84, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 442.1range439.9–445.3139range138–14115,035range14,809–15,1990.95range0.95–0.95
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 321.4range321.1–322.4133range132–13515,732range15,520–15,8851.43range1.43–1.43
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 197.6range191.3–198.4136range135–13815,385range15,184–15,5332.45range2.45–2.45
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 153.6range153.3–153.9283range283–2847,306range7,275–7,3172.97range2.97–2.97
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 91.1range90.0–91.5289range287–2907,147range7,127–7,1965.33range5.33–5.33
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 74, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 700.6range693.0–726.664.1range54.1–64.433,581range33,462–39,9790.47range0.47–0.47
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 77, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 599.8range586.3–603.961.5range61.5–61.734,935range34,835–35,1080.61range0.61–0.61
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 88, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal BF16 · 6/6 validCounted 469.0range429.0–473.763.4range63.2–63.533,834range33,703–33,8800.92range0.91–0.92
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 78, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCounted 25.9range25.8–25.92,823range2,749–3,328727.0range616.2–746.319.07range19.07–19.08
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q8_0 · 6/6 validCounted 18.9range18.7–18.92,748range2,663–3,237746.6range633.6–770.427.63range27.62–27.64
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q2_0 · 6/6 validCounted 44.6range44.2–44.62,672range2,572–2,941767.8range697.4–798.18.12range8.12–8.12
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCheck failed 228.0range226.9–229.351.9range51.3–54.95,074range4,798–5,1321.84range1.84–1.86
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.40.13 · Metal Q4_K_M · 6/6 validCheck failed 224.1range223.7–225.4297range296–2986,966range6,939–6,9721.86range1.86–1.86