Rankings

Engine

MLX LM

Apple (ml-explore) · MIT, open source · Homepage

Top Engines

Desktop
Rank 1, 95 out of 100, range 95 to 96, approximate comparison Speed score · 4 of 4 counted comparisons
Mobile
Not measured
details

Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M. Its weights are 2-bit in another scheme: mlx-affine-2bit-g128, where most use Q2_0. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.

How rankings work

Supporting experiments

MLX LM: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 582.9range574.9–589.728.9range28.6–29.39,024range8,897–9,0920.89range0.87–0.90
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 96, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 361.1range359.7–362.630.7range30.6–31.28,486range8,348–8,5131.44range1.43–1.45
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 199.0range197.4–202.230.7range30.4–30.78,473range8,453–8,5302.44range2.42–2.45
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 192.2range190.7–194.722.1range21.5–22.411,853range11,660–12,162Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 163.1range158.1–163.953.2range53.0–55.14,868range4,703–4,8823.02range2.99–3.07
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 1,102range1,097–1,10523.2range23.2–23.311,258range11,206–11,2720.50range0.50–0.50
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 818.5range814.7–820.523.6range23.4–23.911,082range10,953–11,1680.67range0.65–0.68
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 571.7range565.1–573.218.1range9.3–18.314,490range14,307–28,8520.94range0.91–0.94
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 33.4range33.4–33.5350range349–350734.1range733.2–735.515.48range15.47–15.48
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 97, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 18.4range18.3–18.4428range427–430599.1range596.3–600.627.98range27.98–27.98
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-2bit-g128 · 6/6 validCounted 45.3range44.4–45.3421range419–493609.2range520.6–612.79.65range9.64–9.65
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 533.3range531.6–539.697.7range87.9–98.421,107range20,972–23,4661.83range1.82–1.83
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 338.4range335.9–338.9101range100–10120,476range20,431–20,5772.37range2.35–2.38
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 198.3range197.8–199.294.2range93.9–95.021,872range21,689–21,9563.26range3.25–3.26
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 189.9range187.1–192.887.8range87.6–89.323,503range23,106–23,544Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 160.4range158.8–160.5221range220–2229,292range9,277–9,3214.01range3.92–4.04
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 95, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 897.8range894.8–901.256.2range46.9–56.836,803range36,471–44,2551.78range0.79–2.16
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 90, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 702.2range695.5–703.157.7range57.2–57.935,880range35,780–36,1862.09range1.94–2.38
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 validCounted 526.9range526.0–529.138.7range30.4–38.853,685range53,568–68,7461.01range0.99–1.01
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 33.1range33.0–33.12,145range2,052–2,288956.0range895.4–998.617.87range17.86–17.87
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 97, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 18.3range18.2–18.42,403range2,184–2,451852.7range835.8–938.430.39range30.39–30.40
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 97, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-2bit-g128 · 6/6 validCounted 43.4range40.4–43.73,291range2,966–3,482622.5range588.5–690.914.45range14.45–14.46
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 256 input / 128 output · macOS 27 Not counted
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 565.9range543.6–569.019.9range19.3–21.113,197range12,487–13,611Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 265.2range264.8–265.648.5range48.2–48.85,341range5,313–5,3761.87range1.77–2.01
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 2/6 validPartial 518.9range518.2–519.690.2range89.6–90.822,874range22,714–23,033Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 258.4range257.5–259.3208range208–2119,858range9,759–9,8832.98range2.93–3.03