Engine
MLX LM
Apple (ml-explore) · MIT, open source · Homepage
Top Engines
- Desktop
- Rank 1, 95 out of 100, range 95 to 96, approximate comparison
- Mobile
- Not measured
details
Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M. Its weights are 2-bit in another scheme: mlx-affine-2bit-g128, where most use Q2_0. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 582.9range574.9–589.7 | 28.9range28.6–29.3 | 9,024range8,897–9,092 | 0.89range0.87–0.90 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 96, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 361.1range359.7–362.6 | 30.7range30.6–31.2 | 8,486range8,348–8,513 | 1.44range1.43–1.45 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 199.0range197.4–202.2 | 30.7range30.4–30.7 | 8,473range8,453–8,530 | 2.44range2.42–2.45 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 192.2range190.7–194.7 | 22.1range21.5–22.4 | 11,853range11,660–12,162 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 163.1range158.1–163.9 | 53.2range53.0–55.1 | 4,868range4,703–4,882 | 3.02range2.99–3.07 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 1,102range1,097–1,105 | 23.2range23.2–23.3 | 11,258range11,206–11,272 | 0.50range0.50–0.50 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 818.5range814.7–820.5 | 23.6range23.4–23.9 | 11,082range10,953–11,168 | 0.67range0.65–0.68 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 571.7range565.1–573.2 | 18.1range9.3–18.3 | 14,490range14,307–28,852 | 0.94range0.91–0.94 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 33.4range33.4–33.5 | 350range349–350 | 734.1range733.2–735.5 | 15.48range15.47–15.48 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 97, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 18.4range18.3–18.4 | 428range427–430 | 599.1range596.3–600.6 | 27.98range27.98–27.98 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-2bit-g128 · 6/6 valid | Counted | 45.3range44.4–45.3 | 421range419–493 | 609.2range520.6–612.7 | 9.65range9.64–9.65 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 533.3range531.6–539.6 | 97.7range87.9–98.4 | 21,107range20,972–23,466 | 1.83range1.82–1.83 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 338.4range335.9–338.9 | 101range100–101 | 20,476range20,431–20,577 | 2.37range2.35–2.38 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 198.3range197.8–199.2 | 94.2range93.9–95.0 | 21,872range21,689–21,956 | 3.26range3.25–3.26 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 189.9range187.1–192.8 | 87.8range87.6–89.3 | 23,503range23,106–23,544 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 160.4range158.8–160.5 | 221range220–222 | 9,292range9,277–9,321 | 4.01range3.92–4.04 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 95, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 897.8range894.8–901.2 | 56.2range46.9–56.8 | 36,803range36,471–44,255 | 1.78range0.79–2.16 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 90, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 702.2range695.5–703.1 | 57.7range57.2–57.9 | 35,880range35,780–36,186 | 2.09range1.94–2.38 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 99, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal BF16 · 6/6 valid | Counted | 526.9range526.0–529.1 | 38.7range30.4–38.8 | 53,685range53,568–68,746 | 1.01range0.99–1.01 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 33.1range33.0–33.1 | 2,145range2,052–2,288 | 956.0range895.4–998.6 | 17.87range17.86–17.87 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 97, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 18.3range18.2–18.4 | 2,403range2,184–2,451 | 852.7range835.8–938.4 | 30.39range30.39–30.40 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 97, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-2bit-g128 · 6/6 valid | Counted | 43.4range40.4–43.7 | 3,291range2,966–3,482 | 622.5range588.5–690.9 | 14.45range14.45–14.46 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 565.9range543.6–569.0 | 19.9range19.3–21.1 | 13,197range12,487–13,611 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 265.2range264.8–265.6 | 48.5range48.2–48.8 | 5,341range5,313–5,376 | 1.87range1.77–2.01 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 2/6 valid | Partial | 518.9range518.2–519.6 | 90.2range89.6–90.8 | 22,874range22,714–23,033 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.32.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 258.4range257.5–259.3 | 208range208–211 | 9,858range9,759–9,883 | 2.98range2.93–3.03 |