Engine
Rapid-MLX
Rapid-MLX contributors · Apache-2.0, open source · Homepage
Top Engines
- Desktop
- Not ranked
- Mobile
- Not measured
details
Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 575.8range523.4–582.9 | 34.8range34.2–36.6 | 7,493range7,106–7,622 | 1.12range1.11–1.14 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 97, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 361.9range358.8–364.8 | 33.5range31.3–37.3 | 7,775range6,968–8,328 | 1.69range1.68–1.70 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 99, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 valid | Counted | 198.0range194.6–199.0 | 32.7range31.1–34.0 | 7,969range7,656–8,387 | 2.70range2.67–2.70 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 162.6range161.1–163.6 | 61.2range58.9–61.9 | 4,220range4,175–4,389 | 3.28range3.23–3.41 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 1,108range1,100–1,110 | 24.5range23.9–25.6 | 10,658range10,202–10,949 | 0.72range0.68–0.74 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 799.5range794.3–820.0 | 29.2range25.1–31.3 | 8,949range8,338–10,390 | 0.90range0.88–0.92 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 valid | Counted | 559.0range554.2–560.3 | 26.2range20.2–27.4 | 9,995range9,554–13,067 | 1.17range1.14–1.18 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 538.5range536.5–541.9 | 105range103–106 | 19,715range19,457–20,104 | 1.98range1.94–1.99 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 340.6range340.0–342.4 | 105range100–105 | 19,613range19,565–20,575 | 2.54range2.53–2.54 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 98, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 valid | Counted | 195.1range192.4–197.0 | 100range96.3–102 | 20,564range20,184–21,430 | 3.45range3.42–3.46 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 160.0range159.4–160.8 | 242range240–242 | 8,490range8,484–8,560 | 4.14range4.10–4.22 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Counted | 946.8range941.4–949.5 | 57.6range55.2–57.9 | 35,995range35,755–37,489 | 1.83range1.58–2.17 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 valid | Counted | 712.2range711.3–714.6 | 61.9range59.5–65.3 | 33,458range31,693–34,790 | 2.05range1.91–2.35 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 valid | Counted | 532.2range526.9–535.6 | 46.9range45.9–49.3 | 44,254range42,124–45,258 | 1.28range1.27–1.33 |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 265.2range263.1–265.6 | 58.0range55.3–59.3 | 4,457range4,360–4,684 | 2.12range2.06–2.29 |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 valid | Check failed | 258.3range257.5–259.7 | 234range231–236 | 8,790range8,698–8,909 | 3.15range3.10–3.20 |