Rankings

Engine

Rapid-MLX

Rapid-MLX contributors · Apache-2.0, open source · Homepage

Top Engines

Desktop
Not ranked Results in 2 of 4 comparisons; a rank needs all 4.
Mobile
Not measured
details

Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.

How rankings work

Supporting experiments

Rapid-MLX: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 575.8range523.4–582.934.8range34.2–36.67,493range7,106–7,6221.12range1.11–1.14
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 97, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 361.9range358.8–364.833.5range31.3–37.37,775range6,968–8,3281.69range1.68–1.70
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 99, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 validCounted 198.0range194.6–199.032.7range31.1–34.07,969range7,656–8,3872.70range2.67–2.70
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 162.6range161.1–163.661.2range58.9–61.94,220range4,175–4,3893.28range3.23–3.41
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 1,108range1,100–1,11024.5range23.9–25.610,658range10,202–10,9490.72range0.68–0.74
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 799.5range794.3–820.029.2range25.1–31.38,949range8,338–10,3900.90range0.88–0.92
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 validCounted 559.0range554.2–560.326.2range20.2–27.49,995range9,554–13,0671.17range1.14–1.18
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 538.5range536.5–541.9105range103–10619,715range19,457–20,1041.98range1.94–1.99
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 340.6range340.0–342.4105range100–10519,613range19,565–20,5752.54range2.53–2.54
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 98, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 validCounted 195.1range192.4–197.0100range96.3–10220,564range20,184–21,4303.45range3.42–3.46
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 160.0range159.4–160.8242range240–2428,490range8,484–8,5604.14range4.10–4.22
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCounted 946.8range941.4–949.557.6range55.2–57.935,995range35,755–37,4891.83range1.58–2.17
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-8bit-g64 · 6/6 validCounted 712.2range711.3–714.661.9range59.5–65.333,458range31,693–34,7902.05range1.91–2.35
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal BF16 · 6/6 validCounted 532.2range526.9–535.646.9range45.9–49.344,254range42,124–45,2581.28range1.27–1.33
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 265.2range263.1–265.658.0range55.3–59.34,457range4,360–4,6842.12range2.06–2.29
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 1.0.2 · Metal mlx-affine-4bit-g64 · 6/6 validCheck failed 258.3range257.5–259.7234range231–2368,790range8,698–8,9093.15range3.10–3.20