Rankings

Engine

Magnitude

Magnitude · Apache-2.0, open source · Homepage

Top Engines

Desktop
Not ranked Results in 2 of 4 comparisons; a rank needs all 4. Unsupported in 2.
Mobile
Not measured
details

Approximate comparison: The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.

How rankings work

Supporting experiments

Magnitude: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 82, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 validCounted 444.5range437.1–446.026.6range26.2–26.89,627range9,540–9,7711.56range1.55–1.56
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 88, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 validCounted 330.1range327.3–330.925.5range25.3–25.610,049range9,989–10,1091.90range1.89–1.90
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 validCounted 199.6range194.8–200.428.7range19.6–31.08,926range8,265–13,0762.95range2.94–2.96
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 82, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 validCounted 911.9range904.9–915.918.7range18.4–25.013,679range10,229–13,9221.05range1.05–1.06
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 93, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 validCounted 764.7range759.1–768.918.0range17.8–19.114,188range13,437–14,3491.04range1.03–1.04
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 validCounted 524.1range512.1–526.019.1range18.9–19.513,435range13,137–13,5471.35range1.33–1.35
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q2_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 85, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 validCounted 450.2range441.1–466.3109range96.1–12018,730range17,068–21,3191.56range1.56–1.56
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 89, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 validCounted 329.6range328.4–330.996.3range96.1–96.521,262range21,223–21,3081.90range1.90–1.90
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 validCounted 199.6range197.5–200.8113range113–11918,058range17,233–18,1132.96range2.96–2.96
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 93, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 6/6 validCounted 884.9range879.3–906.646.5range46.3–49.144,058range41,750–44,2661.05range1.05–1.06
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 6/6 validCounted 747.9range731.2–756.844.4range44.3–44.546,119range46,034–46,2751.04range1.04–1.05
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal BF16 · 6/6 validCounted 510.1range491.6–519.048.9range48.7–49.041,885range41,783–42,0281.35range1.34–1.35
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q2_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.2.17 · Metal Q4_K_M · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured