Rankings

Engine

uzu

Mirai · MIT, open source · Homepage

Top Engines

Desktop
Not ranked Results in 2 of 4 comparisons; a rank needs all 4. Unsupported in 2.
Mobile
Not ranked Results in 3 of 12 comparisons; a rank needs all 12.
details

Approximate comparison: The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mirai-l, where most use Q8_0. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m. The experiments use different effective KV cache precision: f16, uzu default. The experiments here use different 8-bit schemes: Q8_0, mirai-l.

How rankings work

Supporting experiments

uzu: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 6/6 validCounted 540.7range539.8–541.475.2range74.9–76.913,205range13,070–13,7000.90range0.90–0.96
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 374.3range373.4–374.775.1range74.3–76.513,385range13,101–14,4261.46range1.46–1.52
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 90, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 6/6 validCounted 1,001range996.6–1,00367.9range67.2–68.719,884range19,381–20,8250.48range0.48–0.53
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 97, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 794.4range790.4–795.569.2range66.9–71.419,693range18,755–20,5230.67range0.67–0.71
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 6/6 validCounted 528.7range525.9–529.0126range124–13031,301range31,245–31,6110.95range0.95–0.95
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 368.8range368.5–369.0127range126–13030,505range30,394–30,6251.51range1.51–1.51
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 100, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 776.1range771.4–779.688.8range86.9–90.366,557range65,117–71,3240.70range0.70–0.71
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 6/6 validCounted 45.4range44.4–46.01,073range1,055–1,117257.7range257.1–259.1Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 25.7range25.4–25.81,048range1,032–1,110260.0range258.8–262.3Not measured
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 100, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 6/6 validCounted 130.8range126.8–131.2384range379–400786.2range777.4–795.5Not measured
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 100, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 74.8range73.9–75.4383range373–399789.4range783.3–802.6Not measured
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Speed score 100, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 6/6 validCounted 70.2range68.8–71.02,356range2,346–2,395895.5range894.2–897.8Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 4/6 validPartial 42.3range41.9–42.67,784range7,768–8,145265.3range253.4–265.9Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 2/6 validPartial 24.7range24.7–24.87,790range7,709–7,871265.1range262.3–267.8Not measured
LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-l · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: 0.6.17 · Metal mirai-m · 0/6 validFailed Not measuredNot measuredNot measuredNot measured