Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Clear filters

Top Engines

Top Engines, Overall, Speed score.
#EngineSpeed score out of 100
176 /100 Score
Not ranked in this view
Not rankedLiteRT-LM Google AI Edge Too few results
Not rankedMagnitude Magnitude Too few results
Not rankedMLX LM Apple (ml-explore) Too few results
Not rankedOllama Ollama Too few results
Not rankedRapid-MLX Rapid-MLX contributors Too few results
Not rankeduzu Mirai Too few results
Not rankedZML ZML Too few results
Not rankedExecuTorch PyTorch (Meta) Not counted
details

Speed Decode speed: tokens generated per second.

Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work

The score combines counted comparisons. Select one comparison to see its measured values.

† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m. The experiments use different effective KV cache precision: f16, uzu default. The experiments here use different 8-bit schemes: Q8_0, mirai-l. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.

LiteRT-LM: Results in 6 of 13 comparisons; a rank needs 7.

Magnitude: Results in 6 of 13 comparisons; a rank needs 7.

MLX LM: Results in 6 of 13 comparisons; a rank needs 7.

Ollama: Results in 6 of 13 comparisons; a rank needs 7.

Rapid-MLX: Results in 6 of 13 comparisons; a rank needs 7.

uzu: Results in 6 of 13 comparisons; a rank needs 7.

ZML: Results in 2 of 13 comparisons; a rank needs 7.

ExecuTorch: Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128.

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp74 out of 100, in 6 of 6 comparisons100 out of 100, in 2 of 2 comparisons90 out of 100, in 2 of 2 comparisons58 out of 100, in 3 of 3 comparisons
LiteRT-LMNot ranked. Results in 2 of 6 comparisons; a rank needs 3.70 out of 100, in 2 of 2 comparisons96 out of 100, in 2 of 2 comparisonsNot measured on this device.
uzu96 out of 100, in 3 of 6 comparisonsNot measured on this device.Not measured on this device.100 out of 100, in 3 of 3 comparisons
Magnitude92 out of 100, in 6 of 6 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
MLX LM97 out of 100, in 6 of 6 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama78 out of 100, in 6 of 6 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Rapid-MLX98 out of 100, in 6 of 6 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
ExecuTorchNot counted Its output check failed: 6/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128.Not counted Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128.Not counted Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128.Not counted Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128.
ZMLNot ranked. Results in 2 of 6 comparisons; a rank needs 3.Not measured on this device.Not measured on this device.Not measured on this device.

Speed

TTFT ms

Generation tok/s

Speed, LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27
RankEngineDecode and TTFT
1uzu, approximate comparison, see details Mirai130.8 tok/s 384 ms TTFT
2llama.cpp, approximate comparison, see details ggml-org56.0 tok/s 314 ms TTFT
Not rankedExecuTorch PyTorch (Meta)Not counted
details

LFM2.5 350M on iPhone 12 · 4 GB · 4-bit · Archive 256 input / 128 output · iOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

uzu: † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

ExecuTorch: Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128. † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

Memory usage by engine

Memory usage by engine, LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27
RankEngineMemory
Not rankedExecuTorch PyTorch (Meta)Not counted
Not rankedllama.cpp ggml-orgNot measured
Not rankeduzu MiraiNot measured
details

LFM2.5 350M on iPhone 12 · 4 GB · 4-bit · Archive 256 input / 128 output · iOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

ExecuTorch: Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128. † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

llama.cpp: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

uzu: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
Comparisonllama.cppLiteRT-LMMagnitudeMLX LMOllamaRapid-MLXuzuZMLExecuTorch
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 43, approximateNot runNot runNot runNot runNot run100, approximateNot runNot counted
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 69, approximateNot runNot runNot runNot runNot run100, approximateNot runNot run
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 67, approximateNot runNot runNot runNot runNot run100, approximateNot runNot run
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 69, approximateNot counted93, approximate95, approximate74, approximate100, approximateFailedNot runNot counted
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 62, approximateNot counted82, approximate100, approximate67, approximate100, approximate90, approximateNot runNot counted
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 76, approximate26, approximate96, approximate90, approximate77, approximate92, approximate100, approximateNot runNot run
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 75, approximate24, approximate93, approximate100, approximate77, approximate98, approximate97, approximateNot runNot run
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 83, approximateNot run96, approximate99, approximate88, approximate100, approximateNot run49, approximateNot run
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 81, approximateNot run92, approximate100, approximate84, approximate98, approximateNot run73, approximateNot run
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 100, approximate76, approximateNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 100, approximate64, approximateNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 81, approximate100, approximateNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 100, approximate93, approximateNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesMeasured, approximateNot runNot runNot runNot runNot runFailedNot runNot counted
LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot runNot runNot runNot runNot runNot runNot counted
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot runNot runNot runNot runNot runNot runNot counted
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedNot countedNot runNot runNot runNot runNot runNot runNot counted
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot runNot runNot runNot runNot runNot runNot counted
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot runNot runNot runNot runNot runNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot runNot runNot runNot run

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work