Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Clear filters

Top Engines

Top Engines, Mobile, Speed score.
#EngineSpeed score out of 100
192 /100 Score
286 /100 Score
Not ranked in this view
Not rankedExecuTorch PyTorch (Meta) Too few results
details

Speed Decode speed: tokens generated per second.

Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work

The score combines counted comparisons. Select one comparison to see its measured values.

† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.

† LiteRT-LM: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.

ExecuTorch: Results in 3 of 10 comparisons; a rank needs 5.

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp85 out of 100, in 24 of 24 comparisons97 out of 100, in 6 of 6 comparisons86 out of 100, in 4 of 4 comparisons69 out of 100, in 5 of 5 comparisons
LiteRT-LMNot ranked. Results in 4 of 24 comparisons; a rank needs 12.78 out of 100, in 4 of 6 comparisons98 out of 100, in 3 of 4 comparisonsNot measured on this device.
uzuNot ranked. Results in 7 of 24 comparisons; a rank needs 12.Not measured on this device.Not measured on this device.100 out of 100, in 5 of 5 comparisons
Magnitude91 out of 100, in 12 of 24 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
MLX LM98 out of 100, in 20 of 24 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama88 out of 100, in 22 of 24 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Rapid-MLX98 out of 100, in 12 of 24 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
ExecuTorchNot ranked. Results in 2 of 24 comparisons; a rank needs 12.Not ranked. Results in 2 of 6 comparisons; a rank needs 3.Not ranked. Results in 1 of 4 comparisons; a rank needs 2.Not ranked. Results in 1 of 5 comparisons; a rank needs 3.
ZMLNot ranked. Results in 4 of 24 comparisons; a rank needs 12.Not measured on this device.Not measured on this device.Not measured on this device.

Speed

Speed, LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16
RankEnginePrefill
1ExecuTorch, approximate comparison, see details PyTorch (Meta)559.0 tok/s
2llama.cpp, approximate comparison, see details ggml-org348.1 tok/s
Not rankedLiteRT-LM Google AI EdgeNot counted
details

LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB · 4-bit · Archive 256 input / 128 output · Android 16

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

ExecuTorch: † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

LiteRT-LM: Its output check failed: 9/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

Memory usage by engine

Memory usage by engine, LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16
RankEngineMemory
Not rankedExecuTorch PyTorch (Meta)Not measured
Not rankedLiteRT-LM Google AI EdgeNot counted
Not rankedllama.cpp ggml-orgNot measured
details

LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB · 4-bit · Archive 256 input / 128 output · Android 16

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

ExecuTorch: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

LiteRT-LM: Its output check failed: 9/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

llama.cpp: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
Comparisonllama.cppLiteRT-LMExecuTorch
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 99, approximateNot counted100, approximate
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 83, approximateNot counted100, approximate
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 100, approximate87, approximateNot run
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 100, approximate86, approximateNot run
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 100, approximate76, approximateNot run
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 100, approximate64, approximateNot run
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 78, approximateNot counted100, approximate
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 87, approximate100, approximateNot run
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 81, approximate100, approximateNot run
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 100, approximate93, approximateNot run
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot run
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot run
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedMeasured, approximateNot run
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedMeasured, approximateNot run
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot run
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot run
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot counted
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot counted
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot run
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot run
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedNot countedNot counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot run
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot run
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot countedNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedMeasured, approximateNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedFailedNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedNot countedNot run
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedNot countedNot counted
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasured, approximateNot countedNot counted
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot run

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work