Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Clear filters

Top Engines

Top Engines, Overall, Speed score.
#EngineSpeed score out of 100
197 /100 Score
293 /100 Score
389 /100 Score
481 /100 Score
579 /100 Score
Not ranked in this view
Not rankedRapid-MLX Rapid-MLX contributors Too few results
Not rankeduzu Mirai Too few results
Not rankedLiteRT-LM Google AI Edge No rival yet
details

Speed Decode speed: tokens generated per second.

Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work

The score combines counted comparisons. Select one comparison to see its measured values.

† MLX LM: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M. The experiments use different effective KV cache precision: f16, model-dtype.

† ExecuTorch: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-4w-g64, where most use Q4_K_M. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

† Magnitude: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype.

† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M.

† Ollama: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. The experiments use different effective KV cache precision: f16, model-dtype.

Rapid-MLX: Results in 1 of 4 comparisons; a rank needs 2.

uzu: Results in 1 of 4 comparisons; a rank needs 2.

LiteRT-LM: No other engine has a result in the same comparison yet.

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp76 out of 100, in 3 of 3 comparisons99 out of 100, in 1 of 1 comparisonsFailed Failed in 1 comparison. Partial: 4 of 6 planned trials were valid.No rival yet No other engine has a result in the same comparison yet.
ExecuTorchNot ranked. Results in 1 of 3 comparisons; a rank needs 2.100 out of 100, in 1 of 1 comparisonsNot counted Partial: 2 of 6 planned trials were valid.Not counted Partial: 4 of 6 planned trials were valid.
Magnitude89 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
MLX LM97 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama79 out of 100, in 3 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
LiteRT-LMNo rival yet No other engine has a result in the same comparison yet.No rival yet No other engine has a result in the same comparison yet.Not counted Its output check failed: 9/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none.Not measured on this device.
uzuNot ranked. Results in 1 of 3 comparisons; a rank needs 2.Not measured on this device.Not measured on this device.Failed Failed in 1 comparison. Partial: 4 of 6 planned trials were valid.
Rapid-MLXNot ranked. Results in 1 of 3 comparisons; a rank needs 2.Not measured on this device.Not measured on this device.Not measured on this device.

Speed

TTFT ms

Generation tok/s

Speed, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27
RankEngineDecode and TTFT
1uzu, approximate comparison, see details Mirai528.7 tok/s 126 ms TTFT
2ExecuTorch, approximate comparison, see details PyTorch (Meta)456.8 tok/s 332 ms TTFT
3Magnitude, approximate comparison, see details Magnitude450.2 tok/s 109 ms TTFT
4Ollama, approximate comparison, see details Ollama442.1 tok/s 139 ms TTFT
5llama.cpp, approximate comparison, see details ggml-org436.7 tok/s 145 ms TTFT
Not rankedLiteRT-LM Google AI EdgeNot counted
Not rankedMLX LM Apple (ml-explore)Not counted
Not rankedRapid-MLX Rapid-MLX contributorsNot counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · Archive 2048 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

uzu: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M.

ExecuTorch: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-4w-g64, where most use Q4_K_M.

Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.

LiteRT-LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: litert-int4, where most use Q4_K_M.

MLX LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.

Rapid-MLX: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.

Memory usage by engine

Memory usage by engine, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27
RankEngineMemory
1Ollama, approximate comparison, see details Ollama0.95 GiB
2llama.cpp, approximate comparison, see details ggml-org0.95 GiB
3uzu, approximate comparison, see details Mirai0.95 GiB
4Magnitude, approximate comparison, see details Magnitude1.56 GiB
Not rankedExecuTorch PyTorch (Meta)Not measured
Not rankedLiteRT-LM Google AI EdgeNot counted
Not rankedMLX LM Apple (ml-explore)Not counted
Not rankedRapid-MLX Rapid-MLX contributorsNot counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · Archive 2048 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.

uzu: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M.

Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

ExecuTorch: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-4w-g64, where most use Q4_K_M.

LiteRT-LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: litert-int4, where most use Q4_K_M.

MLX LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.

Rapid-MLX: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
ComparisonMLX LMExecuTorchMagnitudellama.cppOllamaRapid-MLXuzuLiteRT-LM
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted86, approximate85, approximate83, approximate84, approximateNot counted100, approximateNot counted
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 95, approximateNot counted93, approximate69, approximate74, approximate100, approximateFailedNot counted
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 100, approximateNot runUnsupported76, approximate78, approximateNot runUnsupportedNot run
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not run100, approximateNot run99, approximateNot runNot runNot runNot counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesNot runNot countedNot runMeasured, approximateNot runNot runNot countedNot run
LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesNot runNot runNot runNot countedNot runNot runUnsupportedNot run
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesNot runNot countedNot runMeasured, approximateNot runNot runFailedNot run
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted: fewer than two scored enginesNot countedNot runUnsupportedNot countedNot countedNot countedUnsupportedMeasured, approximate
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot runNot runNot runNot countedNot runNot runNot runMeasured, approximate
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot runNot countedNot runMeasured, approximateNot runNot runNot runNot counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot runNot countedNot runNot countedNot runNot runNot runNot counted
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot runNot runNot runFailedNot runNot runNot runNot counted
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot runNot countedNot runNot countedNot runNot runNot runNot counted

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work