Explore benchmarks
Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.
Top Engines
| # | Engine | Speed score out of 100 |
|---|---|---|
| 1 | uzu†, an approximate comparison, see details | 100 /100 Score |
| ||
| 2 | MLX LM†, an approximate comparison, see details | 95 /100 Score |
| ||
| 3 | Rapid-MLX†, an approximate comparison, see details | 95 /100 Score |
| ||
| 4 | Magnitude†, an approximate comparison, see details | 91 /100 Score |
| ||
| 5 | Ollama†, an approximate comparison, see details | 90 /100 Score |
| ||
| 6 | llama.cpp†, an approximate comparison, see details | 87 /100 Score |
| ||
| Not ranked in this view | ||
| Not ranked | ExecuTorch | Too few results |
| Not ranked | LiteRT-LM | Too few results |
| Not ranked | ZML | Too few results |
details
Speed Decode speed: tokens generated per second.
Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work
The score combines counted comparisons. Select one comparison to see its measured values.
† uzu: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mirai-l, where most use Q8_0.
† MLX LM: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.
† Rapid-MLX: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: mlx-affine-8bit-g64, where most use Q8_0. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.
† Magnitude: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.
† Ollama: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.
† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.
ExecuTorch: Results in 1 of 3 comparisons; a rank needs 2.
LiteRT-LM: Results in 1 of 3 comparisons; a rank needs 2.
ZML: Results in 1 of 3 comparisons; a rank needs 2.
Coverage
| Desktop | Mobile | |||
|---|---|---|---|---|
| M5 Max 64 GiB macOS 27 Apple M5 Max | OnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750) | Pixel 8 8 GB Android 16 Google Tensor G3 | iPhone 12 4 GB iOS 27 Apple A14 Bionic | |
| llama.cpp | 873/3 out of 100, in 3 of 3 comparisons | 992/2 out of 100, in 2 of 2 comparisons | No rival yet No other engine has a result in the same comparison yet. | No rival yet No other engine has a result in the same comparison yet. |
| uzu | 1002/3 out of 100, in 2 of 3 comparisons | Not measured on this device. | Not measured on this device. | Not counted Partial: 4 of 6 planned trials were valid. |
| Magnitude | 913/3 out of 100, in 3 of 3 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| MLX LM | 952/3 out of 100, in 2 of 3 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Ollama | 903/3 out of 100, in 3 of 3 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Rapid-MLX | 952/3 out of 100, in 2 of 3 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| ExecuTorch | 1/3Not ranked. Results in 1 of 3 comparisons; a rank needs 2. | 1/2Not ranked. Results in 1 of 2 comparisons; a rank needs 2. | Not counted Partial: 2 of 6 planned trials were valid. | Not counted Partial: 4 of 6 planned trials were valid. |
| LiteRT-LM | 1/3Not ranked. Results in 1 of 3 comparisons; a rank needs 2. | 1/2Not ranked. Results in 1 of 2 comparisons; a rank needs 2. | Not counted Its output check failed: 9/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. | Not measured on this device. |
| ZML | 1/3Not ranked. Results in 1 of 3 comparisons; a rank needs 2. | Not measured on this device. | Not measured on this device. | Not measured on this device. |
Speed
| Rank | Engine | Decode and TTFT |
|---|---|---|
| 1 | uzu†, approximate comparison, see details | 528.7 tok/s 126 ms TTFT |
| 2 | ExecuTorch†, approximate comparison, see details | 456.8 tok/s 332 ms TTFT |
| 3 | Magnitude†, approximate comparison, see details | 450.2 tok/s 109 ms TTFT |
| 4 | Ollama†, approximate comparison, see details | 442.1 tok/s 139 ms TTFT |
| 5 | llama.cpp†, approximate comparison, see details | 436.7 tok/s 145 ms TTFT |
| Not ranked | LiteRT-LM | Not counted |
| Not ranked | MLX LM | Not counted |
| Not ranked | Rapid-MLX | Not counted |
details
LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · Archive 2048 input / 128 output · macOS 27
Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.
uzu: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M.
ExecuTorch: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-4w-g64, where most use Q4_K_M.
Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.
Ollama: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.
llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.
LiteRT-LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: litert-int4, where most use Q4_K_M.
MLX LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.
Rapid-MLX: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.
Memory usage by engine
| Rank | Engine | Memory |
|---|---|---|
| 1 | Ollama†, approximate comparison, see details | 0.95 GiB |
| 2 | llama.cpp†, approximate comparison, see details | 0.95 GiB |
| 3 | uzu†, approximate comparison, see details | 0.95 GiB |
| 4 | Magnitude†, approximate comparison, see details | 1.56 GiB |
| Not ranked | ExecuTorch | Not measured |
| Not ranked | LiteRT-LM | Not counted |
| Not ranked | MLX LM | Not counted |
| Not ranked | Rapid-MLX | Not counted |
details
LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · Archive 2048 input / 128 output · macOS 27
Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.
Ollama: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.
llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default.
uzu: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mirai-m, where most use Q4_K_M.
Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.
ExecuTorch: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-4w-g64, where most use Q4_K_M.
LiteRT-LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: litert-int4, where most use Q4_K_M.
MLX LM: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.
Rapid-MLX: Its output check failed: 10/12 of the checks the model passes at BF16; needs 11 and the mandatory ones (needle-start, needle-end, continuation-128). Left out, failed by every BF16 configuration: none. † Approximate comparison. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 4-bit in another scheme: mlx-affine-4bit-g64, where most use Q4_K_M.
Every comparison
| Comparison | uzu | MLX LM | Rapid-MLX | Magnitude | Ollama | llama.cpp | ExecuTorch | LiteRT-LM | ZML |
|---|---|---|---|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | 100†, approximate | Not counted | Not counted | 85†, approximate | 84†, approximate | 83†, approximate | 86†, approximate | Not counted | Not run |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 | 100†, approximate | 92†, approximate | 92†, approximate | 89†, approximate | 87†, approximate | 87†, approximate | Not run | 32†, approximate | Not run |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 | Not run | 99†, approximate | 98†, approximate | 100†, approximate | 99†, approximate | 92†, approximate | Not run | Not run | 76†, approximate |
Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work