Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Clear filters

Top Engines

Overall needs mobile results; each platform is ranked on its own until then.

Desktop

Top Engines, Desktop, Speed score.
#EngineSpeed score out of 100
1100 /100 Score
298 /100 Score
396 /100 Score
494 /100 Score
591 /100 Score
684 /100 Score
details

Speed Decode speed: tokens generated per second.

Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work

The score combines counted comparisons. Select one comparison to see its measured values.

† MLX LM: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

† Rapid-MLX: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

† Magnitude: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

† Ollama: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

† ZML: approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp91 out of 100, in 3 of 3 comparisonsNo rival yet No other engine has a result in the same comparison yet.No rival yet No other engine has a result in the same comparison yet.No rival yet No other engine has a result in the same comparison yet.
Magnitude96 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
MLX LM100 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama94 out of 100, in 3 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Rapid-MLX98 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
ZML84 out of 100, in 2 of 3 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.

Speed

TTFT ms

Generation tok/s

Speed, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27
RankEngineDecode and TTFT
1Ollama, approximate comparison, see details Ollama199.6 tok/s 33.8 ms TTFT
2Magnitude, approximate comparison, see details Magnitude199.6 tok/s 28.7 ms TTFT
3MLX LM, approximate comparison, see details Apple (ml-explore)199.0 tok/s 30.7 ms TTFT
4Rapid-MLX, approximate comparison, see details Rapid-MLX contributors198.0 tok/s 32.7 ms TTFT
5ZML, approximate comparison, see details ZML192.6 tok/s 24.9 ms TTFT
6llama.cpp, approximate comparison, see details ggml-org190.5 tok/s 36.7 ms TTFT
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · bf16 · Archive 256 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

MLX LM: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

Rapid-MLX: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

ZML: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

Memory usage by engine

details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · bf16 · Archive 256 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

MLX LM: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

Rapid-MLX: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default.

Magnitude: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

ZML: † Approximate comparison. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. Its phases were timed by the harness clock around the engine API, so they include transport that engine counters leave out.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
Comparisonllama.cppMagnitudeMLX LMOllamaRapid-MLXZML
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 95, approximate100, approximate100, approximate100, approximate99, approximate96, approximate
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 96UnsupportedNot run100Not runUnsupported
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 81, approximate92, approximate100, approximate84, approximate98, approximate73, approximate
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted: fewer than two scored enginesUnsupportedNot runNot runNot runNot runNot run
LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot run
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot run
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot run
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedNot runNot runNot runNot runNot run
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesMeasuredNot runNot runNot runNot runNot run

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work