Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Clear filters

Top Engines

Overall needs mobile results; each platform is ranked on its own until then.

Desktop

Top Engines, Desktop, Speed score.
#EngineSpeed score out of 100
1100 /100 Score
2100 /100 Score
395 /100 Score
490 /100 Score
Not ranked in this view
Not rankedMagnitude Magnitude Unsupported
Not rankeduzu Mirai Unsupported
Not rankedLiteRT-LM Google AI Edge Not counted
details

Speed Decode speed: tokens generated per second.

Score: 100 is the best result in every comparison an engine entered; 80 is 80% of the best, on average. Only complete experiments with a passed output check count. How rankings work

The score combines counted comparisons. Select one comparison to see its measured values.

† MLX LM: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

† Rapid-MLX: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

† Ollama: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

† llama.cpp: approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Magnitude: Unsupported in 2 comparisons. Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).

uzu: Unsupported in 2 comparisons. uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).

LiteRT-LM: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp90 out of 100, in 2 of 2 comparisonsNo rival yet No other engine has a result in the same comparison yet.Failed Failed in 1 comparison. No trial was valid.Unsupported Unsupported in 2 comparisons. llama.cpp errored on the arithmetic check in qualification: The app failed: llama_decode(token 1) returned -3
MLX LM100 out of 100, in 2 of 2 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama95 out of 100, in 2 of 2 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Rapid-MLX100 out of 100, in 2 of 2 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
LiteRT-LMNot counted Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.Not counted Partial: 2 of 6 planned trials were valid.Failed Failed in 1 comparison. No trial was valid.Not measured on this device.
uzuUnsupported Unsupported in 2 comparisons. uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).Not measured on this device.Not measured on this device.Unsupported Unsupported in 2 comparisons. uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).
MagnitudeUnsupported Unsupported in 2 comparisons. Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).Not measured on this device.Not measured on this device.Not measured on this device.

Speed

TTFT ms

Generation tok/s

Speed, LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27
RankEngineDecode and TTFT
1MLX LM, approximate comparison, see details Apple (ml-explore)163.1 tok/s 53.2 ms TTFT
2Rapid-MLX, approximate comparison, see details Rapid-MLX contributors162.6 tok/s 61.2 ms TTFT
3Ollama, approximate comparison, see details Ollama155.1 tok/s 52.7 ms TTFT
4llama.cpp, approximate comparison, see details ggml-org148.8 tok/s 62.8 ms TTFT
Not rankedLiteRT-LM Google AI EdgeNot counted
Not rankedMagnitude MagnitudeUnsupported
Not rankeduzu MiraiUnsupported
details

LFM2.5 2.6B on M5 Max · 64 GiB · 8-bit · Archive 256 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

MLX LM: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Rapid-MLX: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

LiteRT-LM: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Magnitude: Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).

uzu: uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).

Memory usage by engine

Memory usage by engine, LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27
RankEngineMemory
1Ollama, approximate comparison, see details Ollama2.96 GiB
2llama.cpp, approximate comparison, see details ggml-org2.96 GiB
3MLX LM, approximate comparison, see details Apple (ml-explore)3.02 GiB
4Rapid-MLX, approximate comparison, see details Rapid-MLX contributors3.28 GiB
Not rankedLiteRT-LM Google AI EdgeNot counted
Not rankedMagnitude MagnitudeUnsupported
Not rankeduzu MiraiUnsupported
details

LFM2.5 2.6B on M5 Max · 64 GiB · 8-bit · Archive 256 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

Ollama: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

MLX LM: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Rapid-MLX: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

LiteRT-LM: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.

Magnitude: Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).

uzu: uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
ComparisonLiteRT-LMllama.cppMagnitudeMLX LMOllamaRapid-MLXuzu
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Not counted89, approximateUnsupported100, approximate96, approximate100, approximateUnsupported
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Not counted91, approximateUnsupported100, approximate95, approximate100, approximateUnsupported
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted: fewer than two scored enginesNot runUnsupportedNot runNot runNot runNot runUnsupported
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Not counted: fewer than two scored enginesNot runUnsupportedNot runNot runNot runNot runUnsupported
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedMeasured, approximateNot runNot runNot runNot runNot run
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedMeasured, approximateNot runNot runNot runNot runNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted: fewer than two scored enginesFailedFailedNot runNot runNot runNot runNot run
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted: fewer than two scored enginesNot countedNot countedNot runNot runNot runNot runNot run

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work