Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Top Engines

Not enough eligible results to rank.

Speed

Speed, LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27
RankEnginePrefill
Not rankedLiteRT-LM, approximate comparison, see details Google AI Edge166.0 tok/s
Not rankedllama.cpp ggml-orgNot counted
Not rankedMagnitude MagnitudeUnsupported
Not rankedMLX LM Apple (ml-explore)Not counted
Not rankedOllama OllamaNot counted
Not rankedRapid-MLX Rapid-MLX contributorsNot counted
Not rankeduzu MiraiUnsupported
details

LFM2.5 2.6B on M5 Max · 64 GiB · 4-bit · Archive 256 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

LiteRT-LM: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

llama.cpp: Its output check failed: 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Magnitude: Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).

MLX LM: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Ollama: Its output check failed: 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Rapid-MLX: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

uzu: uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).

Memory usage by engine

Memory usage by engine, LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27
RankEngineMemory
Not rankedLiteRT-LM, approximate comparison, see details Google AI Edge2.24 GiB
Not rankedllama.cpp ggml-orgNot counted
Not rankedMagnitude MagnitudeUnsupported
Not rankedMLX LM Apple (ml-explore)Not counted
Not rankedOllama OllamaNot counted
Not rankedRapid-MLX Rapid-MLX contributorsNot counted
Not rankeduzu MiraiUnsupported
details

LFM2.5 2.6B on M5 Max · 64 GiB · 4-bit · Archive 256 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

LiteRT-LM: † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

llama.cpp: Its output check failed: 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Magnitude: Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).

MLX LM: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Ollama: Its output check failed: 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

Rapid-MLX: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end. † Approximate comparison. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 4-bit schemes: litert-int4, Q4_K_M, mlx-affine-4bit-g64.

uzu: uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
ComparisonLiteRT-LMllama.cppMagnitudeMLX LMOllamaRapid-MLXuzu
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted: fewer than two scored enginesMeasured, approximateNot countedUnsupportedNot countedNot countedNot countedUnsupported
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted: fewer than two scored enginesMeasured, approximateNot countedUnsupportedNot countedNot countedNot countedUnsupported

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work