Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Top Engines

Not enough eligible results to rank.

Speed

Speed, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27
RankEnginePrefill
Not rankedllama.cpp, approximate comparison, see details ggml-org10,667 tok/s
Not rankedMLX LM Apple (ml-explore)Not counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 256 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

llama.cpp: † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

MLX LM: Its output check failed: 10/12; needs 11 and all mandatory checks. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

Memory usage by engine

Memory usage by engine, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27
RankEngineMemory
Not rankedllama.cpp ggml-orgNot measured
Not rankedMLX LM Apple (ml-explore)Not counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 256 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

llama.cpp: No peak memory measured. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

MLX LM: Its output check failed: 10/12; needs 11 and all mandatory checks. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
Comparisonllama.cppMLX LM
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 256 input / 128 output · macOS 27 Not counted: fewer than two scored enginesMeasured, approximateNot counted

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work