Rankings

Explore benchmarks

Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.

Top Engines

Not enough eligible results to rank.

Speed

Speed, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27
RankEngineDecode and TTFT
Not rankedllama.cpp ggml-orgNot counted
Not rankedMLX LM Apple (ml-explore)Not counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 2048 input / 128 output · macOS 27

Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.

llama.cpp: Partial: 3 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

MLX LM: Partial: 2 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

Memory usage by engine

Memory usage by engine, LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27
RankEngineMemory
Not rankedllama.cpp ggml-orgNot counted
Not rankedMLX LM Apple (ml-explore)Not counted
details

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 2048 input / 128 output · macOS 27

Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.

llama.cpp: Partial: 3 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

MLX LM: Partial: 2 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.

Every comparison

Comparisons in this view: each engine's score in each comparison, out of 100, or why it has none.
Comparisonllama.cppMLX LM
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted: fewer than two scored enginesNot countedNot counted

Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work