Explore benchmarks
Filter by model, hardware, workload and quantization. Every score links to the comparisons and experiments behind it.
Top Engines
Not enough eligible results to rank.
Speed
| Rank | Engine | Prefill |
|---|---|---|
| Not ranked | llama.cpp | Not counted |
| Not ranked | MLX LM | Not counted |
details
LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 2048 input / 128 output · macOS 27
Medians with the model warmed; model loading is excluded. Each table uses one matched comparison.
llama.cpp: Partial: 3 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.
MLX LM: Partial: 2 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.
Memory usage by engine
| Rank | Engine | Memory |
|---|---|---|
| Not ranked | llama.cpp | Not counted |
| Not ranked | MLX LM | Not counted |
details
LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 4-bit · 2048 input / 128 output · macOS 27
Memory ranks require comparable, calibrated measurements. Unavailable measurements remain unranked.
llama.cpp: Partial: 3 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.
MLX LM: Partial: 2 of 6 planned trials were valid. † Approximate comparison. The experiments use different effective KV cache precision: f16, model-dtype. The experiments here use different 4-bit schemes: Q4_K_M, mlx-affine-4bit-g64.
Every comparison
| Comparison | llama.cpp | MLX LM |
|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted: fewer than two scored engines | Not counted | Not counted |
Each engine’s Speed score in each matched comparison, out of 100, or why it has none. How rankings work