On-device AI leaderboard

Top Engines

Desktop: 3 engines ranked

  1. Rank 1: MLX LM 95 out of 100, range 95 to 96
  2. Rank 2: Ollama 93 out of 100, range 91 to 94
  3. Rank 3: llama.cpp 92 out of 100, range 84 to 93
+ 6 engines with incomplete coverage

Coverage

Speed score per device, out of 100: each device ranked on its own.
DesktopMobile
M5 Max 64 GiB macOS 27 Apple M5 MaxOnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750)Pixel 8 8 GB Android 16 Google Tensor G3iPhone 12 4 GB iOS 27 Apple A14 Bionic
llama.cpp85 out of 100, in 22 of 22 comparisons97 out of 100, in 6 of 6 comparisons86 out of 100, in 4 of 4 comparisons69 out of 100, in 5 of 5 comparisons
LiteRT-LMNot ranked. Results in 4 of 22 comparisons; a rank needs 11.78 out of 100, in 4 of 6 comparisons98 out of 100, in 3 of 4 comparisonsNot measured on this device.
uzuNot ranked. Results in 7 of 22 comparisons; a rank needs 11.Not measured on this device.Not measured on this device.100 out of 100, in 5 of 5 comparisons
Magnitude91 out of 100, in 12 of 22 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
MLX LM98 out of 100, in 18 of 22 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Ollama88 out of 100, in 22 of 22 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
Rapid-MLX98 out of 100, in 12 of 22 comparisonsNot measured on this device.Not measured on this device.Not measured on this device.
ExecuTorchNot ranked. Results in 2 of 22 comparisons; a rank needs 11.Not ranked. Results in 2 of 6 comparisons; a rank needs 3.Not ranked. Results in 1 of 4 comparisons; a rank needs 2.Not ranked. Results in 1 of 5 comparisons; a rank needs 3.
ZMLNot ranked. Results in 4 of 22 comparisons; a rank needs 11.Not measured on this device.Not measured on this device.Not measured on this device.

Performance

Performance scope: Desktop.

LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 8-bit · Archive 256 input / 128 output · macOS 27

Speed

Generation, tok/s

Generation, tok/s. Medians, tok/s.
EngineMedian Range
uzu374.3 373.4 to 374.7
Rapid-MLX361.9 358.8 to 364.8
MLX LM361.1 359.7 to 362.6
Magnitude330.1 327.3 to 330.9
Ollama326.2 294.8 to 332.2
llama.cpp325.0 288.1 to 329.8
LiteRT-LM119.1 117.6 to 119.6
  1. Rank 1: uzu 374.3 tok/s, range 373.4 to 374.7; open the experiment and its raw trials
  2. Rank 2: Rapid-MLX 361.9 tok/s, range 358.8 to 364.8; open the experiment and its raw trials
  3. Rank 3: MLX LM 361.1 tok/s, range 359.7 to 362.6; open the experiment and its raw trials
  4. Rank 4: Magnitude 330.1 tok/s, range 327.3 to 330.9; open the experiment and its raw trials
  5. Rank 5: Ollama 326.2 tok/s, range 294.8 to 332.2; open the experiment and its raw trials
  6. Rank 6: llama.cpp 325.0 tok/s, range 288.1 to 329.8; open the experiment and its raw trials
  7. Rank 7: LiteRT-LM 119.1 tok/s, range 117.6 to 119.6; open the experiment and its raw trials

Lowest Memory

Peak memory, GiB

Peak memory, GiB. Medians, GiB.
EngineMedian Range
Ollama1.41 1.41 to 1.43
llama.cpp1.41 1.41 to 1.43
MLX LM1.44 1.43 to 1.45
uzu1.46 1.46 to 1.52
Rapid-MLX1.69 1.68 to 1.70
LiteRT-LM1.73 1.73 to 1.73
Magnitude1.90 1.89 to 1.90
  1. Rank 1: Ollama 1.41 GiB, range 1.41 to 1.43; open the experiment and its raw trials
  2. Rank 2: llama.cpp 1.41 GiB, range 1.41 to 1.43; open the experiment and its raw trials
  3. Rank 3: MLX LM 1.44 GiB, range 1.43 to 1.45; open the experiment and its raw trials
  4. Rank 4: uzu 1.46 GiB, range 1.46 to 1.52; open the experiment and its raw trials
  5. Rank 5: Rapid-MLX 1.69 GiB, range 1.68 to 1.70; open the experiment and its raw trials
  6. Rank 6: LiteRT-LM 1.73 GiB, range 1.73 to 1.73; open the experiment and its raw trials
  7. Rank 7: Magnitude 1.90 GiB, range 1.89 to 1.90; open the experiment and its raw trials

Performance Breakdowns

Speed score by model, out of 100

Speed score by model, out of 100: 100 is the fastest engine on that model.
LFM2.5 350MLFM2.5 1.2B InstructLFM2.5 2.6BQwen3.8 27BTernary Bonsai 2 27B
MLX LM97971009899
Rapid-MLX9896100No score. Not measured on this model.No score. Not measured on this model.
Magnitude9291No score. Unsupported in 6 comparisons. Magnitude's chat API renders LFM2.5 2.6B's own template, which always opens a thinking block, so it streams reasoning and can't run the closed-think prompts (measured in qualification).No score. Unsupported in 4 comparisons. Magnitude 0.2.17 reports this GGUF as Unsupported (magnitude catalog inspect)No score. Unsupported in 2 comparisons. Magnitude 0.2.17 reports this GGUF as Unsupported (magnitude catalog inspect)
Ollama78909888100
llama.cpp7487938696
uzu96100No score. Unsupported in 4 comparisons. uzu can't run the frozen prompts: Error: uzu counted 23 input tokens, not the frozen prompt's 26 (measured in qualification).No score. Unsupported in 4 comparisons. uzu 0.6.17 renders its own Qwen3.8 chat template: 67 input tokens for the 27-token canonical check prompt, so it can't run the frozen prompts (measured in qualification).No score. Not measured on this model.

Explore Benchmarks

Open the explorer