On-device AI leaderboard
Top Engines
Desktop: 3 engines ranked
+ 6 engines with incomplete coverage
Coverage
| Desktop | Mobile | |||
|---|---|---|---|---|
| M5 Max 64 GiB macOS 27 Apple M5 Max | OnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750) | Pixel 8 8 GB Android 16 Google Tensor G3 | iPhone 12 4 GB iOS 27 Apple A14 Bionic | |
| llama.cpp | 8522/22 out of 100, in 22 of 22 comparisons | 976/6 out of 100, in 6 of 6 comparisons | 864/4 out of 100, in 4 of 4 comparisons | 695/5 out of 100, in 5 of 5 comparisons |
| LiteRT-LM | 4/22Not ranked. Results in 4 of 22 comparisons; a rank needs 11. | 784/6 out of 100, in 4 of 6 comparisons | 983/4 out of 100, in 3 of 4 comparisons | Not measured on this device. |
| uzu | 7/22Not ranked. Results in 7 of 22 comparisons; a rank needs 11. | Not measured on this device. | Not measured on this device. | 1005/5 out of 100, in 5 of 5 comparisons |
| Magnitude | 9112/22 out of 100, in 12 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| MLX LM | 9818/22 out of 100, in 18 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Ollama | 8822/22 out of 100, in 22 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Rapid-MLX | 9812/22 out of 100, in 12 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| ExecuTorch | 2/22Not ranked. Results in 2 of 22 comparisons; a rank needs 11. | 2/6Not ranked. Results in 2 of 6 comparisons; a rank needs 3. | 1/4Not ranked. Results in 1 of 4 comparisons; a rank needs 2. | 1/5Not ranked. Results in 1 of 5 comparisons; a rank needs 3. |
| ZML | 4/22Not ranked. Results in 4 of 22 comparisons; a rank needs 11. | Not measured on this device. | Not measured on this device. | Not measured on this device. |
Performance
Performance scope: Desktop.
LFM2.5 1.2B Instruct on M5 Max · 64 GiB · 8-bit · Archive 256 input / 128 output · macOS 27Speed
- Rank 1: uzu 374.3 tok/s, range 373.4 to 374.7; open the experiment and its raw trials
- Rank 2: Rapid-MLX 361.9 tok/s, range 358.8 to 364.8; open the experiment and its raw trials
- Rank 3: MLX LM 361.1 tok/s, range 359.7 to 362.6; open the experiment and its raw trials
- Rank 4: Magnitude 330.1 tok/s, range 327.3 to 330.9; open the experiment and its raw trials
- Rank 5: Ollama 326.2 tok/s, range 294.8 to 332.2; open the experiment and its raw trials
- Rank 6: llama.cpp 325.0 tok/s, range 288.1 to 329.8; open the experiment and its raw trials
- Rank 7: LiteRT-LM 119.1 tok/s, range 117.6 to 119.6; open the experiment and its raw trials
Lowest Memory
- Rank 1: Ollama 1.41 GiB, range 1.41 to 1.43; open the experiment and its raw trials
- Rank 2: llama.cpp 1.41 GiB, range 1.41 to 1.43; open the experiment and its raw trials
- Rank 3: MLX LM 1.44 GiB, range 1.43 to 1.45; open the experiment and its raw trials
- Rank 4: uzu 1.46 GiB, range 1.46 to 1.52; open the experiment and its raw trials
- Rank 5: Rapid-MLX 1.69 GiB, range 1.68 to 1.70; open the experiment and its raw trials
- Rank 6: LiteRT-LM 1.73 GiB, range 1.73 to 1.73; open the experiment and its raw trials
- Rank 7: Magnitude 1.90 GiB, range 1.89 to 1.90; open the experiment and its raw trials
Performance Breakdowns
Generation by precision on LFM2.5 1.2B Instruct
| Engine | BF16 | 8-bit | 4-bit |
|---|---|---|---|
| uzu | Not measured | 374.3 | 540.7 |
| Rapid-MLX | 198.0 | 361.9 | Not measured |
| MLX LM | 199.0 | 361.1 | Not measured |
| Magnitude | 199.6 | 330.1 | 444.5 |
| Ollama | 199.6 | 326.2 | 454.6 |
| llama.cpp | 190.5 | 325.0 | 438.1 |
| LiteRT-LM | Not measured | 119.1 | Not measured |
| ExecuTorch | Not measured | Not measured | 472.1 |
| ZML | 192.6 | Not measured | Not measured |