On-device AI leaderboard
Top Engines
Mobile: 0 engines ranked
+ 3 engines with incomplete coverage
Coverage
| Desktop | Mobile | |||
|---|---|---|---|---|
| M5 Max 64 GiB macOS 27 Apple M5 Max | OnePlus 13 16 GB Android 16 Qualcomm Snapdragon 8 Elite (SM8750) | Pixel 8 8 GB Android 16 Google Tensor G3 | iPhone 12 4 GB iOS 27 Apple A14 Bionic | |
| llama.cpp | 8522/22 out of 100, in 22 of 22 comparisons | 976/6 out of 100, in 6 of 6 comparisons | 864/4 out of 100, in 4 of 4 comparisons | 695/5 out of 100, in 5 of 5 comparisons |
| LiteRT-LM | 4/22Not ranked. Results in 4 of 22 comparisons; a rank needs 11. | 784/6 out of 100, in 4 of 6 comparisons | 983/4 out of 100, in 3 of 4 comparisons | Not measured on this device. |
| uzu | 7/22Not ranked. Results in 7 of 22 comparisons; a rank needs 11. | Not measured on this device. | Not measured on this device. | 1005/5 out of 100, in 5 of 5 comparisons |
| Magnitude | 9112/22 out of 100, in 12 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| MLX LM | 9818/22 out of 100, in 18 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Ollama | 8822/22 out of 100, in 22 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| Rapid-MLX | 9812/22 out of 100, in 12 of 22 comparisons | Not measured on this device. | Not measured on this device. | Not measured on this device. |
| ExecuTorch | 2/22Not ranked. Results in 2 of 22 comparisons; a rank needs 11. | 2/6Not ranked. Results in 2 of 6 comparisons; a rank needs 3. | 1/4Not ranked. Results in 1 of 4 comparisons; a rank needs 2. | 1/5Not ranked. Results in 1 of 5 comparisons; a rank needs 3. |
| ZML | 4/22Not ranked. Results in 4 of 22 comparisons; a rank needs 11. | Not measured on this device. | Not measured on this device. | Not measured on this device. |
Performance
Performance scope: Mobile.
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB · 4-bit · Archive 256 input / 128 output · iOS 27Speed
Performance Breakdowns
Speed score by model, out of 100
| LFM2.5 350M | LFM2.5 1.2B Instruct | |
|---|---|---|
| llama.cpp | 77 | 90 |
| ExecuTorch | No score. Its output check failed: 5/8 of the checks the model passes at BF16; needs 7 and the mandatory ones (needle-start, needle-end). Left out, failed by every BF16 configuration: arithmetic, sort, word, continuation-128. | 98 |
| LiteRT-LM | 82 | No score. Results in 3 of 8 comparisons; a rank needs 4. |