Engine
llama.cpp
ggml-org · MIT, open source · Homepage
Top Engines
- Desktop
- Rank 3, 92 out of 100, range 84 to 93, approximate comparison
- Mobile
- Not ranked
details
Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m. The experiments use different effective KV cache precision: f16, uzu default. The experiments here use different 8-bit schemes: Q8_0, mirai-l. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 81, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 438.1range430.6–443.7 | 34.2range33.5–34.9 | 7,671range7,514–7,899 | 0.93range0.93–0.95 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 325.0range288.1–329.8 | 34.5range34.0–36.6 | 7,575range7,149–7,697 | 1.41range1.41–1.43 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 190.5range177.9–194.0 | 36.7range35.8–38.5 | 7,122range6,779–7,297 | 2.43range2.43–2.45 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 256 input / 128 output · macOS 27 Speed score 85, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 162.6range140.6–173.6 | 29.4range26.4–31.9 | 8,953range8,216–9,956 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 91, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 148.8range116.0–150.1 | 62.8range61.2–63.7 | 4,126range4,062–4,241 | 2.96range2.96–2.98 |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 96 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 88.1range87.1–88.6 | 68.8range68.7–69.5 | 3,759range3,721–3,767 | 5.31range5.31–5.33 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 62, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 684.9range667.8–718.6 | 23.2range13.0–23.7 | 11,475range11,176–21,451 | 0.46range0.46–0.47 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 75, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 617.2range590.2–655.2 | 22.6range21.8–23.0 | 11,913range11,629–12,350 | 0.60range0.60–0.61 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 81, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 464.2range437.9–478.8 | 23.4range14.2–26.0 | 11,299range10,150–19,058 | 0.91range0.91–0.92 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 75, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 25.1range23.9–25.3 | 421range408–437 | 609.7range586.3–628.8 | 18.90range18.90–19.08 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 18.6range18.5–18.6 | 415range401–417 | 618.1range614.3–640.1 | 27.46range27.46–27.64 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q2_0 · 6/6 valid | Counted | 43.2range40.8–43.5 | 384range382–407 | 668.1range630.7–670.8 | 7.94range7.94–8.12 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 83, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 436.7range416.3–447.6 | 145range144–149 | 14,182range13,797–14,359 | 0.95range0.95–0.95 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 319.6range315.9–323.2 | 139range139–140 | 14,799range14,691–14,847 | 1.43range1.43–1.43 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 184.2range173.0–190.6 | 142range141–144 | 14,523range14,360–14,659 | 2.45range2.45–2.45 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 175.4range156.7–182.0 | 133range132–134 | 15,456range15,369–15,575 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 89, approximate comparison | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 142.4range140.5–147.3 | 298range297–299 | 6,885range6,874–6,908 | 2.98range2.98–2.98 |
| LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 97 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 88.7range86.0–88.9 | 305range304–306 | 6,728range6,704–6,761 | 5.33range5.33–5.33 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 69, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 650.0range639.9–689.2 | 65.2range54.5–70.8 | 31,928range29,289–38,466 | 0.47range0.47–0.47 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 76, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 592.1range568.8–612.1 | 63.4range61.9–70.3 | 32,938range29,505–33,763 | 0.61range0.61–0.61 |
| LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 83, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 valid | Counted | 442.7range427.3–456.7 | 66.2range63.9–69.3 | 31,446range29,964–32,683 | 0.92range0.92–0.92 |
| Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 76, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 25.1range24.9–25.3 | 3,092range2,873–3,345 | 663.8range612.4–713.1 | 19.08range19.08–19.08 |
| Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 98, approximate comparison | |||||
| Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 18.5range14.8–18.5 | 2,996range2,756–4,209 | 683.7range486.8–743.3 | 27.65range27.64–27.65 |
| Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison | |||||
| Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q2_0 · 6/6 valid | Counted | 42.7range42.2–42.8 | 2,814range2,575–3,078 | 728.1range665.6–795.8 | 8.12range8.12–8.12 |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 83, approximate comparison | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 37.6range37.5–37.7 | 987range984–993 | 259.7range257.9–260.4 | Not measured |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 94, approximate comparison | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 24.1range23.7–24.2 | 938range936–998 | 273.1range256.8–273.9 | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 43, approximate comparison | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Counted | 56.0range52.1–57.4 | 314range312–317 | 818.5range809.3–823.7 | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 67, approximate comparison | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 49.8range47.7–50.3 | 297range293–300 | 862.9range857.0–876.7 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Speed score 83, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Counted | 63.5range63.0–64.1 | 744range727–812 | 348.1range319.5–357.8 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 42.1range41.9–42.6 | 484range476–555 | 538.9range467.1–546.9 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 121.1range119.6–124.1 | 151range139–225 | 1,772range1,203–1,940 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Speed score 78, approximate comparison | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Counted | 24.8range19.4–26.5 | 2,328range2,284–2,655 | 110.9range97.2–113.0 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 15.9range13.6–16.9 | 1,923range1,867–2,028 | 134.6range127.6–138.6 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 44.8range41.5–48.3 | 794range762–1,966 | 330.7range131.3–345.1 | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Speed score 69, approximate comparison | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 valid | Counted | 48.3range47.0–49.7 | 2,210range2,203–2,212 | 927.4range926.3–930.2 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Speed score 99, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Counted | 51.5range50.5–55.1 | 8,117range7,464–8,142 | 252.6range251.8–274.7 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 38.5range38.0–38.9 | 5,712range5,164–5,751 | 359.1range356.8–397.3 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 100.1range99.1–101.7 | 1,648range1,587–1,673 | 1,252range1,230–1,298 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 81, approximate comparison | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Counted | 31.5range30.6–34.5 | 10,591range10,155–11,138 | 193.8range184.3–202.2 | Not measured |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Not counted | 425.6range416.4–440.9 | 24.8range23.9–29.7 | 10,667range8,874–11,200 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Check failed | 218.4range212.3–220.4 | 62.5range61.2–65.6 | 4,141range3,950–4,237 | 1.84range1.84–1.86 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 3/6 valid | Partial | 421.8range417.0–436.0 | 137range137–138 | 15,026range14,937–15,064 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Check failed | 212.7range208.7–217.0 | 313range313–316 | 6,555range6,504–6,570 | 1.86range1.86–1.86 |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, bf16 · Archive 256 input / 128 output · iOS 27: b12110 · Metal BF16 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Check failed | 16.1range15.8–17.0 | 2,474range2,300–2,562 | 103.6range100.0–111.4 | Not measured |
| LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, bf16 · Archive 256 input / 128 output · iOS 27: b12110 · Metal BF16 · 6/6 valid | Not counted | 39.7range39.2–40.0 | 304range301–310 | 845.1range829.0–853.2 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 valid | Not counted | 4.7range4.6–4.7 | 42,788range42,756–42,840 | 6.0range6.0–6.0 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Check failed | 28.3range26.9–29.2 | 2,025range1,876–2,206 | 127.4range116.5–137.0 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Not counted | 18.8range18.2–18.9 | 1,517range1,504–1,527 | 170.0range168.4–170.9 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Not counted | 167.5range159.1–174.0 | 209range203–217 | 1,263range1,221–1,301 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 valid | Not counted | 17.4range17.3–17.6 | 10,769range10,678–10,814 | 23.8range23.7–24.0 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 5/6 valid | Partial | 10.2range9.9–10.7 | 4,789range4,736–4,802 | 53.7range53.6–54.2 | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 4/6 valid | Partial | 7.1range6.8–7.5 | 3,990range3,925–5,743 | 64.5range44.8–65.5 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Not counted | 65.0range47.0–73.9 | 972range927–1,119 | 268.8range232.4–281.3 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 valid | Not counted | 5.6range4.8–5.7 | 16,803range13,930–16,903 | 15.3range15.2–18.4 | Not measured |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Not counted | 34.9range34.6–35.1 | 8,775range8,728–8,932 | 233.4range229.3–234.7 | Not measured |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 valid | Not counted | 23.5range23.3–23.6 | 8,222range8,138–8,529 | 249.1range240.2–251.7 | Not measured |
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB bf16 · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, bf16 · Archive 2048 input / 128 output · iOS 27: b12110 · Metal BF16 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 2/6 valid | Partial | 15.1range14.0–16.1 | 20,523range18,848–22,197 | 100.5range92.3–108.7 | Not measured |
| LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 0/6 valid | Unsupported | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 valid | Not counted | 54.3range53.2–60.6 | 2,327range2,324–2,330 | 880.6range879.5–881.9 | Not measured |
| LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 2048 input / 128 output · iOS 27 Not counted | |||||
| LFM2.5 350M on iPhone 12 · 4 GB, bf16 · Archive 2048 input / 128 output · iOS 27: b12110 · Metal BF16 · 6/6 valid | Not counted | 38.9range38.7–39.1 | 2,242range2,234–2,245 | 913.9range912.8–917.2 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 4/6 valid | Partial | 24.5range24.0–24.6 | 18,101range18,091–18,198 | 113.2range112.6–113.3 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Not counted | 17.9range17.7–18.0 | 12,624range12,591–12,634 | 162.4range162.2–162.8 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 valid | Not counted | 133.3range130.8–134.3 | 2,431range2,385–2,499 | 845.5range822.9–861.8 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 valid | Not counted | 14.3range14.1–14.4 | 93,993range92,781–94,093 | 21.8range21.8–22.1 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 4/6 valid | Partial | 17.4range16.7–17.7 | 27,146range26,673–29,106 | 75.5range70.4–76.9 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 valid | Not counted | 12.9range12.4–13.3 | 24,421range23,107–28,048 | 84.1range73.1–88.7 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 5/6 valid | Partial | 39.5range36.7–41.6 | 12,266range12,182–12,717 | 167.3range161.4–168.5 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |