Rankings

Engine

llama.cpp

ggml-org · MIT, open source · Homepage

Top Engines

Desktop
Rank 3, 92 out of 100, range 84 to 93, approximate comparison Speed score · 4 of 4 counted comparisons
Mobile
Not ranked Results in all 12 comparisons; no other engine has completed them yet.
details

Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, model-dtype. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype, uzu default. The experiments use different effective KV cache precision: f16, magnitude default, model-dtype, zml default. The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64. The experiments use different effective KV cache precision: bf16, fp32, litert-lm default, f16, magnitude default, model-dtype. The experiments use different effective KV cache precision: fp32, f16, uzu default. The experiments here use different 4-bit schemes: 8da4w, Q4_K_M, mirai-m. The experiments use different effective KV cache precision: f16, uzu default. The experiments here use different 8-bit schemes: Q8_0, mirai-l. The experiments use different effective KV cache precision: fp32, litert-lm default, f16. The experiments here use different 4-bit schemes: 8da4w, litert-int4, Q4_K_M. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.

How rankings work

Supporting experiments

llama.cpp: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 81, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 438.1range430.6–443.734.2range33.5–34.97,671range7,514–7,8990.93range0.93–0.95
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 325.0range288.1–329.834.5range34.0–36.67,575range7,149–7,6971.41range1.41–1.43
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 190.5range177.9–194.036.7range35.8–38.57,122range6,779–7,2972.43range2.43–2.45
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 256 input / 128 output · macOS 27 Speed score 85, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 162.6range140.6–173.629.4range26.4–31.98,953range8,216–9,956Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 91, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 148.8range116.0–150.162.8range61.2–63.74,126range4,062–4,2412.96range2.96–2.98
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 96
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 88.1range87.1–88.668.8range68.7–69.53,759range3,721–3,7675.31range5.31–5.33
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 62, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 684.9range667.8–718.623.2range13.0–23.711,475range11,176–21,4510.46range0.46–0.47
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 75, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 617.2range590.2–655.222.6range21.8–23.011,913range11,629–12,3500.60range0.60–0.61
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 256 input / 128 output · macOS 27 Speed score 81, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 256 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 464.2range437.9–478.823.4range14.2–26.011,299range10,150–19,0580.91range0.91–0.92
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Speed score 75, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 25.1range23.9–25.3421range408–437609.7range586.3–628.818.90range18.90–19.08
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 98, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 18.6range18.5–18.6415range401–417618.1range614.3–640.127.46range27.46–27.64
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 256 input / 128 output · macOS 27 Speed score 95, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q2_0 · 6/6 validCounted 43.2range40.8–43.5384range382–407668.1range630.7–670.87.94range7.94–8.12
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 83, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 436.7range416.3–447.6145range144–14914,182range13,797–14,3590.95range0.95–0.95
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 319.6range315.9–323.2139range139–14014,799range14,691–14,8471.43range1.43–1.43
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 184.2range173.0–190.6142range141–14414,523range14,360–14,6592.45range2.45–2.45
LFM2.5 1.2B Instruct on M5 Max · 64 GiB bf16 · 2048 input / 128 output · macOS 27 Speed score 92, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, bf16 · 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 175.4range156.7–182.0133range132–13415,456range15,369–15,575Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 89, approximate comparison
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 142.4range140.5–147.3298range297–2996,885range6,874–6,9082.98range2.98–2.98
LFM2.5 2.6B on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 97
LFM2.5 2.6B on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 88.7range86.0–88.9305range304–3066,728range6,704–6,7615.33range5.33–5.33
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 69, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 650.0range639.9–689.265.2range54.5–70.831,928range29,289–38,4660.47range0.47–0.47
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 76, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 592.1range568.8–612.163.4range61.9–70.332,938range29,505–33,7630.61range0.61–0.61
LFM2.5 350M on M5 Max · 64 GiB bf16 · Archive 2048 input / 128 output · macOS 27 Speed score 83, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, bf16 · Archive 2048 input / 128 output · macOS 27: b12110 · Metal BF16 · 6/6 validCounted 442.7range427.3–456.766.2range63.9–69.331,446range29,964–32,6830.92range0.92–0.92
Qwen3.8 27B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Speed score 76, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 25.1range24.9–25.33,092range2,873–3,345663.8range612.4–713.119.08range19.08–19.08
Qwen3.8 27B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 98, approximate comparison
Qwen3.8 27B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 18.5range14.8–18.52,996range2,756–4,209683.7range486.8–743.327.65range27.64–27.65
Ternary Bonsai 2 27B on M5 Max · 64 GiB 2-bit · Archive 2048 input / 128 output · macOS 27 Speed score 96, approximate comparison
Ternary Bonsai 2 27B on M5 Max · 64 GiB, 2-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q2_0 · 6/6 validCounted 42.7range42.2–42.82,814range2,575–3,078728.1range665.6–795.88.12range8.12–8.12
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 83, approximate comparison
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 37.6range37.5–37.7987range984–993259.7range257.9–260.4Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 94, approximate comparison
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 24.1range23.7–24.2938range936–998273.1range256.8–273.9Not measured
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Speed score 43, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 validCounted 56.0range52.1–57.4314range312–317818.5range809.3–823.7Not measured
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Speed score 67, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 49.8range47.7–50.3297range293–300862.9range857.0–876.7Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Speed score 83, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validCounted 63.5range63.0–64.1744range727–812348.1range319.5–357.8Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 42.1range41.9–42.6484range476–555538.9range467.1–546.9Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 121.1range119.6–124.1151range139–2251,772range1,203–1,940Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Speed score 78, approximate comparison
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validCounted 24.8range19.4–26.52,328range2,284–2,655110.9range97.2–113.0Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 15.9range13.6–16.91,923range1,867–2,028134.6range127.6–138.6Not measured
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 44.8range41.5–48.3794range762–1,966330.7range131.3–345.1Not measured
LFM2.5 350M on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Speed score 69, approximate comparison
LFM2.5 350M on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 validCounted 48.3range47.0–49.72,210range2,203–2,212927.4range926.3–930.2Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Speed score 99, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validCounted 51.5range50.5–55.18,117range7,464–8,142252.6range251.8–274.7Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 38.5range38.0–38.95,712range5,164–5,751359.1range356.8–397.3Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 100.1range99.1–101.71,648range1,587–1,6731,252range1,230–1,298Not measured
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 81, approximate comparison
LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validCounted 31.5range30.6–34.510,591range10,155–11,138193.8range184.3–202.2Not measured
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 256 input / 128 output · macOS 27 Not counted
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validNot counted 425.6range416.4–440.924.8range23.9–29.710,667range8,874–11,200Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCheck failed 218.4range212.3–220.462.5range61.2–65.64,141range3,950–4,2371.84range1.84–1.86
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · 2048 input / 128 output · macOS 27 Not counted
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 3/6 validPartial 421.8range417.0–436.0137range137–13815,026range14,937–15,064Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: b12110 · Metal Q4_K_M · 6/6 validCheck failed 212.7range208.7–217.0313range313–3166,555range6,504–6,5701.86range1.86–1.86
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, bf16 · Archive 256 input / 128 output · iOS 27: b12110 · Metal BF16 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 validCheck failed 16.1range15.8–17.02,474range2,300–2,562103.6range100.0–111.4Not measured
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 256 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 256 input / 128 output · iOS 27 Not counted
LFM2.5 350M on iPhone 12 · 4 GB, bf16 · Archive 256 input / 128 output · iOS 27: b12110 · Metal BF16 · 6/6 validNot counted 39.7range39.2–40.0304range301–310845.1range829.0–853.2Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 validNot counted 4.7range4.6–4.742,788range42,756–42,8406.0range6.0–6.0Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validCheck failed 28.3range26.9–29.22,025range1,876–2,206127.4range116.5–137.0Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validNot counted 18.8range18.2–18.91,517range1,504–1,527170.0range168.4–170.9Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validNot counted 167.5range159.1–174.0209range203–2171,263range1,221–1,301Not measured
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 validNot counted 17.4range17.3–17.610,769range10,678–10,81423.8range23.7–24.0Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 5/6 validPartial 10.2range9.9–10.74,789range4,736–4,80253.7range53.6–54.2Not measured
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q8_0 · 4/6 validPartial 7.1range6.8–7.53,990range3,925–5,74364.5range44.8–65.5Not measured
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validNot counted 65.0range47.0–73.9972range927–1,119268.8range232.4–281.3Not measured
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, bf16 · Archive 256 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 validNot counted 5.6range4.8–5.716,803range13,930–16,90315.3range15.2–18.4Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 validNot counted 34.9range34.6–35.18,775range8,728–8,932233.4range229.3–234.7Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 6/6 validNot counted 23.5range23.3–23.68,222range8,138–8,529249.1range240.2–251.7Not measured
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB bf16 · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 1.2B Instruct on iPhone 12 · 4 GB, bf16 · Archive 2048 input / 128 output · iOS 27: b12110 · Metal BF16 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 2/6 validPartial 15.1range14.0–16.120,523range18,848–22,197100.5range92.3–108.7Not measured
LFM2.5 2.6B on iPhone 12 · 4 GB 8-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 2.6B on iPhone 12 · 4 GB, 8-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q8_0 · 0/6 validUnsupported Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on iPhone 12 · 4 GB 4-bit · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 350M on iPhone 12 · 4 GB, 4-bit · Archive 2048 input / 128 output · iOS 27: b12110 · Metal Q4_K_M · 6/6 validNot counted 54.3range53.2–60.62,327range2,324–2,330880.6range879.5–881.9Not measured
LFM2.5 350M on iPhone 12 · 4 GB bf16 · Archive 2048 input / 128 output · iOS 27 Not counted
LFM2.5 350M on iPhone 12 · 4 GB, bf16 · Archive 2048 input / 128 output · iOS 27: b12110 · Metal BF16 · 6/6 validNot counted 38.9range38.7–39.12,242range2,234–2,245913.9range912.8–917.2Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 4/6 validPartial 24.5range24.0–24.618,101range18,091–18,198113.2range112.6–113.3Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validNot counted 17.9range17.7–18.012,624range12,591–12,634162.4range162.2–162.8Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 6/6 validNot counted 133.3range130.8–134.32,431range2,385–2,499845.5range822.9–861.8Not measured
LFM2.5 350M on OnePlus 13 · 16 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 6/6 validNot counted 14.3range14.1–14.493,993range92,781–94,09321.8range21.8–22.1Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 4/6 validPartial 17.4range16.7–17.727,146range26,673–29,10675.5range70.4–76.9Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 6/6 validNot counted 12.9range12.4–13.324,421range23,107–28,04884.1range73.1–88.7Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q8_0 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: b12110 · CPU Q4_K_M · 5/6 validPartial 39.5range36.7–41.612,266range12,182–12,717167.3range161.4–168.5Not measured
LFM2.5 350M on Pixel 8 · 8 GB bf16 · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, bf16 · Archive 2048 input / 128 output · Android 16: b12110 · CPU BF16 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured