Rankings

Engine

LiteRT-LM

Google AI Edge · Apache-2.0, open source · Homepage

Top Engines

Desktop
Not ranked Results in 2 of 4 comparisons; a rank needs all 4.
Mobile
Not ranked Results in 7 of 12 comparisons; a rank needs all 12.
details

Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: litert-int8, where most use Q8_0. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.

How rankings work

Supporting experiments

LiteRT-LM: every experiment, grouped by matched comparison. Medians over valid trials.
ConfigurationStatusGenerationtok/sFirst tokenmsPrefilltok/sMemoryGiB
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validCheck failed 103.5range102.7–103.8716range715–724370.5range366.9–371.31.23range1.23–1.23
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · GPU litert-int4-gpu · 6/6 validCheck failed 330.8range327.6–340.2114range104–1173,438range3,380–3,4971.82range1.82–1.83
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 32, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCounted 119.1range117.6–119.6277range276–2781,015range1,010–1,0201.73range1.73–1.73
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCheck failed 60.3range60.0–60.5541range539–542515.9range514.4–517.43.42range3.42–3.42
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validCheck failed 183.5range179.4–185.6244range244–2541,118range1,072–1,1220.62range0.62–0.62
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 24, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCounted 200.0range197.7–201.0136range136–1392,103range2,050–2,1080.78range0.78–0.78
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validCheck failed 104.3range96.9–114.04,588range4,527–5,127448.9range401.5–454.92.20range2.19–2.20
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · GPU litert-int4-gpu · 6/6 validCheck failed 349.2range347.9–349.5374range369–3775,799range5,773–5,8811.88range1.88–1.89
LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 32, approximate comparison
LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCounted 118.8range117.0–119.31,493range1,484–1,4991,396range1,391–1,4052.69range2.69–2.69
LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCheck failed 60.3range59.8–60.92,923range2,917–2,932711.9range709.8–713.44.42range4.42–4.43
LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27
LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validCheck failed 184.6range181.0–185.31,499range1,498–1,5071,382range1,374–1,3831.09range1.09–1.10
LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 26, approximate comparison
LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 validCounted 198.9range198.3–200.6635range631–6433,308range3,268–3,3301.25range1.25–1.25
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 49.8range49.2–50.2Not measured145.2range142.2–146.9Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 86, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 36.4range36.3–36.4Not measured332.6range325.3–339.7Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 64, approximate comparison
LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 77.0range71.9–77.4Not measured850.8range556.4–898.6Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 25.9range22.7–26.3Not measured97.1range85.4–100.3Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 18.3range16.6–18.8Not measured170.7range153.0–174.8Not measured
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 93, approximate comparison
LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 41.6range37.8–43.2Not measured322.3range304.3–414.2Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 49.7range43.1–50.4Not measured169.7range137.1–173.1Not measured
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 87, approximate comparison
LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 33.4range30.8–35.0Not measured260.5range255.0–285.3Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 76, approximate comparison
LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 76.5range74.4–76.7Not measured851.6range841.9–859.7Not measured
LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison
LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCounted 39.0range35.3–39.4Not measured454.8range438.9–461.7Not measured
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validNot counted 54.1range52.9–54.41,590range1,583–1,618166.0range163.2–166.82.24range2.24–2.24
LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted
LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 validNot counted 54.2range52.4–54.610,200range10,174–10,423201.7range197.4–202.23.20range3.20–3.21
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validNot counted 23.8range23.4–25.5Not measured53.5range51.1–69.4Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 validCheck failed 15.7range15.6–16.6Not measured122.9range121.2–134.2Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 92.2range92.0–92.7Not measured441.8range435.3–446.2Not measured
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validNot counted 11.8range11.6–12.0Not measured43.6range43.2–44.4Not measured
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 3/6 validPartial 8.2range8.1–8.3Not measured99.8range99.5–101.7Not measured
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 51.9range46.2–52.7Not measured248.0range200.9–264.2Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validNot counted 15.0range14.9–23.5Not measured51.4range43.9–65.1Not measured
LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 2/6 validPartial 15.5range15.5–15.6Not measured133.1range129.8–136.4Not measured
LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 92.8range88.4–97.1Not measured527.5range526.1–530.4Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 22.0range21.8–22.3Not measured90.2range89.0–92.4Not measured
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 5/6 validPartial 16.0range15.0–16.1Not measured150.4range146.7–161.7Not measured
LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 3/6 validPartial 9.9range9.7–11.2Not measured45.8range45.3–46.1Not measured
LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 0/6 validFailed Not measuredNot measuredNot measuredNot measured
LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted
LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 validCheck failed 45.4range43.0–48.1Not measured311.9range276.2–314.5Not measured