Rankings
Engine
LiteRT-LM
Google AI Edge · Apache-2.0, open source · Homepage
Top Engines
- Desktop
- Not ranked
- Mobile
- Not ranked
details
Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, magnitude default, model-dtype, uzu default. Its weights are 8-bit in another scheme: litert-int8, where most use Q8_0. The experiments use different effective KV cache precision: litert-lm default, f16. The experiments here use different 8-bit schemes: litert-int8, Q8_0.
Supporting experiments
| Configuration | Status | Generationtok/s | First tokenms | Prefilltok/s | MemoryGiB |
|---|---|---|---|---|---|
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Check failed | 103.5range102.7–103.8 | 716range715–724 | 370.5range366.9–371.3 | 1.23range1.23–1.23 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · GPU litert-int4-gpu · 6/6 valid | Check failed | 330.8range327.6–340.2 | 114range104–117 | 3,438range3,380–3,497 | 1.82range1.82–1.83 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 32, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Counted | 119.1range117.6–119.6 | 277range276–278 | 1,015range1,010–1,020 | 1.73range1.73–1.73 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Check failed | 60.3range60.0–60.5 | 541range539–542 | 515.9range514.4–517.4 | 3.42range3.42–3.42 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Check failed | 183.5range179.4–185.6 | 244range244–254 | 1,118range1,072–1,122 | 0.62range0.62–0.62 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 256 input / 128 output · macOS 27 Speed score 24, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Counted | 200.0range197.7–201.0 | 136range136–139 | 2,103range2,050–2,108 | 0.78range0.78–0.78 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Check failed | 104.3range96.9–114.0 | 4,588range4,527–5,127 | 448.9range401.5–454.9 | 2.20range2.19–2.20 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · GPU litert-int4-gpu · 6/6 valid | Check failed | 349.2range347.9–349.5 | 374range369–377 | 5,799range5,773–5,881 | 1.88range1.88–1.89 |
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 32, approximate comparison | |||||
| LFM2.5 1.2B Instruct on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Counted | 118.8range117.0–119.3 | 1,493range1,484–1,499 | 1,396range1,391–1,405 | 2.69range2.69–2.69 |
| LFM2.5 2.6B on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Check failed | 60.3range59.8–60.9 | 2,923range2,917–2,932 | 711.9range709.8–713.4 | 4.42range4.42–4.43 |
| LFM2.5 350M on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Check failed | 184.6range181.0–185.3 | 1,499range1,498–1,507 | 1,382range1,374–1,383 | 1.09range1.09–1.10 |
| LFM2.5 350M on M5 Max · 64 GiB 8-bit · Archive 2048 input / 128 output · macOS 27 Speed score 26, approximate comparison | |||||
| LFM2.5 350M on M5 Max · 64 GiB, 8-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int8 · 6/6 valid | Counted | 198.9range198.3–200.6 | 635range631–643 | 3,308range3,268–3,330 | 1.25range1.25–1.25 |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 49.8range49.2–50.2 | Not measured | 145.2range142.2–146.9 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 86, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 36.4range36.3–36.4 | Not measured | 332.6range325.3–339.7 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 64, approximate comparison | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 77.0range71.9–77.4 | Not measured | 850.8range556.4–898.6 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 25.9range22.7–26.3 | Not measured | 97.1range85.4–100.3 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 18.3range16.6–18.8 | Not measured | 170.7range153.0–174.8 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Speed score 93, approximate comparison | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 41.6range37.8–43.2 | Not measured | 322.3range304.3–414.2 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 49.7range43.1–50.4 | Not measured | 169.7range137.1–173.1 | Not measured |
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 87, approximate comparison | |||||
| LFM2.5 1.2B Instruct on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 33.4range30.8–35.0 | Not measured | 260.5range255.0–285.3 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 76, approximate comparison | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 76.5range74.4–76.7 | Not measured | 851.6range841.9–859.7 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Speed score 100, approximate comparison | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Counted | 39.0range35.3–39.4 | Not measured | 454.8range438.9–461.7 | Not measured |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 256 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 256 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Not counted | 54.1range52.9–54.4 | 1,590range1,583–1,618 | 166.0range163.2–166.8 | 2.24range2.24–2.24 |
| LFM2.5 2.6B on M5 Max · 64 GiB 4-bit · Archive 2048 input / 128 output · macOS 27 Not counted | |||||
| LFM2.5 2.6B on M5 Max · 64 GiB, 4-bit · Archive 2048 input / 128 output · macOS 27: 0.19.1 · CPU litert-int4 · 6/6 valid | Not counted | 54.2range52.4–54.6 | 10,200range10,174–10,423 | 201.7range197.4–202.2 | 3.20range3.20–3.21 |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Not counted | 23.8range23.4–25.5 | Not measured | 53.5range51.1–69.4 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 6/6 valid | Check failed | 15.7range15.6–16.6 | Not measured | 122.9range121.2–134.2 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 92.2range92.0–92.7 | Not measured | 441.8range435.3–446.2 | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Not counted | 11.8range11.6–12.0 | Not measured | 43.6range43.2–44.4 | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 3/6 valid | Partial | 8.2range8.1–8.3 | Not measured | 99.8range99.5–101.7 | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 256 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 256 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 51.9range46.2–52.7 | Not measured | 248.0range200.9–264.2 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Not counted | 15.0range14.9–23.5 | Not measured | 51.4range43.9–65.1 | Not measured |
| LFM2.5 2.6B on OnePlus 13 · 16 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on OnePlus 13 · 16 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 2/6 valid | Partial | 15.5range15.5–15.6 | Not measured | 133.1range129.8–136.4 | Not measured |
| LFM2.5 350M on OnePlus 13 · 16 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on OnePlus 13 · 16 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 92.8range88.4–97.1 | Not measured | 527.5range526.1–530.4 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 22.0range21.8–22.3 | Not measured | 90.2range89.0–92.4 | Not measured |
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 1.2B Instruct on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 5/6 valid | Partial | 16.0range15.0–16.1 | Not measured | 150.4range146.7–161.7 | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 3/6 valid | Partial | 9.9range9.7–11.2 | Not measured | 45.8range45.3–46.1 | Not measured |
| LFM2.5 2.6B on Pixel 8 · 8 GB 8-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 2.6B on Pixel 8 · 8 GB, 8-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int8 · 0/6 valid | Failed | Not measured | Not measured | Not measured | Not measured |
| LFM2.5 350M on Pixel 8 · 8 GB 4-bit · Archive 2048 input / 128 output · Android 16 Not counted | |||||
| LFM2.5 350M on Pixel 8 · 8 GB, 4-bit · Archive 2048 input / 128 output · Android 16: 0.19.0 · CPU litert-int4 · 6/6 valid | Check failed | 45.4range43.0–48.1 | Not measured | 311.9range276.2–314.5 | Not measured |