LFM2.5 2.6B on Pixel 8 · 8 GB

Experiment Partial Published

LiteRT-LM 0.19.0 on Pixel 8 · 8 GB, CPU

LFM2.5 2.6B as litert-int4 (4-bit), archive 2048 input / 128 output: a 2048-token prompt and 128 tokens out. Measured 13 Nov 2026, 04:28 UTC by Glenn Sonna, Xybrid.

Doesn’t count in rankings: Partial: 3 of 6 planned trials were valid.

Experiment id s4-pixel8-lfm2.5-2.6b-4-bit-litert-2048

Errors

  • trial: No trial was valid.
  • trial: No trial was valid.
  • trial: No trial was valid.

Summary

Over 3 valid trials of 6 attempted (6 planned), computed from the raw trials below.

MetricMedianRangeMean ± SDQuartiles
Decode speed tok/s, higher is better9.9 9.7–11.2 10.2 ± 0.8 9.8–10.5
Time to first token ms, lower is betterNot measured
Peak memory GiB, lower is betterNot measured
Prefill speed tok/s, higher is better45.8 45.3–46.1 45.8 ± 0.4 45.6–46.0

Trials, metric by metric

The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.

Decode speed tok/s, higher is better
Time to first token ms, lower is better
Peak memory GiB, lower is better
Prefill speed tok/s, higher is better

Every trial

Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.

Every attempted trial
TrialTime to first token msPrefill tok/sDecode tok/sPeak memory GiBThermalNote
1not recorded45.32,048 tok in 45,226 ms, engine9.7127 tok in 13,158 ms, enginenot recorded38.7 °C, nominal → 38.7 °C, nominal
2not recorded46.12,048 tok in 44,403 ms, engine9.9127 tok in 12,872 ms, enginenot recorded38.6 °C, nominal → 38.8 °C, nominal
3not recorded45.82,048 tok in 44,673 ms, engine11.2127 tok in 11,380 ms, enginenot recorded36.5 °C, nominal → 39.1 °C, nominal
4Invalidnot recorded48.62,048 tok in 42,132 ms, engine11.2127 tok in 11,362 ms, enginenot recorded36.6 °C, nominal → 39.5 °C, lightThe measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial.
5Invalidnot recorded48.32,048 tok in 42,430 ms, engine11.2127 tok in 11,304 ms, enginenot recorded36.6 °C, nominal → 39.5 °C, lightThe measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial.
6Invalidnot recorded48.12,048 tok in 42,553 ms, engine11.2127 tok in 11,295 ms, enginenot recorded36.7 °C, nominal → 39.5 °C, lightThe measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial.

Caveats

  • Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
  • Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.

Configuration

Device

Device
Google Pixel 8
Cohort
Android phones (mobile)
SoC
Google Tensor G3
CPU
9-core CPU (1× Cortex-X3, 4× Cortex-A715, 4× Cortex-A510)
GPU
Arm Mali-G715
NPU
Not recorded
RAM
8.00 GiB
OS
Android 16 (CP1A.260505.005)
Kernel
6.1.145-android14-11-gfa1d6308d1fe-ab14691759
Power
mains
Low power mode
No
Wakefulness
awake

Engine

Engine
LiteRT-LM
Maintainer
Google AI Edge
License
Apache-2.0, open source
Version
0.19.0
Commit
Not recorded
Binary SHA-256
8e199c6b3fb438cb94943567a279f9e1f994d57c3e1f4667084fc8ae2d06a6eb
Built from
release
Build flags
None
Backend
CPU
Threads
Not recorded
Interface
cli
Settings in effect
backend=cpu process=one per generation chat_template=none (session API) sampler=top_k=1 temperature=0 launch.max_num_tokens=4096 launch.num_cpu_threads=engine default launch.cache_dir=/data/local/tmp/xybrid/litert-cache launch.benchmark=true launch.benchmark_prefill_tokens=0 launch.benchmark_decode_tokens=0 launch.async=false

Model and weights

Base model
LFM2.5 2.6B (LFM2.5, 2.6B)
Upstream
LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
Weights
litert-community/LFM2.5-2.6B@8f87ecede9509fd8d12c3e5b8c46b04b597febfc
Format
litertlm
File 1
LFM2.5-2.6B_int4.litertlm · 1.55 GiB · sha256 7e2194cd81cd82bcbdd6029b3509d99f49c1782afbdfc98c0e4b29b9cc04e6a6
File 2
README.md · 18,500 bytes · sha256 ad39356f0ce401994fed4791f08a33353db1c6e078358f000d3ddcc8ff756bbb
Quantization
litert-int4 (4-bit)
Weight bits
4
Group size
Not recorded
KV cache
litert-lm default
Conversion
Published upstream by litert-community
Conversion command
Not recorded

Workload

Workload
Archive 2048 input / 128 output
Prompt
2,048 tokens, from prompts/source.txt
Prompt SHA-256
c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
Chat template
ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
Output
128 tokens, ignoring end of sequence
Context length
4,096
Batch size
1
Sampling
Greedy
Timing
One streamed request per trial
Warmup
2 runs, not counted
Trials planned
6
Cooldown
60 s between trials

Conditions

Ambient
Not recorded
Thermal at start
36.3 °C, nominal
Thermal at end
39.5 °C, light
Thermal gate
Android thermal status none (0)
Throttling detected
Yes

Provenance

Measured by
Glenn Sonna, Xybrid
Operator affiliation
Xybrid
Run at
13 Nov 2026, 04:28 UTC
Harness
xybrid-bench 0.4.0 (14b0290a7d4391e5dbe45d6bc492b0715990af88)
Methodology
v0.6
Publication
published
Published
16 Nov 2026, 10:23 UTC
Reviewed by
Glenn Sonna

Output check

Result
Passed
Method
sanity-v1-bf16-reference
Reference
checks/sanity-v1.json
Reference SHA-256
a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
Required passing checks
8
Check arithmetic
Passed · sha256 90a647b2379fe43265d36db39c01e0d324059b2e48f24ec4e2c1f6ec1c0d21eb
Check capital
Passed · sha256 7a3bb1905f2039638dff3f8a3da57abf7b1da32596b04c5ed04c5296f155196a
Check opposite
Passed · sha256 69a13e1ba8e284fcc26068ec8aa80e8a7e7f8bbebe86dc7a45424411198a6859
Check weekday
Passed · sha256 b8c6a621f6e216db1d214b8d906c759be744ff3abc6e41f979ff0d24575cbbcd
Check color
Failed · sha256 e7cb21de52f71fbf3756d3e7fbeec92e8b5fa123b1a21476d1fd59911201f7d1
Check sort
Failed · sha256 15f349e050f6fd4d9c4f893f73af531ddb5d7b6242c4516df6401d3df0f653f3
Check animal
Failed · sha256 aaf458b3b76ec5162e6ac43bee3ea73000738a3f4fe5c0316852d2f99886fce3
Check word
Passed · sha256 2e1816dd952a0d4c672ad52a4f32599d4eb0477bf5798060b93fe68f7684dbbd
Check copy
Passed · sha256 d8c711dfa5fef68d1d0f86320daeca065b91bc8d0dbcea210f8b9ef80eefc6c2
Check needle-start
Passed · sha256 56a69645d0ff053b3b7001c224f07a374bf963a22c9fa4706dda5e56e3616870
Check needle-end
Passed · sha256 34b7b9e17d75628c346994d8be48566a2dd011ba8533dc75fc33a579ee705365
Check continuation-128
Passed · sha256 fe1fdf5d49caa68c1ef2f8a341413cb2c3826023aef1113c6c289dcd86193b32
Detail
8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Reproduce it

Commands
pnpm bench run --plan plans/s4-pixel8-lfm2.5-2.6b-4-bit.json

Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.

Raw data