LFM2.5 2.6B on M5 Max · 64 GiB

Experiment Complete Published

LiteRT-LM 0.19.1 on M5 Max · 64 GiB, CPU

LFM2.5 2.6B as litert-int8 (8-bit), archive 256 input / 128 output: a 256-token prompt and 128 tokens out. Measured 12 Nov 2026, 22:29 UTC by Glenn Sonna, Xybrid.

Doesn’t count in rankings: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Experiment id s3-lfm2.5-2.6b-8-bit-litert-256

Summary

Over 6 valid trials of 6 attempted (6 planned), computed from the raw trials below.

MetricMedianRangeMean ± SDQuartiles
Decode speed tok/s, higher is better60.3 60.0–60.5 60.3 ± 0.2 60.1–60.4
Time to first token ms, lower is better541 539–542 541 ± 1.2 540–541
Peak memory GiB, lower is better3.42 3.42–3.42 3.42 ± 0.00 3.42–3.42
Prefill speed tok/s, higher is better515.9 514.4–517.4 515.9 ± 1.1 515.3–516.7

Trials, metric by metric

The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.

Every trial

Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.

Every attempted trial
TrialTime to first token msPrefill tok/sDecode tok/sPeak memory GiBThermalNote
1541515.3256 tok in 497 ms, engine60.3127 tok in 2,105 ms, engine3.42nominal → nominal
2540516.8256 tok in 495 ms, engine60.0127 tok in 2,116 ms, engine3.42nominal → nominal
3540516.4256 tok in 496 ms, engine60.4127 tok in 2,104 ms, engine3.42nominal → nominal
4539517.4256 tok in 495 ms, engine60.5127 tok in 2,099 ms, engine3.42nominal → nominal
5542514.4256 tok in 498 ms, engine60.5127 tok in 2,101 ms, engine3.42nominal → nominal
6541515.4256 tok in 497 ms, engine60.0127 tok in 2,117 ms, engine3.42nominal → nominal

Caveats

  • Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
  • Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.

Configuration

Device

Device
Apple Mac17,6
Cohort
Apple silicon Macs (desktop)
SoC
Apple M5 Max
CPU
18-core Apple CPU
GPU
Not recorded
NPU
Not recorded
RAM
64.00 GiB
OS
macOS 27.2.0 (26B123)
Kernel
27.2.0
Power
mains
Low power mode
No
Performance mode
Not recorded

Engine

Engine
LiteRT-LM
Maintainer
Google AI Edge
License
Apache-2.0, open source
Version
0.19.1
Commit
Not recorded
Binary SHA-256
c61e0faa71026f2875a710a42cbe803226004db811f7ee47ea78198a0499e851
Built from
package
Build flags
None
Backend
CPU
Threads
Not recorded
Interface
python-worker
Settings in effect
backend=cpu max_num_tokens=4096 enable_benchmark=true chat_template=pass-through process=one per generation sampler=top_k=1 temperature=0 stop_token_ids=[[124900], [124895]] cache_dir=[runner]/engines/litert-lm/cache bos_token_id=124894

Model and weights

Base model
LFM2.5 2.6B (LFM2.5, 2.6B)
Upstream
LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
Weights
litert-community/LFM2.5-2.6B@8f87ecede9509fd8d12c3e5b8c46b04b597febfc
Format
litertlm
File 1
LFM2.5-2.6B_int8.litertlm · 2.67 GiB · sha256 02e97ce3a4de444f1b0bbca4494827704f86212bf72c201703e564d96068536a
File 2
README.md · 18,500 bytes · sha256 ad39356f0ce401994fed4791f08a33353db1c6e078358f000d3ddcc8ff756bbb
Quantization
litert-int8 (8-bit)
Weight bits
8
Group size
Not recorded
KV cache
litert-lm default
Conversion
Published upstream by litert-community
Conversion command
Not recorded

Workload

Workload
Archive 256 input / 128 output
Prompt
256 tokens, from prompts/source.txt
Prompt SHA-256
c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
Chat template
ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
Output
128 tokens, ignoring end of sequence
Context length
4,096
Batch size
1
Sampling
Greedy
Timing
One streamed request per trial
Warmup
2 runs, not counted
Trials planned
6
Cooldown
15 s between trials

Conditions

Ambient
Not recorded
Thermal at start
nominal
Thermal at end
nominal
Thermal gate
macOS thermal state nominal
Throttling detected
No

Provenance

Measured by
Glenn Sonna, Xybrid
Operator affiliation
Xybrid
Run at
12 Nov 2026, 22:29 UTC
Harness
xybrid-bench 0.4.0 (777cb3d9fecaa7cc1a30694d98492c33dbc6d693)
Methodology
v0.6
Publication
published
Published
16 Nov 2026, 10:23 UTC
Reviewed by
Glenn Sonna

Output check

Result
Failed
Method
sanity-v1-bf16-reference
Reference
checks/sanity-v1.json
Reference SHA-256
a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
Required passing checks
8
Check arithmetic
Failed · sha256 c0b2a5a0debecb2e388c47cdc10113fcd481f224c0efac592a60e4dd102bed56
Check capital
Passed · sha256 7a3bb1905f2039638dff3f8a3da57abf7b1da32596b04c5ed04c5296f155196a
Check opposite
Passed · sha256 7c90b06eef25984adf77ec226311829bbc1065a033e9ad6b26d9bd0dfdc65f95
Check weekday
Passed · sha256 b8c6a621f6e216db1d214b8d906c759be744ff3abc6e41f979ff0d24575cbbcd
Check color
Failed · sha256 3cde5b4f191c8cd46ee0e9a89b6a7cf42c5cfe733bbf8158ee61992779919dc0
Check sort
Failed · sha256 b9eeb858235b7c5e8af46f7bf38023806dc0df277f67e79c8fdfbe600fc006a2
Check animal
Failed · sha256 e21ef004e72f5ee91147436e546b095269528f2a7dc8cf5395c28183a7eaa9aa
Check word
Passed · sha256 d1bf31457072aaa4c116ea5819a644456232fdd87b15203d3eafdd5e541f4e61
Check copy
Passed · sha256 d8c711dfa5fef68d1d0f86320daeca065b91bc8d0dbcea210f8b9ef80eefc6c2
Check needle-start
Passed · sha256 56a69645d0ff053b3b7001c224f07a374bf963a22c9fa4706dda5e56e3616870
Check needle-end
Failed · sha256 4f3ac3da90b331c87691c05586d5d3b5d29194c2ab1d93c4a82802fec8e40a0e
Check continuation-128
Passed · sha256 fe1fdf5d49caa68c1ef2f8a341413cb2c3826023aef1113c6c289dcd86193b32
Detail
7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Reproduce it

Commands
pnpm bench run --plan plans/s3-lfm2.5-2.6b-8-bit.json

Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.

Raw data