Experiment Failed Published
llama.cpp b12110 on Pixel 8 · 8 GB, CPU
LFM2.5 2.6B as Q4_K_M (4-bit), archive 2048 input / 128 output: a 2048-token prompt and 128 tokens out. Measured 13 Nov 2026, 04:20 UTC by Glenn Sonna, Xybrid.
Doesn’t count in rankings: No trial was valid.
Experiment id s4-pixel8-lfm2.5-2.6b-4-bit-llama-2048
Errors
- trial: No trial was valid.
- trial: No trial was valid.
- trial: No trial was valid.
- trial: No trial was valid.
- trial: No trial was valid.
- trial: No trial was valid.
Summary
Over 0 valid trials of 6 attempted (6 planned), computed from the raw trials below.
| Metric | Median | Range | Mean ± SD | Quartiles |
|---|---|---|---|---|
| Decode speed tok/s, higher is better | Not measured | |||
| Time to first token ms, lower is better | Not measured | |||
| Peak memory GiB, lower is better | Not measured | |||
| Prefill speed tok/s, higher is better | Not measured | |||
Trials, metric by metric
The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.
Every trial
Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.
| Trial | Time to first token ms | Prefill tok/s | Decode tok/s | Peak memory GiB | Thermal | Note |
|---|---|---|---|---|---|---|
| 1Invalid | 54,362 | 37.72,048 tok in 54,334 ms, engine | 8.5127 tok in 14,897 ms, engine | not recorded | 37.0 °C, nominal → 39.9 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
| 2Invalid | 54,238 | 37.82,048 tok in 54,212 ms, engine | 8.2127 tok in 15,447 ms, engine | not recorded | 37.1 °C, nominal → 39.9 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
| 3Invalid | 58,462 | 35.02,048 tok in 58,437 ms, engine | 8.3127 tok in 15,250 ms, engine | not recorded | 36.9 °C, nominal → 39.9 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
| 4Invalid | 56,399 | 36.32,048 tok in 56,374 ms, engine | 6.1127 tok in 20,947 ms, engine | not recorded | 37.1 °C, nominal → 40.1 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
| 5Invalid | 54,182 | 37.82,048 tok in 54,158 ms, engine | 8.8127 tok in 14,437 ms, engine | not recorded | 36.9 °C, nominal → 39.9 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
| 6Invalid | 54,551 | 37.62,048 tok in 54,526 ms, engine | 8.3127 tok in 15,213 ms, engine | not recorded | 37.1 °C, nominal → 39.9 °C, light | The measurement interval lacks nominal thermal evidence or reported throttling. The device reported thermal throttling during the trial. |
Caveats
- Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
- Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.
Configuration
Device
- Device
- Google Pixel 8
- Cohort
- Android phones (mobile)
- SoC
- Google Tensor G3
- CPU
- 9-core CPU (1× Cortex-X3, 4× Cortex-A715, 4× Cortex-A510)
- GPU
- Arm Mali-G715
- NPU
- Not recorded
- RAM
- 8.00 GiB
- OS
- Android 16 (CP1A.260505.005)
- Kernel
- 6.1.145-android14-11-gfa1d6308d1fe-ab14691759
- Power
- mains
- Low power mode
- No
- Wakefulness
- awake
Engine
- Engine
- llama.cpp
- Maintainer
- ggml-org
- License
- MIT, open source
- Version
- b12110
- Commit
- cd959bae9
- Binary SHA-256
- db82377125f69e87c06b402bba1af072e2a4c4cc9945946e4d6378cc213a84d7
- Built from
- release
- Build flags
- None
- Backend
- CPU
- Threads
- Not recorded
- Interface
- http-server
- Settings in effect
- context_length=4096 parallel=1 speculative=false backend=cpu launch.log_verbosity=4 launch.log_colors=off launch.cache_prompt=false launch.cache_ram_mib=0 launch.cache_reuse=0 launch.continuous_batching=false launch.context_shift=false launch.flash_attn=on launch.cache_type_k=f16 launch.cache_type_v=f16 launch.gpu_layers=0 launch.load_mode=mmap offloaded_layers=0 launch.mmap=true n_batch=2048 n_ubatch=512 n_threads=9 n_threads_batch=9 flash_attn=enabled
Model and weights
- Base model
- LFM2.5 2.6B (LFM2.5, 2.6B)
- Upstream
- LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
- Weights
- LiquidAI/LFM2.5-2.6B-GGUF@e7caca5d835a3901a8e0d63e94009429bafafdfc
- Format
- gguf
- File 1
- LFM2.5-2.6B-Q4_K_M.gguf · 1.56 GiB · sha256 02a8b7e17487d326e46d68ce0ba24211e1b80a14c4cd0597fa73c1cd697f52ed
- File 2
- README.md · 3,068 bytes · sha256 d08b79260b3998e96792777085e4175b730667568f3a953ad4ac9099cfa27056
- Quantization
- Q4_K_M (4-bit)
- Weight bits
- 4
- Group size
- Not recorded
- KV cache
- f16
- Conversion
- Published upstream by LiquidAI
- Conversion command
- Not recorded
Workload
- Workload
- Archive 2048 input / 128 output
- Prompt
- 2,048 tokens, from prompts/source.txt
- Prompt SHA-256
- c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
- Chat template
- ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
- Output
- 128 tokens, ignoring end of sequence
- Context length
- 4,096
- Batch size
- 1
- Sampling
- Greedy
- Timing
- One streamed request per trial
- Warmup
- 2 runs, not counted
- Trials planned
- 6
- Cooldown
- 60 s between trials
Conditions
- Ambient
- Not recorded
- Thermal at start
- 34.8 °C, nominal
- Thermal at end
- 40.0 °C, light
- Thermal gate
- Android thermal status none (0)
- Throttling detected
- Yes
Provenance
- Measured by
- Glenn Sonna, Xybrid
- Operator affiliation
- Xybrid
- Run at
- 13 Nov 2026, 04:20 UTC
- Harness
- xybrid-bench 0.4.0 (14b0290a7d4391e5dbe45d6bc492b0715990af88)
- Methodology
- v0.6
- Publication
- published
- Published
- 16 Nov 2026, 10:23 UTC
- Reviewed by
- Glenn Sonna
Output check
- Result
- Failed
- Method
- sanity-v1-bf16-reference
- Reference
- checks/sanity-v1.json
- Reference SHA-256
- a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
- Required passing checks
- 8
- Check arithmetic
- Passed · sha256 68ce25cb2c4b530f85434cd352428b022149c641163300e566f4f9ee9d789bcf
- Check capital
- Passed · sha256 7a3bb1905f2039638dff3f8a3da57abf7b1da32596b04c5ed04c5296f155196a
- Check opposite
- Passed · sha256 69a13e1ba8e284fcc26068ec8aa80e8a7e7f8bbebe86dc7a45424411198a6859
- Check weekday
- Passed · sha256 b8c6a621f6e216db1d214b8d906c759be744ff3abc6e41f979ff0d24575cbbcd
- Check color
- Failed · sha256 a4849893c59e4aa15ff645cadd4cce0fbd882097352a75d8e6f23619578a73c6
- Check sort
- Failed · sha256 f3e9a127eb72ee656a49ae07958aff2ae3343aa843f5b3abc9ae9de9f560a4f8
- Check animal
- Passed · sha256 3e9114b6df29561433c18b3a6abce2c635fbd8e9ff25b5fcee93149bfea52859
- Check word
- Passed · sha256 d1bf31457072aaa4c116ea5819a644456232fdd87b15203d3eafdd5e541f4e61
- Check copy
- Passed · sha256 d8c711dfa5fef68d1d0f86320daeca065b91bc8d0dbcea210f8b9ef80eefc6c2
- Check needle-start
- Passed · sha256 e4b695d890438ad108a6fd9cb2e39bf857a00a883f231741c1f92416c1b218c4
- Check needle-end
- Failed · sha256 7deeb9a4120d4e1813e7bce3bf6d26474752cc1c02487a6ebb0ecf96d9b8e9c6
- Check continuation-128
- Failed · sha256 45734251dc0e5c977efaa337051bd14150a8046421439ce96c90951ea926ed3d
- Detail
- 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.
Reproduce it
pnpm bench run --plan plans/s4-pixel8-lfm2.5-2.6b-4-bit.json
Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.
Raw data
- This record, as committed JSON, schema v0.3.0
- Complete finalized session evidence raw ·
sha256 34ec477ebcf6…