LFM2.5 2.6B on M5 Max · 64 GiB

Experiment Complete Published

llama.cpp b12110 on M5 Max · 64 GiB, Metal

LFM2.5 2.6B as Q4_K_M (4-bit), archive 256 input / 128 output: a 256-token prompt and 128 tokens out. Measured 12 Nov 2026, 22:58 UTC by Glenn Sonna, Xybrid.

Doesn’t count in rankings: Its output check failed: 8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Experiment id s3-lfm2.5-2.6b-4-bit-llama-256

Summary

Over 6 valid trials of 6 attempted (6 planned), computed from the raw trials below.

MetricMedianRangeMean ± SDQuartiles
Decode speed tok/s, higher is better218.4 212.3–220.4 217.1 ± 3.2 214.9–219.1
Time to first token ms, lower is better62.5 61.2–65.6 62.7 ± 1.6 61.7–63.0
Peak memory GiB, lower is better1.84 1.84–1.86 1.84 ± 0.01 1.84–1.84
Prefill speed tok/s, higher is better4,141 3,950–4,237 4,131 ± 102.9 4,110–4,200

Trials, metric by metric

The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.

Every trial

Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.

Every attempted trial
TrialTime to first token msPrefill tok/sDecode tok/sPeak memory GiBThermalNote
161.24,237256 tok in 60.4 ms, engine217.7127 tok in 583 ms, engine1.86nominal → nominal
262.54,142256 tok in 61.8 ms, engine220.4127 tok in 576 ms, engine1.84nominal → nominal
363.24,100256 tok in 62.4 ms, engine219.0127 tok in 580 ms, engine1.84nominal → nominal
462.54,141256 tok in 61.8 ms, engine212.3127 tok in 598 ms, engine1.84nominal → nominal
565.63,950256 tok in 64.8 ms, engine213.9127 tok in 594 ms, engine1.84nominal → nominal
661.44,219256 tok in 60.7 ms, engine219.2127 tok in 579 ms, engine1.84nominal → nominal

Caveats

  • Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
  • Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.

Configuration

Device

Device
Apple Mac17,6
Cohort
Apple silicon Macs (desktop)
SoC
Apple M5 Max
CPU
18-core Apple CPU
GPU
Not recorded
NPU
Not recorded
RAM
64.00 GiB
OS
macOS 27.2.0 (26B123)
Kernel
27.2.0
Power
mains
Low power mode
No
Performance mode
Not recorded

Engine

Engine
llama.cpp
Maintainer
ggml-org
License
MIT, open source
Version
b12110
Commit
cd959bae9
Binary SHA-256
e7e3c60a5e612460d095bfd219047040d6bb9d0b56466aae30a480980b676959
Built from
release
Build flags
None
Backend
Metal
Threads
18
Interface
http-server
Settings in effect
context_length=4096 parallel=1 speculative=false backend=metal launch.log_verbosity=4 launch.log_colors=off launch.cache_prompt=false launch.cache_ram_mib=0 launch.cache_reuse=0 launch.continuous_batching=false launch.context_shift=false launch.flash_attn=on launch.cache_type_k=f16 launch.cache_type_v=f16 launch.gpu_layers=999 launch.load_mode=mmap offloaded_layers=31 launch.mmap=true n_batch=2048 n_ubatch=512 n_threads=18 n_threads_batch=18 flash_attn=enabled

Model and weights

Base model
LFM2.5 2.6B (LFM2.5, 2.6B)
Upstream
LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
Weights
LiquidAI/LFM2.5-2.6B-GGUF@e7caca5d835a3901a8e0d63e94009429bafafdfc
Format
gguf
File 1
LFM2.5-2.6B-Q4_K_M.gguf · 1.56 GiB · sha256 02a8b7e17487d326e46d68ce0ba24211e1b80a14c4cd0597fa73c1cd697f52ed
File 2
README.md · 3,068 bytes · sha256 d08b79260b3998e96792777085e4175b730667568f3a953ad4ac9099cfa27056
Quantization
Q4_K_M (4-bit)
Weight bits
4
Group size
Not recorded
KV cache
f16
Conversion
Published upstream by LiquidAI
Conversion command
Not recorded

Workload

Workload
Archive 256 input / 128 output
Prompt
256 tokens, from prompts/source.txt
Prompt SHA-256
c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
Chat template
ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
Output
128 tokens, ignoring end of sequence
Context length
4,096
Batch size
1
Sampling
Greedy
Timing
One streamed request per trial
Warmup
2 runs, not counted
Trials planned
6
Cooldown
15 s between trials

Conditions

Ambient
Not recorded
Thermal at start
nominal
Thermal at end
nominal
Thermal gate
macOS thermal state nominal
Throttling detected
No

Provenance

Measured by
Glenn Sonna, Xybrid
Operator affiliation
Xybrid
Run at
12 Nov 2026, 22:58 UTC
Harness
xybrid-bench 0.4.0 (bde6325f3e1d617ca13cbb1d008164ccc9b7b7c3)
Methodology
v0.6
Publication
published
Published
16 Nov 2026, 10:23 UTC
Reviewed by
Glenn Sonna

Output check

Result
Failed
Method
sanity-v1-bf16-reference
Reference
checks/sanity-v1.json
Reference SHA-256
a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
Required passing checks
8
Check arithmetic
Passed · sha256 68ce25cb2c4b530f85434cd352428b022149c641163300e566f4f9ee9d789bcf
Check capital
Passed · sha256 7a3bb1905f2039638dff3f8a3da57abf7b1da32596b04c5ed04c5296f155196a
Check opposite
Passed · sha256 69a13e1ba8e284fcc26068ec8aa80e8a7e7f8bbebe86dc7a45424411198a6859
Check weekday
Passed · sha256 b8c6a621f6e216db1d214b8d906c759be744ff3abc6e41f979ff0d24575cbbcd
Check color
Failed · sha256 fe08d89d0b006dfadf3ef0dfdad25eded8e6238dcd5af7ea93748aaec2d565fd
Check sort
Failed · sha256 f3e9a127eb72ee656a49ae07958aff2ae3343aa843f5b3abc9ae9de9f560a4f8
Check animal
Passed · sha256 3e9114b6df29561433c18b3a6abce2c635fbd8e9ff25b5fcee93149bfea52859
Check word
Passed · sha256 d1bf31457072aaa4c116ea5819a644456232fdd87b15203d3eafdd5e541f4e61
Check copy
Passed · sha256 d8c711dfa5fef68d1d0f86320daeca065b91bc8d0dbcea210f8b9ef80eefc6c2
Check needle-start
Passed · sha256 e4b695d890438ad108a6fd9cb2e39bf857a00a883f231741c1f92416c1b218c4
Check needle-end
Passed · sha256 34b7b9e17d75628c346994d8be48566a2dd011ba8533dc75fc33a579ee705365
Check continuation-128
Failed · sha256 da3683ef8034e95c52683294949e3e9c606f4429c3d6e782c3fcadaa386ce12a
Detail
8/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Reproduce it

Commands
pnpm bench run --plan plans/s3-lfm2.5-2.6b-4-bit.json

Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.

Raw data