LFM2.5 2.6B on M5 Max · 64 GiB

Experiment Complete Published

Rapid-MLX 1.0.2 on M5 Max · 64 GiB, Metal

LFM2.5 2.6B as mlx-affine-4bit-g64 (4-bit), archive 256 input / 128 output: a 256-token prompt and 128 tokens out. Measured 12 Nov 2026, 22:59 UTC by Glenn Sonna, Xybrid.

Doesn’t count in rankings: Its output check failed: 7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Experiment id s3-lfm2.5-2.6b-4-bit-rapid-mlx-256

Summary

Over 6 valid trials of 6 attempted (6 planned), computed from the raw trials below.

MetricMedianRangeMean ± SDQuartiles
Decode speed tok/s, higher is better265.2 263.1–265.6 264.7 ± 1.1 263.7–265.5
Time to first token ms, lower is better58.0 55.3–59.3 57.5 ± 1.8 56.0–58.9
Peak memory GiB, lower is better2.12 2.06–2.29 2.14 ± 0.09 2.08–2.15
Prefill speed tok/s, higher is better4,457 4,360–4,684 4,500 ± 143.2 4,387–4,622

Trials, metric by metric

The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.

Every trial

Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.

Every attempted trial
TrialTime to first token msPrefill tok/sDecode tok/sPeak memory GiBThermalNote
158.74,397256 tok in 58.2 ms, engine263.1127 tok in 483 ms, engine2.15nominal → nominal
259.34,360256 tok in 58.7 ms, engine265.6127 tok in 478 ms, engine2.09nominal → nominal
359.04,383256 tok in 58.4 ms, engine264.8127 tok in 480 ms, engine2.06nominal → nominal
455.34,684256 tok in 54.7 ms, engine265.5127 tok in 478 ms, engine2.07nominal → nominal
557.34,516256 tok in 56.7 ms, engine263.4127 tok in 482 ms, engine2.15nominal → nominal
655.54,658256 tok in 55.0 ms, engine265.5127 tok in 478 ms, engine2.29nominal → nominal

Caveats

  • Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
  • Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.

Configuration

Device

Device
Apple Mac17,6
Cohort
Apple silicon Macs (desktop)
SoC
Apple M5 Max
CPU
18-core Apple CPU
GPU
Not recorded
NPU
Not recorded
RAM
64.00 GiB
OS
macOS 27.2.0 (26B123)
Kernel
27.2.0
Power
mains
Low power mode
No
Performance mode
Not recorded

Engine

Engine
Rapid-MLX
Maintainer
Rapid-MLX contributors
License
Apache-2.0, open source
Version
1.0.2
Commit
f1aa011aa91bb38140ed3fb68f6267b77447ee0f
Binary SHA-256
d5c9b638732cda0b2e2a816aa85cc1780e0a3c1b690bb23c8d9b4d3ac81a527c
Built from
package
Build flags
None
Backend
Metal
Threads
Not recorded
Interface
python-worker
Settings in effect
backend=Device(gpu, 0) model_source=artifact folder context_length=4096 batch_size=1 max_num_seqs=1 max_concurrent_requests=1 prefill_batch_size=1 completion_batch_size=1 prefill_step_size=2048 prefix_cache=false disk_caches=false speculative=false spec_decode=none suffix_decoding=false response_cache_entries=0 pflash=off kv_cache_dtype=bf16 kv_quantized=false admission_window_ms=0 stream_interval=1 gpu_memory_utilization=0.9 greedy=true temperature=0 sampler=argmax trust_remote_code=false telemetry=false weight_bits=4 weight_group_size=64

Model and weights

Base model
LFM2.5 2.6B (LFM2.5, 2.6B)
Upstream
LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
Weights
LiquidAI/LFM2.5-2.6B-MLX-4bit@04efa23776ce61ec34ec95ec34c859854c89542b
Format
safetensors
File 1
README.md · 2,648 bytes · sha256 0907976d22f77d4178e10d04002403a94a22ad965ad8431f754fb28a14885f6e
File 2
chat_template.jinja · 5,496 bytes · sha256 aadd425edb1d42fe9288e5de3624c21c1c03099129b9cfea5978f1bc67222c0a
File 3
config.json · 2,202 bytes · sha256 6ab5ad71687f6ca8d3811b9290ac06291f78e1e0f4f3ec504e6644032d58342b
File 4
generation_config.json · 306 bytes · sha256 e08c9a83f4bfe64468c9671a2b69b60549728e2a9481387f3555471f64e958bc
File 5
model.safetensors · 1.47 GiB · sha256 ef350b75815752f8bff8c064c9fc8bd8b5709577d72943436852013d2bcd4691
File 6
model.safetensors.index.json · 41,983 bytes · sha256 620f8efaf53e826757324852e999f6ba9a5d974e193589470a70951fd17a1543
File 7
tokenizer.json · 17.1 MiB · sha256 695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
File 8
tokenizer_config.json · 363 bytes · sha256 11f1de897317b489dd09199284528382eeb17f566398b16e2039bb2266d26b09
Quantization
mlx-affine-4bit-g64 (4-bit)
Weight bits
4
Group size
64
KV cache
model-dtype
Conversion
Published upstream by LiquidAI
Conversion command
Not recorded

Workload

Workload
Archive 256 input / 128 output
Prompt
256 tokens, from prompts/source.txt
Prompt SHA-256
c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
Chat template
ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
Output
128 tokens, ignoring end of sequence
Context length
4,096
Batch size
1
Sampling
Greedy
Timing
One streamed request per trial
Warmup
2 runs, not counted
Trials planned
6
Cooldown
15 s between trials

Conditions

Ambient
Not recorded
Thermal at start
nominal
Thermal at end
nominal
Thermal gate
macOS thermal state nominal
Throttling detected
No

Provenance

Measured by
Glenn Sonna, Xybrid
Operator affiliation
Xybrid
Run at
12 Nov 2026, 22:59 UTC
Harness
xybrid-bench 0.4.0 (bde6325f3e1d617ca13cbb1d008164ccc9b7b7c3)
Methodology
v0.6
Publication
published
Published
16 Nov 2026, 10:23 UTC
Reviewed by
Glenn Sonna

Output check

Result
Failed
Method
sanity-v1-bf16-reference
Reference
checks/sanity-v1.json
Reference SHA-256
a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
Required passing checks
8
Check arithmetic
Passed · sha256 90a647b2379fe43265d36db39c01e0d324059b2e48f24ec4e2c1f6ec1c0d21eb
Check capital
Passed · sha256 b1098d691672b55e98767ac84c6d8375f7bcc0d78154c861ea70ac2ff66e0ba7
Check opposite
Passed · sha256 69a13e1ba8e284fcc26068ec8aa80e8a7e7f8bbebe86dc7a45424411198a6859
Check weekday
Passed · sha256 b8c6a621f6e216db1d214b8d906c759be744ff3abc6e41f979ff0d24575cbbcd
Check color
Failed · sha256 fe08d89d0b006dfadf3ef0dfdad25eded8e6238dcd5af7ea93748aaec2d565fd
Check sort
Failed · sha256 cb12005007583cdcd569a7d96955cb80cb294053fa0ac7cedb3ee401fa09d13b
Check animal
Passed · sha256 3e9114b6df29561433c18b3a6abce2c635fbd8e9ff25b5fcee93149bfea52859
Check word
Passed · sha256 2e1816dd952a0d4c672ad52a4f32599d4eb0477bf5798060b93fe68f7684dbbd
Check copy
Failed · sha256 40ec15ba7f6dd791515c1e570997f3fd45822d5f2c56544e7b35cb2e6fb16729
Check needle-start
Passed · sha256 8b9af2e5b8b4cfddb87d171997b95c471f8d68a68e30fb5177abeb3332372edd
Check needle-end
Passed · sha256 e6589c736af32f81bf239c5eda6de41959122ccc1d05463b75e0eb0c70fc831c
Check continuation-128
Failed · sha256 595ee2e62bb2b0afabce45fc4d707def46db7e116ff6d218da9f9a33e57c4274
Detail
7/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.

Reproduce it

Commands
pnpm bench run --plan plans/s3-lfm2.5-2.6b-4-bit.json

Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.

Raw data