Experiment Complete Published
Rapid-MLX 1.0.2 on M5 Max · 64 GiB, Metal
LFM2.5 2.6B as mlx-affine-8bit-g64 (8-bit), archive 2048 input / 128 output: a 2048-token prompt and 128 tokens out. Measured 12 Nov 2026, 22:42 UTC by Glenn Sonna, Xybrid.
Eligible for ranking when the objective's required metrics are measured and this configuration is selected.Approximate comparison: The experiments use different effective KV cache precision: litert-lm default, f16, model-dtype. The experiments here use different 8-bit schemes: litert-int8, Q8_0, mlx-affine-8bit-g64.
Experiment id s3-lfm2.5-2.6b-8-bit-rapid-mlx-2048
Summary
Over 6 valid trials of 6 attempted (6 planned), computed from the raw trials below.
| Metric | Median | Range | Mean ± SD | Quartiles |
|---|---|---|---|---|
| Decode speed tok/s, higher is better | 160.0 | 159.4–160.8 | 160.1 ± 0.5 | 159.8–160.2 |
| Time to first token ms, lower is better | 242 | 240–242 | 242 ± 0.8 | 242–242 |
| Peak memory GiB, lower is better | 4.14 | 4.10–4.22 | 4.15 ± 0.05 | 4.11–4.19 |
| Prefill speed tok/s, higher is better | 8,490 | 8,484–8,560 | 8,503 ± 29.1 | 8,487–8,505 |
Trials, metric by metric
The line runs from the lowest to the highest valid trial, the dot is the median and each tick is one trial.
Every trial
Raw token counts and durations, with the speeds computed from them. Invalid trials stay in the record with their reason; they are left out of the summary, never silently dropped.
| Trial | Time to first token ms | Prefill tok/s | Decode tok/s | Peak memory GiB | Thermal | Note |
|---|---|---|---|---|---|---|
| 1 | 241 | 8,5092,048 tok in 241 ms, engine | 159.8127 tok in 795 ms, engine | 4.22 | nominal → nominal | |
| 2 | 242 | 8,4872,048 tok in 241 ms, engine | 160.2127 tok in 793 ms, engine | 4.11 | nominal → nominal | |
| 3 | 242 | 8,4912,048 tok in 241 ms, engine | 159.9127 tok in 794 ms, engine | 4.11 | nominal → nominal | |
| 4 | 242 | 8,4892,048 tok in 241 ms, engine | 160.8127 tok in 790 ms, engine | 4.20 | nominal → nominal | |
| 5 | 240 | 8,5602,048 tok in 239 ms, engine | 160.2127 tok in 793 ms, engine | 4.10 | nominal → nominal | |
| 6 | 242 | 8,4842,048 tok in 241 ms, engine | 159.4127 tok in 797 ms, engine | 4.16 | nominal → nominal |
Caveats
- Publisher cards identify the base model but do not always pin the exact export source checkpoint revision.
- Engines may differ in quantization scheme, effective KV cache precision, kernels, and activations.
Configuration
Device
- Device
- Apple Mac17,6
- Cohort
- Apple silicon Macs (desktop)
- SoC
- Apple M5 Max
- CPU
- 18-core Apple CPU
- GPU
- Not recorded
- NPU
- Not recorded
- RAM
- 64.00 GiB
- OS
- macOS 27.2.0 (26B123)
- Kernel
- 27.2.0
- Power
- mains
- Low power mode
- No
- Performance mode
- Not recorded
Engine
- Engine
- Rapid-MLX
- Maintainer
- Rapid-MLX contributors
- License
- Apache-2.0, open source
- Version
- 1.0.2
- Commit
- f1aa011aa91bb38140ed3fb68f6267b77447ee0f
- Binary SHA-256
- d5c9b638732cda0b2e2a816aa85cc1780e0a3c1b690bb23c8d9b4d3ac81a527c
- Built from
- package
- Build flags
- None
- Backend
- Metal
- Threads
- Not recorded
- Interface
- python-worker
- Settings in effect
- backend=Device(gpu, 0) model_source=artifact folder context_length=4096 batch_size=1 max_num_seqs=1 max_concurrent_requests=1 prefill_batch_size=1 completion_batch_size=1 prefill_step_size=2048 prefix_cache=false disk_caches=false speculative=false spec_decode=none suffix_decoding=false response_cache_entries=0 pflash=off kv_cache_dtype=bf16 kv_quantized=false admission_window_ms=0 stream_interval=1 gpu_memory_utilization=0.9 greedy=true temperature=0 sampler=argmax trust_remote_code=false telemetry=false weight_bits=8 weight_group_size=64
Model and weights
- Base model
- LFM2.5 2.6B (LFM2.5, 2.6B)
- Upstream
- LiquidAI/LFM2.5-2.6B@7141b56cd4650af95c7c7b86983443aed050c96a
- Weights
- LiquidAI/LFM2.5-2.6B-MLX-8bit@b372ebbb518c0e81617e25d8824427dd9ee1f08c
- Format
- safetensors
- File 1
- README.md · 2,648 bytes · sha256 0e60b04b3ef5ab4e03e971537ec84cd45256c82f00efce24095ba73bf3f467f0
- File 2
- chat_template.jinja · 5,496 bytes · sha256 aadd425edb1d42fe9288e5de3624c21c1c03099129b9cfea5978f1bc67222c0a
- File 3
- config.json · 1,952 bytes · sha256 6f0d4974b6a17ff0d1140196b89d56107fdb0b88b0a548ce3035c9d99f139c52
- File 4
- generation_config.json · 306 bytes · sha256 e08c9a83f4bfe64468c9671a2b69b60549728e2a9481387f3555471f64e958bc
- File 5
- model.safetensors · 2.67 GiB · sha256 04f7c6997b1501ae2551109aee50d57419f651a80d1a68a58a2c4d53202ee1ca
- File 6
- model.safetensors.index.json · 41,983 bytes · sha256 e18c6098614fe191dd3991c4403ee60830179e3d61c518496549b3890db9140c
- File 7
- tokenizer.json · 17.1 MiB · sha256 695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
- File 8
- tokenizer_config.json · 363 bytes · sha256 11f1de897317b489dd09199284528382eeb17f566398b16e2039bb2266d26b09
- Quantization
- mlx-affine-8bit-g64 (8-bit)
- Weight bits
- 8
- Group size
- 64
- KV cache
- model-dtype
- Conversion
- Published upstream by LiquidAI
- Conversion command
- Not recorded
Workload
- Workload
- Archive 2048 input / 128 output
- Prompt
- 2,048 tokens, from prompts/source.txt
- Prompt SHA-256
- c10d1b7bef62d95322f8f8d229dc66118d9375107bdc49b06f76287d058b2f6c
- Chat template
- ea663864491de7ade391839479860ca95541f892f72665c73251fbd4643b1bef
- Output
- 128 tokens, ignoring end of sequence
- Context length
- 4,096
- Batch size
- 1
- Sampling
- Greedy
- Timing
- One streamed request per trial
- Warmup
- 2 runs, not counted
- Trials planned
- 6
- Cooldown
- 15 s between trials
Conditions
- Ambient
- Not recorded
- Thermal at start
- nominal
- Thermal at end
- nominal
- Thermal gate
- macOS thermal state nominal
- Throttling detected
- No
Provenance
- Measured by
- Glenn Sonna, Xybrid
- Operator affiliation
- Xybrid
- Run at
- 12 Nov 2026, 22:42 UTC
- Harness
- xybrid-bench 0.4.0 (777cb3d9fecaa7cc1a30694d98492c33dbc6d693)
- Methodology
- v0.6
- Publication
- published
- Published
- 16 Nov 2026, 10:23 UTC
- Reviewed by
- Glenn Sonna
Output check
- Result
- Passed
- Method
- sanity-v1-bf16-reference
- Reference
- checks/sanity-v1.json
- Reference SHA-256
- a702dee079525ae0ec90465b1c9e26052c99cb3e1c53e83dea05be4c194fc7ca
- Required passing checks
- 8
- Check arithmetic
- Passed · sha256 68ce25cb2c4b530f85434cd352428b022149c641163300e566f4f9ee9d789bcf
- Check capital
- Passed · sha256 b1098d691672b55e98767ac84c6d8375f7bcc0d78154c861ea70ac2ff66e0ba7
- Check opposite
- Passed · sha256 7c90b06eef25984adf77ec226311829bbc1065a033e9ad6b26d9bd0dfdc65f95
- Check weekday
- Passed · sha256 03db656ee680fb1053be9c98631722488bb10269075159dbae5b000f0359dae2
- Check color
- Failed · sha256 a4849893c59e4aa15ff645cadd4cce0fbd882097352a75d8e6f23619578a73c6
- Check sort
- Failed · sha256 6904619ddb316a20bc1220a9088da9a4181f1f16766b6fdd188defd9cddc4c75
- Check animal
- Passed · sha256 3e9114b6df29561433c18b3a6abce2c635fbd8e9ff25b5fcee93149bfea52859
- Check word
- Passed · sha256 d1bf31457072aaa4c116ea5819a644456232fdd87b15203d3eafdd5e541f4e61
- Check copy
- Passed · sha256 d8c711dfa5fef68d1d0f86320daeca065b91bc8d0dbcea210f8b9ef80eefc6c2
- Check needle-start
- Passed · sha256 56a69645d0ff053b3b7001c224f07a374bf963a22c9fa4706dda5e56e3616870
- Check needle-end
- Failed · sha256 1a8ad5c1a3264694da79d5312eeb8f20a35e3c5522e4cd33bbbe072f6e724877
- Check continuation-128
- Passed · sha256 35b435d7f1c75bb05c07d58cad000525a1d62d6dbe7fd00ae7416953dfa26286
- Detail
- 9/9 of the checks the model passes at BF16; needs 8 and the mandatory ones (needle-start, continuation-128). Left out, failed by every BF16 configuration: color, sort, needle-end.
Reproduce it
pnpm bench run --plan plans/s3-lfm2.5-2.6b-8-bit.json
Three palindromic blocks per quantization class and workload; load, two warmups, one trial, unload per turn.
Raw data
- This record, as committed JSON, schema v0.3.0
- Complete finalized session evidence raw ·
sha256 92db4b58d893…