praveenkumarpranjal/LFM2.5-2.6B-4bit-PathPack-Q

LFM2.5-2.6B 4-bit PathPack-Q

An experimental, training-free 4-bit MLX quantization of LiquidAI/LFM2.5-2.6B.

PathPack-Q exploits exact channel-permutation symmetries in LFM2's double-gated short-convolution blocks. It changes which channels share each 64-weight affine quantization group while preserving the floating-point function, checkpoint size, bit width, and inference graph.

This is a measured research artifact, not a claim of universal superiority. The current evidence compares PathPack-Q with matched MLX round-to-nearest affine 4-bit quantization on one model and one machine.

Result

Both quantized checkpoints are exactly 1,517,616,892 bytes, use 4.501 effective bits/weight, and peak at 2.091 GB in the matched evaluation.

Metric	BF16	Uniform MLX 4-bit	PathPack-Q 4-bit	PathPack-Q vs uniform
WikiText-2 perplexity, 8,160 held-out tokens	73.93	92.69	87.78	-5.30%
Mean teacher KL, 12 fixed prompts	—	0.2220	0.2001	-9.87%
Mean teacher logit cosine	—	0.96641	0.96711	+0.00070
Teacher top-1 agreement, 12 prompts	—	66.7%	75.0%	+8.3 pp

The 200-example-per-task lm-eval subset is directional, not statistically conclusive:

Task	Uniform acc_norm	PathPack-Q acc_norm	Delta
PIQA	69.0%	69.5%	+0.5 pp
HellaSwag	48.0%	49.0%	+1.0 pp
ARC-Easy	42.0%	43.0%	+1.0 pp
Macro average	53.0%	53.83%	+0.83 pp

Raw accuracy is mixed: PIQA -1.0 pp, HellaSwag -0.5 pp, ARC-Easy unchanged. Full benchmark suites and additional architectures are still required.

How it works

For an LFM2 short-convolution channel permutation P, PathPack-Q applies the same permutation to:

• the B, C, and X row blocks of conv.in_proj;
• the depthwise convolution channels; and
• the input columns of conv.out_proj.

This is an exact reparameterization. Before quantization, the maximum measured relative path MSE was 5.74e-13.

A deterministic 5,000-proposal swap search minimizes reconstruction error under MLX's real affine 4-bit/group-64 quantizer. Candidate permutations are then checked on synthetic unit-RMS inputs through the complete gated convolution path. Twenty of 22 convolution blocks passed; layers 28 and 29 were left untouched.

No model training, gradients, text calibration set, extra parameters, high-precision residual, codebook, or runtime permutation is used.

Usage

pip install mlx-lm
mlx_lm.generate \
  --model praveenkumarpranjal/LFM2.5-2.6B-4bit-PathPack-Q \
  --prompt "Explain post-training quantization." \
  --max-tokens 256

Python:

from mlx_lm import generate, load

model, tokenizer = load("praveenkumarpranjal/LFM2.5-2.6B-4bit-PathPack-Q")
messages = [{"role": "user", "content": "Explain post-training quantization."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Reproduction

The method/ folder contains the search, path validation, checkpoint builder, and evaluator. evals/ contains machine-readable results. The core flow is:

python method/search_group_packing.py --model /path/to/LFM2.5-2.6B --layer 0 \
  --swaps 5000 --trials 0 --output packing-layer0-exact5k.json

python method/validate_path_packing.py --model /path/to/LFM2.5-2.6B \
  --packing-dir results --output path-validation.json

python method/build_checkpoint.py --model /path/to/LFM2.5-2.6B \
  --packing-dir results --validation path-validation.json --output mlx_model

Environment used: Apple M4 MacBook Air, 16 GB unified memory, MLX-LM 0.31.3.

Limitations

• Tested only on LFM2.5-2.6B and MLX affine W4/group-64.
• WikiText-2 evaluation covers 8,160 tokens, not the full test split.
• Downstream tasks use fixed 200-example subsets; confidence intervals overlap.
• Prompt agreement uses only 12 fixed prompts.
• Wall-clock runs were thermally confounded, so this card makes no measured speedup claim. The method adds no operators and does not change model bytes.
• Novelty is an engineering/research hypothesis pending broader literature review and external reproduction.

Related work

PathPack-Q is informed by, but technically distinct from, reorder-based PTQ, activation-aware scaling, and SSM-specific PTQ:

• RPTQ
• AWQ
• Quamba
• MLX-LM learned quantization

License

The weights remain subject to the source model's LFM1.0 license.