LFM2.5 / Maple Preview — CPU-Efficient Quantized Variants and a Three-Engine Exploration

Author: Ljubomir Josifovski (ljupco) — Harpenden, UK, 2026. Hardware: thinkpad2 (Lenovo, Intel i5-8350U, 4C/8T, AVX2, 64 GB DDR4-2400) and macbook2 (M2 Max, 96 GB). Scope: quantized GGUF variants of the LiquidAI LFM2.5 family and the DeepGrove Maple Preview, plus a three-engine porting/benchmarking exploration: stock llama.cpp (with the DeepGrove fork for Maple), ik-llama.cpp, and vllm.cpp.

1. llama.cpp (stock + the DeepGrove fork) — the reference engines

1.1 Maple Preview on the DeepGrove fork (`deepgrove-main`)

The best CPU result of the whole project is the Maple Preview model (DeepGrove's ternary 2-bit MoE, ~1B active parameters of ~20B total, 256 experts with 8 routed) running on the DeepGrove fork of llama.cpp. The fork adds the maple architecture (src/models/maple.cpp), the TQ2_0 ternary quantization, and the GGUF conversion support. On top of that upstream work we added:

TQ2_0 ternary Metal kernels (ggml/src/ggml-metal/ggml-metal.metal) for the M2 Max, and
per-machine server wrapper scripts (llama_server_maple-preview_macbook2.sh, llama_server_maple-preview_thinkpad2.sh).

The Maple Preview GGUFs referenced here are the DeepGrove team's own releases; the engine is the DeepGrove fork. We made no code changes that need publishing for this model — the fork is upstream; we only reference it. Measured decode (benchy tg64, 4 threads, thinkpad2): ~28.2 t/s — the fastest model of the four.

1.2 The LFM2.5 quantized variants (this repository's contribution)

For the LiquidAI LFM2.5 family (2.6B dense, 1.2B-Thinking, 8B-A1B MoE) we produced mixed-quant GGUF variants that keep the most sensitive tensors (attention / router / output heads) at a higher precision while the bulk of the weights are 4-bit (Q4_K / Q4_0). The variants we distribute on HuggingFace (ljupco/*) are:

Model	Variant	Body	Heads
LFM2.5-2.6B	`-Q4_K_M`	Q4_K	Q4_K
LFM2.5-1.2B-Thinking	`-Q4_0h`	Q4_0	higher-precision heads
LFM2.5-8B-A1B	`-Q4_0h`	Q4_0	higher-precision heads

These are the files benchmarked in the engine sections below. Full credit and thanks to the original model creators — see the Credits section.

2. `ik-llama.cpp` — the LFM2 port (`lfm2-port` branch)

~/ik-llama.cpp carries a from-scratch LFM2 architecture port into a llama.cpp fork: the LFM2 2.6B dense and 1.2B-Thinking and 8B-A1B MoE, including the shortconv layers, the gated-delta attention, and the NEOX RoPE handling that the LFM2 family requires.

Status: correct and complete — all three LFM2 models run and produce valid output. Benchmarks vs the upstream llama.cpp reference (benchy tg64, 4 threads, thinkpad2):

Model	ik-llama.cpp	upstream llama.cpp	delta
LFM2.5-2.6B-Q4_K_M	13.08 t/s	12.40 t/s	+5 %
LFM2.5-1.2B-Thinking-Q4_0h	25.30 t/s	30.40 t/s	−17 %
LFM2.5-8B-A1B-Q4_0h	13.81 t/s	20.99 t/s	−34 %

The 2.6B dense shows a small win; the 1.2B and 8B trails. The port is the reference used for the vllm.cpp section. No separate code release is planned for this fork beyond the branch itself; it is a research port.

3. `vllm.cpp` — the LFM2/Maple port (`lfm2-maple-port` branch)

~/vllm.cpp/worktrees/lfm2-maple-port (fork: ljubomirj/vllm.cpp, branch lfm2-maple-port) is a port of all four models into the vllm.cpp engine (GGUF-first, C++20, single binary), with a large speed campaign. This work did not beat the llama.cpp references — it is published for reference and as a record of the optimisation arc.

3.1 The port

All four models run and produce correct output: LFM2.5 2.6B / 1.2B-Thinking / 8B-A1B and Maple Preview (general.architecture dispatch in src/vllm/entrypoints/model_loader.cpp).

3.2 The speed campaign — what was done

The iqk tier — vendored the llama.cpp CPU K-quant integer-dot kernels into src/vt/cpu/iqk/ (Q4_K / Q6_K decode dots at ~72 GFLOP/s, DDR4-bandwidth-bound).
Op fusion — the LFM2 layers run at ~4 ops vs ~9 originally:
- the gate/up row-concat merge (per-row block quants make the concat a pure byte copy);
- the silu+down fused kernel (kMatmulBf16DSilu);
- the post-norm+gate_up fusion (kMatmulBf16DPostNorm);
- the in-norm+in_proj and in-norm+first-qkv fusions (the input-layernorm with its residual-add folded into the act quantization);
- the attention bf16-alignment fix (the fused q_proj's act row must match the bf16-rounded normed activation the sibling k/v projections consume).
The R8 gemv arc — vendored the DeepGrove 8x8 LUT gemv (src/vt/cpu/gemv_q4k/). Conclusive negative result: the DeepGrove gemv is structurally tied to the DeepGrove Q4_K sub-block-pair packing (qs[j] = {value (j%32)+64*(j/32), value +32}) and its (2s, 2s+1) scale/min/bsum grouping, which is incompatible with the standard ggml Q4_K packing (sub-blocks (s, s+4)) used by all standard-quantized models. The gemv is correct only for DeepGrove-quantized weights. This is recorded here so the next engineer does not repeat the arc.
Threadpool — the ggml-faithful barrier+steal pool is the local optimum; the poll=0 cond-wait experiment is slower than the spin for the fast decode ops.

3.3 Results (benchy tg64, 4 threads, thinkpad2)

Model	vllm.cpp (best)	llama.cpp reference	delta
LFM2.5-2.6B-Q4_K_M	9.0 t/s	13.08 (ik)	−31 %
LFM2.5-1.2B-Thinking-Q4_0h	~14 t/s	25.30 (ik)	−45 %
LFM2.5-8B-A1B-Q4_0h	9.3 t/s	13.81 (ik)	−33 %
Maple Preview	13.9 t/s	28.2 (DeepGrove)	−51 %

The residual gap is the DDR4-bandwidth-bound dot compute itself (the dots are at the same speed as the reference; the overhead is at the floor after the fusions).

4. Publishing

4.1 HuggingFace — the quantized GGUFs

The mixed-quant LFM2.5 GGUFs are published under ljupco on HuggingFace. Each repository's model card declares the base_model metadata pointing at the original LiquidAI model, so HuggingFace lists our variants on the originals' Quantizations pages.

4.2 Code

Maple Preview: the engine is the DeepGrove fork (upstream); our additions are the TQ2_0 ternary Metal kernels (a non-trivial, reusable contribution — ~400+ lines of ggml-metal support for the ternary quantization that could be upstreamed to llama.cpp) and the per-machine wrapper scripts, which remain in the deepgrove-main worktree.
ik-llama.cpp: the lfm2-port branch remains the reference.
vllm.cpp: the lfm2-maple-port branch is pushed to the ljubomirj/vllm.cpp fork for reference.

5. Credits and Acknowledgements

We are deeply grateful to the teams whose work this project builds on:

Liquid AI — for the LFM2.5 family (2.6B, 1.2B-Thinking, 8B-A1B), the gated-delta / shortconv architecture, and their open weights. We thank them for making these models available.
DeepGrove AI — for the Maple Preview model, the TQ2_0 ternary quantization, and their llama.cpp fork with the Maple architecture support. Their 8x8 gemv design and the fork itself made the Maple results possible.
llama.cpp / ggml — the core inference engine and its maintainers and contributors.
vllm.cpp (mudler) — the engine we ported into.
The HuggingFace / GGUF ecosystem — the format, the tooling, and the platform.

Without their work, none of this project would exist. Any remaining errors are ours.

Report written 2026-08-08. Benchmarks are indicative single-machine measurements; expect day-to-day noise of ±10–20 % on the thinkpad2.