Myric/KAT-Coder-V2.5-Dev-MTP-APEX-GGUF

KAT-Coder-V2.5-Dev — APEX GGUF with MTP head

MoE-aware, mixed-precision APEX quantization of Kwaipilot/KAT-Coder-V2.5-Dev, an agentic-coding fine-tune on the Qwen3.6-35B-A3B backbone (qwen3_5_moe: 40 layers, 256 routed

• 1 shared expert, hybrid GatedDeltaNet linear-attention with periodic full attention). Kwaipilot reports SWE-bench Verified 69.40 for this size class, against Qwen3.6-35B-A3B's 64.40.

What makes this build different: it has a working MTP head. KAT-Coder ships mtp_num_hidden_layers: 0 — no multi-token-prediction head at all, so no speculative decoding is possible out of the box, and that is true of the vendor release and of every other quant of this model I am aware of. This repo transplants Qwen3.6-35B-A3B's own trained MTP head onto KAT-Coder and quantizes the result. Measured below: 2.03x on a hard agentic-coding suite, with correctness unchanged.

Files

file	size	what it is
KAT-Coder-V2.5-Dev-MTP-APEX-i-quality.gguf	20.72 GB	recommended default — APEX i-quality (~4.67 bpw) with the MTP head included
KAT-Coder-V2.5-Dev-APEX-dynamic.gguf	11.86 GiB	role-aware allocation targeting a 16GB card; no MTP head. See the size/quality note below — this one beats the stock baseline on both axes at once
kat-coder.imatrix	192 MB	importance matrix used to build both; standard GGUF format, works with any llama-quantize --imatrix
model-00014-of-mtp.safetensors	1.69 GB	the bf16 MTP head shard, if you want to do the transplant yourself
config.json, model.safetensors.index.json	—	already merged to reference all 14 shards
TOOL_CALLING.md	—	reproducible chained tool-calling transcript

Pick MTP-APEX-i-quality unless you are VRAM-constrained, in which case APEX-dynamic is the best quality available at its size. To run either you need only the .gguf. The safetensors shard and index are there for anyone who wants to redo the splice against the original checkpoint: drop those three files into a local copy of Kwaipilot/KAT-Coder-V2.5-Dev alongside its own 13 model-*-of-00013.safetensors shards. KAT-Coder's original shards are untouched and byte-identical; this is a drop-in addition, not a re-upload of the 65 GB checkpoint.

The MTP head — honest framing

This is the naive, un-fine-tuned transplant. The head's weights are copied straight from stock Qwen3.6 with zero training on KAT-Coder's own hidden-state distribution. It is architecturally clean rather than a hack — hidden_size, expert count, and attention/MoE shapes match exactly, because KAT-Coder is itself built on the Qwen3.6-35B-A3B backbone — but a fine-tuning pass (freezing KAT-Coder's 40 layers, training only the ~850M-parameter head) is a planned follow-up. These numbers are the baseline before that.

Draft-acceptance rate

llama-server --spec-type draft-mtp --spec-draft-n-max 2, 5 real from-scratch coding prompts, n_predict=400 each:

prompt	draft tokens	accepted	rate	tok/s
binary search tree (insert/delete/traverse)	286	255	89.2%	105.5
CSV parser (stdlib only)	114	84	73.7%	94.6
retry decorator w/ exponential backoff	188	152	80.9%	100.6
merge two sorted linked lists	74	58	78.4%	97.6
thread-safe LRU cache	360	218	60.6%	85.9
overall	1022	767	75.1%	—

The same quant tier without any MTP head runs ~70.8 tok/s solo. Acceptance tracks predictability: BST traversal boilerplate hits 89%, the LRU cache with genuine locking and eviction logic is hardest at 61%.

Agentic coding benchmarks

Two task suites run through opencode's real tool-calling loop, MTP build vs the same quant without MTP. Correctness was identical — 100% on both suites either way (59/59 and 28/28 test cases), which is what the theory predicts: the MTP head only drafts, the main model always verifies against its own computation, so a bad draft is rejected and never reaches the output. Only speed moves.

hard-tier task	no MTP	with MTP	speedup
batch_scheduler	22.5s	20.8s	1.08x
buddy_allocator	123.5s	63.5s	1.94x
circuit_breaker	50.7s	14.0s	3.62x
expr_parser_advanced	54.7s	21.1s	2.59x
inventory_fix_and_extend	16.4s	12.3s	1.33x
total	267.8s	131.7s	2.03x

tier-4 task	no MTP	with MTP	speedup
lru_ttl_cache	20.5s	25.2s	0.81x
rate_limiter_fix_and_extend	19.7s	12.1s	1.63x
weighted_interval_scheduling	22.5s	17.4s	1.29x
total	62.7s	54.7s	1.15x

The single slower result (lru_ttl_cache, 0.81x) is most likely run-to-run variance in how much iteration that attempt needed rather than a systematic MTP cost — every other task improved.

The imatrix

llama-imatrix is impractical on this architecture: GatedDeltaNet linear-attention is a serial state-space recurrence, and the imatrix collection callback breaks the GPU path, falling back to a single CPU thread. kat-coder.imatrix was instead generated with a band-serialized PyTorch forward pass over a general text/code corpus. It is a standard GGUF-format importance matrix (in_sum2 + counts per tensor), bit-compatible with llama-quantize --imatrix, so it is directly reusable if you want to build your own tiers.

One checkpoint quirk worth knowing if you do: KAT-Coder stores MoE experts as fully separate per-expert tensors (mlp.experts.{e}.{gate,up,down}_proj.weight × 256) rather than the fused gate_up_proj/down_proj layout the HF module expects. Both imatrix generation and GGUF conversion have to fuse them.

Allocation

APEX structural mode: routed experts are bucketed into edge / near / middle depth bands (edge = first and last layers, generally most load-bearing) with decreasing precision toward the middle. For i-quality that is routed edge → Q6_K, near → Q4_K, middle → Q3_K. Attention, shared-expert, and router tensors stay at higher fixed precision (Q6_K–Q8_0).

Measured quality

Wikitext perplexity across the tiers that were built. The two marked hosted are in this repo; the other two were measured for comparison and are not uploaded:

tier	file size	bits/weight	wikitext PPL	vs bf16
bf16 (reference)	64.61 GiB	16.01	7.028	—
APEX-i-quality	19.23 GiB	4.65	7.117	+1.27%	recipe used by the hosted MTP build
APEX-dynamic	12.13 GiB	2.94	7.543	+7.33%	hosted
IQ3_XXS (stock, imatrix-guided)	12.68 GB	~2.98	7.881	+12.13%	stock baseline, for comparison

Two things worth drawing out. i-quality lands within 1.3% of full bf16 at under a third of the size. And APEX-dynamic beats the stock IQ3_XXS baseline on both axes at once — smaller (12.13 vs 12.68 GiB) and lower perplexity (7.543 vs 7.881) — which is why it is the one hosted alongside the flagship rather than the stock tier.

That result is cleaner than the same method got on Laguna-XS, where it traded size for a small PPL cost. Whether role/depth-aware allocation beats a well-tuned stock quant appears to depend on the architecture and calibration, and is not something to assume without measuring.

The hosted MTP build uses the i-quality recipe; its file is slightly larger than the 19.23 GiB row because it carries the MTP head as well.

Attribution

• KAT-Coder base model: Kwaipilot — Kwaipilot/KAT-Coder-V2.5-Dev (Apache-2.0).
• MTP head weights: Qwen — Qwen/Qwen3.6-35B-A3B (Apache-2.0), transplanted unmodified.
• APEX recipe & toolkit: LocalAI — localai-org/apex-quant.
• Quantization engine: llama.cpp (ggml-org).

Unofficial community quantization and experiment; not affiliated with or endorsed by Kwaipilot or Qwen.