Myric/KAT-Coder-V2.5-Dev-MTP-head

KAT-Coder-V2.5-Dev — MTP head (experimental, un-fine-tuned baseline)

Kwaipilot/KAT-Coder-V2.5-Dev ships with mtp_num_hidden_layers: 0 — no MTP/NextN head at all, so no speculative-decoding speedup is possible out of the box. This repo adds one back by transplanting Qwen/Qwen3.6-35B-A3B's own trained MTP head onto KAT-Coder. This is architecturally clean, not a hack: every relevant dimension (hidden_size, expert count, attention/MoE shapes) matches exactly between the two models, since KAT-Coder is itself built on the Qwen3.6-35B-A3B backbone.

This is the naive, un-fine-tuned transplant — the MTP head's weights are copied directly from stock Qwen3.6 with zero training on KAT-Coder's own hidden-state distribution. A fine-tuning pass (freezing KAT-Coder's own 40 layers entirely, training only this ~850M-parameter head) is a planned follow-up; this repo documents the honest baseline before that.

What's actually in this repo

Just one new shard (model-00014-of-mtp.safetensors, 1.69GB) containing the 19 MTP-related tensors, plus a config.json/model.safetensors.index.json already merged to reference it alongside KAT-Coder's own 13 original shards. Nothing else was touched — KAT-Coder's own 13 shards are byte-identical to the original release; this is a drop-in addition, not a full re-upload of the 65GB checkpoint.

To use: put this repo's 3 files into a local copy of Kwaipilot/KAT-Coder-V2.5-Dev alongside its own 13 model-*-of-00013.safetensors shards (the config.json/index here already reference all 14 shards together).

Baseline: draft-acceptance rate (before fine-tuning)

Measured via llama-server --spec-type draft-mtp --spec-draft-n-max 2 on an APEX i-quality quant (4.67 bpw) built from this checkpoint, 5 real from-scratch coding prompts, n_predict=400 each:

prompt	draft tokens	accepted	rate	tok/s
binary search tree (insert/delete/traverse)	286	255	89.2%	105.5
CSV parser (stdlib only)	114	84	73.7%	94.6
retry decorator w/ exponential backoff	188	152	80.9%	100.6
merge two sorted linked lists	74	58	78.4%	97.6
thread-safe LRU cache	360	218	60.6%	85.9
overall	1022	767	75.1%	—

For reference, the same KAT-Coder quant tier without any MTP head runs at ~70.8 tok/s solo decode speed (measured separately, no speculative decoding available at all). So even this un-fine-tuned transplant is already a real ~20-40% speedup, not a wash — acceptance rate varies by task in a way that tracks predictability (the BST task, mostly standard tree-traversal boilerplate, hits 89%; the LRU cache, with genuine locking/eviction logic, is the hardest at 61%).

Full coding-benchmark validation (real agentic tasks, not just completion speed)

Ran the same two task suites used elsewhere in this project's Qwen3.6-lineage work (an existing 5-task hard tier, plus a new 3-task tier designed to require genuine algorithmic reasoning rather than spec-transcription) against this MTP-enabled build vs the same quant without MTP, via opencode's real agentic tool-calling loop (not just raw completion). Correctness was identical — 100% on both suites, both with and without MTP — exactly what the theory predicts: the MTP head only drafts candidate tokens, the main model always verifies against its own true computation, so a bad draft just gets rejected and never reaches the output. Speed differed substantially:

hard-tier task	no MTP	with MTP	speedup
batch_scheduler	22.5s	20.8s	1.08x
buddy_allocator	123.5s	63.5s	1.94x
circuit_breaker	50.7s	14.0s	3.62x
expr_parser_advanced	54.7s	21.1s	2.59x
inventory_fix_and_extend	16.4s	12.3s	1.33x
total	267.8s	131.7s	2.03x

tier-4 task	no MTP	with MTP	speedup
lru_ttl_cache	20.5s	25.2s	0.81x
rate_limiter_fix_and_extend	19.7s	12.1s	1.63x
weighted_interval_scheduling	22.5s	17.4s	1.29x
total	62.7s	54.7s	1.15x

Both suites hit 100% (59/59 and 28/28 test cases respectively) in both configurations. The one slower result (lru_ttl_cache, 0.81x) is most likely ordinary run-to-run variance in how much iteration that specific attempt needed, not a systematic MTP cost — every other task improved, and the hard-tier set as a whole is a genuine 2x.

Attribution

• KAT-Coder base model: Kwaipilot — Kwaipilot/KAT-Coder-V2.5-Dev (Apache-2.0).
• MTP head weights: Qwen — Qwen/Qwen3.6-35B-A3B (Apache-2.0), transplanted unmodified.
• Quantization/serving: llama.cpp (ggml-org).

Unofficial community experiment; not affiliated with or endorsed by Kwaipilot or Qwen.