Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,65 @@
+---
+license: apache-2.0
+base_model: Kwaipilot/KAT-Coder-V2.5-Dev
+tags:
+  - mtp
+  - speculative-decoding
+  - qwen3_5_moe
+  - experiment
+---
+
+# KAT-Coder-V2.5-Dev — MTP head (experimental, un-fine-tuned baseline)
+
+[Kwaipilot/KAT-Coder-V2.5-Dev](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev) ships with
+`mtp_num_hidden_layers: 0` — no MTP/NextN head at all, so no speculative-decoding speedup is
+possible out of the box. This repo adds one back by transplanting
+[Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)'s own trained MTP head onto
+KAT-Coder. This is architecturally clean, not a hack: every relevant dimension (hidden_size,
+expert count, attention/MoE shapes) matches exactly between the two models, since KAT-Coder is
+itself built on the Qwen3.6-35B-A3B backbone.
+
+**This is the naive, un-fine-tuned transplant** — the MTP head's weights are copied directly from
+stock Qwen3.6 with zero training on KAT-Coder's own hidden-state distribution. A fine-tuning pass
+(freezing KAT-Coder's own 40 layers entirely, training only this ~850M-parameter head) is a
+planned follow-up; this repo documents the honest baseline before that.
+
+## What's actually in this repo
+
+Just **one new shard** (`model-00014-of-mtp.safetensors`, 1.69GB) containing the 19 MTP-related
+tensors, plus a `config.json`/`model.safetensors.index.json` already merged to reference it
+alongside KAT-Coder's own 13 original shards. Nothing else was touched — KAT-Coder's own 13
+shards are byte-identical to the original release; this is a drop-in addition, not a full
+re-upload of the 65GB checkpoint.
+
+**To use**: put this repo's 3 files into a local copy of
+[Kwaipilot/KAT-Coder-V2.5-Dev](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev) alongside its
+own 13 `model-*-of-00013.safetensors` shards (the `config.json`/index here already reference all
+14 shards together).
+
+## Baseline: draft-acceptance rate (before fine-tuning)
+
+Measured via `llama-server --spec-type draft-mtp --spec-draft-n-max 2` on an APEX i-quality quant
+(4.67 bpw) built from this checkpoint, 5 real from-scratch coding prompts, `n_predict=400` each:
+
+| prompt | draft tokens | accepted | rate | tok/s |
+|---|---:|---:|---:|---:|
+| binary search tree (insert/delete/traverse) | 286 | 255 | 89.2% | 105.5 |
+| CSV parser (stdlib only) | 114 | 84 | 73.7% | 94.6 |
+| retry decorator w/ exponential backoff | 188 | 152 | 80.9% | 100.6 |
+| merge two sorted linked lists | 74 | 58 | 78.4% | 97.6 |
+| thread-safe LRU cache | 360 | 218 | 60.6% | 85.9 |
+| **overall** | **1022** | **767** | **75.1%** | — |
+
+For reference, the same KAT-Coder quant tier **without** any MTP head runs at ~70.8 tok/s solo
+decode speed (measured separately, no speculative decoding available at all). So even this
+un-fine-tuned transplant is already a real ~20-40% speedup, not a wash — acceptance rate varies
+by task in a way that tracks predictability (the BST task, mostly standard tree-traversal
+boilerplate, hits 89%; the LRU cache, with genuine locking/eviction logic, is the hardest at 61%).
+
+## Attribution
+
+- KAT-Coder base model: **Kwaipilot** — [Kwaipilot/KAT-Coder-V2.5-Dev](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev) (Apache-2.0).
+- MTP head weights: **Qwen** — [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) (Apache-2.0), transplanted unmodified.
+- Quantization/serving: **llama.cpp** ([ggml-org](https://github.com/ggml-org/llama.cpp)).
+
+Unofficial community experiment; not affiliated with or endorsed by Kwaipilot or Qwen.