best158 (step95400, K2.6 val-best): overwrite ckpt100; README updated with real accept_len + distribution-shift note

README.md

@@ -1,54 +0,0 @@
----
-license: other
-base_model: moonshotai/Kimi-K2.6
-tags:
-- text-generation
-- speculative-decoding
-- eagle3
-- kimi-k2.6
-- mla
-- torchspec
----
-
-# kimi-k2.6-eagle3-mla
-
-Eagle3 MTP draft model with MLA (Multi-Latent Attention) for accelerating
-inference of [Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6).
-
-This is a fine-tuned draft, anchored to the official
-[lightseekorg/kimi-k2.6-eagle3-mla](https://huggingface.co/lightseekorg/kimi-k2.6-eagle3-mla)
-initialization. It targets multi-hop (downstream-position) acceptance while
-preserving the first-hop gain, evaluated by runtime accept-length on a frozen
-full-context held-out set.
-
-## Fine-tune setup
-
-- **Init**: lightseekorg/kimi-k2.6-eagle3-mla (official MLA weights)
-- **Objective**: Eagle3 distillation + multi-step TTT supervision
-  (`ttt_steps=4`, `ttt_step_loss_decay=1.0`, off-policy downstream tokens)
-- **Anti-over-specialization**: L2-SP weight-space anchor toward the init
-  (penalize trainable-param drift; lambda=1e-4)
-- **Optimizer**: lr 2e-5, cosine schedule
-- **Checkpoint**: best by held-out runtime accept-length
-
-## Performance
-
-Primary metric is **accept_length** — average tokens accepted per speculation
-step with `num_speculative_tokens=3` (higher is better). Per-position numbers
-are conditional acceptance rates at hop 0/1/2. Evaluated on a frozen
-full-context held-out judge set (912 prompts, greedy), vLLM 0.20.0, 8x H200,
-TP=8, max-model-len 32768.
-
-| Model | accept_len | pos-0 | pos-1 | pos-2 |
-|-------|-----------:|------:|------:|------:|
-| lightseek (official init) | 2.30 | 0.633 | 0.404 | 0.264 |
-| this model | **2.345** | **0.648** | **0.419** | **0.278** |
-
-This draft improves first-hop acceptance over the official init while also
-lifting the downstream positions (pos-1, pos-2), yielding a higher overall
-accept length.
-
-## Usage
-
-Serve with vLLM as the speculative draft for Kimi-K2.6, with
-`num_speculative_tokens=3` in the speculative-config.

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:57cf1a3b1cd6153619f37a42410e6123fd899f15d2f95d230dafe68399825d04
+oid sha256:5b5829babeb37d5b1f982b3f6f759e45370f82e5d7b7938398e3a48ecb5e32c4
 size 6031210296