| license: other | |
| base_model: moonshotai/Kimi-K2.7-Code | |
| library_name: transformers | |
| tags: | |
| - speculative-decoding | |
| - eagle3 | |
| - eagle | |
| - draft-model | |
| - kimi-k2 | |
| - specforge | |
| # Kimi-K2.7-Code EAGLE3 draft | |
| EAGLE3 speculative-decoding **draft model** for [moonshotai/Kimi-K2.7-Code](https://huggingface.co/moonshotai/Kimi-K2.7-Code), | |
| trained with [SpecForge](https://github.com/sgl-project/SpecForge) (PR #593) on NVIDIA Nemotron-Post-Training-Dataset-v2 (stem+chat+math+code). | |
| - Single-layer Llama-style EAGLE3 draft (`LlamaForCausalLMEagle3`, hidden 7168); consumes target aux hidden states at layers [1,29,57]. | |
| - Target vocab/tokenizer: Kimi-K2.7-Code (vocab 163840); reduced draft vocab 32000 (t2d/d2t mapping, 97.5% token coverage on the training mix). | |
| - Checkpoint: epoch_1_step_98000 — **Work-in-progress snapshot (epoch_1_step_98000)** — training still running. | |
| Intended as the EAGLE3 draft in SGLang speculative decoding paired with the Kimi-K2.7-Code target. Trained as an A/B counterpart to the DFlash draft ([cm00cm/Kimi-K2.7-Code-DFlash](https://huggingface.co/cm00cm/Kimi-K2.7-Code-DFlash)). | |