---
license: other
base_model: moonshotai/Kimi-K2.7-Code
library_name: transformers
tags:
- speculative-decoding
- eagle3
- eagle
- draft-model
- kimi-k2
- specforge
---

# Kimi-K2.7-Code EAGLE3 draft

EAGLE3 speculative-decoding **draft model** for [moonshotai/Kimi-K2.7-Code](https://huggingface.co/moonshotai/Kimi-K2.7-Code),
trained with [SpecForge](https://github.com/sgl-project/SpecForge) (PR #593) on NVIDIA Nemotron-Post-Training-Dataset-v2 (stem+chat+math+code).

- Single-layer Llama-style EAGLE3 draft (`LlamaForCausalLMEagle3`, hidden 7168); consumes target aux hidden states at layers [1,29,57].
- Target vocab/tokenizer: Kimi-K2.7-Code (vocab 163840); reduced draft vocab 32000 (t2d/d2t mapping, 97.5% token coverage on the training mix).
- Checkpoint: epoch_1_step_98000 — **Work-in-progress snapshot (epoch_1_step_98000)** — training still running.

Intended as the EAGLE3 draft in SGLang speculative decoding paired with the Kimi-K2.7-Code target. Trained as an A/B counterpart to the DFlash draft ([cm00cm/Kimi-K2.7-Code-DFlash](https://huggingface.co/cm00cm/Kimi-K2.7-Code-DFlash)).