Super-squash branch 'main' using huggingface_hub

0729020 3 days ago

952 Bytes

license: other
base_model: moonshotai/Kimi-K2.7-Code
library_name: transformers
tags:
  - speculative-decoding
  - dflash
  - eagle
  - draft-model
  - kimi-k2
  - specforge

Kimi-K2.7-Code DFlash draft

DFlash speculative-decoding draft model for moonshotai/Kimi-K2.7-Code, trained with SpecForge (PR #593) on NVIDIA Nemotron-Post-Training-Dataset-v2 (stem+chat+math+code).

6-layer Qwen3-style draft (hidden 7168); consumes target hidden states at layers [1,12,24,35,47,58]; block_size 8.
Target vocab/tokenizer: Kimi-K2.7-Code (vocab 163840, mask_token_id 163838).
Checkpoint: epoch_4_step_334000 — Work-in-progress snapshot (epoch_4_step_334000) — training still running.

Load with trust_remote_code=True (model code in dflash.py). Intended as the draft in SGLang DFlash speculative decoding paired with the Kimi-K2.7-Code target.