Inferact/Kimi-K3-DSpark-Block5

Kimi-K3 DSpark — Block 5

A DSpark draft model for speculative decoding with moonshotai/Kimi-K3. Three full-causal MLA layers, conditioned on hidden states extracted from three of the target's layers and trained at block_size=5: one forward pass drafts the full five-token block.

Training

The draft completed a four-epoch run with speculators, the speculative-decoding library from the vLLM project, against Kimi-K3 hidden states streamed live from vLLM.

Acceptance

Evaluated against Kimi-K3 on 8×B300 (TP8), drafting 5 tokens with probabilistic sampling and block rejection at temperature 1.0 and top-p 0.95. All 1,604 requests succeeded. The aggregate is computed from summed counters, not an average of arm rates.

Workload	Requests	Acceptance	Mean accepted length (max 6)
GSM8K	256	71.62%	4.581
HumanEval	164	63.74%	4.187
MBPP	256	56.38%	3.819
MATH-500	500	47.48%	3.374
SWE-bench Pro	128	38.09%	2.904
MT-Bench	80	33.19%	2.659
AIME26	30	29.78%	2.489
AA-LCR (~100K)	100	41.04%	3.052
BEAM 100K	20	37.17%	2.858
BEAM 500K	35	30.26%	2.513
BEAM 1M	35	31.37%	2.569
Aggregate	1,604	46.10%	3.305

The separate five-shot GSM8K accuracy check scored 96.51% exact match on all 1,319 examples with zero request errors.

The full 1M context window is supported through YaRN, evaluated on every row of the BEAM 100K, 500K, and 1M splits above. The fixed 1M acceptance corpus left-truncates oldest turns in 14 of 35 over-limit source conversations to keep prompt plus output within 1,048,576 tokens; its longest measured prompt is 1,039,871 tokens. Two successful completions were flagged as degenerate by the client (BEAM 500K conversation 4 and BEAM 1M conversation 18). Full counters, per-position rates, and quality flags are in benchmark_results.json.

Serving

vllm serve moonshotai/Kimi-K3 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --max-model-len 1048576 \
  --speculative-config '{
    "method": "dspark",
    "model": "Inferact/Kimi-K3-DSpark-Block5",
    "num_speculative_tokens": 5,
    "attention_backend": "FLASHINFER_MLA",
    "draft_sample_method": "probabilistic",
    "rejection_sample_method": "block"
  }'