Inferact/Kimi-K3.1-Pro-DSpark-preview

Preview checkpoint. We are working on a stronger version of this draft and will release it soon.

Kimi K3.1 Pro DSpark (preview)

DSpark draft model for Kimi K3.1 Pro.

Component	Value
Layers	5 dense decoder layers, 2.7B parameters (BF16)
Attention	MLA, full causal; 96 heads, query/KV LoRA rank 1536/512
Hidden size / FFN	7168 / 14336
Input	hidden states from 5 target layers
Draft block	7 tokens per forward pass
Heads	Markov head (rank 512), confidence head; embedding and LM head shared with the target
Context	1,048,576 tokens (YaRN, factor 32)

Evaluation settings

• Server: --speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3.1-Pro-DSpark-preview","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}'
• Client: vllm-bench chat completions, temperature 1.0, top_p 0.95, full test set per benchmark.
• Metric: acceptance length = 1 + accepted draft tokens / drafts (counts the bonus token).

Results

Benchmark	Prompts	Max tokens	Acceptance length	Accept rate, positions 1-7
GSM8K	1,319	1,024	4.597	0.86 0.71 0.59 0.48 0.39 0.32 0.26
HumanEval	164	1,024	3.585	0.76 0.56 0.41 0.30 0.23 0.18 0.14
MBPP	974	1,024	3.520	0.77 0.56 0.41 0.29 0.21 0.16 0.12
MATH-500	500	2,048	3.824	0.80 0.62 0.46 0.35 0.26 0.19 0.14
SPEED-Bench low entropy (10K-token inputs)	512	1,536	2.930	0.70 0.46 0.30 0.19 0.13 0.09 0.06
MT-Bench	80	1,024	2.812	0.69 0.44 0.27 0.17 0.11 0.07 0.05
SWE-bench Pro	642	2,048	2.741	0.67 0.43 0.27 0.16 0.10 0.06 0.04
AIME 2026	30	4,096	2.973	0.71 0.49 0.32 0.20 0.13 0.08 0.05
Pooled			3.154	0.72 0.50 0.34 0.23 0.16 0.11 0.08

Long context

Benchmark	Prompts	Prompt tokens	Max tokens	Acceptance length	Accept rate, positions 1-7
AA-LCR	100	95K mean	1,024	3.501	0.75 0.57 0.42 0.30 0.22 0.15 0.10
BEAM 128K	400	97K-125K	2,048	2.954	0.71 0.48 0.31 0.20 0.13 0.08 0.05
BEAM 1M	700	826K-1.04M	2,048	2.765	0.70 0.45 0.28 0.16 0.09 0.05 0.03