Kimi K3.1 Pro DSpark
DSpark draft model for speculative decoding with Kimi K3.1 Pro. This is our latest checkpoint that replaces Kimi-K3.1-Pro-DSpark-preview.
This checkpoint is a joint effort with the Red Hat AI Speculators team, using the vLLM project's Speculators library.
| Component | Value |
|---|---|
| Layers | 5 dense decoder layers, 2.5B parameters (BF16) |
| Attention | Grouped-query attention with a 2,048-token sliding window in every layer; 96 query heads, 16 KV heads, head dim 64 |
| Hidden size / FFN | 7168 / 14336 |
| Input | hidden states from 5 target layers |
| Draft block | 7 tokens per forward pass |
| Heads | Markov head (rank 512), confidence head; embedding and LM head shared with the target |
| Context | 1,048,576 tokens; each draft layer attends to the last 2,048 |
Sliding-window attention
Every draft layer uses a 2,048-token sliding window, so the draft's attention cost and KV cache stay fixed as the context grows. This keeps drafting efficient at long context, up to Kimi K3.1 Pro's 1M tokens.
Evaluation settings
- Server:
--speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3.1-Pro-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}' - Client: chat completions, temperature 1.0, top_p 0.95, full test set per benchmark.
- Metric: acceptance length = 1 + accepted draft tokens / drafts (counts the bonus token).
Serving with vLLM
VLLM_KIMI_K3_AUX_ATTN_RES_STREAM=0 \
vllm serve moonshotai/Kimi-K3.1-Pro \
--host 127.0.0.1 --port 19142 \
--trust-remote-code \
--tensor-parallel-size 8 \
--load-format fastsafetensors \
--reasoning-parser kimi_k3 \
--max-model-len 65536 \
--max-num-seqs 128 \
--max-num-batched-tokens 8192 \
--gpu-memory-utilization 0.90 \
--no-enable-prefix-caching \
--per-request-spec-decode-metrics summary \
--speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3.1-Pro-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'
Per-position draft acceptance
Positions 1β7 refer to the proposed draft tokens. Rates are percentages of verification steps.
| Benchmark | Prompts | Max tokens | Acceptance length | Position 1 | Position 2 | Position 3 | Position 4 | Position 5 | Position 6 | Position 7 |
|---|---|---|---|---|---|---|---|---|---|---|
| GSM8K | 1,319 | 1,024 | 5.566 | 91% | 82% | 72% | 64% | 56% | 49% | 43% |
| HumanEval | 164 | 1,024 | 4.660 | 86% | 71% | 59% | 49% | 40% | 34% | 28% |
| MBPP | 974 | 1,024 | 4.376 | 84% | 68% | 55% | 44% | 35% | 28% | 23% |
| MATH-500 | 500 | 2,048 | 4.445 | 84% | 69% | 56% | 45% | 37% | 30% | 24% |
| SPEED-Bench low entropy | 302 | 1,536 | 3.619 | 78% | 58% | 42% | 31% | 23% | 17% | 13% |
| SPEED-Bench coding | 80 | 4,096 | 4.159 | 81% | 64% | 50% | 40% | 32% | 27% | 22% |
| MT-Bench | 80 | 1,024 | 3.393 | 76% | 54% | 38% | 27% | 20% | 14% | 11% |
| SWE-bench Pro | 642 | 2,048 | 3.302 | 75% | 53% | 37% | 26% | 18% | 13% | 9% |
| AIME 2026 | 30 | 4,096 | 3.429 | 77% | 56% | 40% | 28% | 20% | 14% | 10% |
| Pooled | β | β | 3.864 | 79% | 61% | 46% | 35% | 27% | 21% | 17% |
SpeedBench acceptance
| Category | Conversations | Model calls | Max tokens | Acceptance length | Draft acceptance |
|---|---|---|---|---|---|
| Coding | 80 | 89 | 4,096 | 4.0957 | 44.22% |
| Humanities | 80 | 88 | 4,096 | 3.0354 | 29.08% |
| Math | 80 | 98 | 4,096 | 3.2025 | 31.46% |
| QA | 80 | 80 | 4,096 | 3.4800 | 35.43% |
| RAG | 80 | 105 | 4,096 | 4.2890 | 46.99% |
| Reasoning | 80 | 190 | 4,096 | 3.5954 | 37.08% |
| STEM | 80 | 86 | 4,096 | 3.1874 | 31.25% |
| Writing | 80 | 84 | 4,096 | 2.8243 | 26.06% |
| Multilingual | 80 | 80 | 4,096 | 4.0903 | 44.15% |
| Summarization | 80 | 80 | 4,096 | 4.1758 | 45.37% |
| Roleplay | 80 | 158 | 4,096 | 3.0310 | 29.01% |
| Pooled | 880 | 1,138 | 4,096 | 3.2690 | 32.41% |
Long context acceptance
| Benchmark | Prompts | Prompt tokens | Max tokens | Acceptance length | Position 1 | Position 2 | Position 3 | Position 4 | Position 5 | Position 6 | Position 7 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| AA-LCR | 100 | 95K mean | 1,024 | 3.935 | 81% | 64% | 49% | 37% | 27% | 21% | 15% |
| BEAM 128K | 400 | 97Kβ125K | 2,048 | 3.516 | 78% | 58% | 41% | 29% | 20% | 14% | 10% |
| BEAM 1M | 700 | 826Kβ1.04M | 2,048 | 3.421 | 77% | 56% | 40% | 28% | 19% | 13% | 9% |
Throughput and interactivity
- Downloads last month
- 962
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support
