update license

README.md

@@ -16,46 +16,28 @@ tags:
 
 ## Model Overview
 
-**Inferact/Kimi-K3-DSpark** is an **MLA-native DSpark** draft model that accelerates [Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3) on **[vLLM](https://github.com/vllm-project/vllm)**, served through vLLM's native `dspark` speculative method.
+**Inferact/Kimi-K3-DSpark** is an **MLA-native DSpark** draft model that accelerates inference of [Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3) on **[vLLM](https://github.com/vllm-project/vllm)**, which serves it through its native `dspark` speculative method.
 
-The draft trains on **target hidden states extracted from vLLM itself** — the same engine that serves it, so the numerics it learns from are the numerics it meets at inference. [TorchSpec](https://github.com/lightseekorg/TorchSpec) provides the loop, streaming those hidden states from live target inference into concurrent FSDP draft training.
+The draft is trained on **target hidden states extracted from vLLM itself** — the same engine that serves it — so it learns from exactly the distribution it will later draft against. [TorchSpec](https://github.com/lightseekorg/TorchSpec) provides the training loop, streaming those hidden states from live vLLM target inference into concurrent FSDP draft training.
 
-**DSpark** = a block-diffusion backbone of 5 dense layers with non-causal attention, drafting 7 tokens in a single parallel pass, a low-rank sequential **Markov head** supplying the intra-block dependency, and a confidence head for resource-aware scheduling. Mirroring Kimi-K3's own MLA attention means draft and target share one KV layout (a compact 576-element latent per token), so the draft's pages unify with the target's cache — KV offloading and P/D disaggregation work with no separate page format.
+**DSpark** = a block-diffusion backbone (5 dense layers that consume 5 target auxiliary hidden states and draft a block of 7 tokens in a single parallel pass), plus a low-rank sequential **Markov head** (`rank=256`) that supplies the intra-block token dependency the parallel backbone misses, plus a confidence head.
+
+**Why MLA:** the draft mirrors Kimi-K3's own MLA attention, so draft and target share a single KV layout (a compact 576-element latent per token). The draft's pages unify with the target's cache, letting it reuse the same KV management and PD-disaggregated serving paths with no separate page format.
 
 ---
 
 ## Performance
 
-### Peak bs=1 decode: 464 tok/s
-
-Under low-entropy real reasoning workload — Kimi-K3 + DSpark on vLLM can achieve **464 tok/s** using the public `vllm/vllm-openai:kimi-k3` image on 4 × GB300 at `bs=1 & tensor-parallel-size=16`.
+Measured end-to-end against Kimi-K3 at `tensor-parallel-size=8` (2 × GB300), `num_speculative_tokens=7`. **Sampling matches the production serving setting (`temperature=1.0` / `top_p=0.95`), so acceptance reflects real serving conditions.**
 
-### Speculator acceptance
 
-Speculators do best on predictable, low-entropy work like the reasoning workload behind the 464 tok/s above, and worst on open-ended, high-entropy generation. So we measured **14 benchmarks** spanning math, code synthesis, real-world software engineering, multi-turn chat, RAG and QA, multilingual text, creative writing and long-context generation — all with the **Kimi-K3 chat template enabled** and production sampling parameters, with `temperature=0` reported alongside for reproducibility. Acceptance stays strong at long context too, verified on AA-LCR's ~95k-token multi-document prompts. That is the most comprehensive view we can give of how the speculator performs on real-world workloads.
+![Screenshot 2026-07-26 at 8.55.09 PM](https://cdn-uploads.huggingface.co/production/uploads/649921b1c789488720308b62/dQv6SqAHA1x_Oo5t6Q22G.png)
 
-Acceptance length, with 7 speculative tokens:
+Acceptance tracks how predictable the output is: it is strongest on deterministic, structured text such as code, and weaker on high-entropy generation such as creative writing. Confidence-based scheduling — using the draft's confidence head to prune low-confidence drafted tokens before they are sent for verification — is planned future work aimed at lifting the harder categories.
 
-| benchmark | `temperature=0` | `temperature=1.0`, `top_p=0.95` | prompts |
-|---|---|---|---|
-| GSM8K | 5.64 | 5.44 | 1319 |
-| HumanEval | 5.34 | 5.07 | 164 |
-| MBPP | 4.44 | 4.31 | 256 |
-| SPEED-Bench · coding | 4.38 | 4.22 | 80 |
-| SPEED-Bench · multilingual | 4.21 | 4.10 | 80 |
-| SPEED-Bench · RAG | 4.11 | 3.97 | 80 |
-| MATH-500 | 3.82 | 3.77 | 500 |
-| SPEED-Bench · low-entropy, 10k input | 3.72 | 3.66 | 512 |
-| SWE-bench Pro | 3.35 | 3.11 | 128 |
-| AA-LCR · ~95k input | 3.19 | 3.23 | 100 |
-| MT-Bench | 3.14 | 3.06 | 80 |
-| SPEED-Bench · QA | 3.07 | 2.98 | 80 |
-| SPEED-Bench · writing | 2.79 | 2.69 | 80 |
-| AIME 2026 | 2.72 | 2.64 | 30 |
-| **mean** | **3.85** | **3.73** | |
+**Benchmarks:** [MT-Bench](https://huggingface.co/datasets/philschmid/mt-bench) (80 dialogue prompts) and [NVIDIA SPEED-Bench](https://huggingface.co/datasets/nvidia/SPEED-Bench) — its `throughput_16k` low-entropy split (302 prompts at 10k-token input) plus five qualitative categories.
 
-
-**Benchmarks:** [GSM8K](https://huggingface.co/datasets/openai/gsm8k), [MATH-500](https://huggingface.co/datasets/HuggingFaceH4/MATH-500), [AIME 2026](https://huggingface.co/datasets/math-ai/aime26), [HumanEval](https://huggingface.co/datasets/openai/openai_humaneval), [MBPP](https://huggingface.co/datasets/Muennighoff/mbpp), [SWE-bench Pro](https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro), [MT-Bench](https://huggingface.co/datasets/philschmid/mt-bench), [AA-LCR](https://huggingface.co/datasets/ArtificialAnalysis/AA-LCR) — 100 multi-document prompts of 71k–115k tokens — and six splits of [NVIDIA SPEED-Bench](https://huggingface.co/datasets/nvidia/SPEED-Bench): its `throughput_16k` low-entropy split at 10k-token input, plus five qualitative categories.
+vLLM supports **block verification** (`rejection_sample_method=block`) and **probabilistic draft sampling** (`draft_sample_method=probabilistic`) — verifying the drafted block as a unit, and sampling the draft from its own distribution rather than its argmax. Both are enabled in the numbers above and push acceptance higher at `temperature=1.0`.
 
 ---
 
@@ -67,7 +49,7 @@ Acceptance length, with 7 speculative tokens:
 - [`nvidia/OpenCodeInstruct`](https://huggingface.co/datasets/nvidia/OpenCodeInstruct) — single-turn coding
 - A nine-category prompt pool from the NVIDIA **Nemotron** SFT/RL collections and [`CohereLabs/aya_dataset`](https://huggingface.co/datasets/CohereLabs/aya_dataset), spanning chat, code, multilingual, RAG/QA, math, structured output and safety. Evaluation-set prompts are excluded.
 
-**Method:** DSpark with block_size=7, trained on a combined CE + L1 distribution-distillation objective against the target's post-final-norm hidden state, bf16. The draft consumes Kimi-K3 auxiliary hidden states from target layers (2, 23, 47, 71, 89) of 93. Roughly **two epochs** in total, on GB300 nodes.
+**Method:** DSpark with `block_size=7`, `max_seq_length=16384`, trained on a combined CE + L1 distribution-distillation objective against the target's post-final-norm hidden state, bf16. The draft consumes Kimi-K3 auxiliary hidden states from target layers (2, 23, 47, 71, 89) of 93. Roughly **two epochs** in total, on GB300 nodes.
 
 Draft architecture, block size, sequence length, and loss weights are YAML-configurable — see the [TorchSpec repo](https://github.com/lightseekorg/TorchSpec).
 
@@ -86,12 +68,3 @@ Add to your Kimi-K3 `vllm serve` command:
 ```bash
 --speculative-config '{"method": "dspark", "model": "Inferact/Kimi-K3-DSpark", "num_speculative_tokens": 7, "attention_backend": "FLASHINFER_MLA", "draft_sample_method": "probabilistic", "rejection_sample_method": "block"}'
 ```
-
-### Sampling options
-
-vLLM offers two knobs on top of the defaults, both used in the `temperature=1.0` column above:
-
-- **`draft_sample_method`** — `probabilistic` samples the draft from its own distribution instead of taking its argmax. Pair it with a sampling client; use `greedy` when serving at `temperature=0` so the draft matches the client.
-- **`rejection_sample_method`** — `block` verifies the drafted block as a unit rather than token by token. It is a no-op under greedy decoding (at `temperature=0` verification reduces to a deterministic argmax match), so it only applies to the sampling configuration.
-
-The `temperature=0` column was produced with `{"draft_sample_method": "greedy"}` and no `rejection_sample_method`.