Qwen3.8 Flash-Next DFlash2 drafter for Tokspur
Website | Get started | Model guide | Results | GitHub | Main model
An optional draft model that makes Tokspur generate faster with Qwen3.8 Flash-Next. It proposes up to 7 tokens per round and the main model checks every one before it is kept. Better drafts mean more tokens kept per round, so the drafter changes the speed and not the answer.
| Works with | san-digital/Qwen3.8-Flash-Next-Q4_K-Tokspur-GGUF in Tokspur 0.3.1 or later |
| Fine-tuned from | incoai/Qwen3.8-Flash-Next-DFlash2 at revision a22a174 |
| Tuned for | Japanese and other CJK text, <think> reasoning and code |
| Download | 1.21 GB, quantised to 788 MB when loaded |
| Hardware | Apple M5 Max with a 40-core GPU and 128 GB of unified memory |
| Licence | Qwen Community License 1.0 and Apache License 2.0 |
Quick start
With the main model already set up:
tokspur pull --dflash2
tokspur serve --speed-preset --drafter dflash2 --shared-prefix-cache
For Japanese, other CJK text or multilingual prompts alongside <think> reasoning, use the wider draft vocabulary:
tokspur serve --speed-preset --drafter dflash2 --shared-prefix-cache \
--dflash2-vocab-gather cjk64k --dflash2-lookup hybrid --ple-ram-resident
To keep the files in your own folder, download this repository and add --dflash2-dir /path/to/folder. The configuration guide covers the other drafter options.
Speed
Live HTTP throughput in tokens/sec on an M5 Max with a 40-core GPU and 128 GB.
| Workload | Built-in MTP head | Upstream DFlash2 | This drafter | Gain over MTP |
|---|---|---|---|---|
| Japanese cloud MSA, 1,024 thinking then 768 prose tokens, greedy | 51.3 | 60.4 | 73.6 to 76.1 | 1.4 to 1.5ร |
| Same, temperature 0.6 | 45.6 | 57.6 | 64.0 to 66.3 | 1.4 to 1.5ร |
| Japanese contract redline, 2,560 tokens, greedy | 44.1 | 57.4 | 68.2 | 1.5ร |
| Japanese direct review without thinking, 1,536 tokens | 43.0 | 58.8 | 72.4 | 1.7ร |
| Systems Rust, greedy | 60.2 to 66.7 | 135.4 | 148.5 to 158.8 | 2.2 to 2.6ร |
The MTP column is Tokspur 0.2.2 drafting 2 tokens ahead over the full vocabulary. Both DFlash2 columns use the cjk64k vocabulary and 7 drafts per round, and the Rust row also covers the default 32k vocabulary. In the first row, the Japanese prose portion alone ran at 74.5 to 77.5 tokens/sec.
Tokspur's published peak of 165 tokens/sec over a complete Rust request, recorded on 15th November 2026, also used this drafter. See results.
Does it change the output
No. Tokspur verifies every drafted token against the main model. Greedy decoding keeps the main model's own choice, and sampling uses rejection sampling that follows the main model's distribution.
--speed-preset is separate. It switches Metal kernels, which can change output slightly compared with the default server whichever drafter you use.
Training
Fine-tuned from the upstream incoai drafter against hidden states captured from Qwen/Qwen3.8-Flash-Next at revision 34567a4712bc9766c4449e2e98e4468bfa24d915.
- Data. 2,013 self-distilled conversations, about 1.75M generated tokens. They cover Japanese legal, contract and regulatory review, Japanese systems and engineering analysis,
<think>reasoning traces, direct answers without thinking, and English, Rust, Python, C++ and TypeScript code. - Signals. Main model hidden states at layers 5, 15, 25, 35 and 45, together with its token distributions.
- Starting point. The upstream 5-layer non-causal sliding-window
DFlash2DraftModelwith its rank-256, top-16 pairwise candidate selector. - Method. A head loss over a gathered vocabulary of 65,536 rows named
cjk64k, covering ASCII, English and code, Hiragana and Katakana, CJK ideographs and punctuation, and Qwen control tokens such as<think>and<|im_end|>. Rank-256 semantic delta projections were trained on the frozen main model embeddings and folded back into the standard 81-tensor BF16 schema, so they add no cost at run time.
Held-out acceptance
Measured on 6,164 held-out anchors, 2,531 of them CJK. Accepted is the mean number of drafted tokens kept per round at a draft depth of 4 or 7.
| Drafter | Top-k recall | Accepted, depth 4 | Accepted, depth 7 | CJK accepted, depth 4 | CJK accepted, depth 7 |
|---|---|---|---|---|---|
Upstream incoai |
80.10% | 1.83 | 2.27 | 1.46 | 1.87 |
| This drafter | 84.34% | 1.97 (+8.1%) | 2.505 (+10.4%) | 1.57 (+7.5%) | 2.03 (+8.6%) |
Files
| File | Bytes | Contents | SHA-256 |
|---|---|---|---|
model.safetensors |
1,206,139,992 | 81 BF16 drafter and selector tensors | a75fee73e46545c71e40808d3de14f8b45d4237763e768aaf4aa78db914f433c |
config.json |
1,015 | Architecture and hidden state tap configuration | 74b388fb752a0b55515b8cc1d47fdf8211e686a38da1086e6b9b0679eeb460e7 |
tokspur pull --dflash2 checks both digests for you.
Limits
- This is not a standalone model. It reads the main model's hidden states and only works with the Tokspur build linked above.
- The figures above cover Japanese and Rust workloads. Gains on other languages and domains have not been published.
Licence
Parts derived from Qwen/Qwen3.8-Flash-Next are under the Qwen Community License 1.0 (LICENSE). Parts from incoai/Qwen3.8-Flash-Next-DFlash2 are under Apache License 2.0 (LICENSE-APACHE). See NOTICE. Apache 2.0 does not lift the Qwen conditions, so read Sections 1 and 2 of the Qwen Community License before commercial use. The Tokspur runtime is licensed separately under Apache License 2.0.
- Downloads last month
- 253
Model tree for san-digital/Qwen3.8-Flash-Next-DFlash2
Base model
Qwen/Qwen3.8-Flash-Next