Qwen3.8 Flash-Next DFlash2 drafter for Tokspur

Website | Get started | Model guide | Results | GitHub | Main model

An optional draft model that makes Tokspur generate faster with Qwen3.8 Flash-Next. It proposes up to 7 tokens per round and the main model checks every one before it is kept. Better drafts mean more tokens kept per round, so the drafter changes the speed and not the answer.

Works with san-digital/Qwen3.8-Flash-Next-Q4_K-Tokspur-GGUF in Tokspur 0.3.1 or later
Fine-tuned from incoai/Qwen3.8-Flash-Next-DFlash2 at revision a22a174
Tuned for Japanese and other CJK text, <think> reasoning and code
Download 1.21 GB, quantised to 788 MB when loaded
Hardware Apple M5 Max with a 40-core GPU and 128 GB of unified memory
Licence Qwen Community License 1.0 and Apache License 2.0

Quick start

With the main model already set up:

tokspur pull --dflash2
tokspur serve --speed-preset --drafter dflash2 --shared-prefix-cache

For Japanese, other CJK text or multilingual prompts alongside <think> reasoning, use the wider draft vocabulary:

tokspur serve --speed-preset --drafter dflash2 --shared-prefix-cache \
  --dflash2-vocab-gather cjk64k --dflash2-lookup hybrid --ple-ram-resident

To keep the files in your own folder, download this repository and add --dflash2-dir /path/to/folder. The configuration guide covers the other drafter options.

Speed

Live HTTP throughput in tokens/sec on an M5 Max with a 40-core GPU and 128 GB.

Workload Built-in MTP head Upstream DFlash2 This drafter Gain over MTP
Japanese cloud MSA, 1,024 thinking then 768 prose tokens, greedy 51.3 60.4 73.6 to 76.1 1.4 to 1.5ร—
Same, temperature 0.6 45.6 57.6 64.0 to 66.3 1.4 to 1.5ร—
Japanese contract redline, 2,560 tokens, greedy 44.1 57.4 68.2 1.5ร—
Japanese direct review without thinking, 1,536 tokens 43.0 58.8 72.4 1.7ร—
Systems Rust, greedy 60.2 to 66.7 135.4 148.5 to 158.8 2.2 to 2.6ร—

The MTP column is Tokspur 0.2.2 drafting 2 tokens ahead over the full vocabulary. Both DFlash2 columns use the cjk64k vocabulary and 7 drafts per round, and the Rust row also covers the default 32k vocabulary. In the first row, the Japanese prose portion alone ran at 74.5 to 77.5 tokens/sec.

Tokspur's published peak of 165 tokens/sec over a complete Rust request, recorded on 15th November 2026, also used this drafter. See results.

Does it change the output

No. Tokspur verifies every drafted token against the main model. Greedy decoding keeps the main model's own choice, and sampling uses rejection sampling that follows the main model's distribution.

--speed-preset is separate. It switches Metal kernels, which can change output slightly compared with the default server whichever drafter you use.

Training

Fine-tuned from the upstream incoai drafter against hidden states captured from Qwen/Qwen3.8-Flash-Next at revision 34567a4712bc9766c4449e2e98e4468bfa24d915.

  • Data. 2,013 self-distilled conversations, about 1.75M generated tokens. They cover Japanese legal, contract and regulatory review, Japanese systems and engineering analysis, <think> reasoning traces, direct answers without thinking, and English, Rust, Python, C++ and TypeScript code.
  • Signals. Main model hidden states at layers 5, 15, 25, 35 and 45, together with its token distributions.
  • Starting point. The upstream 5-layer non-causal sliding-window DFlash2DraftModel with its rank-256, top-16 pairwise candidate selector.
  • Method. A head loss over a gathered vocabulary of 65,536 rows named cjk64k, covering ASCII, English and code, Hiragana and Katakana, CJK ideographs and punctuation, and Qwen control tokens such as <think> and <|im_end|>. Rank-256 semantic delta projections were trained on the frozen main model embeddings and folded back into the standard 81-tensor BF16 schema, so they add no cost at run time.

Held-out acceptance

Measured on 6,164 held-out anchors, 2,531 of them CJK. Accepted is the mean number of drafted tokens kept per round at a draft depth of 4 or 7.

Drafter Top-k recall Accepted, depth 4 Accepted, depth 7 CJK accepted, depth 4 CJK accepted, depth 7
Upstream incoai 80.10% 1.83 2.27 1.46 1.87
This drafter 84.34% 1.97 (+8.1%) 2.505 (+10.4%) 1.57 (+7.5%) 2.03 (+8.6%)

Files

File Bytes Contents SHA-256
model.safetensors 1,206,139,992 81 BF16 drafter and selector tensors a75fee73e46545c71e40808d3de14f8b45d4237763e768aaf4aa78db914f433c
config.json 1,015 Architecture and hidden state tap configuration 74b388fb752a0b55515b8cc1d47fdf8211e686a38da1086e6b9b0679eeb460e7

tokspur pull --dflash2 checks both digests for you.

Limits

  • This is not a standalone model. It reads the main model's hidden states and only works with the Tokspur build linked above.
  • The figures above cover Japanese and Rust workloads. Gains on other languages and domains have not been published.

Licence

Parts derived from Qwen/Qwen3.8-Flash-Next are under the Qwen Community License 1.0 (LICENSE). Parts from incoai/Qwen3.8-Flash-Next-DFlash2 are under Apache License 2.0 (LICENSE-APACHE). See NOTICE. Apache 2.0 does not lift the Qwen conditions, so read Sections 1 and 2 of the Qwen Community License before commercial use. The Tokspur runtime is licensed separately under Apache License 2.0.

Downloads last month
253
Safetensors
Model size
0.6B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for san-digital/Qwen3.8-Flash-Next-DFlash2

Finetuned
(135)
this model