Instructions to use DaoCloud/Qwen3.8-27B-DFlash2-Exp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DaoCloud/Qwen3.8-27B-DFlash2-Exp with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DaoCloud/Qwen3.8-27B-DFlash2-Exp", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("DaoCloud/Qwen3.8-27B-DFlash2-Exp", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DaoCloud/Qwen3.8-27B-DFlash2-Exp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DaoCloud/Qwen3.8-27B-DFlash2-Exp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Qwen3.8-27B-DFlash2-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DaoCloud/Qwen3.8-27B-DFlash2-Exp

SGLang

How to use DaoCloud/Qwen3.8-27B-DFlash2-Exp with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DaoCloud/Qwen3.8-27B-DFlash2-Exp" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Qwen3.8-27B-DFlash2-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DaoCloud/Qwen3.8-27B-DFlash2-Exp" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Qwen3.8-27B-DFlash2-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DaoCloud/Qwen3.8-27B-DFlash2-Exp with Docker Model Runner:
```
docker model run hf.co/DaoCloud/Qwen3.8-27B-DFlash2-Exp
```

Qwen3.8-27B-DFlash2-Exp

An experimental DFlash2 drafter for Qwen/Qwen3.8-27B.

This checkpoint uses a 7-query sample_from_anchor=true layout that produces seven draft proposals from seven query positions instead of eight, reducing draft-side query compute.

Results

Setting	Value
Engine	vLLM
Hardware	NVIDIA H200, TP1
Concurrency	1
Thinking sampling	`temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=0.0`, `repetition_penalty=1.0`
Thinking mode	`reasoning_effort=xhigh`, `enable_thinking=true`
Non-thinking sampling	`temperature=0.7`, `top_p=0.8`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`
Non-thinking mode	`enable_thinking=false`
Max new tokens	4,096
Context	native 262,144
Draft tokens	7 per verification step

Dataset	Samples	Thinking (`xhigh`) AL	Non-thinking AL
AA-LCR	100	3.8475	4.3578
AIME26	30	2.9922	5.2214
GSM8K	1,319	3.8272	5.5458
HumanEval	164	3.3457	6.5176
LiveCodeBench	1,055	2.7684	4.5785
LongBench-v2	402	3.4772	3.7079
MATH500	500	3.4836	5.7381
MBPP	257	3.2696	5.4883
MMSpec	600	3.1216	3.3236
MT-Bench	80	2.8316	3.5207
SWE-bench Pro	731	2.6905	3.8953

Accepted length is calculated from the raw server counters:

accepted_length = 1 + accepted_tokens / draft_calls

Architecture

Setting	Value
Draft layers	5
Auxiliary hidden-state IDs	`[6, 20, 34, 48, 62]`
Hidden size	5,120
FFN intermediate	17,408
Attention	32 Q heads / 8 KV heads
Head dimension	128
Attention window	sliding window 2,048
Draft attention	non-causal inside the draft block
RoPE theta	1e7
Norm / activation	RMSNorm eps 1e-6 / SiLU
Block	7 draft tokens, `sample_from_anchor=true`
Dynamic convolution	kernel 2, group 16
Candidate selector	rank 256, top-k 16
Draft vocabulary	248,320
Parameters	1,924,404,480

Training data

A decontaminated, redistributable subset of the training corpus is released as DaoCloud/Qwen3.8-27B-Drafter-SFT.

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Qwen3.8-27B or a compatible quantized version:

vllm serve Qwen/Qwen3.8-27B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/Qwen3.8-27B-DFlash2-Exp",
    "num_speculative_tokens": 7
  }'

This checkpoint uses the 7-query sample_from_anchor=true layout and currently requires vLLM PR #54154.

Checkpoint

The BF16 checkpoint contains 81 tensors and 1,924,404,480 parameters. It is released as a 3,848,817,920-byte (3.584 GiB) safetensors file.

File	SHA256
`model.safetensors`	`9e1aad9323c3ffecc35b1130e222476f722238233263e7ef9f08389bf1ff2a04`

Future work

Adaptive verification: explore confidence-head-driven adaptive draft budgets for high-concurrency serving, including varlen GDN support (vLLM #51869).
MoE drafter: explore sparsely activated drafter architectures for large-scale serving, increasing model capacity without proportionally increasing active compute.

License

Apache-2.0.

Downloads last month: 240

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for DaoCloud/Qwen3.8-27B-DFlash2-Exp

Base model

Qwen/Qwen3.8-27B

Finetuned

(244)

this model