Instructions to use z-lab/Qwen3-Coder-Next-DFlash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use z-lab/Qwen3-Coder-Next-DFlash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="z-lab/Qwen3-Coder-Next-DFlash", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("z-lab/Qwen3-Coder-Next-DFlash", trust_remote_code=True)
model = AutoModel.from_pretrained("z-lab/Qwen3-Coder-Next-DFlash", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use z-lab/Qwen3-Coder-Next-DFlash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "z-lab/Qwen3-Coder-Next-DFlash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "z-lab/Qwen3-Coder-Next-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/z-lab/Qwen3-Coder-Next-DFlash

SGLang

How to use z-lab/Qwen3-Coder-Next-DFlash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "z-lab/Qwen3-Coder-Next-DFlash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "z-lab/Qwen3-Coder-Next-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "z-lab/Qwen3-Coder-Next-DFlash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "z-lab/Qwen3-Coder-Next-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use z-lab/Qwen3-Coder-Next-DFlash with Docker Model Runner:
```
docker model run hf.co/z-lab/Qwen3-Coder-Next-DFlash
```

jianchen0311 commited on Apr 7

Commit

0ef7624

verified ·

1 Parent(s): 485ed4f

Update README.md

Browse files

Files changed (1) hide show

README.md +19 -9

README.md CHANGED Viewed

@@ -26,18 +26,33 @@ tags:
 ### Installation
 ```bash
 uv pip install "git+https://github.com/sgl-project/sglang.git@refs/pull/20547/head#subdirectory=python"
 ```
 ### Launch Server
-Use `--speculative-num-draft-tokens` to set the block size (8 or **16**).
 ```bash
-export SGLANG_ENABLE_SPEC_V2=1
-export SGLANG_ENABLE_DFLASH_SPEC_V2=1
-export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1
 python -m sglang.launch_server \
     --model-path Qwen/Qwen3-Coder-Next \
@@ -50,7 +65,6 @@ python -m sglang.launch_server \
     --mamba-scheduler-strategy extra_buffer \
     --trust-remote-code
 ```
 > **Tip:** For long-context or agentic workloads, add `--speculative-dflash-draft-window-size WINDOW_SIZE` to enable sliding-window attention for the drafter.
 ### Usage
@@ -68,10 +82,6 @@ response = client.chat.completions.create(
 print(response.choices[0].message.content)
 ```
-### vLLM
-Community-contributed support is available. See PRs [#36847](https://github.com/vllm-project/vllm/pull/36847) and [#36767](https://github.com/vllm-project/vllm/pull/36767) for details.
 ## Acceptance Length
 - Max new tokens: 4096

 ### Installation
+vLLM:
+```bash
+uv pip install vllm
+uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly
+```
+SGLang:
 ```bash
 uv pip install "git+https://github.com/sgl-project/sglang.git@refs/pull/20547/head#subdirectory=python"
 ```
 ### Launch Server
+vLLM:
+```bash
+vllm serve Qwen/Qwen3-Coder-Next \
+  --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3-Coder-Next-DFlash", "num_speculative_tokens": 15}' \
+  --attention-backend flash_attn \
+  --max-num-batched-tokens 32768
+```
+SGLang:
 ```bash
+# Optional: enable schedule overlapping (experimental, may not be stable)
+# export SGLANG_ENABLE_SPEC_V2=1
+# export SGLANG_ENABLE_DFLASH_SPEC_V2=1
+# export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1
 python -m sglang.launch_server \
     --model-path Qwen/Qwen3-Coder-Next \
     --mamba-scheduler-strategy extra_buffer \
     --trust-remote-code
 ```
 > **Tip:** For long-context or agentic workloads, add `--speculative-dflash-draft-window-size WINDOW_SIZE` to enable sliding-window attention for the drafter.
 ### Usage
 print(response.choices[0].message.content)
 ```
 ## Acceptance Length
 - Max new tokens: 4096