Instructions to use incoai/GLM-5.3-Flash-DFlash2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use incoai/GLM-5.3-Flash-DFlash2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="incoai/GLM-5.3-Flash-DFlash2")

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("incoai/GLM-5.3-Flash-DFlash2")
model = AutoModel.from_pretrained("incoai/GLM-5.3-Flash-DFlash2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use incoai/GLM-5.3-Flash-DFlash2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "incoai/GLM-5.3-Flash-DFlash2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "incoai/GLM-5.3-Flash-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/incoai/GLM-5.3-Flash-DFlash2

SGLang

How to use incoai/GLM-5.3-Flash-DFlash2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "incoai/GLM-5.3-Flash-DFlash2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "incoai/GLM-5.3-Flash-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "incoai/GLM-5.3-Flash-DFlash2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "incoai/GLM-5.3-Flash-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use incoai/GLM-5.3-Flash-DFlash2 with Docker Model Runner:
```
docker model run hf.co/incoai/GLM-5.3-Flash-DFlash2
```

Running on 2x DGX Spark (GB10) via SGLang — recipe + 4 day-0 fixes

by randomllama - opened about 13 hours ago

Discussion

randomllama

about 13 hours ago

Got this drafter serving GLM-5.3-Flash (LibertAIDAI NVFP4) on 2x DGX Spark (GB10, sm_121), SGLang TP=2, using the PR #36507 branch stack (incl. the #36708 capture adapter).

Measured (warmed, temp 0, n=5 medians, 800 tok): 27.6 tok/s code decode vs 14.7 no-spec on the same stack = 1.88x (prose 1.41x). Accept len up to 5.6 of the 8-token block during code decode — consistent with the card's expectations.

Four GB10-specific issues had to be fixed to get here (multi-node SGLANG_HOST_IP hang; mamba/KDA state pool needs per_req*(1+D) + ratio~5 with this drafter attached; DSA tilelang smem overflow at the 8-token verify shape; residual=None crash in the capture adapter during graph capture). All documented with patches and probes:

https://github.com/beastllama/GLM-5.3-Flash-DFlash2-SGLang-2x-DGX-Spark

Note for GB10 users: we do not yet claim bit-identical greedy outputs on this stack (1 near-tie token flip observed in a 5-prompt temp-0 comparison; full matrix in progress).

randomllama

about 7 hours ago

Follow-up with a finding your team may want: on our stack (SGLang PR #36507 branch, 2x GB10 sm_121, LibertAIDAI NVFP4 target, tilelang DSA, bf16 KV), DFlash2-on is NOT bit-identical to speculative-off at temperature 0. A 20-prompt matrix capturing both content and reasoning_content: 1/20 exactly identical, 19/20 diverge at near-tie tokens that cascade through long reasoning traces. Output quality is preserved (all correctness gates passed) - this is exactness, not quality. The card's "greedy output matches the target model" does not reproduce here; candidate causes are verify-path numerics on sm_121 or NVFP4 quant interaction - we couldn't isolate which. Full data + method: https://github.com/beastllama/GLM-5.3-Flash-DFlash2-SGLang-2x-DGX-Spark (RESULTS.md, G6 section). Happy to run diagnostics if useful.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment