Instructions to use OpenGVLab/SDLM-32B-D4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenGVLab/SDLM-32B-D4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenGVLab/SDLM-32B-D4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenGVLab/SDLM-32B-D4", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("OpenGVLab/SDLM-32B-D4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenGVLab/SDLM-32B-D4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenGVLab/SDLM-32B-D4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/SDLM-32B-D4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenGVLab/SDLM-32B-D4

SGLang

How to use OpenGVLab/SDLM-32B-D4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenGVLab/SDLM-32B-D4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/SDLM-32B-D4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenGVLab/SDLM-32B-D4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/SDLM-32B-D4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenGVLab/SDLM-32B-D4 with Docker Model Runner:
```
docker model run hf.co/OpenGVLab/SDLM-32B-D4
```

lll2343 commited on Oct 3, 2025

Commit

841f283

verified ·

1 Parent(s): ec105fd

Update README.md

Browse files

Files changed (1) hide show

README.md +2 -30

README.md CHANGED Viewed

@@ -34,7 +34,7 @@ We propose a **S**equential **D**iffusion **L**anguage **M**odel (**SDLM**), to
 SDLM delivers strong performance with significantly faster decoding speed. It operates approximately 2x faster than comparable autoregressive models while matching their accuracy, and achieves up to 5x speedup over other diffusion language models, as evidenced by results on the MATH-500 benchmark.
-![Overall Framework](https://huggingface.co/OpenGVLab/SDLM-32B-D4/resolve/main/assets/framwork_compare.png)
 -   Autoregression: Predicts tokens one by one.
 -   Diffusion: Regenerates all tokens each step.
@@ -191,40 +191,12 @@ Trade-off between performance and speed under different confidence thresholds τ
     *   `attn_implementation`: Attention implementation type. Options include sdpa, eager, or flex_attn. Using Flex Attention requires additional setup. Prefer to use `sdpa` for a quick start.
     *   `causal_attn`: Whether to use causal attention within the window. Currently set to non-causal (`False`).
-    Our training setting is:
-    <p align="center">
-        <img src="https://github.com/OpenGVLab/SDLM/blob/main/assets/hyper-param.png" width="50%"></a>
-    </p>
-    The training loss of our 3B model. loss_pos_`i` refers to the loss at the `i`-th position of each block. The loss at `i=0` is close to the SFT loss of AR's NTP.
-    Here, we display the loss corresponding to each position within the window during the training process. When bs=8, only the first 4 are shown.
-    The correspondence is as follows:
-    bs = 4 (red):
-    | x | m | m | m |
-    | :-- | :-- | :-- | :-- |
-    | loss_pos_1 | loss_pos_2 | loss_pos_3 | loss_pos_4 |
-    bs = 8 (orange):
-    | x | m | m | m | m | m | m | m |
-    | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |
-    | loss_pos_1 | loss_pos_2 | loss_pos_3 | loss_pos_4 | -- | -- | -- | -- |
-    ![](https://github.com/OpenGVLab/SDLM/blob/main/assets/train_log_3b.png)
 ## Evaluation
 Currently, we use [Opencompass](https://github.com/open-compass/opencompass) for evaluation. For more details, please refer to the [evaluation guide](https://github.com/OpenGVLab/SDLM/blob/main/eval/with_opencompass/readme.md).
-## Case
-<p align="center">
-    <img src="https://github.com/OpenGVLab/SDLM/blob/main/assets/case.gif" width="70%"></a>
-</p>
 ## Acknowledge

 SDLM delivers strong performance with significantly faster decoding speed. It operates approximately 2x faster than comparable autoregressive models while matching their accuracy, and achieves up to 5x speedup over other diffusion language models, as evidenced by results on the MATH-500 benchmark.
+![Overall Framework](https://huggingface.co/OpenGVLab/SDLM-32B-D4/resolve/main/assets/three_framework.png)
 -   Autoregression: Predicts tokens one by one.
 -   Diffusion: Regenerates all tokens each step.
     *   `attn_implementation`: Attention implementation type. Options include sdpa, eager, or flex_attn. Using Flex Attention requires additional setup. Prefer to use `sdpa` for a quick start.
     *   `causal_attn`: Whether to use causal attention within the window. Currently set to non-causal (`False`).
+    More details about training please refer to [github](https://github.com/OpenGVLab/SDLM).
 ## Evaluation
 Currently, we use [Opencompass](https://github.com/open-compass/opencompass) for evaluation. For more details, please refer to the [evaluation guide](https://github.com/OpenGVLab/SDLM/blob/main/eval/with_opencompass/readme.md).
 ## Acknowledge