Instructions to use zai-org/GLM-5-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zai-org/GLM-5-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zai-org/GLM-5-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5-FP8")
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use zai-org/GLM-5-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zai-org/GLM-5-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zai-org/GLM-5-FP8

SGLang

How to use zai-org/GLM-5-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zai-org/GLM-5-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zai-org/GLM-5-FP8 with Docker Model Runner:
```
docker model run hf.co/zai-org/GLM-5-FP8
```

GLM-5-FP8

Commit History

Update README.md

fb0d065
verified

UnicornChan commited on Feb 12

update

949e09d

zRzRzRzRzRzRzR commited on Feb 11

line

3f790a1

zRzRzRzRzRzRzR commited on Feb 11

sglang update

69f9dbb

zRzRzRzRzRzRzR commited on Feb 11

work

5e400dd

zRzRzRzRzRzRzR commited on Feb 11

work

f40945c

zRzRzRzRzRzRzR commited on Feb 11

init3

9fadc95

zRzRzRzRzRzRzR commited on Feb 11

Add files using upload-large-folder tool

be01624
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

cd05fa9
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

7429144
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

f713e45
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

6d71d3b
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

6f12c3a
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

5c88522
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

76fb160
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

94aeb1e
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

0fb49fe
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

ea92814
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

edb3d50
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

3d2a1e0
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

fe67182
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

ec5f115
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

c58f114
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

d57f561
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

a28bc56
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

404b679
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

083348a
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

cf2d1a8
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

7e3aa1c
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

2659378
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

ddf89d9
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

eee4fbe
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

b65f187
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

1b7a826
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

8ebfc38
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

bbda59b
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

2ff73da
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

9df30e2
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

a21875a
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

31723fe
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

2546d52
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

8d8ba89
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

f393c14
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

4ebafef
verified

ZHANGYUXUAN-zR commited on Feb 11

Add files using upload-large-folder tool

0ae1a5b
verified

ZHANGYUXUAN-zR commited on Feb 11

initial commit

0a0b52c
verified

ZHANGYUXUAN-zR commited on Feb 11

Commit History

Update README.md fb0d065 verified

update 949e09d

line 3f790a1

sglang update 69f9dbb

work 5e400dd

work f40945c

init3 9fadc95

Add files using upload-large-folder tool be01624 verified

Add files using upload-large-folder tool cd05fa9 verified

Add files using upload-large-folder tool 7429144 verified

Add files using upload-large-folder tool f713e45 verified

Add files using upload-large-folder tool 6d71d3b verified

Add files using upload-large-folder tool 6f12c3a verified

Add files using upload-large-folder tool 5c88522 verified

Add files using upload-large-folder tool 76fb160 verified

Add files using upload-large-folder tool 94aeb1e verified

Add files using upload-large-folder tool 0fb49fe verified

Add files using upload-large-folder tool ea92814 verified

Add files using upload-large-folder tool edb3d50 verified

Add files using upload-large-folder tool 3d2a1e0 verified

Add files using upload-large-folder tool fe67182 verified

Add files using upload-large-folder tool ec5f115 verified

Add files using upload-large-folder tool c58f114 verified

Add files using upload-large-folder tool d57f561 verified

Add files using upload-large-folder tool a28bc56 verified

Add files using upload-large-folder tool 404b679 verified

Add files using upload-large-folder tool 083348a verified

Add files using upload-large-folder tool cf2d1a8 verified

Add files using upload-large-folder tool 7e3aa1c verified

Add files using upload-large-folder tool 2659378 verified

Add files using upload-large-folder tool ddf89d9 verified

Add files using upload-large-folder tool eee4fbe verified

Add files using upload-large-folder tool b65f187 verified

Add files using upload-large-folder tool 1b7a826 verified

Add files using upload-large-folder tool 8ebfc38 verified

Add files using upload-large-folder tool bbda59b verified

Add files using upload-large-folder tool 2ff73da verified

Add files using upload-large-folder tool 9df30e2 verified

Add files using upload-large-folder tool a21875a verified

Add files using upload-large-folder tool 31723fe verified

Add files using upload-large-folder tool 2546d52 verified

Add files using upload-large-folder tool 8d8ba89 verified

Add files using upload-large-folder tool f393c14 verified

Add files using upload-large-folder tool 4ebafef verified

Add files using upload-large-folder tool 0ae1a5b verified

initial commit 0a0b52c verified

Update README.md

fb0d065
verified

update

949e09d

line

3f790a1

sglang update

69f9dbb

work

5e400dd

work

f40945c

init3

9fadc95

Add files using upload-large-folder tool

be01624
verified

Add files using upload-large-folder tool

cd05fa9
verified

Add files using upload-large-folder tool

7429144
verified

Add files using upload-large-folder tool

f713e45
verified

Add files using upload-large-folder tool

6d71d3b
verified

Add files using upload-large-folder tool

6f12c3a
verified

Add files using upload-large-folder tool

5c88522
verified

Add files using upload-large-folder tool

76fb160
verified

Add files using upload-large-folder tool

94aeb1e
verified

Add files using upload-large-folder tool

0fb49fe
verified

Add files using upload-large-folder tool

ea92814
verified

Add files using upload-large-folder tool

edb3d50
verified

Add files using upload-large-folder tool

3d2a1e0
verified

Add files using upload-large-folder tool

fe67182
verified

Add files using upload-large-folder tool

ec5f115
verified

Add files using upload-large-folder tool

c58f114
verified

Add files using upload-large-folder tool

d57f561
verified

Add files using upload-large-folder tool

a28bc56
verified

Add files using upload-large-folder tool

404b679
verified

Add files using upload-large-folder tool

083348a
verified

Add files using upload-large-folder tool

cf2d1a8
verified

Add files using upload-large-folder tool

7e3aa1c
verified

Add files using upload-large-folder tool

2659378
verified

Add files using upload-large-folder tool

ddf89d9
verified

Add files using upload-large-folder tool

eee4fbe
verified

Add files using upload-large-folder tool

b65f187
verified

Add files using upload-large-folder tool

1b7a826
verified

Add files using upload-large-folder tool

8ebfc38
verified

Add files using upload-large-folder tool

bbda59b
verified

Add files using upload-large-folder tool

2ff73da
verified

Add files using upload-large-folder tool

9df30e2
verified

Add files using upload-large-folder tool

a21875a
verified

Add files using upload-large-folder tool

31723fe
verified

Add files using upload-large-folder tool

2546d52
verified

Add files using upload-large-folder tool

8d8ba89
verified

Add files using upload-large-folder tool

f393c14
verified

Add files using upload-large-folder tool

4ebafef
verified

Add files using upload-large-folder tool

0ae1a5b
verified

initial commit

0a0b52c
verified