Instructions to use zai-org/GLM-4.7-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zai-org/GLM-4.7-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zai-org/GLM-4.7-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-4.7-FP8")
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-4.7-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zai-org/GLM-4.7-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zai-org/GLM-4.7-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-4.7-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zai-org/GLM-4.7-FP8

SGLang

How to use zai-org/GLM-4.7-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-4.7-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-4.7-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zai-org/GLM-4.7-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-4.7-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zai-org/GLM-4.7-FP8 with Docker Model Runner:
```
docker model run hf.co/zai-org/GLM-4.7-FP8
```

GLM-4.7-FP8

Commit History

Update EvalEval source links to flat datastore

f478645
verified

EvalEvalBot commited on 8 days ago

Add EvalEval community eval results

40ef115
verified

EvalEvalBot commited on Jun 5

Add EvalEval community eval results

35a45ca
verified

EvalEvalBot commited on Jun 5

Added endif in add_generation_prompt part (#1)

7b3b5f8
verified

ZHANGYUXUAN-zR

aivashov commited on Dec 23, 2025

update

54380d8

zRzRzRzRzRzRzR commited on Dec 23, 2025

readme

e0262b4

zRzRzRzRzRzRzR commited on Dec 23, 2025

fix readme

d782ea0

zRzRzRzRzRzRzR commited on Dec 22, 2025

new readme

470cd15

zRzRzRzRzRzRzR commited on Dec 22, 2025

Update README.md

bb69988
verified

ZHANGYUXUAN-zR commited on Dec 22, 2025

Update README.md

79355a8
verified

ZHANGYUXUAN-zR commited on Dec 22, 2025

Add files using upload-large-folder tool

205a915
verified

ZHANGYUXUAN-zR commited on Dec 22, 2025

Add files using upload-large-folder tool

abb7419
verified

ZHANGYUXUAN-zR commited on Dec 22, 2025

initial commit

f3e6137
verified

ZHANGYUXUAN-zR commited on Dec 22, 2025

Commit History

Update EvalEval source links to flat datastore f478645 verified

Add EvalEval community eval results 40ef115 verified

Add EvalEval community eval results 35a45ca verified

Added endif in add_generation_prompt part (#1) 7b3b5f8 verified

update 54380d8

readme e0262b4

fix readme d782ea0

new readme 470cd15

Update README.md bb69988 verified

Update README.md 79355a8 verified

Add files using upload-large-folder tool 205a915 verified

Add files using upload-large-folder tool abb7419 verified

initial commit f3e6137 verified

Update EvalEval source links to flat datastore

f478645
verified

Add EvalEval community eval results

40ef115
verified

Add EvalEval community eval results

35a45ca
verified

Added endif in add_generation_prompt part (#1)

7b3b5f8
verified

update

54380d8

readme

e0262b4

fix readme

d782ea0

new readme

470cd15

Update README.md

bb69988
verified

Update README.md

79355a8
verified

Add files using upload-large-folder tool

205a915
verified

Add files using upload-large-folder tool

abb7419
verified

initial commit

f3e6137
verified