Instructions to use RedHatAI/GLM-5.2-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RedHatAI/GLM-5.2-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RedHatAI/GLM-5.2-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RedHatAI/GLM-5.2-NVFP4-FP8")
model = AutoModelForCausalLM.from_pretrained("RedHatAI/GLM-5.2-NVFP4-FP8", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RedHatAI/GLM-5.2-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RedHatAI/GLM-5.2-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/GLM-5.2-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RedHatAI/GLM-5.2-NVFP4-FP8

SGLang

How to use RedHatAI/GLM-5.2-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RedHatAI/GLM-5.2-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/GLM-5.2-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RedHatAI/GLM-5.2-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/GLM-5.2-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RedHatAI/GLM-5.2-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/RedHatAI/GLM-5.2-NVFP4-FP8
```

GLM-5.2-NVFP4-FP8

Commit History

Update README.md

fe85fc4
verified

dsikka commited on 5 days ago

Update README.md

25859dc
verified

kylesayrs commited on Jul 15

Update README.md

2cefadd
verified

kylesayrs commited on Jul 15

Fix vLLM snippet in `README.md` (#2)

8408569

kylesayrs

alvarobartt HF Staff commited on Jul 13

Upload config.json with huggingface_hub

87766bb
verified

kylesayrs commited on Jul 13

Upload config.json with huggingface_hub

ab41a32
verified

kylesayrs commited on Jul 13

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json

4957064
verified

kylesayrs commited on Jul 10

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors

379ab2f
verified

kylesayrs commited on Jul 10

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json

82a6cba
verified

kylesayrs commited on Jul 10

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors

5edbc76
verified

kylesayrs commited on Jul 10

Upload config.json with huggingface_hub

10bd28b
verified

kylesayrs commited on Jul 10

Upload config.json with huggingface_hub

9926965
verified

kylesayrs commited on Jul 9

Update README.md

cc67f34
verified

kylesayrs commited on Jul 9

Update README.md

20edbd7
verified

kylesayrs commited on Jul 7

Update README.md

411e3c2
verified

kylesayrs commited on Jul 7

Update README.md

f6c4264
verified

kylesayrs commited on Jul 6

Update README.md

98e9596
verified

kylesayrs commited on Jul 6

Add files using upload-large-folder tool

23b34c1
verified

kylesayrs commited on Jul 1

Add files using upload-large-folder tool

75bc60c
verified

kylesayrs commited on Jul 1

Add files using upload-large-folder tool

540e27e
verified

kylesayrs commited on Jun 30

Add files using upload-large-folder tool

5ee42c1
verified

kylesayrs commited on Jun 30

Update README.md

adb8a56
verified

kylesayrs commited on Jun 29

Update README.md

6252298
verified

kylesayrs commited on Jun 29

Update README.md

a162014
verified

kylesayrs commited on Jun 29

Update README.md

e0e68e2
verified

kylesayrs commited on Jun 29

Update README.md

902e457
verified

kylesayrs commited on Jun 29

Create README.md

73b8173
verified

kylesayrs commited on Jun 29

Add files using upload-large-folder tool

36a34ce
verified

kylesayrs commited on Jun 29

initial commit

9b2b23e
verified

kylesayrs commited on Jun 29

Commit History

Update README.md fe85fc4 verified

Update README.md 25859dc verified

Update README.md 2cefadd verified

Fix vLLM snippet in `README.md` (#2) 8408569

Upload config.json with huggingface_hub 87766bb verified

Upload config.json with huggingface_hub ab41a32 verified

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json 4957064 verified

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors 379ab2f verified

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json 82a6cba verified

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors 5edbc76 verified

Upload config.json with huggingface_hub 10bd28b verified

Upload config.json with huggingface_hub 9926965 verified

Update README.md cc67f34 verified

Update README.md 20edbd7 verified

Update README.md 411e3c2 verified

Update README.md f6c4264 verified

Update README.md 98e9596 verified

Add files using upload-large-folder tool 23b34c1 verified

Add files using upload-large-folder tool 75bc60c verified

Add files using upload-large-folder tool 540e27e verified

Add files using upload-large-folder tool 5ee42c1 verified

Update README.md adb8a56 verified

Update README.md 6252298 verified

Update README.md a162014 verified

Update README.md e0e68e2 verified

Update README.md 902e457 verified

Create README.md 73b8173 verified

Add files using upload-large-folder tool 36a34ce verified

initial commit 9b2b23e verified

Update README.md

fe85fc4
verified

Update README.md

25859dc
verified

Update README.md

2cefadd
verified

Fix vLLM snippet in `README.md` (#2)

8408569

Upload config.json with huggingface_hub

87766bb
verified

Upload config.json with huggingface_hub

ab41a32
verified

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json

4957064
verified

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors

379ab2f
verified

Add MTP layers from zai-org/GLM-5.2: model.safetensors.index.json

82a6cba
verified

Add MTP layers from zai-org/GLM-5.2: mtp.safetensors

5edbc76
verified

Upload config.json with huggingface_hub

10bd28b
verified

Upload config.json with huggingface_hub

9926965
verified

Update README.md

cc67f34
verified

Update README.md

20edbd7
verified

Update README.md

411e3c2
verified

Update README.md

f6c4264
verified

Update README.md

98e9596
verified

Add files using upload-large-folder tool

23b34c1
verified

Add files using upload-large-folder tool

75bc60c
verified

Add files using upload-large-folder tool

540e27e
verified

Add files using upload-large-folder tool

5ee42c1
verified

Update README.md

adb8a56
verified

Update README.md

6252298
verified

Update README.md

a162014
verified

Update README.md

e0e68e2
verified

Update README.md

902e457
verified

Create README.md

73b8173
verified

Add files using upload-large-folder tool

36a34ce
verified

initial commit

9b2b23e
verified