Instructions to use ginsongsong/GLM-5.2-FP8-PTPC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ginsongsong/GLM-5.2-FP8-PTPC with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ginsongsong/GLM-5.2-FP8-PTPC")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ginsongsong/GLM-5.2-FP8-PTPC")
model = AutoModelForCausalLM.from_pretrained("ginsongsong/GLM-5.2-FP8-PTPC")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ginsongsong/GLM-5.2-FP8-PTPC with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ginsongsong/GLM-5.2-FP8-PTPC"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ginsongsong/GLM-5.2-FP8-PTPC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ginsongsong/GLM-5.2-FP8-PTPC

SGLang

How to use ginsongsong/GLM-5.2-FP8-PTPC with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ginsongsong/GLM-5.2-FP8-PTPC" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ginsongsong/GLM-5.2-FP8-PTPC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ginsongsong/GLM-5.2-FP8-PTPC" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ginsongsong/GLM-5.2-FP8-PTPC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ginsongsong/GLM-5.2-FP8-PTPC with Docker Model Runner:
```
docker model run hf.co/ginsongsong/GLM-5.2-FP8-PTPC
```

About this checkpoint

This is a PTPC (Per-Token activation + Per-Channel weight) FP8 variant of GLM-5.2, requantized from the block-scaled baseline zai-org/GLM-5.2-FP8 for AMD Instinct MI300X / MI308X / MI325X (gfx942) inference with SGLang + aiter rowwise FP8 GEMM.

Quantization: w8a8_fp8 — per-channel static weight scales (FP8 E4M3) + dynamic per-token activation scales
Derived from: block-scaled zai-org/GLM-5.2-FP8 (weight_block_size=[128,128]) via offline per-channel requantization (weight_block_size removed). Every block-quantized Linear was dequantized to float and re-quantized with one FP8 scale per output channel; the activation path is quantized dynamically per token at runtime by aiter.
Quantized modules: all MoE routed/shared experts, dense MLP, attention projections (q_a_proj, q_b_proj, kv_a_proj_with_mqa, kv_b_proj, o_proj) and DSA indexer projections (indexer.wk, indexer.wq_b). Kept in BF16: layernorms, router gate (+bias), indexer.weights_proj, indexer.k_norm, kv_a_layernorm, q_a_layernorm, the MTP/next-token layer norms (eh_proj, enorm, hnorm, shared_head.norm), lm_head, model.embed_tokens.
Validated (GSM8K, 5-shot, lm-eval-harness, exact_match):

Checkpoint flexible-extract strict-match

zai-org/GLM-5.2-FP8 (block-scale source) 0.9507 ± 0.006 0.9507 ± 0.006

GLM-5.2-FP8-PTPC (this model) 0.9462 ± 0.0062 0.9462 ± 0.0062

The −0.45 pp delta is within the confidence interval → no meaningful regression. Measured on 8× AMD Instinct MI308X (gfx942) with the SGLang PTPC stack.

Checkpoint	flexible-extract	strict-match
`zai-org/GLM-5.2-FP8` (block-scale source)	0.9507 ± 0.006	0.9507 ± 0.006
GLM-5.2-FP8-PTPC (this model)	0.9462 ± 0.0062	0.9462 ± 0.0062

Serving with SGLang

PTPC routes the MoE/dense FP8 GEMMs through aiter's tuned rowwise (per-token × per-channel) FP8 kernels. Use SGLang with --quantization w8a8_fp8 and SGLANG_USE_AITER_FP8_PER_TOKEN=1:

export SGLANG_USE_AITER=1
export SGLANG_USE_AITER_FP8_PER_TOKEN=1
export SGLANG_ADAPTIVE_FP8_DISPATCH=1

python3 -m sglang.launch_server \
    --model-path ginsongsong/GLM-5.2-FP8-PTPC --tp-size 8 \
    --quantization w8a8_fp8 --attention-backend aiter \
    --nsa-prefill-backend tilelang --nsa-decode-backend tilelang \
    --mem-fraction-static 0.85 --served-model-name glm-5.2-fp8 \
    --trust-remote-code --disable-custom-all-reduce --disable-radix-cache \
    --port 30000

On gfx942 the 8 GPUs must be in SPX compute-partition mode for TP=8 (rocm-smi --showcomputepartition). CPX mode wedges RCCL init at TP=8.

GLM-5.2

👋 Join the GLM Discord community. 📖 See the GLM-5.2 blog and GLM-5 Technical report. 📍 Use GLM-5.2 API services on the Z.ai API Platform.

[Paper] [GitHub]

Introduction

GLM-5.2 is Z.ai's latest flagship model for long-horizon tasks, delivering a substantial leap in long-horizon capability over GLM-5.1 on a solid 1M-token context. Highlights:

Solid 1M context that stably sustains long-horizon work.
Advanced coding with flexible effort — multiple thinking-effort levels to balance performance and latency.
Improved architecture — IndexShare reuses the same indexer across every four sparse attention layers (−2.9× per-token FLOPs at 1M context), and an improved MTP layer raises speculative-decoding acceptance length by up to 20%.
Pure open — MIT license.

GlmMoeDsaForCausalLM: 78 layers, 256 routed experts (+1 shared), MLA + DSA sparse attention with IndexShare, 1 MTP (next-token-prediction) layer.

Benchmark (upstream BF16, for reference)

Benchmark	GLM-5.2	GLM-5.1
HLE	40.5	31.0
AIME 2026	99.2	95.3
GPQA-Diamond	91.2	86.2
SWE-bench Pro	62.1	58.4
Terminal-Bench 2.1 (Terminus-2)	81.0	63.5
MCP-Atlas (Public)	76.8	71.8

See the original model card for the full benchmark suite and evaluation settings.

Citation

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}

Acknowledgement

PTPC requantization and AMD ROCm (SGLang + aiter) validation methodology follow the AMD ROCm blog "Further Accelerating Kimi-K2.5 on AMD Instinct MI325X: W4A8 & W8A8 Quantization with AMD Quark" and the GLM-5.1-FP8-PTPC recipe. Original model © Z.ai, released under the MIT license.

Downloads last month: 495

Safetensors

Model size

754B params

Tensor type

F32

BF16

F8_E4M3

Model tree for ginsongsong/GLM-5.2-FP8-PTPC

Base model

zai-org/GLM-5.2

Quantized

(97)

this model

Paper for ginsongsong/GLM-5.2-FP8-PTPC

GLM-5: from Vibe Coding to Agentic Engineering

Paper • 2602.15763 • Published Feb 17 • 196