I researched the practical GPU specs for GLM-5.2 — a few findings that surprised me more...
Well, I spent some time going through the GLM-5.2 checkpoints, vLLM recipes, quantization options, and GPU layouts to figure out what you actually need to self-host it.
GLM-5.2 is a ~743B parameter MoE with ~39B active per token, using 256 routed experts with 8 active, and it supports a native 1,048,576-token context window.
A few useful things I found:
FP8 is probably the most straightforward production deployment. The checkpoint is roughly 893 GB, and the verified single-node setup is 8× H200/H20 with TP8.
The big gotcha is the 1M context window. Fitting the model weights is one thing; fitting the KV cache is another. If you actually want to push toward the full context length, the GPU memory requirements increase significantly.
NVFP4 is the interesting Blackwell option. The quantized checkpoint is around 465 GB, with vLLM recipes targeting B200/B300-class GPUs.
AMD also has an MXFP4 path, at roughly 446 GB, with a verified 8× MI355X setup.
Another useful feature is MTP speculative decoding. GLM-5.2 supports using multiple speculative tokens, which can improve generation throughput without changing the model itself.
---
Here's the hardware picture I ended up with:
BF16
~1.79 TB model footprint
Generally a multi-node deployment
FP8
~893 GB
8× H200 / H20
8× B200 if you're targeting very long context
NVFP4
~465 GB checkpoint
8× B200 / B300
MXFP4
~446 GB checkpoint
8× MI355X
The main thing to remember is that 743B total / 39B active doesn't mean you only need memory for 39B parameters. All of the expert weights still need to live in GPU memory.
And once you start serving hundreds of thousands of tokens of context, KV cache becomes a major part of the deployment math too.
I collected the GPU layouts, available quants, and actual vLLM serving commands in a longer breakdown here:
Full breakdown:
https://blog.gpus.market/running-glm-5-2-in-production-with-vllm-quants-gpu-pods-and-verified-serving-recipes