File size: 2,996 Bytes
1448231 91dd68e 1448231 3f7f370 1448231 91dd68e 1448231 91dd68e 1448231 1ac8bba 1448231 1ac8bba 1448231 c54bc0f 386bd0e 1448231 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | ---
license: mit
base_model:
- zai-org/GLM-5.2
---
# Model Overview
- **Model Architecture:** GLM-5.2
- **Input:** Text
- **Output:** Text
- **Supported Hardware Microarchitecture:** AMD MI350/MI355
- **ROCm:** 7.0.0
- **PyTorch:** 2.9.0
- **Transformers:** 5.8.1
- **Operating System(s):** Linux
- **Inference Engine:** [SGLang](https://docs.sglang.ai/)/[vLLM](https://docs.vllm.ai/en/latest/)
- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.11)
- **Weight quantization:** MOE-only (shared experts quantized), OCP MXFP4, Static
- **Activation quantization:** MOE-only, OCP MXFP4, Dynamic
This model was built with GLM-5.2 model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 quantization.
# Model Quantization
The model was quantized from [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). The weights and activations are quantized to MXFP4.
**Quantization scripts:**
```bash
cd Quark/examples/torch/language_modeling/llm_ptq/
python quantize_quark.py \
--model_dir zai-org/GLM-5.2 \
--output_dir GLM-5.2-MXFP4 \
--quant_scheme mxfp4 \
--exclude_layers "*self_attn*" "*mlp.gate" "*lm_head" \
"*mlp.gate_proj" "*mlp.up_proj" "*mlp.down_proj" \
"*layers.78.*" \ # Exclude the MTP layer (layer 78)
--file2file_quantization
```
# Deployment
### Use with SGLang/vLLM
This model can be deployed efficiently using the [SGLang](https://docs.sglang.ai/) or [vLLM](https://docs.vllm.ai/en/latest/) backends.
## Evaluation
The model was evaluated on GSM8K benchmarks.
### Accuracy
<table>
<tr>
<td><strong>Benchmark</strong>
</td>
<td><strong>GLM-5.2 </strong>
</td>
<td><strong>GLM-5.2-MXFP4(this model)</strong>
</td>
<td><strong>Recovery</strong>
</td>
</tr>
<tr>
<td>GSM8K (flexible-extract)
</td>
<td>94.09
</td>
<td>93.93
</td>
<td>99.8%
</td>
</tr>
</table>
### Reproduction
The GSM8K results were obtained using the `lm-evaluation-harness` framework, based on the Docker image `lmsysorg/sglang:v0.5.13.post1-rocm700-mi35x`, with SGLang pre-installed inside the image and lm-eval compiled and installed from source.
```
lm_eval --model sglang \
--model_args pretrained=amd/GLM-5.2-MXFP4,tp_size=4 \
--tasks gsm8k \
--batch_size auto
```
The Docker image `rocm/vllm-dev:nightly_main_20260616` with vLLM pre-installed can also be used for reproducing using vLLM backend.
```
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_FP8BMM=0
export VLLM_ROCM_USE_AITER_FP4BMM=0
lm_eval --model vllm \
--model_args 'pretrained=amd/GLM-5.2-MXFP4,tensor_parallel_size=4,dtype=auto,quantization='quark',gpu_memory_utilization=0.9,max_model_len=32768,trust_remote_code=True' \
--tasks gsm8k \
--batch_size auto
```
# License
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
|