---
language:
- en
- zh
library_name: transformers
license: other
license_name: glm-5.3
pipeline_tag: text-generation
base_model:
- zai-org/GLM-5.3
base_model_relation: quantized
---
# Model Overview

- **Model Architecture:** GLM-5.3
  - **Input:** Text
  - **Output:** Text
- **Supported Hardware Microarchitecture:** AMD MI325X (gfx942)
- **ROCm:** 7.2
- **PyTorch:** 2.11.0
- **Inference Engine:** [SGLang](https://docs.sglang.ai/)/[vLLM](https://docs.vllm.ai/en/latest/)
- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12.post1)
  - **Weight quantization:** MOE-only (shared experts quantized), OCP MXFP4, Static
  - **Activation quantization:** MOE-only, OCP MXFP4, Dynamic

This model was built with GLM-5.3 model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 quantization.

# Model Quantization

The model was quantized from [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). The weights and activations are quantized to MXFP4.

Weights use OCP MXFP4 with a per-group block size of 32 and E8M0 scales computed from the weight tensors themselves. Activations use the same block format with scales computed at runtime. No calibration dataset is involved.

The upstream checkpoint ships in FP8 (`e4m3`, 128x128 weight blocks, dynamic activation scaling) rather than BF16. Quark's file-to-file pipeline dequantizes those weights through a Triton kernel before re-quantizing them to MXFP4, so this conversion requires a GPU; the CPU path raises `ImportError`.

Attention, dense MLP projections, router gates, `lm_head`, and the MTP layer (layer 78) are excluded.

**Quantization script:**

```python
import json
from quark.torch import LLMTemplate, ModelQuantizer

SRC = "zai-org/GLM-5.3"
DST = "GLM-5.3-MXFP4"

model_type = json.load(open(f"{SRC}/config.json"))["model_type"]

quant_config = LLMTemplate.get(model_type).get_config(
    scheme="mxfp4",
    exclude_layers=[
        "*self_attn*",
        "*mlp.gate",
        "*lm_head",
        "*mlp.gate_proj", "*mlp.up_proj", "*mlp.down_proj",
        "*layers.78.*",          # MTP layer
    ],
)

ModelQuantizer(quant_config).direct_quantize_checkpoint(
    pretrained_model_path=SRC,
    save_path=DST,
    device="cuda",
)
```

The equivalent CLI form:

```bash
cd Quark/examples/torch/language_modeling/llm_ptq/
  python quantize_quark.py \
      --model_dir zai-org/GLM-5.3 \
      --output_dir GLM-5.3-MXFP4 \
      --quant_scheme mxfp4 \
      --exclude_layers "*self_attn*" "*mlp.gate" "*lm_head" \
          "*mlp.gate_proj" "*mlp.up_proj" "*mlp.down_proj" \
          "*layers.78.*" \
      --file2file_quantization
```

# Deployment
### Use with SGLang/vLLM

This model can be deployed efficiently using the [SGLang](https://docs.sglang.ai/) or [vLLM](https://docs.vllm.ai/en/latest/) backends.

## Evaluation

No accuracy evaluation has been run on this checkpoint. The quantization recipe follows [amd/GLM-5.2-MXFP4](https://huggingface.co/amd/GLM-5.2-MXFP4), but that model's reported GSM8K figures were measured on GLM-5.2 and do not describe this checkpoint. Anyone relying on this model for accuracy-sensitive work should measure it first.

To reproduce a GSM8K run with `lm-evaluation-harness`:

```
lm_eval --model sglang \
    --model_args pretrained=<repo_id>,tp_size=4 \
    --tasks gsm8k \
    --batch_size auto
```

# License

This checkpoint is derived from [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) and is distributed under the same license terms as the base model.