Hugging Face's logo

amd
/
GLM-4.7-MXFP4

Safetensors
glm4_moe
8-bit precision
quark
Model card Files
xet
Community
2

Missing Docker image and MI300X

#2
by twinsen123 - opened
Discussion
twinsen123

Two questions for using this model

  1. The rocm/vllm-private:vllm_dev_base_mxfp4_20260122 image specified model card is not accessible, could you update the model card which VLLM is compatible to this quantization
  2. It is possible to run this model using MI300X?
jiaxwang
 AMD org
  1. In this docker images we use this vllm commit 74c583bc508c2dafb9e95bab3b635884e4a021f3 and modify the following code.

https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/glm4_moe.py
image (1)

  1. We haven't tested it on the mi300, you can give it a try.

Sign up or log in to comment