amd

/

MiniMax-M2.1-MXFP4

like 1

Follow

AMD 2.43k

Hi, this is a model support issue in vLLM for MiniMax-M2.
Would you please add a patch by applying the packed_modules_mapping to MiniMaxM2Model?
Take this as a reference: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/minimax_vl_01.py#L182

Sorry for late update. But the patch you recommended does not work (not sure if I did something wrong)

Patched code

@support_torch_compile
class MiniMaxM2Model(nn.Module):
+    packed_modules_mapping = {
+       "qkv_proj": ["q_proj", "k_proj", "v_proj"],
+        "gate_up_proj": ["gate_proj", "up_proj"],
+    }

Tested docker version

rocm/vllm:rocm7.0.0_vllm0.11.2_20251210
rocm/vllm:v0.14.0_amd_dev

It is possible to have you patch so that you can run this model on MI3XX series? I think it is huge boost for other to onboard more for AMD MI3XX series

Can you please try adding the patched code in class MiniMaxM2ForCausalLM(nn.Module, SupportsLoRA, SupportsPP):instead of class MiniMaxM2Model(nn.Module):?

You may also try adding the following in the MiniMaxM2Model class's __init__ function:

            vllm_config.quant_config.packed_modules_mapping.update({
                "qkv_proj": ["q_proj", "k_proj", "v_proj"],
                "gate_up_proj": ["gate_proj", "up_proj"],
            })

We are able to pass through the mismatch model but hit another issue. The quark seemed access some invalid memory address and cause GPU hang.

(EngineCore_DP0 pid=5992) 
(EngineCore_DP0 pid=5992) [QUARK-INFO]: C++ kernel compilation check start.
(EngineCore_DP0 pid=5992) 
(EngineCore_DP0 pid=5992) [QUARK-INFO]: C++ kernel build directory: /root/.cache/torch_extensions/py312_cpu/kernel_ext. First-time compilation may take a few minutes... Building for architectures PYTORCH_ROCM_ARCH='gfx90a;gfx942;gfx950;gfx1100;gfx1101;gfx1200;gfx1201;gfx1150;gfx1151'.
(EngineCore_DP0 pid=5992) Successfully preprocessed all matching files.
(APIServer pid=5927) DEBUG 03-12 00:00:00 [v1/engine/utils.py:950] Waiting for 1 local, 0 remote core engine proc(s) to start.
(APIServer pid=5927) DEBUG 03-12 00:00:00 [v1/engine/utils.py:950] Waiting for 1 local, 0 remote core engine proc(s) to start.
(APIServer pid=5927) DEBUG 03-12 00:00:00 [v1/engine/utils.py:950] Waiting for 1 local, 0 remote core engine proc(s) to start.
(EngineCore_DP0 pid=5992) 
(EngineCore_DP0 pid=5992) [QUARK-INFO]: C++ kernel compilation is already complete. Ending the C++ kernel compilation check. Total time: 211.4942 seconds
HW Exception by GPU node-1 (Agent handle: 0x30d01ce0) reason :GPU Hang
HW Exception by GPU node-1 (Agent handle: 0x2848ab50) reason :GPU Hang

The testing is running on MI300x1 (192gb) VM. Please see if anything I could try and use this model.

@twinsen123 MI300 does not support native MXFP4 kernel. Do you use emulative or native kernel?

Thanks for the prompt reply.

I got this warning when starting up so I assumed it is running emulative kernel

(EngineCore_DP0 pid=5992) WARNING 03-12 00:00:00 [model_executor/.../quark/quark_moe.py:441] The current mode (supports_mx=False, use_mxfp4_aiter_moe=True, ocp_mx_scheme=OCP_MX_Scheme.w_mxfp4_a_mxfp4) does not support native MXFP4/MXFP6 computation. Simulated weight dequantization and activation QDQ (quantize and dequantize) will be used, with the linear layers computed in high precision.

and

VLLM Environment variable used 
-------------------------------------------
VLLM_LOGGING_LEVEL=DEBUG
VLLM_ROCM_USE_AITER_FP4BMM=0
VLLM_DISABLE_COMPILE_CACHE=1
VLLM_USE_TRITON_FLASH_ATTN=0
VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1
------------------------------------------------------------------

@twinsen123 Yes. It uses emulative QDQ.
We haven’t tested this on MI300, so I’m not sure about the root cause of the issue. If you have multiple GPUs, you could try increasing the TP size, as a single MI300 (192 GB) may not be sufficient for emulative mode. Alternatively, you can launch the vLLM server in eager mode instead of using CUDA Graphs to avoid some potential issues:

vllm serve "$MODEL" \
    --tensor-parallel-size 4 \
    --trust-remote-code \
    --max-model-len 32768 \
   --enforce-eager \
    --port 8899  

No luck. Still memory access fault by GPU node-1 (Agent handle: 0x3801e4a0) on address 0x74cce6200000. Reason: Unknown.

Script for booting up vllm

VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_USE_AITER_FP4BMM=0 \
VLLM_DISABLE_COMPILE_CACHE=1 \
VLLM_ROCM_USE_AITER_MOE=0 \
VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1 \
vllm serve "amd/MiniMax-M2.1-MXFP4" \
    --tensor-parallel-size 1 \
    --trust-remote-code \
    --max-model-len 32768 \
    --enforce-eager \
    --compilation-config '{"cudagraph_mode": "PIECEWISE"}' \
    --port 8899

I think able to run optimized quanitization model in MI300x1 would boost up the AMD GPU adoption as MI300x1 is widely available and price-friendly compared to NVIDIA GPU. See whether there is any roadmap for better support MI300x

1. The GPU hang issue may be due to memory limitations, as in emulation mode the model weights are dequantized to BF16, which is four times the size of the MXFP4 format. The pretrained FP8 model is about 230G: see https://huggingface.co/MiniMaxAI/MiniMax-M2.1/tree/main.
2. For the MI300X roadmap, would you please refer to this link: https://rocm.docs.amd.com/en/latest/

Thanks @twinsen123 for reporting the issue. The "memory access fault" issue can be reproduced, we needs some extra time to fix it.
As a suggestion, please consider using MI350/355 on which the model can be successfully launched.