README.md

  ---
  license: other
  license_name: minimax-community
  license_link: https://huggingface.co/MiniMaxAI/MiniMax-M3/blob/main/LICENSE
  base_model: MiniMaxAI/MiniMax-M3
  tags:
  - fp8
  - modelopt
  - quantized
  ---

  # MiniMax-M3-FP8 (per-tensor static, experts-quantized)

  Built with MiniMax M3.

  FP8 W8A8 quantization of [MiniMaxAI/MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3)
  produced with NVIDIA TensorRT Model Optimizer (per-tensor static scales,
  calibrated on a 512-sample proprietary domain-matched dataset). Scope:

  - **Routed MoE experts: FP8** (weights + static input scales) — 97% of parameters.
  - **Attention projections: BF16** — vLLM cannot load per-tensor scales on
    M3's fused QKV, so attention is kept high-precision by design.
  - **KV cache: no checkpoint-side scheme** — use `--kv-cache-dtype` as desired.

  ## IMPORTANT: serving requires a vLLM fix

  vLLM releases to date drop MiniMax-M3's swigluoai activation constants
  (`swiglu_alpha=1.702`, `swiglu_beta=1.0`) in the ModelOpt FP8 MoE path, which
  makes this checkpoint (and any FP8 M3 checkpoint) produce incoherent output
  while loading without error. Fixes: vllm-project/vllm#49473 (ModelOpt path)
  and vllm-project/vllm#46845 (compressed-tensors path). Serve only with a build
  containing #49473 or with the equivalent patch applied.

  vllm serve <this-repo> --tensor-parallel-size 8 --enable-expert-parallel
    --trust-remote-code --language-model-only

  ## Quality

  On an internal SWE-bench-based agentic coding evaluation (300 instances,
  identical harness and hardware), this checkpoint ties the official
  `MiniMaxAI/MiniMax-M3-MXFP8` cumulative pass@1-3 (77 = 77) and resolves more
  instances on the first attempt (pass@1 54 vs 51).

  This model inherits the MiniMax Community License of the base model,
  including its use restrictions and the commercial-use authorization
  requirement for large entities.