File size: 1,946 Bytes
f6887ab | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | ---
license: other
license_name: minimax-community
license_link: https://huggingface.co/MiniMaxAI/MiniMax-M3/blob/main/LICENSE
base_model: MiniMaxAI/MiniMax-M3
tags:
- fp8
- modelopt
- quantized
---
# MiniMax-M3-FP8 (per-tensor static, experts-quantized)
Built with MiniMax M3.
FP8 W8A8 quantization of [MiniMaxAI/MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3)
produced with NVIDIA TensorRT Model Optimizer (per-tensor static scales,
calibrated on a 512-sample proprietary domain-matched dataset). Scope:
- **Routed MoE experts: FP8** (weights + static input scales) — 97% of parameters.
- **Attention projections: BF16** — vLLM cannot load per-tensor scales on
M3's fused QKV, so attention is kept high-precision by design.
- **KV cache: no checkpoint-side scheme** — use `--kv-cache-dtype` as desired.
## IMPORTANT: serving requires a vLLM fix
vLLM releases to date drop MiniMax-M3's swigluoai activation constants
(`swiglu_alpha=1.702`, `swiglu_beta=1.0`) in the ModelOpt FP8 MoE path, which
makes this checkpoint (and any FP8 M3 checkpoint) produce incoherent output
while loading without error. Fixes: vllm-project/vllm#49473 (ModelOpt path)
and vllm-project/vllm#46845 (compressed-tensors path). Serve only with a build
containing #49473 or with the equivalent patch applied.
vllm serve <this-repo> --tensor-parallel-size 8 --enable-expert-parallel
--trust-remote-code --language-model-only
## Quality
On an internal SWE-bench-based agentic coding evaluation (300 instances,
identical harness and hardware), this checkpoint ties the official
`MiniMaxAI/MiniMax-M3-MXFP8` cumulative pass@1-3 (77 = 77) and resolves more
instances on the first attempt (pass@1 54 vs 51).
This model inherits the MiniMax Community License of the base model,
including its use restrictions and the commercial-use authorization
requirement for large entities.
|