| --- |
| license: other |
| license_name: minimax-m2.7-license |
| license_link: LICENSE |
| base_model: MiniMaxAI/MiniMax-M2.7 |
| tags: |
| - hipfire |
| - moe |
| - minimax |
| - quantized |
| library_name: hipfire |
| --- |
| |
| # hipfire-MiniMax-M2.7 |
|
|
| > **License:** Non-commercial use only per the upstream MiniMax license; |
| > commercial use requires prior written authorization from MiniMax (see the |
| > [LICENSE](LICENSE) file in this repo, copied verbatim from |
| > [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/blob/main/LICENSE)). |
|
|
| [hipfire](https://github.com/Kaden-Schutt/hipfire)-native quantizations of |
| [MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) — a 229B-parameter |
| (≈10B active) MoE: GQA attention with per-layer QK-norm, partial rotate_half |
| RoPE, and a 256-expert top-8 sigmoid+bias router (DeepSeek-V3-style), SwiGLU |
| experts, **no shared expert**. 62 layers, hidden 3072, vocab 200064. |
| |
| These `.mq*` files run with the hipfire inference engine (HIP/ROCm-direct, no |
| Python in the hot path). They are **not** GGUF/safetensors and won't load in |
| llama.cpp / transformers. Attention, router, embedding, and lm_head are kept at |
| **Q8** throughout; only the routed-expert precision varies below, and gate/up |
| AWQ is baked in on every quantized tier. |
|
|
| ## Files |
|
|
| A 5-tier quality/size spectrum. Each routed expert is `gate/up · down`; the |
| **down projection is promoted independently** (see below). |
|
|
| | File | gate/up · down | Size | Notes | |
| |------|----------------|------|-------| |
| | `MiniMax-M2.7.mq4` | MQ4 · MQ4 | 123.6 GB | best quality | |
| | `MiniMax-M2.7.mq3-lloyd` | MQ3-Lloyd+AWQ · **MQ4** | 109.6 GB | high quality | |
| | `MiniMax-M2.7.mq3` | MQ3-Lloyd uniform | 102.6 GB | balanced 3-bit | |
| | `MiniMax-M2.7.mq2-lloyd` | MQ2-Lloyd+AWQ · **MQ4** | 86.2 GB | **fits 128 GB unified (Strix Halo / gfx1151)** | |
| | `MiniMax-M2.7.mq2` | MQ2-Lloyd+AWQ · **MQ3-Lloyd** | 79.2 GB | smallest coherent | |
|
|
| ## Why the down projection is promoted |
|
|
| MiniMax-M2 has **no shared expert**, so routed-expert quantization error has no |
| high-precision path to hide behind, and uniform 2-bit experts collapse into |
| repetition loops. The cause is specific: the **down projection's** input (the |
| SwiGLU intermediate) carries **~588× the activation energy** of the gate/up |
| input, so a low-bit error in `w2` is amplified ~24× more at the block output. It |
| behaves like a precision-critical conv1d — it doesn't degrade gracefully. |
|
|
| The fix (in every sub-mq4 tier here): keep gate/up cheap (2- or 3-bit + AWQ) but |
| **promote only the down projection**. Promoting down to as little as 3-bit |
| restores coherence; 4-bit and 6-bit progressively add richness. Per-expert |
| promotion does **not** help — MiniMax's router is load-balanced (no dominant |
| experts to anchor), so the lever is the *projection*, not the expert. |
|
|
| ## Validation |
|
|
| Forward pass validated against a PyTorch (`transformers` `MiniMaxM2`) reference |
| on a dimension-faithful tiny random-weight oracle (per-layer hidden-state cosine |
| **0.9996** mq4 / **0.9994** mq3-lloyd; attention isolated **0.99990**; routing |
| exact). Real 229B generation is coherent on all five tiers (greedy): *"why is the |
| sky blue"* → correct Rayleigh-scattering answer; *"factorial of n"* → correct |
| Python. Greedy decode can show mild repetition on open-ended prompts — use light |
| sampling + a repeat penalty + the chat template for production. |
|
|
| ## Quantization |
|
|
| Source: the official FP8 (E4M3 + F32 block-[128,128] `weight_scale_inv`) |
| checkpoint, dequantized to F32 then re-quantized. gate/up AWQ uses a shared |
| per-layer activation scale from an unsloth imatrix (math `W·s @ x/s = W·x` is |
| exact). Weights are stored at the listed bit-width and **dequantized to f32 on |
| the fly inside the GEMV kernels** — the math is f32; the bit-width is a |
| storage/bandwidth dial. The MQ3-Lloyd indexed-decode MoE GEMV kernel and the |
| per-projection mixed-dtype dispatch were added to hipfire for this model. |
|
|
| ## Usage (hipfire) |
|
|
| ``` |
| # Chat serving — the chat template is embedded in every tier: |
| HIPFIRE_JINJA_CHAT=1 hipfire serve --model MiniMax-M2.7.mq2-lloyd |
| # or a single raw forward (no chat template): |
| # examples/infer_minimax --model MiniMax-M2.7.mq2-lloyd --prompt "..." |
| ``` |
|
|
| arch_id 10 in the hipfire HFQ header. |
| |
