Upload README.md with huggingface_hub

README.md

@@ -19,64 +19,63 @@ experts, **no shared expert**. 62 layers, hidden 3072, vocab 200064.
 
 These `.mq*` files run with the hipfire inference engine (HIP/ROCm-direct, no
 Python in the hot path). They are **not** GGUF/safetensors and won't load in
-llama.cpp / transformers.
+llama.cpp / transformers. Attention, router, embedding, and lm_head are kept at
+**Q8** throughout; only the routed-expert precision varies below.
 
 ## Files
 
-| File | Expert format | Size | Target |
-|------|---------------|------|--------|
-| `MiniMax-M2.7.mq4` | MQ4G256 (4-bit FWHT) experts, Q8 attn/router/head | 123.6 GB | best quality |
-| `MiniMax-M2.7.mq3-lloyd` | MQ3G256-Lloyd (3-bit + 8-entry codebook) experts + gate-up AWQ, Q8 attn/router/head | 102.6 GB | **fits 128 GB unified (e.g. Strix Halo / gfx1151)** |
+| File | Routed experts (gate/up · down) | Size | Target |
+|------|----------------------------------|------|--------|
+| `MiniMax-M2.7.mq4` | MQ4 · MQ4 | 123.6 GB | best quality |
+| `MiniMax-M2.7.mq3-lloyd` | MQ3-Lloyd · MQ3-Lloyd | 102.6 GB | high quality |
+| `MiniMax-M2.7.mq2-lloyd` | MQ2-Lloyd+AWQ · **MQ4** | 86.2 GB | **smallest coherent — fits 128 GB unified (Strix Halo / gfx1151)** |
 
-## Validation
-
-The hipfire MiniMax-M2 forward pass was validated against a PyTorch
-(`transformers` built-in `MiniMaxM2`) reference on a dimension-faithful tiny
-random-weight oracle (per-layer post-residual hidden-state cosine):
+(More tiers may be added as the precision sweep completes.)
 
-- **mq4** experts: **0.9996** vs reference
-- **mq3-lloyd** experts: **0.9994**
-- attention isolated **0.99990**; routing (sigmoid + bias top-8) matches exactly.
+## The down-projection is the precision-critical component
 
-Real 229B generation is coherent on both files (greedy): e.g. *"Explain why the
-sky is blue"* → "shorter blue wavelengths of sunlight scatter more than other
-colors … this scattered light reaches our eyes from every part of the sky";
-*"factorial of n"* → correct Python with a docstring. (Greedy decode can show
-mild repetition on open-ended prompts; use light sampling / a repeat penalty and
-the chat template for production.)
+MiniMax-M2 has **no shared expert**, so routed-expert quantization error has no
+high-precision path to hide behind. Pushing experts to a uniform 2-bit makes the
+model collapse into repetition loops on instruction prompts. The cause is
+specific and measurable: the **down projection's** input (the SwiGLU
+intermediate) carries **~588× the activation energy** of the gate/up input, so a
+2-bit error in `w2` is amplified ~24× more at the block output than the same
+error in `w1/w3`. It behaves like the precision-critical conv1d in other
+architectures — it doesn't degrade gracefully, it derails generation.
 
-## Quantization
+The fix in `mq2-lloyd` here: keep gate/up at 2-bit (MQ2-Lloyd + gate/up AWQ) but
+promote **only the down projection to MQ4**. That restores coherence (correct
+factual answers, working code, no loops) at 86 GB — smaller than the uniform
+3-bit build. Per-expert promotion does **not** work: MiniMax's router is
+load-balanced (no dominant experts to anchor), so the lever is the projection,
+not the expert.
 
-Source: the official FP8 (E4M3 + F32 block-[128,128] `weight_scale_inv`)
-checkpoint, dequantized to F32 then re-quantized. Attention projections, router,
-embedding, and lm_head are kept at **Q8**; routed experts are MQ4G256 (mq4) or
-MQ3G256-Lloyd (mq3-lloyd). The mq3-lloyd build bakes in **gate/up AWQ** (shared
-per-layer activation scale from an unsloth imatrix; the math `W·s @ x/s = W·x`
-is exact, redistributing quant difficulty toward the salient input channels).
+## Validation
 
-## Why only two formats
+Forward pass validated against a PyTorch (`transformers` `MiniMaxM2`) reference
+on a dimension-faithful tiny random-weight oracle (per-layer hidden-state cosine):
+mq4 **0.9996**, mq3-lloyd **0.9994**; attention isolated **0.99990**; routing
+(sigmoid + bias top-8) exact. Real 229B generation is coherent on all three files
+(greedy): *"why is the sky blue"* → correct Rayleigh-scattering answer;
+*"factorial of n"* → correct Python. (Greedy decode can show mild repetition on
+open-ended prompts; use light sampling + a repeat penalty + the chat template for
+production.)
 
-MiniMax-M2 has **no shared expert**, so there is no higher-precision path to
-buffer routed-expert quantization error. Empirically:
+## Quantization
 
-- **2-bit experts (mq2 / mq2-lloyd) are not shippable** — they stay grammatical
-  but collapse into repetition loops on instruction prompts (per-block 2-bit MoE
-  error reaches ~80–100% of the block's signal). AWQ alone does not rescue them.
-- **MQ3-Lloyd (3-bit) is the floor for coherent MiniMax** — 3-bit cuts the
-  per-block error to ~40–50% and recovers near-mq4 fluency, at 102.6 GB.
-- Plain (non-Lloyd) `mq3` / `mq2` are omitted because hipfire ships indexed-decode
-  MoE GEMV kernels only for MQ4G256, MQ6G256, MQ2G256-Lloyd, and MQ3G256-Lloyd.
+Source: the official FP8 (E4M3 + F32 block-[128,128] `weight_scale_inv`)
+checkpoint, dequantized to F32 then re-quantized. gate/up AWQ uses a shared
+per-layer activation scale from an unsloth imatrix (math `W·s @ x/s = W·x` is
+exact; redistributes quant difficulty to the salient input channels). The
+MQ3-Lloyd indexed-decode MoE GEMV kernel and the per-projection mixed-dtype
+dispatch were added to hipfire for this model.
 
 ## Usage (hipfire)
 
 ```
-hipfire serve --model MiniMax-M2.7.mq3-lloyd
+hipfire serve --model MiniMax-M2.7.mq2-lloyd
 # or a single forward:
-# examples/infer_minimax --model MiniMax-M2.7.mq3-lloyd --prompt "..."
+# examples/infer_minimax --model MiniMax-M2.7.mq2-lloyd --prompt "..."
 ```
 
-## Notes
-
-- arch_id 10 in the hipfire HFQ header.
-- The MQ3G256-Lloyd indexed-decode MoE GEMV kernel was added in hipfire for this
-  model (first 3-bit-Lloyd MoE decode kernel in the engine).
+arch_id 10 in the hipfire HFQ header.