Upload README.md with huggingface_hub

README.md

@@ -20,55 +20,56 @@ experts, **no shared expert**. 62 layers, hidden 3072, vocab 200064.
 These `.mq*` files run with the hipfire inference engine (HIP/ROCm-direct, no
 Python in the hot path). They are **not** GGUF/safetensors and won't load in
 llama.cpp / transformers. Attention, router, embedding, and lm_head are kept at
-**Q8** throughout; only the routed-expert precision varies below.
+**Q8** throughout; only the routed-expert precision varies below, and gate/up
+AWQ is baked in on every quantized tier.
 
 ## Files
 
-| File | Routed experts (gate/up · down) | Size | Target |
-|------|----------------------------------|------|--------|
-| `MiniMax-M2.7.mq4` | MQ4 · MQ4 | 123.6 GB | best quality |
-| `MiniMax-M2.7.mq3-lloyd` | MQ3-Lloyd · MQ3-Lloyd | 102.6 GB | high quality |
-| `MiniMax-M2.7.mq2-lloyd` | MQ2-Lloyd+AWQ · **MQ4** | 86.2 GB | **smallest coherent — fits 128 GB unified (Strix Halo / gfx1151)** |
+A 5-tier quality/size spectrum. Each routed expert is `gate/up · down`; the
+**down projection is promoted independently** (see below).
 
-(More tiers may be added as the precision sweep completes.)
+| File | gate/up · down | Size | Notes |
+|------|----------------|------|-------|
+| `MiniMax-M2.7.mq4` | MQ4 · MQ4 | 123.6 GB | best quality |
+| `MiniMax-M2.7.mq3-lloyd` | MQ3-Lloyd+AWQ · **MQ4** | 109.6 GB | high quality |
+| `MiniMax-M2.7.mq3` | MQ3-Lloyd uniform | 102.6 GB | balanced 3-bit |
+| `MiniMax-M2.7.mq2-lloyd` | MQ2-Lloyd+AWQ · **MQ4** | 86.2 GB | **fits 128 GB unified (Strix Halo / gfx1151)** |
+| `MiniMax-M2.7.mq2` | MQ2-Lloyd+AWQ · **MQ3-Lloyd** | 79.2 GB | smallest coherent |
 
-## The down-projection is the precision-critical component
+## Why the down projection is promoted
 
 MiniMax-M2 has **no shared expert**, so routed-expert quantization error has no
-high-precision path to hide behind. Pushing experts to a uniform 2-bit makes the
-model collapse into repetition loops on instruction prompts. The cause is
-specific and measurable: the **down projection's** input (the SwiGLU
-intermediate) carries **~588× the activation energy** of the gate/up input, so a
-2-bit error in `w2` is amplified ~24× more at the block output than the same
-error in `w1/w3`. It behaves like the precision-critical conv1d in other
-architectures — it doesn't degrade gracefully, it derails generation.
-
-The fix in `mq2-lloyd` here: keep gate/up at 2-bit (MQ2-Lloyd + gate/up AWQ) but
-promote **only the down projection to MQ4**. That restores coherence (correct
-factual answers, working code, no loops) at 86 GB — smaller than the uniform
-3-bit build. Per-expert promotion does **not** work: MiniMax's router is
-load-balanced (no dominant experts to anchor), so the lever is the projection,
-not the expert.
+high-precision path to hide behind, and uniform 2-bit experts collapse into
+repetition loops. The cause is specific: the **down projection's** input (the
+SwiGLU intermediate) carries **~588× the activation energy** of the gate/up
+input, so a low-bit error in `w2` is amplified ~24× more at the block output. It
+behaves like a precision-critical conv1d — it doesn't degrade gracefully.
+
+The fix (in every sub-mq4 tier here): keep gate/up cheap (2- or 3-bit + AWQ) but
+**promote only the down projection**. Promoting down to as little as 3-bit
+restores coherence; 4-bit and 6-bit progressively add richness. Per-expert
+promotion does **not** help — MiniMax's router is load-balanced (no dominant
+experts to anchor), so the lever is the *projection*, not the expert.
 
 ## Validation
 
 Forward pass validated against a PyTorch (`transformers` `MiniMaxM2`) reference
-on a dimension-faithful tiny random-weight oracle (per-layer hidden-state cosine):
-mq4 **0.9996**, mq3-lloyd **0.9994**; attention isolated **0.99990**; routing
-(sigmoid + bias top-8) exact. Real 229B generation is coherent on all three files
-(greedy): *"why is the sky blue"* → correct Rayleigh-scattering answer;
-*"factorial of n"* → correct Python. (Greedy decode can show mild repetition on
-open-ended prompts; use light sampling + a repeat penalty + the chat template for
-production.)
+on a dimension-faithful tiny random-weight oracle (per-layer hidden-state cosine
+**0.9996** mq4 / **0.9994** mq3-lloyd; attention isolated **0.99990**; routing
+exact). Real 229B generation is coherent on all five tiers (greedy): *"why is the
+sky blue"* → correct Rayleigh-scattering answer; *"factorial of n"* → correct
+Python. Greedy decode can show mild repetition on open-ended prompts — use light
+sampling + a repeat penalty + the chat template for production.
 
 ## Quantization
 
 Source: the official FP8 (E4M3 + F32 block-[128,128] `weight_scale_inv`)
 checkpoint, dequantized to F32 then re-quantized. gate/up AWQ uses a shared
 per-layer activation scale from an unsloth imatrix (math `W·s @ x/s = W·x` is
-exact; redistributes quant difficulty to the salient input channels). The
-MQ3-Lloyd indexed-decode MoE GEMV kernel and the per-projection mixed-dtype
-dispatch were added to hipfire for this model.
+exact). Weights are stored at the listed bit-width and **dequantized to f32 on
+the fly inside the GEMV kernels** — the math is f32; the bit-width is a
+storage/bandwidth dial. The MQ3-Lloyd indexed-decode MoE GEMV kernel and the
+per-projection mixed-dtype dispatch were added to hipfire for this model.
 
 ## Usage (hipfire)