--- license: other license_name: minimax-community license_link: https://huggingface.co/MiniMaxAI/MiniMax-M3/blob/main/LICENSE base_model: MiniMaxAI/MiniMax-M3 base_model_relation: quantized pipeline_tag: image-text-to-text tags: - mlx - mlx-vlm - multimodal - vision - minimax_m3_vl - apple-silicon - mixed-precision --- # MiniMax-M3-Alis-MLX-Dynamic **The first MLX quantization of MiniMax-M3 that keeps the full vision-language model.** Sensitivity-graded mixed-precision builds of [MiniMaxAI/MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3) (**427B total** / ~23B active MoE VL, MiniMax Sparse Attention, 1M context) for Apple Silicon, sized for 512 GiB and 256 GiB machines. > Exact parameter count, measured across all 59 bf16 source shards: **427.04B** > (language model 426.18B + vision tower 0.63B + patch-merge 0.19B + projector 0.05B). Existing MLX quants of M3 are text-only extractions — the vision tower, multimodal projector and patch-merge MLP are deleted. These builds keep **all of it**: image and video prompts work end-to-end through `mlx-vlm`. ## Builds (branches) **Default branches load on stock `mlx-vlm` and on oMLX — no fork, no patch.** | branch | routed experts | shared expert | attn/dense | embed / head | vision | size | fits | |---|---|---|---|---|---|---|---| | `main` (**T256**) | 3-bit g64 | 3-bit (packed) | 6-bit | 6b / 8b | bf16 | **192.6 GB** (3.65 bpw) | 256 GiB Macs, no sysctl needed | | `t512` (**T512**) | 6-bit g64 | 6-bit (packed) | 8-bit | 8b / 8b | bf16 | **350.8 GB** (6.57 bpw) | 512 GiB Macs | | `t512ref` (**T512REF**) | 8-bit g64 | 8-bit | 8-bit | 8b / 8b | bf16 | **454.9 GB** (8.52 bpw) | 512 GiB Macs (max quality / reference) | All three load on stock mlx-vlm and oMLX with no patch (`t512ref` keeps the unpacked layout but is uniform 8-bit, so its shared expert is already 8-bit *and* concatenates cleanly). Never quantized, in every build: - **Router gates + `e_score_correction_bias` — fp32** (discrete top-4 expert selection) - **MSA index projections (`index_q_proj`/`index_k_proj`) — bf16** (they pick the top-16 attention blocks; a flipped selection reads different history, so this control path stays exact) - **Vision tower, multimodal projector, patch-merge MLP — bf16** (~1.4 GB total; multimodality is the point) ### Note on the shared expert (compatibility over a marginal lever) M3's always-on shared expert sees **100% of tokens** (a routed expert sees ~3%), so holding it at 8-bit while the routed bank drops to 3/6-bit is a real quality-per-GB lever (+0.4% size). But MLX stores it *packed* into the same 129-wide `SwitchLinear` as the routed experts, and a packed bank can hold **only one bit-width** — so an 8-bit shared expert over a 3/6-bit routed bank needs the *unpacked* layout, which stock mlx-vlm and oMLX cannot load (they unconditionally re-pack and can't concatenate mixed bit-widths — see [discussion #1](https://huggingface.co/avlp12/MiniMax-M3-Alis-MLX-Dynamic/discussions/1)). These builds therefore keep the shared expert **packed at routed bits** so they load everywhere with no fork. If you want the 8-bit-shared variant, rebuild it with [`pack_shared_expert=false`](https://github.com/Blaizzy/mlx-vlm/pull/1544) on the patched mlx-vlm — it is not hosted here. ## Install & run **Stock, no fork:** ```sh pip install mlx-vlm # >= 0.6.5 ``` ```sh # text mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \ --prompt "Explain MoE routing in three sentences." --max-tokens 300 # image mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \ --image photo.jpg --prompt "Describe this image." --max-tokens 300 # video mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \ --video clip.mp4 --prompt "What happens in this clip?" --max-tokens 300 ``` Python: ```python import mlx.core as mx from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template model, processor = load("avlp12/MiniMax-M3-Alis-MLX-Dynamic") # or revision="t512" mx.set_wired_limit(mx.metal.device_info()["max_recommended_working_set_size"]) prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1) out = generate(model, processor, prompt, image=["photo.jpg"], max_tokens=300) print(out.text) ``` Sampling: MiniMax recommends `temperature=1.0, top_p=0.95`. The model thinks in `…` before answering; budget `max_tokens` accordingly. ## Measured quality (vs T512REF 8-bit reference) Same-machine (M3 Ultra 512 GB), deterministic contexts (EN/KO/code/econ prose), KL on the reference's top-256 support; long-context on a 16K document exercising the sparse-attention path (MSA only activates beyond ~2.2K tokens — short evals cannot see it). > These numbers were measured on an earlier **8-bit-shared-expert** prototype. The published > packed builds keep the shared expert at routed bits (T256 → 3-bit, T512 → 6-bit): the T512 > delta is negligible (routed already 6-bit), the T256 delta is bounded (one always-on FFN > among a 3-bit bank). Figures are indicative; the qualitative ranking holds. | metric | T512 | T256 (main) | |---|---|---| | KL vs REF, short (512 tok) | **0.0184 nats** | 0.1243 nats | | top-1 agreement, short | 97.4% | 90.4% | | KL vs REF @16K | **0.0087 nats** | 0.0011 nats | | top-1 agreement @16K | 99.2% | 100.0% | | NIAH @16K (3 depths) | 3/3 | 3/3 | | sparse block-selection overlap vs REF @16K | 54% | 48% | | vision (figure description + OCR of axis labels/annotations) | pass | pass | | decode tok/s (short / 2.4K ctx) | 22.5 / 17.7 | 28.1 / 20.6 | | prefill tok/s @2.4K | 344 | 355 | | peak memory @2.4K | 362 GB | 205 GB | Reference itself: PPL 2.851 on the eval slice (T512 2.900, T256 2.994), NIAH 3/3, decode 20.6 tok/s. Notes, honestly stated: - T256's short-context KL (0.124) is dominated by a code context (0.217); prose contexts run 0.05–0.12. Target was <0.10 — close but a miss. A v2 with measured per-layer expert promotion may close it. - Sparse block-selection overlap vs REF @16K (Jaccard on exact top-16 picks) reads low for both builds (T512 54%, T256 48%) — and nearly identically low, despite T512 being 2.9 bpw richer. Combined with 16K KL of 0.0011–0.0087 and 3/3 NIAH, this says the flips happen among near-tied blocks on repetitive text: selection differs, retrieval doesn't degrade. ## Memory planning KV cache ≈ **134 KiB/token** fp16 (60L × 4 KV heads × 128d K+V, + 57 index-key caches) → ~13.7 GB per 100K tokens. | machine | default GPU limit (~75% RAM) | with `sudo sysctl iogpu.wired_limit_mb=...` | |---|---|---| | 512 GiB | ~412 GB — T512 fits, REF needs bump | up to ~475 GB: REF + ~20 GB ctx, T512 + 1M-token ctx | | 256 GiB | ~206 GB — **T256 fits stock**, ~11 GB ctx (~80K tok) | up to ~240 GB: ~300K+ tok | A 128 GiB build was evaluated and cut: the routed-expert floor alone (2-bit g128 ≈ 117 GB) plus essentials exceeds a 128 GiB Mac's realistic wired ceiling — it cannot load, so we won't ship it. ## Provenance & method - Converted from the bf16 release (869 GB, 59 shards, all files size-verified + sha256 sampled against the Hub) with `mlx_vlm.convert` + a custom per-module quant predicate. - MTP heads: not present in the source release (verified against the weight index). - Playbook: sensitivity-graded mixed precision, 3-lens adversarial design review, ship gates defined before conversion. Same lineage as [Hy3-Alis-MLX-Dynamic](https://huggingface.co/avlp12/Hy3-Alis-MLX-Dynamic) and [GLM-5.2-Alis-MLX-Dynamic-4.5bpw](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-4.5bpw).