intel-ai/MiniMax-H3-MXFP8-CT-AutoRound

MiniMax-H3 — MXFP8 DiT + Text-Encoder

An MXFP8 (OCP microscaling FP8) weight-quantized build of MiniMaxAI/MiniMax-H3 — the omni-modal (video + native stereo audio) generative DiT — produced with Intel auto-round.

** both diffusion transformers and the Qwen3-VL text encoder are quantized to MXFP8. (The companion repo MiniMax-H3-mxfp8-native quantizes only the two DiTs and keeps the text encoder in BF16 — see "Relationship to the other variants".) The large AdaLN branch of each DiT is deliberately kept in BF16.

TL;DR

• Each DiT ~62 GB (bf16) → ~43.65 GB (≈0.71×, −29 %); AdaLN (≈13 B params/DiT) stays BF16.
• Text encoder (Qwen3-VL, 64 decoder layers) ~63 GB (bf16) → ~34 GB (≈0.54×, −46 %): the model.language_model.layers linears are MXFP8 (448 × F8_E4M3 + 448 × U8 scale); the vision tower, embeddings and norms stay BF16.
• Serves end-to-end on 2×H200 (TP2) and produces video with synchronized audio. H200 runs the FP8 weights through a Marlin dequant kernel (correct, memory-saving, not faster).

---

Showcase — official T2VA example

Reproduction of the official MiniMax-H3 Text-to-Audio-Video (T2VA) showcase (assets/t2va.mp4), generated on our own vLLM-Omni stack with identical settings (1344×768, 24 fps, 32 steps, 10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task t2va) so the only variable is the quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt.

bf16 (reference)	MXFP8 DiT+TE (this model)
assets/t2va_official_bf16.mp4	assets/t2va_official_mxfp8_full.mp4

Both clips: 1344×768, 24 fps, 243 frames (10.12 s), video + stereo audio, matching the official t2va.mp4 geometry. The MXFP8 clip is faithful to the prompt (starship bridge; silver-haired female captain close-up) — the quantized text encoder is genuinely active.

Quantitative quality (dense93 VBench) is in the Quality section below: imaging_quality 0.6284 (this model, DiT+TE) vs 0.6306 (bf16), i.e. −0.35%.

---

What was quantized

H3 has two DiTs sharing one architecture, plus one shared text encoder:

Component	Role	Quantized?
transformer (FL2VA / T2VA DiT, 33.12 B)	video+audio denoiser	✅ MXFP8 (from BF16)
Ref2VA DiT (33.12 B)	reference-conditioned denoiser	✅ MXFP8 (from BF16)
text_encoder (Qwen3-VL)	text / vision encoding	✅ MXFP8 (language-model layers)
video_vae / audio_vae	video / audio decode	❌ unchanged
adaln_proj.linear (×52/DiT)	AdaLN modulation (~13 B)	❌ kept BF16 on purpose

Per DiT — quantized Linears (6 per block × 52 = 312): attn.{to_q,to_k,to_v,to_out.0} + ff.net.0.proj (SwiGLU gate+up) + ff.net.2 (down). Per DiT: 312 × F8_E4M3 weights + 312 × U8 E8M0 block scales + 314 × BF16 (AdaLN, norms, embedders, biases). AdaLN stays BF16 because vLLM-Omni precomputes and caches its modulation — pure inference never runs it through a matmul, so quantizing it would cost accuracy for zero benefit.

Text encoder — quantized Linears (7 per layer × 64 = 448): every model.language_model.layers.N.{self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj}. 448 × F8_E4M3 weights (≈29.06 GiB) + 448 × U8 block scales (≈0.91 GiB) + 610 × BF16 (≈4.01 GiB: vision tower, embeddings, final norm, lm_head). The vision encoder is not quantized.

Quant format (per component config.json → quantization_config): quant_method=auto-round, data_type=mx_fp, bits=8, group_size=32, sym=True; activations act_data_type=mx_fp, act_bits=8, act_dynamic=True, act_group_size=32; DiT block_name_to_quantize="token_refiner.blocks,blocks", TE block_name_to_quantize="model.language_model.layers", packing_format="auto_round:llm_compressor"; autoround_version=0.17.0.

---

Size / compression

Component (per instance)	bf16	MXFP8	ratio
DiT (transformer, ×2)	~62 GiB	43.65 GiB	0.71× (−29 %)
text encoder (Qwen3-VL)	~63 GiB	~34 GiB	0.54× (−46 %)

Per-DiT breakdown (safetensors header scan): 312 × F8_E4M3 = 18.66 GiB, 312 × U8 = 0.58 GiB, 314 × BF16 (AdaLN + norms + embedders) = 24.34 GiB, 12 × F32 = 0.06 GiB → 43.65 GiB. Because the source is already BF16 and AdaLN (55 % of bytes) stays BF16, the per-DiT ratio is a modest ~0.71×; the text encoder, with no cached-modulation carve-out, compresses further (0.54×).

On-disk total ≈ 142 GiB for this repo (both partitions). The two pipeline folders share one 34 GiB quantized text encoder via hardlinks, so the TE is stored once rather than duplicated

---

Directory layout

Native, serve-ready layout (what vLLM-Omni's combined mode loads). All entries are real files (materialized via hardlinks; find -type l is empty):

MiniMax-H3-mxfp8-full-native/
├── model_index.json          # pipeline manifest
├── README.md                 # this card
├── assets/                   # showcase mp4s
├── transformer/config.json   # ~1 KB shim: carries DiT quantization_config for auto-detect
├── FL2VA/                     # self-contained T2VA/FL2VA pipeline
│   ├── transformer/           # 11 shards, MXFP8 DiT (+ .weight_scale)
│   ├── text_encoder/          #  7 shards, MXFP8 Qwen3-VL (+ quantization_config)
│   ├── video_vae/  audio_vae/ tokenizer/  processor/  model_index.json
└── Ref2VA/                    # same structure; text_encoder shares inodes with FL2VA

Each partition's text_encoder is the quantized Qwen3-VL (its config.json carries the quantization_config with data_type=mx_fp), not the BF16 encoder shipped by the base model.

---

---

Serving (vLLM-Omni, 2×H200 / TP2)

CUDA_VISIBLE_DEVICES=2,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve intel-ai/MiniMax-H3-MXFP8-CT-AutoRound --omni \
  --host 0.0.0.0 --port 8092 --trust-remote-code \
  --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \
  --enforce-eager --diffusion-attention-backend CUDNN_ATTN \
  --init-timeout 1800 --stage-init-timeout 1200
# no --task-type ⇒ combined (loads FL2VA + Ref2VA; required for T2VA)

• TP2 is mandatory: create_weights allocates the BF16 skeleton first (two DiTs ≈ 176 GB), so a single 141 GB H200 OOMs; --enable-cpu-offload is incompatible (Marlin repack is CUDA-only).

Required vLLM-Omni patch (text encoder)

Serving the quantized text encoder needs a small, un-upstreamed patch to vllm_omni/diffusion/models/minimax_h3/encoder.py (4 fixes):

1. Component-level quant routing — read the serialized TE config.json → quantization_config (data_type=mx_fp) and build a text-encoder quant config scoped to text_encoder.text_model.layers (the pipeline otherwise passes the DiT config, so the TE loads unquantized).
2. RowParallel scale-aware loader — shard the grouped weight_scale (in/32) on the same dim as the weight, so scales stay aligned under TP.
3. Suffix-agnostic fused mapping — fuse q/k/v → qkv_proj and gate/up → gate_up_proj for both .weight and .weight_scale.
4. Post-load repack — run process_weights_after_loading on the TE's quantized Linears after device placement (the pipeline skips the generic post_load_weights, and MarlinMxfp8 repack needs CUDA).

Without the patch: ~700 weight_scale has no target parameter warnings, the TE falls back to an unquantized/zeroed path, and the model ignores the prompt.

(The two DiTs additionally need the serving fixes: a top-level transformer/config.json quant anchor, grouped-qkv / fc1 scale reordering in load_weights, and TP2 without cpu-offload.)

---

Hardware note (important)

H200 (Hopper) has no native MXFP8 tensor core; vLLM-Omni runs these weights through a Marlin weight-only dequant GEMM (documented as MXFP8 W8A16 GEMM via Marlin): it dequantizes the MXFP8 weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies against unquantized bf16 activations. It is functionally correct and memory-saving, but not faster than bf16 on Hopper. The checkpoints' act_data_type=mx_fp / act_bits=8 is quantization-recipe metadata that is not exercised on H200 (activations stay bf16). True W8A8 (FP8 activations on a native microscaling tensor core) requires Blackwell/B300. Dequant does not change stored values, so accuracy matches a native kernel.

---

Quality — VBench (dense93, H200, 2026-10-14, n=93)

Both models generate the same 93 dense-prompt videos (512×288, 16 steps, 24 fps, 4 s, seeds 1000+i) on the same vLLM-Omni code path, then are scored with VBench imaging_quality (MUSIQ-SPAQ, no-reference per-frame).

① imaging_quality (0–1, higher = better):

Model	imaging_quality	/100	vs bf16
bf16 (native, DiT+TE bf16)	0.6306	63.06	—
MXFP8 DiT-only (-native)	0.6313	63.13	+0.12%
MXFP8 DiT+TE (this model)	0.6284	62.84	−0.35%

Quantizing the text encoder as well keeps no-reference image quality essentially on par with bf16 (Δ = −0.0022, −0.35%); the ±0.3 scatter is diffusion sampling noise, not a quality collapse.

② Reference metrics vs bf16 (same prompt + same seed ⟹ frame-aligned; PSNR↑ / SSIM↑ / LPIPS↓):

Pair	PSNR (dB)	SSIM	LPIPS
MXFP8 DiT+TE vs bf16	≈ 20.05¹	0.652	0.254

¹ Mean over 92 of 93 clips. One clip ("Van Gogh's selfie") has a pixel-identical first frame → a per-frame PSNR of inf that poisons that video's dB-average; the remaining frames diverge normally, so it is excluded from the PSNR mean (SSIM/LPIPS are unaffected). These are not per-pixel-identical because a ~2–3% MXFP8 weight perturbation is amplified by the 16-step denoiser into a different but equally plausible sample (same scene/soundscape, pixels slightly shifted) — divergence grows with scene complexity/motion, consistent with the DiT-only ablation.

---

Limitations

• Not faster than bf16 on Hopper; needs 2 GPUs to serve.
• Serving the quantized text encoder requires the un-upstreamed encoder.py patch above.
• VAEs and AdaLN kept BF16; TE vision tower kept BF16.
• Native/diffusers dual packaging means serving needs the conversion + the fixes above.

License & attribution

Inherits the base model's license: MiniMax-H3 Community License Agreement (license: other, see LICENSE). This is a derivative (quantized) checkpoint of MiniMaxAI/MiniMax-H3; all model credit belongs to MiniMax. Quantization by Intel auto-round (zero-shot RTN, MXFP8) with PR #2481. Please also comply with the upstream MiniMax-H3 usage terms.