Ar4ikov/MiniMax-H3-transformer-W4A16-RTN

MiniMax-H3 transformer, W4A16 (AutoRound RTN)

The joint video/audio diffusion transformer of MiniMaxAI/MiniMax-H3, weight-only quantized to 4 bits. 66.28 GB → 37.76 GB (0.570 of the original).

This repository holds only the transformer. Every other component of the pipeline (the Qwen3-VL conditioner, the video and audio VAEs, tokenizer, processor, both schedulers) is unchanged and has to come from the original repository.

What is quantized and what is not

Group	Params	Precision
transformer_blocks.*.attn.to_{q,k,v}, attn.to_out.0, ff.net.0.proj, ff.net.2	19.27 B (300 layers)	INT4, group 128, symmetric
transformer_blocks.*.adaln_proj.linear	13.01 B (50 layers)	BF16
proj_in, proj_out, audio_proj_in, audio_proj_out, context_embedder, time_embedder, token_refiner.*, norm_out.linear	~0.7 B	BF16

adaln_proj is deliberately left alone, and it is the reason the file is 37.76 GB and not ~12 GB. It is 13.0 B of the model's 33.1 B parameters, and quantizing it is a bad trade: its input is the timestep embedding, whose row count is num_timesteps * MODALITY_NUM rather than the token count, so it costs almost nothing to compute no matter the precision. Meanwhile the diffusers implementation notes that a rounding applied before its SiLU biases every block's modulation parameters identically at every sampling step, so the error accumulates coherently along the denoising trajectory instead of averaging out.

The keep-list for the remaining layers is the one the diffusers integration uses for its own int8 recipe, and it matches what SGLang keeps in FP32 for this model: patch projections, the timestep MLP, and the final video/audio heads.

Honest limitations

• This is RTN, not calibrated AutoRound. iters=0, disable_opt_rtn=True, so no calibration data was used at all and quality is round-to-nearest, comparable to a GGUF Q4_K_M produced without an importance matrix. A calibrated run needs calibration inputs for a DiT (hidden_states, temb, adaln_indices, rotary_emb), which AutoRound's diffusion driver cannot produce for H3 today: it detects diffusion models by model_index.json and H3 ships modular_model_index.json, and its loader goes through AutoPipelineForText2Image, which has no H3 mapping.
• No generated video has been compared against the BF16 baseline. The BF16 pipeline needs far more accelerator memory than the machine this was produced on has, so there is no reference to diff against. What has been verified is that the weights are physically packed and that diffusers loads the checkpoint back.
• Expect no speedup. A video DiT pushes tens to hundreds of thousands of latent positions through every step, so its GEMMs are compute bound, and W4A16 there is a memory play rather than a throughput one. Speed on this class of model comes from quantizing activations too (FP8/W8A8 on Hopper or Blackwell), not from 4-bit weights.

Loading

import torch
from diffusers import AutoRoundConfig, MiniMaxH3Transformer3DModel, ModularPipeline

transformer = MiniMaxH3Transformer3DModel.from_pretrained(
    "Ar4ikov/MiniMax-H3-transformer-W4A16-RTN",
    quantization_config=AutoRoundConfig(backend="auto"),
    dtype=torch.bfloat16,
)

pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3")
pipe.update_components(transformer=transformer)
pipe.load_components(dtype=torch.bfloat16)

MiniMax-H3 is not in a diffusers release yet, so this needs diffusers from its pull request:

pip install "git+https://github.com/huggingface/diffusers.git@refs/pull/14355/head"
pip install "auto-round>=0.13.0"

For faster CUDA kernels also install gptqmodel>=5.8.0 and pass AutoRoundConfig(backend="marlin").

Recipe

from auto_round import AutoRound

# layer_config maps every Linear outside the 6 targeted leaves of each block
# (and every adaln_proj.linear) to {"bits": 16, "act_bits": 16}
ar = AutoRound(
    model=transformer,              # MiniMaxH3Transformer3DModel, bf16
    scheme="W4A16",
    group_size=128,
    sym=True,
    layer_config=layer_config,
    to_quant_block_names="transformer_blocks",
    low_gpu_mem_usage=True,
    batch_size=1,
    iters=0,
    disable_opt_rtn=True,
)
ar.quantize_and_save(out_dir, format="auto_round", inplace=True)

Produced on 2x RTX 3090: 50 blocks in 16m53s, peak 13.9 GB host RAM and 4.05 GB VRAM, since AutoRound onloads one block at a time.

Note that quantize_and_save raises AttributeError: 'FrozenDict' object has no attribute 'save_pretrained' at the very end on this path: AutoRound's LLM export calls model.config.save_pretrained(...), which a diffusers config does not implement, and its diffusion export path is not reached because H3 is not detected as a diffusion model. The shards are already written when this happens; config.json, quantization_config.json and the diffusers shard names were written afterwards, with the layer lists in extra_config read back from the tensors actually present in the shards.

auto_round 0.14.2, diffusers 0.40.0.dev0 (PR 14355), torch 2.11.0+cu128.

License and modifications

These weights are a modified version of MiniMax-H3 and stay under the MiniMax H3 Community License Agreement, a copy of which is included as LICENSE. NOTICE states which files were modified and how, as the license requires. Anyone receiving these weights receives them under that same agreement.

Two conditions in it are easy to miss and are on you as a user, not on this repository: the license restricts use and distribution to its Applicable Territory, which excludes the EU, the UK, South Korea and the USA; and products built on it must display "Powered by MiniMax H3" and mark generated media with an AI-generation identifier. Read LICENSE rather than taking this paragraph as a summary of it.