abhishekchohan/minimax-h3-fp8

MiniMax-H3 — FP8 Weight-Only Quantized (torchao)

An FP8 weight-only quantized version of MiniMaxAI/MiniMax-H3 — the 33B video+audio omni diffusion model — repackaged to shrink its ~130 GB BF16 footprint toward consumer GPUs. Quantized with torchao (Float8WeightOnlyConfig, e4m3) at load time. FP8 is near-lossless and, on GPUs with native FP8 tensor cores (Ada / Hopper / Blackwell — RTX 4090/5090, H100), can also be faster than BF16.

INT8 vs FP8: our minimax-h3-int8 repo is the primary consumer target (INT8 version=2 is the pinnable recipe we recommend). This FP8 repo is the near-lossless alternative — and we validated it end-to-end on a consumer GPU with the same streamed group-offload recipe (see the measured table below), so it runs on 24–32 GB cards too. On FP8-native hardware (RTX 4090/5090, H100) it can also be faster.

Sample output (FP8, t2va, on a 32 GB consumer card)

A validated 124-frame (~5 s), 544×960 t2va clip with its generated stereo soundtrack — prompt: "a golden retriever running through tall grass at sunset, cinematic". This run peaked at 16.36 GB VRAM with group offload — well inside a 24 GB budget.

What was quantized (and what was protected)

Weight-only FP8 keeps the sensitive I/O boundary in BF16:

Component	Class	Quantized	Protected (kept BF16)
transformer (t2va + fl2va)	MiniMaxH3Transformer3DModel	block attention/FFN linears	proj_in, audio_proj_in, context_embedder, time_embedder, time_proj, token_refiner, norm_out, proj_out, audio_proj_out
transformer_ref (ref2va)	MiniMaxH3Transformer3DModel	block attention/FFN linears	same as above
text_encoder	Qwen3VLForConditionalGeneration	attention/MLP linears	model.visual, model.language_model.embed_tokens, model.language_model.norm, lm_head
vae / audio_vae	AutoencoderKLMiniMaxH3(Audio)	—	full precision
tokenizer / processor / schedulers	—	—	full precision

All three workflows are supported: t2va / fl2va (transformer/) and ref2va (transformer_ref/).

Footprint

Metric	BF16 (base)	FP8 (this repo, measured)
Transformer (t2va/fl2va)	~66 GB	31.7 GB
Transformer_ref (ref2va)	~66 GB	31.7 GB
Text encoder (Qwen3-VL-32B)	~65 GB	33.1 GB
Total heavy weights	~197 GB	96.5 GB (~2×)
Full repo (incl. VAEs)	~215 GB	106.8 GB

Validated on a consumer GPU (RTX 5090, 32 GB)

Loaded drop-in from this repo (ModularPipeline.from_pretrained(...) + load_components), then ran the streamed group-offload recipe below and generated real clips. Peak VRAM measured with torch.cuda.max_memory_allocated():

Workflow	Call	Peak VRAM	Result
t2va	124 frames, 544×960, 20 steps	16.36 GB	✅ coherent video + stereo audio
ref2va	+ 1 image reference, 124 frames, 544×960, 20 steps	18.21 GB	✅ coherent video + stereo audio

Both are well under 24 GB (fl2va uses the same transformer/ partition as t2va). So despite the earlier pinnability uncertainty, FP8 + streamed group offload works on a 24–32 GB consumer card. The bulk of the weights live in host RAM (~66 GB resident on the t2va path; plan for ≈75 GB of system RAM).

Requirements

• diffusers from main: pip install git+https://github.com/huggingface/diffusers.git
• pip install torchao transformers accelerate
• CUDA is required to use FP8 weights (the float8 tensor subclass is CUDA-only). Native FP8 speedup needs compute capability ≥ 8.9 (RTX 4090/5090, H100); on other CUDA GPUs it's a memory-saver with dequant-on-compute.

Usage

import torch
from diffusers import ModularPipeline
from diffusers.hooks import apply_group_offloading

pipe = ModularPipeline.from_pretrained("abhishekchohan/minimax-h3-fp8")
pipe.load_components(workflow="t2va", dtype=torch.bfloat16)

pipe.transformer.requires_grad_(False)
pipe.text_encoder.requires_grad_(False)

# Group offload is validated for FP8 (16.4 GB peak on a 32 GB card at 544x960).
# If you have the VRAM you can instead keep the transformer on-device.
offload = dict(onload_device=torch.device("cuda"), offload_device=torch.device("cpu"), use_stream=True)
pipe.transformer.enable_group_offload(offload_type="block_level", num_blocks_per_group=1, **offload)
apply_group_offloading(pipe.text_encoder.model, offload_type="leaf_level", **offload)
pipe.vae.to("cuda")
pipe.audio_vae.to("cuda")

out = pipe(
    prompt="a golden retriever running through tall grass at sunset, cinematic",
    num_frames=124,
    height=544,   # validated canvas; 768x1344 also works on 32 GB (multiples of 32)
    width=960,
    generator=torch.Generator().manual_seed(42),
    output=["videos", "audio", "sampling_rate"],
)

⚠️ Loading notes

• The transformer/ and transformer_ref/ weights are stored as pickle .bin (torchao FP8 tensor subclasses can't be written to safetensors). Load with the standard from_pretrained above; diffusers re-materializes the FP8 weights automatically (requires torchao). Your client may prompt for pickle trust.
• The text_encoder/ is stored as safetensors and reloads quantized.
• Experimental diffusers main modular pipeline; API may shift.

Quality

FP8 weight-only is near-lossless. We validated the full quantize → save → reload → generate round-trip on a consumer GPU for both t2va and ref2va (coherent, prompt-aligned video + stereo audio; see the measured table above). A full VBench-style benchmark is out of scope; treat quality as indicative and spot-check your own prompts.

License

Redistributes quantized weights derived from MiniMaxAI/MiniMax-H3, governed by the MiniMax H3 Community License Agreement (license:other). Open-weight use is region-limited to the US / EU / UK / South Korea; other regions apply via MiniMax's platform. See the base model's license terms.

Limitations

• INT8 remains our recommended primary recipe; FP8 is validated to work with group offload on 24–32 GB cards too, and shines on FP8-native GPUs with more VRAM.
• Requires CUDA; FP8 speedups are hardware-dependent (≥ CC 8.9).
• Requires ~75 GB+ host RAM for the streamed-offload consumer path.
• Base-model license and region restrictions (above) still apply.