MiniMax H3 two-track mixed resolution (MPEG-Diffusion)
Research weights, help wanted. Working now, no DiT fine-tune: the fitted K/V maps (
kv_maps/) and region head v3 (region_head_v3/) give sharp subjects on 22-frame 1344Γ768 video at 1.94Γ the speed of full resolution (RTX 5090). The LoRA is not published: its 500-step run changes the whole picture. Help and results: scrya-com/mpeg-diffusion issue #2.
License and territory. Derived from MiniMax H3 under the MiniMax H3 Community License Agreement (see
NOTICE). Use and redistribution are allowed only outside the European Union, the United Kingdom, the Republic of Korea and the United States of America. Commercial use above US$20M yearly revenue needs MiniMax's written authorization. Do not use these weights to impersonate anyone, and disclose machine-generated output.
Weights for two-track mixed-resolution sampling on the MiniMax H3 DiT. The method is MPEG-Diffusion: Mixed-Resolution Video Generation by Dennis Menn, Tianpei Gu, Guande He, Aryan V S, Yanke Song, Nan Liu, Desai Xie, Xun Huang, Chenfeng Xu and Diana Marculescu (UT Austin and Roblox), arXiv 2611.12069. This repository is an independent port to H3, not their release.
How it works: the unmodified H3 renders the whole frame at half resolution. A region, such as the face, gets full-resolution tokens. Dual-resolution attention lets every query read keys and values at its own resolution. The paper fine-tunes the model so the full-resolution track can start earlier and redraw detail. These weights are that fine-tune, made affordable as a rank-16 LoRA plus the K/V up-maps and two resolution vectors.
| Folder | What |
|---|---|
kv_maps/ |
Fitted f_up K/V maps (rank 128, 50 blocks). They remove the diamond-grid grain of copied keys in the full-resolution region. Use with render --preset quality or the ComfyUI node's maps input. |
region_head_v3/ |
Region head v3 (see its card); the one the quality recipe uses. |
region_head/ |
The self-selected region head: one linear layer on block 25 of the frozen DiT that marks the subject partway through sampling, and so sets the full-resolution region and the Level-of-Token layout. Trained on 2,354 pictures with YOLOE subject labels; held-out mIoU 0.468. It favours the centre of the frame (see its card). |
| LoRA (not published) | Lets the full-resolution track start earlier. First run (2,346 pairs, RTX 5090) stopped at step 500 with held-out loss down 21% (half resolution) and 26% (full resolution). |
- Code, sampler and training pipeline: scrya-com/mpeg-diffusion (
lot/two_track_h3.py,lot/train_tt.py, one command:lot/train_tt_all.sh). - Precomputed caption embeddings for the Pexels photos, which skip the 32B text encoder: johndpope/MiniMax-H3-pexels-people-text.
- Write-up: scrya.com/mpeg_diffusion.
- The earlier Level-of-Token adapters are in johndpope/MiniMax-H3-LoT.
Model tree for johndpope/MiniMax-H3-MPEG-Diffusion
Base model
MiniMaxAI/MiniMax-H3