larryvrh/MiniMax-H3-Turbo-Lora

MiniMax-H3 Turbo LoRA — 4-step audio-video generation (early preview)

A LoRA for MiniMax-H3 that renders joint video + synchronized stereo audio in 4 sampling steps instead of the usual ~20 — roughly a 5× speedup in sampling wall-clock.

⚠️ This is a demo / preview checkpoint, not a finished model. It is an early snapshot from an in-progress run — under-trained, the time-averaged (EMA) variant hasn't matured, and overall quality is nowhere near what a full run will produce. What it does already show is a clear improvement over the base model at 4 steps: sharper detail and cleaner, better-synced audio than the base gives you in the same 4 steps. Treat it as a taste of the direction, not the destination. No comfyui support yet, will add later.

If you get completely broken audio after loading this in ComfyUI, it's almost certainly a scheduler mismatch. With the correct sampler the audio here is a bit rough but coherent (this is an early preview checkpoint); completely broken — blown-out, distorted, or noise-like audio — is the signature of the wrong sampler. MiniMax-H3 runs video and audio on two different flow schedules (shift 12 for video, 3 for audio). The model returns the audio velocity pre-scaled so that a plain single-schedule ("flat") sampler — what ComfyUI's stock samplers do — can step both streams with one schedule. That approximation holds at the usual ~20 steps, but at ultra-low step counts like 4 the flat step massively overshoots the audio schedule on the last steps, and the audio energy explodes. To make audio work at 4 steps you have to replace the flat single-schedule sampler with a dual-clock one that integrates the audio stream on its own shifted schedule (recovering the raw audio velocity by dividing the schedule derivative back out). That is exactly what generate.py in this repo does, and it matches MiniMax-H3's official dual-scheduler behavior.

Two weight files are included:

file	notes
minimax_h3_turbo_4step.safetensors	trained weights — crisper, holds up better on fast motion
minimax_h3_turbo_4step_ema.safetensors	time-averaged weights — smoother, but immature at this checkpoint (EMA hasn't warmed up), so it can look soft

Both are bf16, ~744 MB, apply as a standard low-rank update (W_eff = W + lora_B @ lora_A, alpha = rank so no extra scaling).

Quick start

The base model, VAEs and text encoder come from the official MiniMax-H3 release, and the model definitions live in ComfyUI, so you need a ComfyUI checkout:

# 1. ComfyUI (pinned to the commit these weights were validated against)
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI && git checkout 14b05228cef127ce529bc0c08660770d4af3e9a8
pip install -r requirements.txt && cd ..

# 2. this release
pip install -r requirements.txt        # torch, safetensors, imageio-ffmpeg, ...

# 3. base weights (from Comfy-Org/MiniMax-H3): the bf16 DiT, the int8 Qwen3-VL
#    text encoder, and the video + audio VAEs, into a models/ tree.

# 4. generate
python generate.py \
  --comfyui ./ComfyUI \
  --base   models/diffusion_models/minimax_h3_fl2va_bf16.safetensors \
  --lora   minimax_h3_turbo_4step.safetensors \
  --te     models/text_encoders/qwen3vl_32b_minimax_h3_int8_convrot.safetensors \
  --video-vae models/vae/minimax_h3_video_vae_fp16.safetensors \
  --audio-vae models/vae/minimax_h3_audio_vae_fp32.safetensors \
  --prompt "A corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark." \
  --width 1344 --height 768 --frames 124 --out corgi.mp4

generate.py is a single self-contained file: it loads the base DiT and merges the LoRA, encodes the prompt with Qwen3-VL, runs 4-step sampling on the model's native dual video/audio schedule, decodes both streams and muxes an mp4.

Notes

• Resolution / duration: width and height are multiples of 16 (the canvas is 32-based); the short edge is typically 768. Frame count is at 24 fps and snaps up to the model's 17·k+5 grid (124 ≈ 5 s). The validated range is 124–362 frames (5–15 s).
• VRAM: the base model is large (~33 B). --offload-adaln keeps the biggest timestep-conditioned weights in CPU fp32, saving ~13 GB; an 80 GB GPU runs comfortably, smaller cards can try with offload on.
• Steps: 4 is the design point. --steps 8 is a little cleaner if you have the budget; the schedule is the same either way.
• Audio: the model emits 32 kHz stereo aligned to the video; the two streams ride different flow schedules and generate.py integrates each on its own.