Duplicate from larryvrh/MiniMax-H3-Turbo-Lora

Co-authored-by: larryvrh <larryvrh@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,152 @@
+---
+license: apache-2.0
+base_model: Comfy-Org/MiniMax-H3
+tags:
+  - text-to-video
+  - text-to-audio
+  - audio-video
+  - lora
+  - minimax-h3
+  - comfyui
+pipeline_tag: text-to-video
+---
+
+# MiniMax-H3 Turbo LoRA — few-step audio-video generation
+
+A LoRA for [MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3) that renders
+joint **video + synchronized stereo audio** in as few as **4 sampling steps**
+instead of the usual ~20 — a ~5× sampling speedup — and keeps getting better as
+you add steps.
+
+## Which checkpoint — `v4` (step 600) or `v1` (850)?
+
+For **most** work, use **`minimax_h3_turbo_v4_step600_ema.safetensors`**. It's the
+strongest checkpoint we've released: much better static and small-motion shots,
+markedly better micro-detail (faces, fingers, fine texture), and the
+over-sharpening / plastic look of the earlier `v1` (~850) line is **fully
+resolved**.
+
+v4 introduced a **static-frame enhancement** — a big win for static and
+small-motion content. The one trade-off shows up **only at 4 steps with large,
+fast motion**, where v4 can produce **motion-smear / trailing ghosting** (we're
+actively fixing this). Two things address it:
+
+- **Use 6–8 steps.** This **largely removes the smear** and is where v4 looks its
+  best. v4 also tolerates higher step counts better than v1, which tends to
+  over-sharpen at high steps + strength 1.0.
+- For the specific case of **4 steps *and* heavy motion**, the older **`v1` ~850**
+  checkpoint can still be the friendlier pick.
+
+```
+Using 6–8 steps?        ── yes ──►  v4-600  (recommended)
+   │ no (4 steps)
+   ▼
+Heavy / fast motion?    ── no  ──►  v4-600  (recommended)
+   │ yes
+   ▼
+                                    v1-850  (friendlier at 4-step heavy motion)
+```
+
+Still a preview — training continues; the two areas still being improved are
+**audio** and **behaviour under fast, intense motion**.
+
+## Steps and strength — read this
+
+- **4 steps is the recommended *minimum*; 4–8 is the useful range.** 6–8 steps
+  look noticeably better than 4, so add steps if you can afford them. Past **8
+  steps** it stops helping and can start to introduce **over-sharp artifacts** —
+  there's no benefit to going higher, so stay in **4–8**.
+- **Keep strength at `1.0`.** It's tuned for 1.0 and holds up well across the 4–8
+  step range. Only reach for the strength dial if a *specific* clip misbehaves —
+  then **blurry ghosting / smear → nudge up** (`~1.05–1.2`), **over-sharp grain →
+  nudge down** (`~0.8–0.95`).
+- Keep the scheduler on `simple`.
+
+## Use it in ComfyUI (recommended)
+
+Custom nodes: **[Larryvrh/ComfyUI-MiniMax-H3-Turbo](https://github.com/Larryvrh/ComfyUI-MiniMax-H3-Turbo)**
+— or search **"MiniMax-H3 Turbo"** in ComfyUI-Manager. (Keep the node updated; it
+evolves alongside these weights.)
+
+1. Install the nodes (Manager, or `git clone` into `ComfyUI/custom_nodes`) and put
+   a `.safetensors` from this repo into `ComfyUI/models/loras/`. You also need the
+   base MiniMax-H3 model, VAEs and text encoder — see the
+   [MiniMax-H3 tutorial](https://docs.comfy.org/tutorials/video/minimax/minimax-h3).
+2. Start from the official MiniMax-H3 workflow (t2v or i2v) and make two changes:
+   - insert **MiniMax-H3 Turbo LoRA** between the model loader and the sampler;
+   - feed `SamplerCustomAdvanced` from **MiniMax-H3 Turbo Sampler**, and set the
+     scheduler to `simple` at **≥ 4 steps**.
+
+Everything else stays as in the official graph, so both text-to-video and
+image-to-video work. A ready-made t2v workflow ships in the
+[node repo](https://github.com/Larryvrh/ComfyUI-MiniMax-H3-Turbo/tree/main/example_workflows)
+(and here as `minimax_h3_t2v_turbo.json`) — drag it in.
+
+- **Base model**: any MiniMax-H3 base — full (`bf16`, `int8_convrot`) **and the
+  pruned/curve variants** (`pruned_int8`, `pruned_fp8`). The node auto-detects a
+  pruned base and re-injects the time-conditioning at run time, so **one LoRA file
+  covers every base**.
+- **`low_vram`** switch: **off** applies the LoRA at run time (sharpest,
+  recommended); **on** merges it into the weights for the lowest peak VRAM (a bit
+  softer on quantized bases). Turn it on only if you run out of memory.
+- The custom sampler **auto-adapts to your ComfyUI version**: MiniMax-H3 runs
+  video and audio on two different flow schedules; recent ComfyUI handles that
+  natively (`ModelSamplingAV`) and older ComfyUI doesn't — the Turbo Sampler
+  detects which and does the right thing either way, so nothing to change when you
+  update ComfyUI.
+
+## Weights
+
+All bf16, ~744 MB, applied as a plain low-rank update
+(`W_eff = W + lora_B @ lora_A`, alpha = rank, so no extra scaling). **Prefer the
+EMA files**; the non-EMA ones are for comparison.
+
+| file | notes |
+|---|---|
+| **`minimax_h3_turbo_v4_step600_ema.safetensors`** | **recommended — current best.** Strong static/small-motion, good micro-detail, no over-sharpening. |
+| `minimax_h3_turbo_v4_step600.safetensors` | v4-600 non-EMA (comparison). |
+| `minimax_h3_turbo_v4_step150_ema.safetensors` | earlier v4 checkpoint. |
+| `minimax_h3_turbo_4step_ema_ckpt850.safetensors` | `v1` line (~850) — over-sharpened / plastic in general, but the friendlier pick for **4-step heavy motion** (see above). |
+| `minimax_h3_turbo_4step_ema_ckpt500.safetensors` | older `v1` (~500), softer. |
+| `minimax_h3_turbo_4step_ema.safetensors` | initial release (~200). |
+
+*Naming:* `v4` is the current training recipe and `stepN` is the training step.
+Older files carry the previous `4step_ckptN` naming, where `4step` referred to the
+sampler-step count.
+
+## Standalone (no ComfyUI graph)
+
+`generate.py` is a single self-contained file — it loads the base DiT + a LoRA,
+encodes the prompt, runs the few-step dual-schedule sampler, decodes and muxes an
+mp4. It still needs a ComfyUI checkout for the H3 model / VAE / text-encoder
+definitions:
+
+```bash
+git clone https://github.com/comfyanonymous/ComfyUI
+cd ComfyUI && pip install -r requirements.txt && cd ..
+pip install -r requirements.txt          # this repo: torch, safetensors, imageio-ffmpeg
+
+# base weights from Comfy-Org/MiniMax-H3 into a models/ tree, then:
+python generate.py \
+  --comfyui ./ComfyUI \
+  --base   models/diffusion_models/minimax_h3_fl2va_bf16.safetensors \
+  --lora   minimax_h3_turbo_v4_step600_ema.safetensors \
+  --te     models/text_encoders/qwen3vl_32b_minimax_h3_int8_convrot.safetensors \
+  --video-vae models/vae/minimax_h3_video_vae_fp16.safetensors \
+  --audio-vae models/vae/minimax_h3_audio_vae_fp32.safetensors \
+  --prompt "A corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark." \
+  --width 1344 --height 768 --frames 124 --steps 6 --out corgi.mp4
+```
+
+## Notes
+
+- **Resolution / duration**: width and height are multiples of 32 (short edge
+  typically 768). Frame count is at 24 fps and snaps to the model's 17·k+5 grid
+  (124 ≈ 5 s). Validated range ~124–362 frames (~5–15 s).
+- **VRAM**: the base model is large (~33 B); an 80 GB GPU is comfortable at the
+  largest resolutions. The ComfyUI node streams the base and adds the `low_vram`
+  switch, so it runs on much smaller GPUs. In the standalone script,
+  `--offload-adaln` trades ~13 GB of VRAM for CPU RAM.
+- **Audio**: 32 kHz stereo, aligned to the video; the two streams ride different
+  flow schedules and are integrated each on its own clock. (Audio is one of the
+  two areas still being improved — see the top.)

experimental_step_149.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e6684c0c041c70e00c86d2da7d19b32386e78f7a5287a76ace842974ef49bc9
+size 10918430752

experimental_step_490.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e95764286503eaf4998da4c8b489447c5c2f90da5425380023dbae87fa7b1cf2
+size 10918430752

experimental_step_729.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4523eccc68fe474c76494f19844c37e2d02859d078ecae39a086bcb943824ee3
+size 10918440992

experimental_step_850.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c07df0e2c4df7121096da3ddb1209c4e07b70e24c94f313a4cf0c2d39f22234
+size 10918440992

experimental_step_922.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7026879187f3e29a1c0107df74d1f1d6e7e9dae802f6d73febaada4553698d53
+size 10918440992

experimental_v2_step_298.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02091565e94ac7618ccf1ef5c2dafe83e485e6d7717b2ef8028e5891924c32ac
+size 7798814752

experimental_v3_step_300.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afb009bf49588ee5aa9250d1c64f2630934bec193d2d2fc136f93b41c278ac25
+size 10918430752

experimental_v4_step_150.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78487982233383dd91e8cae158f6d64ac68580835733a4262e8bfa97b4fabdca
+size 10918430752

experimental_v4_step_600.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5717f204f819505795de519467e20dd23948b5da3ba97732795094fb566a6aa2
+size 10918430752

experimental_v5_step_600.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a57e51bce769d8d3934c0e90926e18f84fa915dd08f9981864d8bafceb4a3551
+size 10918430752

generate.py

@@ -0,0 +1,517 @@
+"""MiniMax-H3 Turbo LoRA — 4-step text-to-audio-video generation.
+
+A lightweight LoRA that lets MiniMax-H3 render joint video + stereo audio in
+**4 sampling steps** instead of the usual ~20, at a fraction of the wall-clock
+cost. This single file is a self-contained generator: it loads the base H3 DiT
+plus this LoRA, encodes the prompt with the Qwen3-VL text encoder, runs the
+model's native dual-schedule sampler for 4 steps, decodes both streams and muxes
+a playable mp4.
+
+The audio stream runs on its own shifted flow schedule (video shift 12, audio
+shift 3); each stream is integrated on its own clock, which is the schedule
+semantics MiniMax-H3 was designed around. That is the only non-obvious part of
+sampling — everything else is a plain Euler flow sampler.
+
+Dependencies (see requirements.txt), plus a ComfyUI checkout for the H3 model /
+VAE / text-encoder module definitions:
+
+  git clone https://github.com/comfyanonymous/ComfyUI
+  cd ComfyUI && git checkout 14b05228cef127ce529bc0c08660770d4af3e9a8
+
+Base weights come from the official MiniMax-H3 release
+(Comfy-Org/MiniMax-H3 on the Hugging Face Hub): the bf16 DiT, the int8 Qwen3-VL
+text encoder, and the video + audio VAEs.
+
+Usage:
+  python generate.py \
+    --comfyui /path/to/ComfyUI \
+    --base   models/diffusion_models/minimax_h3_fl2va_bf16.safetensors \
+    --lora   minimax_h3_turbo_4step.safetensors \
+    --te     models/text_encoders/qwen3vl_32b_minimax_h3_int8_convrot.safetensors \
+    --video-vae models/vae/minimax_h3_video_vae_fp16.safetensors \
+    --audio-vae models/vae/minimax_h3_audio_vae_fp32.safetensors \
+    --prompt "A corgi in a tiny chef hat flipping a pancake, sizzling sounds." \
+    --width 1344 --height 768 --frames 124 --out corgi.mp4
+
+`minimax_h3_turbo_4step.safetensors` is the trained LoRA; the accompanying
+`minimax_h3_turbo_4step_ema.safetensors` is a time-averaged variant — try both,
+the trained one tends to be crisper on fast motion, the averaged one smoother.
+"""
+
+import argparse
+import math
+import os
+import subprocess
+import sys
+import time
+import wave
+
+import torch
+import torch.nn.functional as F
+
+
+def log(msg):
+    print(f"[{time.strftime('%H:%M:%S')}] {msg}", flush=True)
+
+
+# ======================================================================
+# Flow schedule (video shift 12 / audio shift 3, closed-form dual mapping)
+# ======================================================================
+SHIFT_VIDEO = 12.0
+SHIFT_AUDIO = 3.0
+
+
+def shift_sigma(u, shift):
+    return shift * u / (1.0 + (shift - 1.0) * u)
+
+
+def time_shift_sigma(sigma, from_shift, to_shift):
+    base = sigma / (from_shift + sigma * (1.0 - from_shift))
+    return to_shift * base / (1.0 + (to_shift - 1.0) * base)
+
+
+def time_shift_slope(sigma, from_shift, to_shift):
+    base = sigma / (from_shift + sigma * (1.0 - from_shift))
+    return (to_shift * (1.0 + (from_shift - 1.0) * base) ** 2) / (
+        from_shift * (1.0 + (to_shift - 1.0) * base) ** 2)
+
+
+def timesteps(n, shift=SHIFT_VIDEO):
+    """n-step video sigma grid: ts[0]=1 (pure noise) > ... > ts[n]=0."""
+    return [shift_sigma(1.0 - i / n, shift) for i in range(n + 1)]
+
+
+def audio_sigma(sigma_v):
+    return time_shift_sigma(sigma_v, SHIFT_VIDEO, SHIFT_AUDIO)
+
+
+def audio_slope(sigma_v):
+    return time_shift_slope(sigma_v, SHIFT_VIDEO, SHIFT_AUDIO)
+
+
+@torch.no_grad()
+def sample(vfn, xv, xa, ts):
+    """4-step Euler on the joint flow. The model returns the audio velocity
+    already scaled by d(sigma_a)/d(sigma_v), so video steps on its own sigma
+    delta while audio steps on its own schedule's delta (recovering the raw
+    audio velocity by dividing out the slope). This dual-clock stepping is the
+    schedule MiniMax-H3 expects; a single flat step on the video clock would
+    over/under-shoot the audio stream badly at 4 steps.
+    """
+    for i in range(len(ts) - 1):
+        ov, oa = vfn(xv, xa, ts[i])
+        hv = ts[i + 1] - ts[i]
+        sl = audio_slope(max(ts[i], 1e-6))
+        ha = audio_sigma(ts[i + 1]) - audio_sigma(ts[i])
+        xv = xv + hv * ov
+        xa = xa + ha * (oa / sl)
+    return xv, xa
+
+
+# ======================================================================
+# Functional forward (out-of-place, mirrors the reference module math)
+# ======================================================================
+def _rms(x, weight, eps):
+    return F.rms_norm(x, (x.shape[-1],), weight, eps)
+
+
+def _attn(attn, x, rope_cos, rope_sin):
+    s = x.shape[0]
+    heads, hd = attn.heads, attn.head_dim
+    q, k, v = attn.qkv_proj(x).split(heads * hd, dim=-1)
+    q = _rms(q.view(s, heads, hd), attn.q_norm.weight, attn.q_norm.eps)
+    k = _rms(k.view(s, heads, hd), attn.k_norm.weight, attn.k_norm.eps)
+    v = v.view(s, heads, hd)
+    if rope_cos is not None:
+        c, si = rope_cos[:, None, :], rope_sin[:, None, :]
+
+        def rot(t):
+            t96 = t[..., :96].float()
+            x1, x2 = t96[..., :48], t96[..., 48:]
+            return torch.cat([(x1 * c - x2 * si).to(t.dtype),
+                              (x1 * si + x2 * c).to(t.dtype),
+                              t[..., 96:]], dim=-1)
+
+        q, k = rot(q), rot(k)
+    q, k, v = (t.transpose(0, 1).unsqueeze(0) for t in (q, k, v))
+    out = F.scaled_dot_product_attention(q, k, v)
+    return attn.out_proj(out.squeeze(0).transpose(0, 1).reshape(s, heads * hd))
+
+
+def _mlp(mlp, x):
+    x1, x2 = mlp.fc1(x).chunk(2, dim=-1)
+    return mlp.fc2(F.silu(x1) * x2)
+
+
+def _refiner(refiner, x):
+    for blk in refiner.blocks:
+        x = x + _attn(blk.attn, _rms(x, blk.norm1.weight, blk.norm1.eps),
+                      None, None)
+        x = x + _mlp(blk.mlp, _rms(x, blk.norm2.weight, blk.norm2.eps))
+    return _rms(x, refiner.final_norm.weight, refiner.final_norm.eps)
+
+
+def _apply_mod(h, shift, scale, segments):
+    parts = []
+    for a, b, row in segments:
+        parts.append(h[a:b] * (1.0 + scale[row].to(h.dtype)) + shift[row].to(h.dtype))
+    return torch.cat(parts)
+
+
+def _apply_gate(x, gate, other, segments):
+    parts = []
+    for a, b, row in segments:
+        parts.append(x[a:b] + other[a:b] * gate[row].to(x.dtype))
+    return torch.cat(parts)
+
+
+def _block(blk, h, mods, segments, rope_cos, rope_sin):
+    sh_msa, sc_msa, g_msa, sh_mlp, sc_mlp, g_mlp = mods.unbind(dim=1)
+    hn = _apply_mod(_rms(h, blk.norm1.weight, blk.norm1.eps), sh_msa, sc_msa, segments)
+    h = _apply_gate(h, g_msa, _attn(blk.attn, hn, rope_cos, rope_sin), segments)
+    hn = _apply_mod(_rms(h, blk.norm2.weight, blk.norm2.eps), sh_mlp, sc_mlp, segments)
+    return _apply_gate(h, g_mlp, _mlp(blk.mlp, hn), segments)
+
+
+class LoRALinear(torch.nn.Module):
+    """Applies the low-rank update at run time in activation space:
+    y = base(x) + B(A(x)). Folding it into the (bf16) base weight instead would
+    round most of the update away when it is small relative to the weight, so we
+    keep it as a separate matmul — same as how the update is meant to act."""
+
+    def __init__(self, base, a, b):
+        super().__init__()
+        self.base = base
+        self.a, self.b = a, b  # [rank, in], [out, rank]; alpha == rank -> scale 1
+
+    def forward(self, x):
+        return self.base(x) + F.linear(F.linear(x, self.a), self.b)
+
+
+# ======================================================================
+# Model load + LoRA (applied at run time, not merged)
+# ======================================================================
+def load_model(comfyui, base_path, lora_path, device, offload_adaln):
+    import comfy.ldm.minimax.model as h3ref
+    import comfy.ops
+    import comfy.utils
+    from safetensors.torch import load_file
+
+    log(f"loading base DiT: {base_path}")
+    sd = comfy.utils.load_torch_file(base_path)
+    model = h3ref.MiniMaxH3Model(dtype=torch.bfloat16, device="cpu",
+                                 operations=comfy.ops.disable_weight_init)
+    missing, unexpected = model.load_state_dict(sd, strict=True, assign=True)
+    assert not missing and not unexpected, (missing[:3], unexpected[:3])
+    model.requires_grad_(False)
+    model.eval()
+
+    for i, blk in enumerate(model.blocks):
+        blk.to(device)
+    for mod in (model.token_refiner, model.final_layer, model.condition_proj,
+                model.video_patch_proj, model.audio_patch_proj,
+                model.time_embedder, model.rope):
+        mod.to(device)
+
+    log(f"applying LoRA: {lora_path}")
+    lora = load_file(lora_path)
+    names = sorted({k.rsplit(".lora_", 1)[0] for k in lora})
+    # adaLN projections are read weight-first (bypassing their module), so their
+    # LoRA can't ride a wrapper — stash it and add the delta where adaLN is built.
+    model._adaln_lora = {}      # block index -> (a, b)
+    model._final_adaln_lora = None
+    n_wrap = 0
+    for name in names:
+        a = lora[name + ".lora_A.weight"].to(device, torch.bfloat16)
+        b = lora[name + ".lora_B.weight"].to(device, torch.bfloat16)
+        if name.endswith("adaln_proj.linear"):
+            if name.startswith("final_layer"):
+                model._final_adaln_lora = (a, b)
+            else:
+                model._adaln_lora[int(name.split(".")[1])] = (a, b)
+        else:
+            parent = model.get_submodule(name.rsplit(".", 1)[0])
+            setattr(parent, name.rsplit(".", 1)[1],
+                    LoRALinear(model.get_submodule(name), a, b))
+            n_wrap += 1
+    log(f"LoRA: {n_wrap} wrapped + {len(model._adaln_lora)} adaLN "
+        f"+ {1 if model._final_adaln_lora else 0} final")
+
+    if offload_adaln:
+        # The per-layer adaLN projection is huge (2688 -> 96768) but depends only
+        # on the timestep, of which there are a handful per denoise. Keep it in
+        # CPU fp32 to save ~13 GB of VRAM; the matmul is cheap at 4 steps.
+        for blk in model.blocks:
+            lin = blk.adaln_proj.linear
+            lin.weight.data = lin.weight.data.float().cpu()
+            lin.bias.data = lin.bias.data.float().cpu()
+    return model, h3ref
+
+
+VISUAL_COND_T = 0.999
+
+
+def timestep_rows(model, sigma_v):
+    sigma_v = float(max(sigma_v, 1e-6))
+    t_v = 1.0 - sigma_v
+    t_a = 1.0 - time_shift_sigma(sigma_v, model.sigma_shift_video,
+                                 model.sigma_shift_audio)
+    seg_t = {"text": t_v, "video": t_v, "audio": t_a}
+    unique_t = sorted({t_v, t_a})
+    return seg_t, unique_t, {t: i for i, t in enumerate(unique_t)}
+
+
+def adaln_mods(model, unique_t, device, offload, cache):
+    key = tuple(round(t, 9) for t in unique_t)
+    if key in cache:
+        return cache[key]
+    ts = torch.tensor(unique_t, dtype=torch.float32, device=device)
+    with torch.no_grad():
+        temb = model.time_embedder(ts).float()                 # [M, 2688] GPU
+        si = F.silu(temb)
+        si_base = si.cpu() if offload else si.to(torch.bfloat16)
+        outs = torch.stack([F.linear(si_base, b.adaln_proj.linear.weight,
+                                     b.adaln_proj.linear.bias)
+                            for b in model.blocks])             # [50, M, 96768]
+        mods = outs.to(device, torch.bfloat16)
+        if getattr(model, "_adaln_lora", None):
+            # run-time low-rank delta, on GPU (base built in CPU fp32 under offload)
+            si_g = si.to(torch.bfloat16)
+            for idx, (a, b) in model._adaln_lora.items():
+                mods[idx] = mods[idx] + F.linear(F.linear(si_g, a), b)
+    M, H = len(unique_t), model.hidden_size
+    mods = mods.view(len(model.blocks), M, 3, 6, H).reshape(
+        len(model.blocks), M * 3, 6, H)
+    temb_bf = temb.to(torch.bfloat16)
+    cache[key] = (mods, temb_bf)
+    return mods, temb_bf
+
+
+class Prepared:
+    """Static packed-sequence structure for one (text_len, shape) signature."""
+
+    def __init__(self, model, h3ref, text_len, video_shape, audio_t, tags,
+                 device):
+        _, _, lt, lh, lw = video_shape
+        self.video_shape = tuple(video_shape)
+        self.lat_pad = ((lh + 1) // 2 * 2, (lw + 1) // 2 * 2)
+        self.layout = h3ref.PackedLayout(text_len, lt, *self.lat_pad, audio_t)
+        pos = self.layout.position_ids.to(torch.float32).to(device)
+        inv = model.rope.inv_freq.to(device)
+        ang = (pos.unsqueeze(-1) * inv.view(1, 1, -1)).flatten(1)
+        self.rope_cos, self.rope_sin = torch.cos(ang), torch.sin(ang)
+
+        segs = []
+        for a, b, kind in self.layout.segments:
+            if kind == "text" and tags is not None:
+                tg = tags.view(-1).tolist()
+                run = 0
+                for i in range(1, b - a + 1):
+                    if i == b - a or tg[i] != tg[run]:
+                        segs.append((a + run, a + i, int(tg[run]), kind))
+                        run = i
+            else:
+                tag = {"text": 1, "video": 0, "audio": 2}[kind]
+                segs.append((a, b, tag, kind))
+        self.seg_template = segs
+        (self.video_seg,) = [(a, b) for a, b, k in self.layout.segments if k == "video"]
+        (self.audio_seg,) = [(a, b) for a, b, k in self.layout.segments if k == "audio"]
+
+
+@torch.no_grad()
+def forward(model, h3ref, prep, video_x, audio_x, sigma_v, context, device,
+            offload, cache):
+    """One denoise evaluation in the sigma_v domain. Returns
+    (video_velocity, audio_velocity * slope), matching what the sampler wants."""
+    import comfy.ldm.common_dit
+    video_x = comfy.ldm.common_dit.pad_to_patch_size(video_x, model.patch_size)
+    orig_t, orig_h, orig_w = prep.video_shape[2:]
+
+    sigma_v = float(max(sigma_v, 1e-6))
+    seg_t, unique_t, t_row = timestep_rows(model, sigma_v)
+    segments = [(a, b, t_row[seg_t[k]] * 3 + tag)
+                for a, b, tag, k in prep.seg_template]
+
+    base_mods, t_emb = adaln_mods(model, unique_t, device, offload, cache)
+    silu_temb = F.silu(t_emb)
+
+    video_rows = h3ref.patchify_video(video_x.to(torch.float32), model.patch_size)
+    audio_rows = h3ref.pack_audio(audio_x.to(torch.float32))
+    video_embed = model.video_patch_proj(video_rows).to(torch.bfloat16)
+    audio_embed = model.audio_patch_proj(audio_rows).to(torch.bfloat16)
+
+    with torch.autocast("cuda", dtype=torch.bfloat16):
+        text_states = context[0]
+        if text_states.shape[-1] != model.hidden_size:
+            text_states = _refiner(model.token_refiner,
+                                   model.condition_proj(text_states))
+        pieces = []
+        for a, b, kind in prep.layout.segments:
+            if kind == "text":
+                pieces.append(text_states)
+            elif kind == "video":
+                pieces.append(video_embed)
+            else:
+                pieces.append(audio_embed)
+        h = torch.cat(pieces)
+        for i, blk in enumerate(model.blocks):
+            h = _block(blk, h, base_mods[i], segments,
+                       prep.rope_cos, prep.rope_sin)
+
+    fl = model.final_layer
+    with torch.autocast("cuda", dtype=torch.bfloat16):
+        si_t = F.silu(t_emb)
+        f_mod = fl.adaln_proj.linear(si_t)
+        if getattr(model, "_final_adaln_lora", None):
+            a, b = model._final_adaln_lora
+            f_mod = f_mod + F.linear(F.linear(si_t, a), b)
+    f_shift, f_scale = f_mod.view(len(unique_t), 2, model.hidden_size).unbind(1)
+    (va, vb), (aa, ab) = prep.video_seg, prep.audio_seg
+    vrow, arow = t_row[seg_t["video"]], t_row[seg_t["audio"]]
+    hn = _rms(h, fl.norm.weight, fl.norm.eps)
+    hv = (hn[va:vb] * (1.0 + f_scale[vrow]) + f_shift[vrow]).to(torch.float32)
+    ha = (hn[aa:ab] * (1.0 + f_scale[arow]) + f_shift[arow]).to(torch.float32)
+    v_rows, a_rows = fl.video_out(hv), fl.audio_out(ha)
+
+    lt = video_x.shape[2]
+    video_out = h3ref.unpatchify_video(v_rows, lt, prep.lat_pad[0] // 2,
+                                       prep.lat_pad[1] // 2, model.latents_dim,
+                                       model.patch_size)[:, :, :orig_t, :orig_h, :orig_w]
+    audio_out = h3ref.unpack_audio(a_rows)
+    slope_a = time_shift_slope(sigma_v, model.sigma_shift_video,
+                               model.sigma_shift_audio)
+    return -video_out.to(video_x.dtype), (-slope_a) * audio_out.to(audio_x.dtype)
+
+
+# ======================================================================
+# Text encode / decode / mux
+# ======================================================================
+def encode_prompt(comfyui, te_path, prompt, device):
+    import comfy.model_management
+    import comfy.sd
+    log(f"loading text encoder: {te_path}")
+    clip = comfy.sd.load_clip([te_path], clip_type=comfy.sd.CLIPType.MINIMAX)
+    cond = clip.encode_from_tokens_scheduled(clip.tokenize(prompt))
+    ca, ex = cond[0][0], cond[0][1]
+    tags = ex.get("minimax_token_tags")
+    ctx = ca.to(device, torch.bfloat16)
+    tags = tags.to(device) if torch.is_tensor(tags) else tags
+    del clip
+    comfy.model_management.unload_all_models()
+    comfy.model_management.soft_empty_cache()
+    return ctx, tags
+
+
+def _write_wav(path, waveform, sr):
+    w = waveform.detach().cpu().float()
+    if w.ndim == 3:
+        w = w[0]
+    w = w.clamp(-1.0, 1.0)
+    ch = w.shape[0]
+    pcm = (w.transpose(0, 1).contiguous().numpy() * 32767.0).astype("<i2")
+    with wave.open(path, "wb") as f:
+        f.setnchannels(ch)
+        f.setsampwidth(2)
+        f.setframerate(int(sr))
+        f.writeframes(pcm.tobytes())
+
+
+def save_mp4(images, waveform, sr, fps, out_path):
+    import imageio.v2 as imageio
+    import imageio_ffmpeg
+    frames = images.detach().cpu().float().clamp(0, 1).mul(255).round().to(
+        torch.uint8).numpy()
+    tv, ta = out_path + ".v.mp4", out_path + ".a.wav"
+    writer = imageio.get_writer(tv, fps=fps, codec="libx264", quality=8,
+                                pixelformat="yuv420p", macro_block_size=1,
+                                ffmpeg_log_level="error")
+    for fr in frames:
+        writer.append_data(fr)
+    writer.close()
+    _write_wav(ta, waveform, sr)
+    ffmpeg = imageio_ffmpeg.get_ffmpeg_exe()
+    subprocess.run([ffmpeg, "-y", "-loglevel", "error", "-i", tv, "-i", ta,
+                    "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", "-shortest",
+                    out_path], check=True)
+    os.remove(tv)
+    os.remove(ta)
+
+
+# ======================================================================
+# Main
+# ======================================================================
+def main():
+    ap = argparse.ArgumentParser(description="MiniMax-H3 Turbo LoRA 4-step generator")
+    ap.add_argument("--comfyui", required=True, help="path to a ComfyUI checkout @14b05228")
+    ap.add_argument("--base", required=True, help="H3 bf16 DiT safetensors")
+    ap.add_argument("--lora", required=True, help="turbo LoRA safetensors")
+    ap.add_argument("--te", required=True, help="Qwen3-VL text encoder safetensors")
+    ap.add_argument("--video-vae", required=True)
+    ap.add_argument("--audio-vae", required=True)
+    ap.add_argument("--prompt", required=True)
+    ap.add_argument("--out", default="out.mp4")
+    ap.add_argument("--width", type=int, default=1344, help="multiple of 16 (canvas is 32-based)")
+    ap.add_argument("--height", type=int, default=768)
+    ap.add_argument("--frames", type=int, default=124, help="24 fps; snaps to the 17k+5 grid")
+    ap.add_argument("--steps", type=int, default=4)
+    ap.add_argument("--seed", type=int, default=42)
+    ap.add_argument("--offload-adaln", action="store_true",
+                    help="keep the timestep-projection weights in CPU fp32 (saves ~13GB VRAM)")
+    args = ap.parse_args()
+    sys.path.insert(0, args.comfyui)  # ComfyUI supplies the H3 module definitions
+
+    dev = "cuda"
+    frames = args.frames
+    while frames % 17 != 5:
+        frames += 1
+    lt = (frames - 5) // 17 * 5 + 2
+    lh, lw = args.height // 16, args.width // 16
+    audio_t = round(frames / 24 * 40)
+    v_shape, a_shape = (1, 24, lt, lh, lw), (1, 32, 2, audio_t)
+    ts = timesteps(args.steps)
+    log(f"{args.width}x{args.height}x{frames}f ({frames/24:.1f}s) -> "
+        f"video{v_shape} audio{a_shape}; {args.steps}-step grid "
+        f"{['%.3f' % t for t in ts]}")
+
+    ctx, tags = encode_prompt(args.comfyui, args.te, args.prompt, dev)
+    model, h3ref = load_model(args.comfyui, args.base, args.lora, dev,
+                              args.offload_adaln)
+    prep = Prepared(model, h3ref, ctx.shape[1], v_shape, audio_t, tags, dev)
+
+    g = torch.Generator(dev).manual_seed(args.seed)
+    ga = torch.Generator(dev).manual_seed(args.seed + 1)
+    nv = torch.randn(v_shape, generator=g, device=dev, dtype=torch.bfloat16)
+    na = torch.randn(a_shape, generator=ga, device=dev, dtype=torch.bfloat16)
+
+    cache = {}
+
+    def vfn(xv, xa, sv):
+        return forward(model, h3ref, prep, xv, xa, sv, ctx, dev,
+                       args.offload_adaln, cache)
+
+    log("sampling ...")
+    t0 = time.time()
+    with torch.inference_mode():
+        zv, za = sample(vfn, nv, na, ts)
+    log(f"sampled in {time.time()-t0:.1f}s")
+
+    import comfy.sd
+    import comfy.utils
+    video_vae = comfy.sd.VAE(sd=comfy.utils.load_torch_file(args.video_vae))
+    audio_vae = comfy.sd.VAE(sd=comfy.utils.load_torch_file(args.audio_vae))
+    with torch.inference_mode():
+        images = video_vae.decode(zv.float())
+        if images.ndim == 5:
+            images = images.reshape(-1, *images.shape[-3:])
+        waveform = audio_vae.decode(za.float()).movedim(-1, 1)
+        std = torch.std(waveform, dim=[1, 2], keepdim=True) * 5.0
+        std[std < 1.0] = 1.0
+        waveform = waveform / std
+    sr = getattr(audio_vae, "audio_sample_rate_output",
+                 getattr(audio_vae, "audio_sample_rate", 44100))
+    save_mp4(images, waveform, sr, 24, args.out)
+    log(f"done -> {args.out} ({os.path.getsize(args.out)/2**20:.1f}MB)")
+
+
+if __name__ == "__main__":
+    main()

minimax_h3_t2v_turbo.json

@@ -0,0 +1,1072 @@
+{
+  "id": "e3f2b845-8f2c-4b5a-9caf-eac1029d3e7e",
+  "revision": 0,
+  "last_node_id": 135,
+  "last_link_id": 252,
+  "nodes": [
+    {
+      "id": 119,
+      "type": "VAELoader",
+      "pos": [
+        -2393.7830229552733,
+        5342.158989520116
+      ],
+      "size": [
+        640,
+        70
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            229,
+            241
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader"
+      },
+      "widgets_values": [
+        "minimax_h3_video_vae_fp16.safetensors"
+      ]
+    },
+    {
+      "id": 120,
+      "type": "VAELoader",
+      "pos": [
+        -2393.7830229552733,
+        5472.158989520116
+      ],
+      "size": [
+        650,
+        70
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            228
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader"
+      },
+      "widgets_values": [
+        "minimax_h3_audio_vae_fp32.safetensors"
+      ]
+    },
+    {
+      "id": 121,
+      "type": "VAEDecodeAudio",
+      "pos": [
+        -243.2871350097655,
+        5020.092864990236
+      ],
+      "size": [
+        230,
+        60
+      ],
+      "flags": {
+        "collapsed": false
+      },
+      "order": 18,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 248
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 228
+        }
+      ],
+      "outputs": [
+        {
+          "name": "AUDIO",
+          "type": "AUDIO",
+          "links": [
+            239
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecodeAudio"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 122,
+      "type": "VAEDecode",
+      "pos": [
+        -243.2871350097655,
+        4900.092864990236
+      ],
+      "size": [
+        230,
+        60
+      ],
+      "flags": {
+        "collapsed": false
+      },
+      "order": 17,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 247
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 229
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            238
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 125,
+      "type": "SamplerCustomAdvanced",
+      "pos": [
+        -553.2871350097655,
+        4960.092864990236
+      ],
+      "size": [
+        230,
+        140
+      ],
+      "flags": {},
+      "order": 16,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "noise",
+          "type": "NOISE",
+          "link": 231
+        },
+        {
+          "name": "guider",
+          "type": "GUIDER",
+          "link": 232
+        },
+        {
+          "name": "sampler",
+          "type": "SAMPLER",
+          "link": 252
+        },
+        {
+          "name": "sigmas",
+          "type": "SIGMAS",
+          "link": 234
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 235
+        }
+      ],
+      "outputs": [
+        {
+          "name": "output",
+          "type": "LATENT",
+          "links": [
+            247,
+            248
+          ]
+        },
+        {
+          "name": "denoised_output",
+          "type": "LATENT",
+          "links": []
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SamplerCustomAdvanced"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 126,
+      "type": "BasicGuider",
+      "pos": [
+        -983.2871350097655,
+        4940.092864990236
+      ],
+      "size": [
+        360,
+        60
+      ],
+      "flags": {},
+      "order": 15,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 250
+        },
+        {
+          "name": "conditioning",
+          "type": "CONDITIONING",
+          "link": 237
+        }
+      ],
+      "outputs": [
+        {
+          "name": "GUIDER",
+          "type": "GUIDER",
+          "links": [
+            232
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "BasicGuider"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 129,
+      "type": "RandomNoise",
+      "pos": [
+        -983.2871350097655,
+        4800.092864990236
+      ],
+      "size": [
+        360,
+        90
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "NOISE",
+          "type": "NOISE",
+          "links": [
+            231
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "RandomNoise"
+      },
+      "widgets_values": [
+        953478908379578,
+        "randomize"
+      ]
+    },
+    {
+      "id": 128,
+      "type": "CLIPLoader",
+      "pos": [
+        -2393.7830229552733,
+        5152.158989520116
+      ],
+      "size": [
+        640,
+        120
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            240
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader"
+      },
+      "widgets_values": [
+        "qwen3vl_32b_minimax_h3_int8_convrot.safetensors",
+        "minimax",
+        "default"
+      ]
+    },
+    {
+      "id": 127,
+      "type": "UNETLoader",
+      "pos": [
+        -2393.7830229552733,
+        5002.158989520116
+      ],
+      "size": [
+        640,
+        90
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            249
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader"
+      },
+      "widgets_values": [
+        "minimax_h3_fl2va_int8_convrot.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 134,
+      "type": "MiniMaxH3TurboLoRA",
+      "pos": [
+        -1537.408311244107,
+        4546.676307402845
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 249
+        }
+      ],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            250,
+            251
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "MiniMaxH3TurboLoRA"
+      },
+      "widgets_values": [
+        "minimax_h3_turbo_4step_ema_ckpt500.safetensors",
+        1
+      ]
+    },
+    {
+      "id": 130,
+      "type": "CreateVideo",
+      "pos": [
+        76.39285156250011,
+        5059.452871704105
+      ],
+      "size": [
+        270,
+        110
+      ],
+      "flags": {},
+      "order": 19,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 238
+        },
+        {
+          "name": "audio",
+          "shape": 7,
+          "type": "AUDIO",
+          "link": 239
+        }
+      ],
+      "outputs": [
+        {
+          "name": "VIDEO",
+          "type": "VIDEO",
+          "links": [
+            242
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CreateVideo"
+      },
+      "widgets_values": [
+        24,
+        8
+      ]
+    },
+    {
+      "id": 92,
+      "type": "SaveVideo",
+      "pos": [
+        529.3816487426757,
+        4466.255195938108
+      ],
+      "size": [
+        1070,
+        692.4444444444445
+      ],
+      "flags": {},
+      "order": 20,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "video",
+          "type": "VIDEO",
+          "link": 242
+        }
+      ],
+      "outputs": [
+        {
+          "name": "video",
+          "type": "VIDEO",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "video/MiniMax_H3",
+        "auto",
+        "auto"
+      ]
+    },
+    {
+      "id": 131,
+      "type": "MiniMaxH3ImageToVideo",
+      "pos": [
+        -1547.4412168457013,
+        4768.115383134615
+      ],
+      "size": [
+        410,
+        510
+      ],
+      "flags": {},
+      "order": 14,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 240
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 241
+        },
+        {
+          "name": "first_frame",
+          "shape": 7,
+          "type": "IMAGE",
+          "link": null
+        },
+        {
+          "name": "last_frame",
+          "shape": 7,
+          "type": "IMAGE",
+          "link": null
+        },
+        {
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": 244
+        },
+        {
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": 245
+        },
+        {
+          "name": "length",
+          "type": "INT",
+          "widget": {
+            "name": "length"
+          },
+          "link": 243
+        }
+      ],
+      "outputs": [
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            237
+          ]
+        },
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            235
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "MiniMaxH3ImageToVideo"
+      },
+      "widgets_values": [
+        "Vaporwave title sequence look: pink and blue gradient palette, VHS tracking artifacts, Greek statue motifs, chrome palm trees, RGB chromatic aberration, lo-fi retro atmosphere, mood languid and nostalgic.\n\nTimeline:\n[0s-1s] VHS static opens the frame, the title \"COMFYUI\" appears with RGB split and a slight horizontal jitter.\n[1s-2.5s] Hard cut, a Greek plaster bust close-up, pink-purple gradient sky, a pixelated sun.\n[2.5s-4s] Clean \"STARRING\" credits appear, \"LATENT\" and \"CONTROLNET\" each shown exactly once.\n[4s-5s] Final card \"DIRECTED BY COMFYUI\" holds, one VHS tracking glitch settling into stability.\n\nHard cuts only, transitions landing with tape jumps, no push-ins, no dissolves.\n\nAudio: lo-fi vaporwave score, slow drum machine with soft bass, VHS tape-noise sample joins at 2.5s, melody fading for the last 1s.\n\nAll text must be clearly legible, do not misspell English, no Chinese characters, do not repeat names or job titles, no soft dissolves, no subtitle bars.",
+        1344,
+        768,
+        73
+      ]
+    },
+    {
+      "id": 124,
+      "type": "BasicScheduler",
+      "pos": [
+        -983.2871350097655,
+        5170.092864990236
+      ],
+      "size": [
+        370,
+        130
+      ],
+      "flags": {},
+      "order": 13,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 251
+        }
+      ],
+      "outputs": [
+        {
+          "name": "SIGMAS",
+          "type": "SIGMAS",
+          "links": [
+            234
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "BasicScheduler"
+      },
+      "widgets_values": [
+        "simple",
+        6,
+        1
+      ]
+    },
+    {
+      "id": 116,
+      "type": "MarkdownNote",
+      "pos": [
+        -3481.88621199646,
+        4686.487353762512
+      ],
+      "size": [
+        450,
+        740
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "title": "Note: MiniMax H3",
+      "properties": {},
+      "widgets_values": [
+        "## MiniMax H3\n\n[MiniMax H3](https://www.minimax.io/blog/minimax-h3) is MiniMax's general-purpose, omni-modal generation model. It jointly understands text, image, video, and audio, and generates video with **native stereo audio**: voice, sound effects, and music are modeled jointly in a single forward pass, not layered on afterward. Output is up to 2K resolution, 24fps, and up to about 15 seconds.\n\n## ComfyUI links\n- [ComfyUI#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224)\n- [🤗 Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3)\n\n## About this workflow\n\n**Key inputs**\n\n- **prompt**: describe the shots, camera moves, and the accompanying audio (dialogue, SFX, music) in one block\n- **width / height**: set via Resolution Selector. H3's native canvas is a 768px short edge, capped at 768x1344 pixels, rounded to a multiple of 32\n- **duration (seconds)**: converted to a valid frame `length` by the Math Expression node, snapping up to the model's 17-frame-per-block (17k+5) grid at 24fps\n"
+      ],
+      "color": "#222",
+      "bgcolor": "#000"
+    },
+    {
+      "id": 117,
+      "type": "MarkdownNote",
+      "pos": [
+        -2949.206052406617,
+        4685.120018463135
+      ],
+      "size": [
+        440,
+        740
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "title": "Note: Model Links",
+      "properties": {},
+      "widgets_values": [
+        "## Model Links\n\n**vae**\n\n- [minimax_h3_video_vae_fp16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors)\n- [minimax_h3_audio_vae_fp32.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors)\n\n**diffusion_models**\n\n- [minimax_h3_fl2va_pruned_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors)\n\n**text_encoders**\n\n- [qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors)\n\n\n## Model Storage Location\n\n```\n📂 ComfyUI/\n├── 📂 models/\n│   ├── 📂 vae/\n│   │   ├── minimax_h3_video_vae_fp16.safetensors\n│   │   └── minimax_h3_audio_vae_fp32.safetensors\n│   ├── 📂 diffusion_models/\n│   │   └── minimax_h3_fl2va_pruned_int8_convrot.safetensors\n│   └── 📂 text_encoders/\n│       └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors\n```\n\n## Report Issue\n\nNote: Please update ComfyUI first ([guide](https://docs.comfy.org/installation/update_comfyui)) and prepare required models. Desktop/Cloud updates follow stable releases, so some nightly-supported models may not be available yet.\n\n- Cannot run / runtime errors: [ComfyUI/issues](https://github.com/comfyanonymous/ComfyUI/issues)\n- UI / frontend issues: [ComfyUI_frontend/issues](https://github.com/Comfy-Org/ComfyUI_frontend/issues)\n- Workflow issues: [workflow_templates/issues](https://github.com/Comfy-Org/workflow_templates/issues)\n"
+      ],
+      "color": "#222",
+      "bgcolor": "#000"
+    },
+    {
+      "id": 133,
+      "type": "PrimitiveFloat",
+      "pos": [
+        -2415.5672499884035,
+        5729.984664990233
+      ],
+      "size": [
+        270,
+        70
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": [
+            246
+          ]
+        }
+      ],
+      "title": "Float (duration)",
+      "properties": {
+        "Node name for S&R": "PrimitiveFloat"
+      },
+      "widgets_values": [
+        5
+      ]
+    },
+    {
+      "id": 132,
+      "type": "ComfyMathExpression",
+      "pos": [
+        -2056.6077066693106,
+        5696.486027500916
+      ],
+      "size": [
+        360,
+        160
+      ],
+      "flags": {
+        "collapsed": false
+      },
+      "order": 12,
+      "mode": 0,
+      "inputs": [
+        {
+          "label": "a",
+          "name": "values.a",
+          "type": "FLOAT,INT,BOOLEAN",
+          "link": 246
+        },
+        {
+          "label": "b",
+          "name": "values.b",
+          "shape": 7,
+          "type": "FLOAT,INT,BOOLEAN",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": []
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": [
+            243
+          ]
+        },
+        {
+          "name": "BOOL",
+          "type": "BOOLEAN",
+          "links": []
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ComfyMathExpression"
+      },
+      "widgets_values": [
+        "max(5, round(a * 24)) + (5 - (max(5, round(a * 24)) % 17)) % 17"
+      ]
+    },
+    {
+      "id": 118,
+      "type": "MarkdownNote",
+      "pos": [
+        -1521.0404566809086,
+        5724.123689148864
+      ],
+      "size": [
+        300,
+        520
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "title": "Note: Size Settings Reference",
+      "properties": {},
+      "widgets_values": [
+        "| megapixels | Aspect | Output (multiple=32) |\n|---|---|---|\n| 0.2 | 16:9 | 608 x 352 |\n| 0.3 | 16:9 | 736 x 416 |\n| 0.4 | 16:9 | 864 x 480 |\n| 0.5 | 16:9 | 960 x 544 |\n| 0.6 | 16:9 | 1056 x 608 |\n| 0.7 | 16:9 | 1152 x 640 |\n| 0.8 | 16:9 | 1216 x 672 |\n| 0.9 | 16:9 | 1280 x 736 |\n| 0.98 | 16:9 | 1344 x 768 |\n| 1.0 | 16:9 | 1376 x 768 |\n| 1.2 | 16:9 | 1504 x 832 |\n| 1.5 | 16:9 | 1664 x 928 |\n| 1.8 | 16:9 | 1824 x 1024 |\n| 2.0 | 16:9 | 1920 x 1088 |\n"
+      ],
+      "color": "#222",
+      "bgcolor": "#000"
+    },
+    {
+      "id": 115,
+      "type": "ResolutionSelector",
+      "pos": [
+        -1580.9217133618172,
+        5487.988854170227
+      ],
+      "size": [
+        270,
+        170
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "showAdvanced": true,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "width",
+          "type": "INT",
+          "links": [
+            244
+          ]
+        },
+        {
+          "name": "height",
+          "type": "INT",
+          "links": [
+            245
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ResolutionSelector"
+      },
+      "widgets_values": [
+        "16:9 (Widescreen)",
+        0.4,
+        32
+      ]
+    },
+    {
+      "id": 135,
+      "type": "MiniMaxH3TurboSampler",
+      "pos": [
+        -826.1238619112232,
+        5411.853328422132
+      ],
+      "size": [
+        280.2447265625,
+        26
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "SAMPLER",
+          "type": "SAMPLER",
+          "links": [
+            252
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "MiniMaxH3TurboSampler"
+      }
+    }
+  ],
+  "links": [
+    [
+      228,
+      120,
+      0,
+      121,
+      1,
+      "VAE"
+    ],
+    [
+      229,
+      119,
+      0,
+      122,
+      1,
+      "VAE"
+    ],
+    [
+      231,
+      129,
+      0,
+      125,
+      0,
+      "NOISE"
+    ],
+    [
+      232,
+      126,
+      0,
+      125,
+      1,
+      "GUIDER"
+    ],
+    [
+      234,
+      124,
+      0,
+      125,
+      3,
+      "SIGMAS"
+    ],
+    [
+      235,
+      131,
+      1,
+      125,
+      4,
+      "LATENT"
+    ],
+    [
+      237,
+      131,
+      0,
+      126,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      238,
+      122,
+      0,
+      130,
+      0,
+      "IMAGE"
+    ],
+    [
+      239,
+      121,
+      0,
+      130,
+      1,
+      "AUDIO"
+    ],
+    [
+      240,
+      128,
+      0,
+      131,
+      0,
+      "CLIP"
+    ],
+    [
+      241,
+      119,
+      0,
+      131,
+      1,
+      "VAE"
+    ],
+    [
+      242,
+      130,
+      0,
+      92,
+      0,
+      "VIDEO"
+    ],
+    [
+      243,
+      132,
+      1,
+      131,
+      6,
+      "INT"
+    ],
+    [
+      244,
+      115,
+      0,
+      131,
+      4,
+      "INT"
+    ],
+    [
+      245,
+      115,
+      1,
+      131,
+      5,
+      "INT"
+    ],
+    [
+      246,
+      133,
+      0,
+      132,
+      0,
+      "FLOAT"
+    ],
+    [
+      247,
+      125,
+      0,
+      122,
+      0,
+      "LATENT"
+    ],
+    [
+      248,
+      125,
+      0,
+      121,
+      0,
+      "LATENT"
+    ],
+    [
+      249,
+      127,
+      0,
+      134,
+      0,
+      "MODEL"
+    ],
+    [
+      250,
+      134,
+      0,
+      126,
+      0,
+      "MODEL"
+    ],
+    [
+      251,
+      134,
+      0,
+      124,
+      0,
+      "MODEL"
+    ],
+    [
+      252,
+      135,
+      0,
+      125,
+      2,
+      "SAMPLER"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 1,
+      "title": "Models",
+      "bounding": [
+        -2423.7830229552733,
+        4912.158989520116,
+        700,
+        670
+      ],
+      "color": "#3f789e",
+      "flags": {}
+    },
+    {
+      "id": 2,
+      "title": "Sampling",
+      "bounding": [
+        -1003.2871350097655,
+        4680.092864990236,
+        690,
+        670
+      ],
+      "color": "#3f789e",
+      "flags": {}
+    },
+    {
+      "id": 3,
+      "title": "Conditioning",
+      "bounding": [
+        -1545.4973350097653,
+        4700.707397905124,
+        480,
+        670
+      ],
+      "color": "#3f789e",
+      "flags": {}
+    },
+    {
+      "id": 4,
+      "title": "Decoding and create video",
+      "bounding": [
+        -283.2871350097655,
+        4680.092864990236,
+        670,
+        670
+      ],
+      "color": "#3f789e",
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "frontendVersion": "1.47.12",
+    "VHS_latentpreview": false,
+    "VHS_latentpreviewrate": 0,
+    "VHS_MetadataImage": true,
+    "VHS_KeepIntermediate": true,
+    "ds": {
+      "scale": 0.6209213230591556,
+      "offset": [
+        2494.182489857376,
+        -4486.16972718581
+      ]
+    }
+  },
+  "version": 0.4
+}

minimax_h3_turbo_4step.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c468c61ebf715699b5a710fed016654e4a244195ce2e1c4f9f84dc63f82da905
+size 779849872

minimax_h3_turbo_4step_ckpt500.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82d0acff583b04ad9a4238a7440b584b56094bfb7c4fdb2981f67c7a4784b62d
+size 779849872

minimax_h3_turbo_4step_ckpt850.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6d1be058a76ec7b228a2e2ef36b2cdcdfa65f32e98ccb4009cbdc9224914bc3
+size 779849816

minimax_h3_turbo_4step_ema.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d645b67e606874e9179b277cea721c1f1e75830532fcc2206e23353cb33edc5
+size 779849872

minimax_h3_turbo_4step_ema_ckpt500.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6183247bacc434f382f1e0e3e03265f677c50928313ea806756d18fba5cc9160
+size 779849872

minimax_h3_turbo_4step_ema_ckpt850.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a6eeba171cf183020a4ad48774bb2968f29f8168afd6ec17a04987f3528b4ea
+size 779849816

minimax_h3_turbo_v4_step600.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c280b58f03b5dc89e791347679b821b8a85615bedae04d3c65bbbe753e07ce8
+size 779849816

minimax_h3_turbo_v4_step600_ema.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f3a626cd72c93a8b9318d6760c510bc5092d2ab13aaba1f932c5bab07a416d3
+size 779849816

requirements.txt

@@ -0,0 +1,10 @@
+# MiniMax-H3 Turbo LoRA 4-step generator.
+# The base model / VAE / text-encoder module definitions come from a ComfyUI
+# checkout (see generate.py header): git clone + checkout 14b05228, then install
+# ComfyUI's own requirements. The packages below are what generate.py itself
+# needs on top of that.
+torch>=2.4
+safetensors>=0.4
+imageio>=2.34
+imageio-ffmpeg>=0.5
+numpy>=1.24