| <!doctype html><html><head><meta charset="utf-8">
|
| <title>MiniMax-H3 Hard Mode — reference to video</title><style>
|
| :root{--bg:#141518;--card:#1c1e22;--card2:#212429;--line:#2e3238;--txt:#e8e9eb;
|
| --dim:#a8aeb8;--acc:#68b5fb;--good:#5fd08a;--warn:#e0a13a;--bad:#e06c6c;--code:#9fd8ff}
|
| *{box-sizing:border-box}
|
| body{max-width:960px;margin:0 auto;padding:44px 26px 90px;background:var(--bg);color:var(--txt);
|
| font:16px/1.62 -apple-system,"Segoe UI",Roboto,sans-serif}
|
| h1{font-size:2.05em;margin:0 0 .12em;letter-spacing:-.02em}
|
| .sub{color:var(--dim);font-size:1.04em;margin:0 0 26px}
|
| h2{font-size:1.28em;margin:2.3em 0 .6em;padding-bottom:.32em;border-bottom:1px solid var(--line)}
|
| h3{font-size:1.03em;margin:1.5em 0 .4em;color:#fff}
|
| p{margin:.7em 0}
|
| code{background:#262a30;padding:.13em .4em;border-radius:3px;font-size:.9em;color:var(--code);
|
| font-family:ui-monospace,Consolas,monospace}
|
| pre{background:#262a30;padding:13px 15px;border-radius:6px;overflow-x:auto;font-size:.87em;
|
| font-family:ui-monospace,Consolas,monospace;line-height:1.5}
|
| a{color:var(--acc)}
|
| table{border-collapse:collapse;width:100%;margin:1em 0;font-size:.93em}
|
| th,td{border:1px solid var(--line);padding:.5em .72em;text-align:left;vertical-align:top}
|
| th{background:#262a30;color:#fff;font-weight:600}
|
| ul,ol{padding-left:1.35em}li{margin:.4em 0}
|
|
|
| .status{background:var(--card);border:1px solid var(--line);border-radius:9px;padding:16px 20px;margin:0 0 8px}
|
| .status h3{margin:0 0 .5em;font-size:.85em;letter-spacing:.09em;text-transform:uppercase;color:var(--dim)}
|
| .grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(168px,1fr));gap:11px}
|
| .chip{background:var(--card2);border:1px solid var(--line);border-radius:7px;padding:10px 13px}
|
| .chip .k{font-size:.74em;letter-spacing:.05em;text-transform:uppercase;color:var(--dim)}
|
| .chip .v{font-size:1.0em;color:#fff;margin-top:2px;font-weight:600}
|
| .ok{color:var(--good)}.no{color:var(--bad)}.mid{color:var(--warn)}
|
|
|
| .callout{border-left:3px solid var(--line);padding:.55em 0 .55em 15px;margin:1.15em 0;color:#cfd4dc}
|
| .callout.bad{border-color:var(--bad)}
|
| .callout.warn{border-color:var(--warn)}
|
| .callout.good{border-color:var(--good)}
|
| .callout b{color:#fff}
|
|
|
| ol.steps{counter-reset:s;list-style:none;padding-left:0}
|
| ol.steps>li{counter-increment:s;position:relative;padding:13px 0 13px 46px;border-top:1px solid var(--line)}
|
| ol.steps>li:before{content:counter(s);position:absolute;left:0;top:12px;width:29px;height:29px;
|
| border-radius:50%;background:var(--card2);border:1px solid var(--line);color:var(--acc);
|
| display:flex;align-items:center;justify-content:center;font-size:.86em;font-weight:700}
|
| ol.steps>li b{color:#fff}
|
| .small{font-size:.9em;color:var(--dim)}
|
| </style></head><body>
|
|
|
| <h1>Hard Mode — reference to video</h1>
|
| <p class="sub">Identity from reference <em>images, video and audio</em> instead of a start frame.
|
| Up to 9 images, 3 videos, 3 soundtracks and 3 standalone audio clips.</p>
|
|
|
| <h2>Is this the workflow you want?</h2>
|
| <p>Three graphs, three jobs. Picking wrong is the most common way to waste a render.</p>
|
| <table>
|
| <tr><th>You want</th><th>Use</th></tr>
|
| <tr><td>A specific opening or closing frame, or timed anchors</td><td><code>H3_Keyframes.json</code></td></tr>
|
| <tr><td><strong>Identity carried by reference material</strong> — a person, a place, a voice</td><td><strong>this graph</strong></td></tr>
|
| <tr><td>2–5 minute runtime</td><td><code>H3_Multishot_MEMORY.json</code></td></tr>
|
| </table>
|
|
|
| <div class="callout bad"><b>References and keyframes are mutually exclusive.</b> Not a style
|
| preference — a hard limit in ComfyUI's core. <code>model_base.py</code> writes
|
| <code>cond_video_latents</code> for keyframes, then <em>assigns</em> it again for references, so the
|
| keyframe latents are discarded while the keyframe layout rows survive. The sequence desyncs and the
|
| sampler dies on a shape mismatch. There is no "reference images + start frame" mode, and asking for
|
| both crashes rather than degrading gracefully.</div>
|
|
|
| <h2>First render, start to finish</h2>
|
| <ol class="steps">
|
| <li><b>Load the graph.</b> <code>H3_HardMode_R2V.json</code>. It opens clean — the image slots point
|
| at ComfyUI's bundled <code>example.png</code> so nothing is missing on load.</li>
|
|
|
| <li><b>Swap in your own reference images.</b> The three <code>LoadImage</code> nodes are the entire
|
| point of this mode. The zip ships three views of one subject in <code>reference_images/</code> — drop them into <code>ComfyUI/input/</code> to try it immediately, then replace them with your own. Two or three angles of the same subject is the sweet spot; a third reference costs nothing measurable.</li>
|
|
|
| <li><b>Pick the model.</b> On a 24 GB card use <code>MiniMax-H3-ref2va-curve-Q8_0.gguf</code> —
|
| measured on a 24 GB card at <strong>6.8 min</strong> for 123 frames, sitting <em>resident</em> at 21.7 GB
|
| of 24.0, not streaming. It is the highest quality tier that fits, and it is both smaller and faster
|
| than the original-form Q4_0 people usually run.</li>
|
|
|
| <li><b>Write the prompt so it names the references.</b> This is the part that is not guessable —
|
| see the section below. If you skip it the model has references but no instruction about what they
|
| <em>are</em>.</li>
|
|
|
| <li><b>Leave the audio branch muted for the first run.</b> <code>LoadAudio</code> and the stereo
|
| guard ship muted because there is no example clip to point them at. Identity from images alone works
|
| fine. Add voice later.</li>
|
|
|
| <li><b>Queue it.</b> Expect roughly 7 minutes at 124 frames / 480×864 / 20 steps. If it is
|
| dramatically slower, jump to Troubleshooting — the cause is almost always VRAM, and the tell is
|
| <em>power draw</em>, not utilisation.</li>
|
| </ol>
|
|
|
| <h2>Naming your references in the prompt</h2>
|
| <p>Reference blocks are labelled in the prompt text, and <b>the numbering is 1-based while the input
|
| slots are 0-based</b>. So <code>ref_image_0</code> is <code><Picture 1></code>. Write prompts
|
| that actually refer to them:</p>
|
| <pre><Picture 1> is the woman. <Picture 2> is the room she is standing in.
|
| She walks to the window in the style of <Video 1>.</pre>
|
|
|
| <p>Ordering is fixed by the node no matter how you wire it: <b>all images, then videos, then
|
| standalone audios.</b> Empty slots are skipped and the numbering compacts — leaving
|
| <code>ref_image_0</code> empty and filling <code>ref_image_1</code> still gives you
|
| <code><Picture 1></code>.</p>
|
|
|
| <div class="callout warn"><b>The ordinal trap.</b> A reference video <em>with a soundtrack</em>
|
| consumes an <code><Audio j></code> ordinal <em>before</em> your standalone clips. One ref video
|
| with sound plus one <code>ref_audio_0</code> makes your voice clip <code><Audio 2></code>, not
|
| <code><Audio 1></code>. Get this wrong and the model binds the wrong voice with no error.</div>
|
|
|
| <h2>Three rules that will bite you</h2>
|
|
|
| <h3>1. Reference audio must be STEREO</h3>
|
| <p>The audio VAE encodes <code>[B, 2, L]</code> and the layout reserves exactly two channels. A
|
| <b>mono</b> reference produces half the rows it reserved and dies deep in the model with no useful
|
| error message. The stereo-guard node forces 2 channels at 32 kHz and trims length —
|
| <b>keep it wired</b> whenever you unmute the audio branch.</p>
|
|
|
| <h3>2. The eviction node is ON — leave it on</h3>
|
| <p><code>H3FreeTextEncoder</code> (node 146) frees the ~16.5 GB text encoder once conditioning is
|
| computed, before the DiT loads. The encoder and the ~24 GB DiT do not co-fit in 32 GB, so without it
|
| the DiT loads partially and streams the remainder from system RAM on every sampling step.</p>
|
| <p>Measured on a 32 GB card, <b>images only</b>, <code>POST /free</code> before every run, runs
|
| alternated so machine drift could not fake the result:</p>
|
| <ul>
|
| <li>eviction bypassed — <strong>4.5 min</strong> (4.6, 4.5)</li>
|
| <li><strong>eviction active — 3.4 min</strong> (3.5, 3.3)</li>
|
| </ul>
|
| <p><b>24% faster, and bit-identical output</b> — every pairing scored 0.00 mean pixel distance
|
| with motion 7.19 across all four runs. It costs nothing and changes nothing.</p>
|
| <p>With a reference <b>video</b> the gap is far wider, because a reference video goes through the
|
| encoder's <em>vision</em> path:</p>
|
| <ul>
|
| <li>no eviction — <span class="no">46 min, killed</span>, 98% utilisation at ~172 W</li>
|
| <li>eviction — <span class="ok"><strong>6.5 and 7.7 min</strong></span>, 355–390 W, zero frozen frames</li>
|
| </ul>
|
| <p class="small">Clip length is not the driver — a 24-frame reference thrashed just as badly as
|
| a 124-frame one. The video's mere presence is what does it.</p>
|
| <div class="callout"><b>If you ever disable it, use Bypass — never Mute.</b> A muted node
|
| produces nothing, so <code>H3ConditionStrength</code> loses its conditioning input entirely and the
|
| graph stops validating. Bypass passes conditioning straight through, which is the correct idiom for
|
| a pass-through node.</div>
|
|
|
| <h3>3. Reference video has shape rules</h3>
|
| <ul>
|
| <li>minimum <b>5 frames</b>, or it raises at validation</li>
|
| <li>silently truncated to the generation length, then trimmed to <code>n % 17 == 5</code></li>
|
| <li>shown to the encoder at <b>2 fps</b> (a stride of 12 over 24 fps)</li>
|
| </ul>
|
|
|
| <h2>What the settings cost</h2>
|
| <p>Reference rows sit in the packed sequence on <em>every sampling step</em>, not once up front — so
|
| more references and a bigger <code>ref_image_size</code> both cost time per step.</p>
|
| <p>Measured, 124 frames at 480×864, two reference images:</p>
|
| <table>
|
| <tr><th>Setting</th><th>Time</th><th>Identity margin</th></tr>
|
| <tr><td><code>ref_image_size = match</code></td><td class="ok"><strong>2.9 min</strong></td><td>+9.42</td></tr>
|
| <tr><td><code>ref_image_size = max</code></td><td>4.0 min</td><td>+9.10</td></tr>
|
| </table>
|
| <p>So <b>1.38×</b>, not the "several times slower" core's own tooltip warns about — and <code>max</code>
|
| did <em>not</em> improve identity here, scoring fractionally worse while costing 38% more. It does
|
| change the output, so it is doing something; it just did not buy fidelity at this resolution.
|
| <b>Start with <code>match</code> and 2–3 images.</b></p>
|
| <p class="small">Caveat worth knowing: that margin is a blunt measure (distance to the reference
|
| images versus distance to a control person the model never saw). It confirms identity carried, but
|
| may be too coarse to see the fine facial fidelity <code>max</code> targets. At higher output
|
| resolutions it is likelier to earn its cost.</p>
|
| <p>Reference <b>audio</b>, same test: about <b>14%</b> extra wall clock, and it trims motion roughly
|
| 17–27%. The audio conditioning constrains the performance rather than stalling it — it does
|
| <em>not</em> freeze the render.</p>
|
|
|
| <h2>Troubleshooting</h2>
|
| <table>
|
| <tr><th>Symptom</th><th>Cause / fix</th></tr>
|
| <tr><td>Render crawling, GPU shows ~98% utilisation</td><td><b>Check power draw, not utilisation.</b>
|
| A card thrashing weights between system RAM and VRAM still reports ~98% while pulling a fraction of
|
| its rated watts. 140–240 W means it did not fit. 355–600 W means it is genuinely sampling.
|
| <br><b>Sample more than once.</b> The encoder/conditioning phase at the start of every render is
|
| legitimately low-power — about 100 W at ~35% utilisation with only the encoder resident. One
|
| unlucky reading there looks exactly like thrashing. Watch for 20–30 s: if it climbs past 400 W once
|
| sampling begins, nothing is wrong.</td></tr>
|
| <tr><td>Sampler dies on a shape mismatch</td><td>You wired references <em>and</em> keyframes. Pick one.</td></tr>
|
| <tr><td>Dies deep in the model, no useful error, audio connected</td><td>Mono reference audio. Wire the stereo guard.</td></tr>
|
| <tr><td>"Required input is missing → conditioning"</td><td>You muted a pass-through node instead of bypassing it.</td></tr>
|
| <tr><td>Wrong voice bound to the wrong reference</td><td>Ordinal shift — a ref video's soundtrack took <code><Audio 1></code>.</td></tr>
|
| <tr><td>Timing looks impossibly good, then never repeats</td><td>Warm session. <code>POST /free</code> before any timing you intend to compare.</td></tr>
|
| </table>
|
|
|
| <h2>Settled late, and it reversed the default</h2>
|
| <div class="callout good">
|
| <p style="margin-top:0"><b>Does eviction cost anything on the ordinary images-only path?</b> No
|
| — it <em>saves</em>. Bypassed 4.5 min, active 3.4 min, with bit-identical output. So the node
|
| now ships <b>active</b>, and the old "turn it on only for reference video" instruction is gone.</p>
|
| <p style="margin-bottom:0">The previous default rested on a single run that stalled at 19 minutes
|
| and was killed before the cause could be isolated. That run followed an interrupt, so it inherited
|
| resident models — the same contamination that produced the discredited "2.9 min" baseline
|
| below. It was never evidence about eviction at all.</p>
|
| </div>
|
| <p class="small">Also note: a warm-session baseline of "2.9 min" for the images-only graph did not
|
| survive a restart — the same graph cold took <b>17.5 min</b>. Wall-clock numbers taken mid-session
|
| are best-case. Pixel comparisons are unaffected: re-running the baseline after restart gave a
|
| distance of 0.00, so sampling is deterministic across restarts.</p>
|
|
|
| <h2>Everything above that was verified by reading the source</h2>
|
| <p class="small">Rather than by spending renders on it — <code>comfy_extras/nodes_minimax_h3.py</code>
|
| and <code>comfy/model_base.py</code>:</p>
|
| <table>
|
| <tr><th>Claim</th><th>Where</th></tr>
|
| <tr><td>minimum 5 frames or it raises</td><td><code>nodes_minimax_h3.py:249</code></td></tr>
|
| <tr><td>truncate to generation length, then trim to <code>n % 17 == 5</code></td><td><code>nodes_minimax_h3.py:246-253</code></td></tr>
|
| <tr><td>shown to the encoder at 2 fps</td><td><code>nodes_minimax_h3.py:260-262</code></td></tr>
|
| <tr><td>a ref video's soundtrack takes an <code><Audio j></code> before standalone clips</td><td><code>nodes_minimax_h3.py:256-274</code></td></tr>
|
| <tr><td>refs and keyframes collide</td><td><code>model_base.py:2094</code> and <code>2098</code></td></tr>
|
| </table>
|
|
|
| </body></html>
|
|
|
