COMFYUI.md

Viggle Remake: ComfyUI, int8 and GGUF

Part of Viggle Remake; the reference sampler sample.py and its setup are in the README's Quick start.

ComfyUI

ComfyUI runs MiniMax-H3 natively. Remake adds a small node pack, comfyui/ComfyUI-Viggle-Remake, for four things it does the way sample.py does and stock nodes do not: the text-only presentation of the references, keyframes pinned at real frame indices, the source soundtrack kept as clean audio, and the starting noise for a seed. Everything else is stock ComfyUI. Tested with ComfyUI 0.42.0.

Install

cd ComfyUI

# the base model, as Comfy-Org's single-file repack (about 124 GB)
hf download Comfy-Org/MiniMax-H3 \
  diffusion_models/minimax_h3_fl2va_bf16.safetensors \
  text_encoders/qwen3vl_32b_minimax_h3_bf16.safetensors \
  vae/minimax_h3_video_vae_fp16.safetensors \
  vae/minimax_h3_audio_vae_fp32.safetensors \
  --local-dir models

# the LoRAs, the nodes, the workflows and the example inputs from this repository
hf download Viggle/Viggle-Remake --include "comfyui/*" "examples/inputs/*" --local-dir ./Viggle-Remake
cp -r Viggle-Remake/comfyui/ComfyUI-Viggle-Remake custom_nodes/
cp Viggle-Remake/comfyui/viggle_remake_*_comfyui.safetensors models/loras/

File	Folder	Node
minimax_h3_fl2va_bf16.safetensors	models/diffusion_models	Load Diffusion Model
qwen3vl_32b_minimax_h3_bf16.safetensors	models/text_encoders	Load CLIP, type minimax
minimax_h3_video_vae_fp16.safetensors, minimax_h3_audio_vae_fp32.safetensors	models/vae	Load VAE, one each
viggle_remake_swap_comfyui.safetensors, viggle_remake_edit_comfyui.safetensors	models/loras	LoraLoaderBypassModelOnly

Use the fl2va diffusion model, not ref2va: the LoRAs apply to the base transformer, the same weights sample.py loads. The two ComfyUI LoRA files hold the same weights as lora/swap and lora/edit, repacked to ComfyUI's layer names by comfyui/to_comfyui.py.

Run the examples

Copy the inputs under the names the workflows expect:

cp Viggle-Remake/examples/inputs/swap-pineapple-mascot/source.mp4 input/remake_swap_source.mp4
cp Viggle-Remake/examples/inputs/swap-pineapple-mascot/ref1.png   input/remake_swap_ref1.png
cp Viggle-Remake/examples/inputs/swap-pineapple-mascot/ref2.png   input/remake_swap_ref2.png
cp Viggle-Remake/examples/inputs/edit-remove-turbines/source.mp4  input/remake_edit_source.mp4
cp Viggle-Remake/examples/inputs/edit-remove-turbines/ref1.png    input/remake_edit_ref1.png
cp Viggle-Remake/examples/inputs/edit-remove-turbines/ref2.png    input/remake_edit_ref2.png

Then drag comfyui/workflows/remake_swap_ui.json or remake_edit_ui.json onto the canvas and queue it. The *_api.json files are the same graphs for ComfyUI's /prompt API. Each is the same render as examples/demo.sh with seed 0: 226 frames at 832×480, two references pinned as keyframes, the prompt from examples/prompts/ already filled in. The video lands in output/viggle_remake/.

The graph:

Load Diffusion Model → LoraLoaderBypassModelOnly → Viggle Remake Keyframe Until (0.9) → BasicGuider
Load Video → Get Video Components ─┐
Load Image ×2 ─────────────────────┼→ Viggle Remake Conditioning → BasicGuider, Viggle Remake Noise
Load CLIP, Load VAE ×2 ────────────┘
KSamplerSelect (euler) + ManualSigmas → SamplerCustomAdvanced → VAE Decode + VAE Decode Audio → Create Video (24 fps) → Save Video

Two settings change with the mode:

	swap	edit
LoRA	viggle_remake_swap_comfyui.safetensors	viggle_remake_edit_comfyui.safetensors
ManualSigmas	1.0, 0.990476, 0.965243, 0.923077, 0.8, 0.631579, 0.0	1.0, 0.983607, 0.96, 0.923077, 0.857143, 0.705882, 0.0

Your own clip

Viggle Remake Conditioning takes the same inputs as sample.py:

• source: the clip's frames. They are used as they come, so convert the clip to 24 fps first (ffmpeg -i in.mp4 -r 24 source.mp4); sample.py does this itself. A clip that is not already on the canvas is scaled and centre-cropped onto it.
• audio: the soundtrack kept in the output, usually the source's own from Get Video Components. Silence when it is not connected.
• ref_1 … ref_4: the reference images, <Picture 1> … <Picture 4> in the prompt.
• pins: one output frame per reference, in order, such as 12, 184 (--pin). Leave it empty to use the references without keyframes.
• frames: the output length, 17n+5 from 90 to 345 (--frames). The prompt's timestamps are written for it.
• short_edge: 480 (832×480 for 16:9) or 736 (1280×736).
• prompt: the six-section prompt, used verbatim.

Speed and memory

Measured on one B200 at 832×480 with 226 frames and two pinned references:

	ComfyUI	sample.py
One render, models already loaded	about 95 s (6 steps at about 13.5 s each, plus encoding and decoding)	about 58 s
First render, including loading	about 2.5–3 min	about 2 min
GPU memory	runs at full speed in 80 GB	87.2 GiB allocated, 91.0 GiB reserved (832×480, 226 frames)
System RAM	about 130 GB

ComfyUI loads weights onto the GPU as it needs them and keeps the rest in system RAM, so it runs on a smaller card than sample.py does. With ComfyUI capped to 92 and 76 GiB (--reserve-vram on a 183 GB card), the card peaked at 92.8 and 76.8 GiB and every step took as long as with no cap. Given a larger card, it keeps more of the weights resident (120–135 GB on the B200).

Differences from sample.py

• Load the LoRA with LoraLoaderBypassModelOnly, not LoraLoaderModelOnly. The plain loader adds the LoRA into the bf16 weights, and at bf16 precision much of the update rounds away: on average only 37–73% of it survives in the attention and MLP layers. On the swap example the merged render is 7% less sharp and further from sample.py (21.2 dB against 24.7 dB). The bypass loader keeps the LoRA as a separate branch, for 2–3% more time per render.
• Same seed, same video, not the same pixels. Viggle Remake Noise draws the same starting noise as sample.py for a given seed, so the two renders show the same result. They are not bit-identical, because the attention kernels and the order of operations differ. On the examples, ComfyUI's frames are 41.6 dB (edit) and 24.7 dB (swap) from sample.py's. The swap clip is the sensitive one: in ComfyUI alone, a change in the text embeddings the size of bf16 rounding moves it by about as much (27 dB). Sharpness is within 3% of sample.py, and the pinned keyframes are reproduced to within 0.2 dB.
• Slower. ComfyUI tries flash attention before cuDNN attention, which sample.py uses. On B200, putting cuDNN first in SDPA_BACKEND_PRIORITY in comfy/ops.py halves the step time (6.8 s against 13.5 s) and the render matches sample.py as closely (25.4 and 41.7 dB). We have only measured this on B200.
• Audio. Viggle Remake Conditioning encodes the soundtrack itself. ComfyUI's stock audio encode trims a track to whole 800-sample blocks by cutting from both ends, which shifts it by up to 400 samples against the video. The node pads the end instead, as sample.py does.

Smaller GPUs: int8 and GGUF

For cards below 80 GB, each mode also comes as a single transformer file for ComfyUI. Each file is the base transformer with that mode's LoRA already merged in, then quantized. It replaces both the base diffusion model and the LoRA loader in the ComfyUI graph above, so do not load a Remake LoRA on top of it.

The merge is done once, at fp32, and the sum is quantized directly. No bf16 copy is made in between, so these files do not lose part of the LoRA the way LoraLoaderModelOnly does.

Files

file	size	ComfyUI node
int8/viggle_remake_swap_pruned_int8_convrot.safetensors	21.0 GB	Load Diffusion Model (core)
int8/viggle_remake_edit_pruned_int8_convrot.safetensors	21.0 GB	Load Diffusion Model (core)
gguf/viggle_remake_swap-Q8_0.gguf, gguf/viggle_remake_edit-Q8_0.gguf	22.3 GB each	Unet Loader (GGUF), from ComfyUI-GGUF
gguf/viggle_remake_swap-Q6_K.gguf, gguf/viggle_remake_edit-Q6_K.gguf	17.6 GB each	Unet Loader (GGUF)
gguf/viggle_remake_swap-Q4_K_M.gguf, gguf/viggle_remake_edit-Q4_K_M.gguf	14.3 GB each	Unet Loader (GGUF)

• int8 uses ComfyUI's native int8 format, with the same layout as Comfy-Org's minimax_h3_fl2va_pruned_int8_convrot.
  • The four attention and MLP linears in each of the 50 blocks are int8 with one scale per output row (int8_tensorwise, convrot, group 256).
  • Every other tensor keeps the base file's precision.
  • adaLN is Comfy-Org's compact pruned form, unchanged, because the LoRAs do not touch adaLN.
• GGUF quantizes the same four linears per block.
  • Q8_0 and Q6_K use that type for all four.
  • Q4_K_M uses Q5_K for the qkv projection, Q6_K for the second MLP layer and Q4_K for the other two. This follows llama.cpp's treatment of the matching layers.
  • These stay unquantized: patch embedding, output layer, adaLN, norms, token refiner and condition projection.
• The GGUF files carry the wan architecture tag. ComfyUI-GGUF only checks this tag against a list of names it accepts, and ComfyUI identifies the model as MiniMax-H3 from its tensor names.

Install

cd ComfyUI

# int8: no extra node needed
hf download Viggle/Viggle-Remake --include "int8/*" --local-dir ./Viggle-Remake
mv Viggle-Remake/int8/*.safetensors models/diffusion_models/

# or GGUF, for example Q6_K
git clone https://github.com/city96/ComfyUI-GGUF custom_nodes/ComfyUI-GGUF
pip install -r custom_nodes/ComfyUI-GGUF/requirements.txt
hf download Viggle/Viggle-Remake --include "gguf/*Q6_K*" --local-dir ./Viggle-Remake
mv Viggle-Remake/gguf/*.gguf models/diffusion_models/

The text encoder, the VAEs, the node pack and the inputs are the same as in the ComfyUI section.

Change two nodes in the workflow.

1. Remove Load Diffusion Model and LoraLoaderBypassModelOnly.
2. Add Load Diffusion Model with the int8 file, or Unet Loader (GGUF) with a .gguf file.
3. Connect its MODEL output to Viggle Remake Keyframe Until.

Leave everything else as it is. Use the swap file with the swap sigmas and the edit file with the edit sigmas.

Which file for which card

We measured on B200s. A second process held the rest of the card, so ComfyUI saw only the stated amount of free memory. The test was the edit example at 226 frames with two pinned references and the bf16 text encoder. ComfyUI ran with default settings unless a flag is shown.

card	832×480	1280×736
12 GB	Q4_K_M or Q8_0 with --reserve-vram 8. int8: 124 frames only	not tested
16 GB	int8. Q4_K_M or Q6_K with --reserve-vram 6	Q4_K_M with --reserve-vram 10: 124 frames. At 226 frames it aborted even with --reserve-vram 12
24 GB	int8 or Q4_K_M. Q6_K or Q8_0 with --reserve-vram 6	Q4_K_M or Q8_0 with --reserve-vram 20. int8: 124 frames only
32 GB	int8 or Q8_0	Q8_0 with --reserve-vram 20. int8: up to 175 frames
48 GB	any	Q8_0. int8 runs out of memory at 226 frames
80 GB	bf16 + LoRA, as in the ComfyUI section	bf16 + LoRA. int8 runs out of memory at 226 frames

• At 832×480 on 16 GB or more, start with int8.
  • On these examples it scores the same as Q8_0, and it needs no flags.
  • ComfyUI loads it with dynamic VRAM: it moves each weight onto the card when it is needed and keeps the rest in system RAM.
• For 12 GB, or for 1280×736 below 80 GB, use a GGUF file with --reserve-vram. With these settings ComfyUI keeps the weights in system RAM and streams them to the card every step. That costs little time:
  • at 832×480, 15.9 s per step against about 14 s with the weights on the card;
  • at 1280×736, 67–72 s per step against 64 s.
• Why GGUF needs the flag. ComfyUI-GGUF puts as much of the model on the card as ComfyUI's memory estimate allows, and for MiniMax-H3 that estimate leaves out the reference and keyframe tokens.
  • When the weights fit but the activations do not, ComfyUI aborts with CUDA error: invalid argument instead of falling back to offloading.
  • --reserve-vram N keeps N GB free for the activations. If a run aborts, raise N.
• Why int8 stops at 1280×736 × 226 frames. Without a CUDA 13 build of PyTorch, comfy-kitchen falls back to a plain int8 matmul. That matmul keeps 32-bit sums for the whole layer and assembles the output in pieces, so one layer briefly needs several times the memory of its output. At 1280×736 × 226 frames this ran out of memory even with 80 GB free, while bf16 + LoRA fit in 80 GB and Q8_0 ran in 24 GB with --reserve-vram 20. We have not tested a CUDA 13 build.
• Q4_K_M is the smallest file. It is visibly further from the reference than int8 (see Quality).

Quality

Each file rendered both examples with seed 0 at 832×480 and 226 frames. We compared each render with sample.py's bf16 + LoRA render of the same inputs. The first comparison runs both in the same diffusers pipeline, so only the weights differ. PSNR is over every frame. Sharpness is the variance of the Laplacian, as a percentage of the reference's.

weights	swap, pinned	swap, refs only	edit, pinned	edit, refs only	sharpness
bf16 + LoRA (reference)	—	—	—	—	100%
int8	27.2 dB	34.0 dB	46.8 dB	49.7 dB	100–104%
Q8_0	26.9 dB	36.6 dB	49.8 dB	52.0 dB	99–102%
Q6_K	24.6 dB	30.5 dB	44.6 dB	46.0 dB	99–102%
Q4_K_M	20.6 dB	23.8 dB	38.3 dB	40.7 dB	97–109%
for scale: same bf16 weights, rounded differently	29.6 dB		50.4 dB		101%

• Pinned is the way the examples run. Refs only drops the keyframe pins, so the LoRA alone has to carry the edit. It is the harder test for a quantized file.
• Every file makes the edit: the turbines are removed and the pizza maker is replaced. Sharpness stays between 3% below and 9% above the reference.
• The swap clip is sensitive. The last row is the same model with its bf16 weights rounded a second, slightly different way, and that change alone moves the swap render to 29.6 dB. Read swap differences of a few dB as noise. The edit column separates the formats more clearly.
• In ComfyUI we compared each file with the LoRA applied through LoraLoaderBypassModelOnly, on the same graph:
  • int8 scored 24.0 dB on swap and 46.5 dB on edit.
  • Q8_0 scored 23.8 / 48.9.
  • Q6_K scored 20.0 / 43.6.
  • Q4_K_M scored 20.0 / 38.5.
  • For comparison, LoraLoaderModelOnly, which merges the LoRA at bf16, scored 22.0 / 46.1.
  • Contact sheets of every render match the reference by eye.
• At 1280×736 (edit, 226 frames, in ComfyUI against bypass-LoRA bf16), Q8_0 scored 42.8 dB and Q4_K_M 31.6 dB. Sharpness was 101% and 98% of the reference.
• Card size and offloading do not change the output. The same file and seed gave the same frames on a 12 GB budget as with the whole card free.

Text encoder

MiniMax-H3 uses the first 50 layers of Qwen3-VL 32B, and Comfy-Org's text-encoder files contain only those layers. They come in three versions:

• qwen3vl_32b_minimax_h3_bf16 (51.5 GB);
• _int8_convrot (27.1 GB);
• _nvfp4_awq (15.7 GB).

ComfyUI runs the encoder once, before sampling, and streams it through the card, so it does not decide which card you need. Every result in the card table used the bf16 encoder, except the 12 GB int8 runs, which used int8.

A smaller encoder saves disk and loading time. On 16 GB, the first render took 144 s with nvfp4 against 192 s with bf16.

We rendered both examples with each encoder and the int8 transformer, and compared each render with the bypass-LoRA render:

encoder	edit	swap
bf16	46.5 dB	24.0 dB
int8	47.0 dB	25.0 dB
nvfp4	46.7 dB	23.1 dB

Neither smaller encoder costs anything we can measure. Community GGUF versions of the encoder exist, but we have not tested them.

Speed and memory

All times are from one B200.

• 832×480, 226 frames.
  • With the whole file on the card, a step takes about 14 s for every file, so the six steps take about 85 s.
  • With the weights streamed from RAM, a step takes 15.9–17.4 s.
  • A first render, which includes loading the bf16 text encoder from local NVMe, takes 170–210 s from queue to saved frames.
• 1280×736, 226 frames. A step takes 64–72 s for every file and for bf16 + LoRA. A render takes 510–585 s.
• We have not timed consumer cards. Expect them to be slower.
• System RAM. We measured the peak resident memory of the ComfyUI process. Memory-mapped weight files count toward it.
  • 832×480: 47–55 GB, whichever encoder.
  • 1280×736 with an int8 or GGUF file: 55–80 GB.
  • 1280×736 with bf16 + LoRA: 127 GB.
• int8 ran on comfy-kitchen's fallback path. It ran on a PyTorch built for CUDA 12.8. There, comfy-kitchen's optimized int8 kernels are disabled and ComfyUI logs "You need pytorch with cu130 or higher". The fallback path gives correct output. We have not measured the optimized kernels.

Caveats

• Tested with ComfyUI 0.42.0, ComfyUI-GGUF at commit 6ea2651, PyTorch 2.9.1+cu128 and comfy-kitchen 0.2.40, on B200s only.
• The quality numbers come from the two example clips. The card table comes from the edit example only.
• The small cards were emulated. ComfyUI still reported the B200's full 183 GB as total VRAM, so it ran in its normal-VRAM mode. On a real 12–24 GB card, ComfyUI may pick its low-VRAM defaults by itself, and the flags above may need adjusting.
• GGUF runs log a harmless warning, "Mismatch dtype between input and weight … Cannot dispatch to fused implementation". It comes from the norm weights, which are kept at F32.