Viggle-Remake / MODIFICATIONS.md
yycc's picture
Viggle Remake
21665fa
|
Raw History Blame Contribute Delete
4.79 kB

Modified files

Section III.2 of the MiniMax H3 Community License Agreement requires that modified files carry a prominent notice saying so. This file is that notice.

Everything below is derived from MiniMaxAI/MiniMax-H3.

Modified: the base transformer/, in int8/ and gguf/

The eight files in int8/ and gguf/ are modified copies of the base model's transformer/ (the FL2VA transformer). Each one is that transformer with one of the two LoRAs below merged into its weights at fp32, then quantized:

  • int8/viggle_remake_{swap,edit}_pruned_int8_convrot.safetensors: the merged transformer in ComfyUI's int8 format, following the layout of Comfy-Org's repackaged minimax_h3_fl2va_pruned_int8_convrot (itself derived from MiniMax H3). The attention and MLP linears are int8 with per-row scales; adaLN is that file's pruned form, unchanged.
  • gguf/viggle_remake_{swap,edit}-{Q8_0,Q6_K,Q4_K_M}.gguf: the merged transformer with the attention and MLP linears quantized to GGUF types, and the wan architecture tag so that ComfyUI-GGUF loads it.

Changes against the base transformer/: the LoRA deltas added to 302 linear layers, the quantization, tensor names repacked to ComfyUI's layout, and the adaLN pruning above. Nothing else.

Not modified

The two LoRAs below and the ComfyUI LoRA files are deltas that apply to the base model's transformer/ at load time. That transformer/ stays in your own copy of the base model and is not changed on disk.

lora/swap/pytorch_lora_weights.safetensors — new

Not a MiniMax file. Made by us. A PEFT LoRA, rank 256 and alpha 256, over 302 linear layers of the base model's transformer/ (the FL2VA transformer), for replacing people and characters. It is stored in bf16, except for the 4 tensors on proj_in and proj_out, which are fp32.

It is a delta on the stock base transformer/. Load it with load_lora_adapter(..., prefix=None) and put the result in the transformer_ref slot of the ref2va workflow, with keyframes pinned inside the generated span. We have not run it on any other base, including the base model's own transformer_ref/.

lora/edit/pytorch_lora_weights.safetensors — new

Not a MiniMax file. Made by us. Same shape, base and loading as lora/swap/, for general video edits: object swap, removal, insertion, background, weather and light.

lora_lowrank/{swap,edit}-r{128,64}/pytorch_lora_weights.safetensors — new

Not MiniMax files. Made by us. The two LoRAs above cut to rank 128 or 64 (alpha = rank): each layer's delta B @ A is replaced by its best rank-k approximation from an SVD, and nothing else changes. Same base, layers and loading as lora/swap/ and lora/edit/; inference/sample.py --rank 128 or --rank 64 picks them.

comfyui/ — new

Not MiniMax files. Made by us. viggle_remake_{swap,edit}_comfyui.safetensors hold the same LoRA weights as lora/swap and lora/edit, repacked to ComfyUI's layer names by comfyui/to_comfyui.py. The custom nodes and workflows are code.

inference/, examples/demo.sh, examples/prompts/ — new

Not MiniMax files. Written by us. The prompts follow the six-section format that MiniMax documents in the base model's docs/.

LICENSE-CODE, NOTICE — new

Not MiniMax files. LICENSE-CODE is the Apache 2.0 text, and it covers the code in inference/, examples/ and comfyui/ (the .py files and the workflow .json files) only. Each script there carries an SPDX-License-Identifier: Apache-2.0 header.

NOTICE records that the weights are not Apache 2.0.

LICENSE is the MiniMax H3 Community License Agreement itself, included unmodified as Section III.1 requires.

examples/media/, examples/inputs/ — new

Not MiniMax files. They show third-party footage rendered by this model. Most clips in examples/media/ are side-by-sides, the source on the left and the render on the right; white-house-funk.mp4 is the render alone. examples/inputs/ holds two source spans at 832 × 480 with their soundtracks, plus the frames we painted over them with OpenAI's gpt-image models.

The footage is credited in the "Example footage" section of README.md. Neither LICENSE nor LICENSE-CODE grants any rights to it.

Not redistributed here

The following are not included in this repository and are not modified:

  • the unmodified transformers, transformer/ and transformer_ref/ (the merged, quantized copies in int8/ and gguf/ are listed above);
  • the text encoder;
  • the VAE and audio VAE;
  • the schedulers;
  • the tokenizer and processor.

They are loaded at runtime from your own copy of MiniMaxAI/MiniMax-H3, which you must download separately and under its own licence terms.