Instructions to use Viggle/Viggle-Animate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Viggle/Viggle-Animate with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Viggle/Viggle-Animate", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Modified files
Section III.2 of the MiniMax H3 Community License Agreement requires that modified files carry a prominent notice saying so. This file is that notice.
Everything below is derived from
MiniMaxAI/MiniMax-H3.
transformer/ β modified
Every weight file in transformer/ has been modified. It started as the base
model's transformer_ref/ (the ref2va transformer, 33.1 B parameters) and every
parameter was updated by a full finetune on a character-replacement objective. The
architecture, config.json and tensor names are unchanged, so it is a drop-in
replacement for the base transformer_ref/; the numbers in it are not the base
model's numbers.
The file layout also differs: the finetune was written as one 61.7 GiB safetensors file and re-sharded here into 14 parts, because HuggingFace rejects single files above 50 GB. The 638 tensors and their contents are unchanged by that re-sharding.
lora/pytorch_lora_weights.safetensors β new
Not a MiniMax file. A rank-128 LoRA over 302 linear layers of transformer/,
trained by us with DMD2 distillation. It is a delta on the finetuned transformer
above, not on the base model β loading it onto stock transformer_ref/ produces
garbage.
assets/fixed_embed_fwd_anyframe.pt β new
Not a MiniMax file. A frozen 362 Γ 5120 text-conditioning tensor we computed once
with the base model's own text encoder, so that inference never has to load
Qwen3-VL. It is an output of the base model's encoder in the sense of Section
I.12, computed from the prompt in assets/fixed_prompt.txt.
assets/fixed_prompt.txt β new
Not a MiniMax file. The prompt text the tensor above was computed from, included so that what conditions every render is readable rather than opaque.
inference/sample.py, examples/demo.sh β new
Not MiniMax files. Written by us against the public diffusers API.
LICENSE-CODE, NOTICE β new
Not MiniMax files. LICENSE-CODE is the Apache 2.0 text, and it covers inference/ and
examples/ only β each file there carries an SPDX-License-Identifier: Apache-2.0 header.
NOTICE records that the weights are not Apache 2.0. LICENSE is the MiniMax H3
Community License Agreement itself, included unmodified as Section III.1 requires.
examples/media/ β new
Two kinds of media here, with different provenance.
The demo β reference.png, output.mp4, before-after.png β derives from
assets/ref2va.mp4, a video MiniMax published with the base model. That clip is itself a
MiniMax H3 generation rather than camera footage. All three files are a 512 Γ 768 portrait
crop of it (crop=512:768:389:0, no scaling):
reference.pngβ the crop's first frame with the young man repainted as an invented elderly woman. Produced with OpenAI'sgpt-image-2; the character is fictional and is not a real person or an existing property.output.mp4β that reference propagated across 124 frames by this model.before-after.pngβ frames from the driving crop above frames from the output.
The driving clip itself is not bundled. examples/demo.sh rebuilds it, with the
documented crop, from your own copy of the base model.
The comparison clips β compare-prop.mp4, compare-costume.mp4, compare-style.mp4 β are a
different matter. Their driving videos are
real filmed footage that we hold the rights to, and they are among the clips this model was
evaluated against. Each file is a four-panel stack: painted reference, driving video, this model,
Wan2.2-Animate-14B. The Wan2.2-Animate panels were rendered by us from the official
Wan-AI/Wan2.2-Animate-14B weights and code,
unmodified, at the replacement-mode settings its own README documents (20 steps, sample_shift 5.0,
--refert_num 1 --replace_flag --use_relighting_lora, preprocessing at --w_len 1 --h_len 1) β
that panel is Wan's output, not ours.
Wan's generate.py hardcodes a 30 fps output timebase regardless of the source, so its raw files
claim 4.10 s for motion that is 24 fps. The panels are retimed (setpts), not resampled, so no
frames are dropped and both models play at the same speed. Every panel is letterboxed into the
driving clip's own geometry; nothing is stretched.
Not redistributed here
The VAE, audio VAE, schedulers, text encoder, tokenizer and processor are not
included in this repository and are not modified. They are loaded at runtime from
your own copy of MiniMaxAI/MiniMax-H3, which you must download separately and
under its own license terms.