Instructions to use Viggle/Viggle-Animate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Viggle/Viggle-Animate with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Viggle/Viggle-Animate", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
| # Modified files | |
| Section III.2 of the MiniMax H3 Community License Agreement requires that modified | |
| files carry a prominent notice saying so. This file is that notice. | |
| Everything below is derived from | |
| [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3). | |
| ## `transformer/` β modified | |
| **Every weight file in `transformer/` has been modified.** It started as the base | |
| model's `transformer_ref/` (the `ref2va` transformer, 33.1 B parameters) and every | |
| parameter was updated by a full finetune on a character-replacement objective. The | |
| architecture, `config.json` and tensor names are unchanged, so it is a drop-in | |
| replacement for the base `transformer_ref/`; the numbers in it are not the base | |
| model's numbers. | |
| The file layout also differs: the finetune was written as one 61.7 GiB safetensors | |
| file and re-sharded here into 14 parts, because HuggingFace rejects single files | |
| above 50 GB. The 638 tensors and their contents are unchanged by that re-sharding. | |
| ## `lora/pytorch_lora_weights.safetensors` β new | |
| Not a MiniMax file. A rank-128 LoRA over 302 linear layers of `transformer/`, | |
| trained by us with DMD2 distillation. It is a delta on the finetuned transformer | |
| above, not on the base model β loading it onto stock `transformer_ref/` produces | |
| garbage. | |
| ## `assets/fixed_embed_fwd_anyframe.pt` β new | |
| Not a MiniMax file. A frozen 362 Γ 5120 text-conditioning tensor we computed once | |
| with the base model's own text encoder, so that inference never has to load | |
| Qwen3-VL. It is an *output* of the base model's encoder in the sense of Section | |
| I.12, computed from the prompt in `assets/fixed_prompt.txt`. | |
| ## `assets/fixed_prompt.txt` β new | |
| Not a MiniMax file. The prompt text the tensor above was computed from, included so | |
| that what conditions every render is readable rather than opaque. | |
| ## `inference/sample.py`, `examples/demo.sh` β new | |
| Not MiniMax files. Written by us against the public `diffusers` API. | |
| ## `LICENSE-CODE`, `NOTICE` β new | |
| Not MiniMax files. `LICENSE-CODE` is the Apache 2.0 text, and it covers `inference/` and | |
| `examples/` only β each file there carries an `SPDX-License-Identifier: Apache-2.0` header. | |
| `NOTICE` records that the weights are *not* Apache 2.0. `LICENSE` is the MiniMax H3 | |
| Community License Agreement itself, included unmodified as Section III.1 requires. | |
| ## `examples/media/` β new | |
| Two kinds of media here, with different provenance. | |
| **The demo** β `reference.png`, `output.mp4`, `before-after.png` β derives from | |
| `assets/ref2va.mp4`, a video MiniMax published with the base model. That clip is itself a | |
| MiniMax H3 generation rather than camera footage. All three files are a 512 Γ 768 portrait | |
| crop of it (`crop=512:768:389:0`, no scaling): | |
| - `reference.png` β the crop's first frame with the young man repainted as an invented | |
| elderly woman. Produced with OpenAI's `gpt-image-2`; the character is fictional and is | |
| not a real person or an existing property. | |
| - `output.mp4` β that reference propagated across 124 frames by this model. | |
| - `before-after.png` β frames from the driving crop above frames from the output. | |
| The driving clip itself is **not** bundled. `examples/demo.sh` rebuilds it, with the | |
| documented crop, from your own copy of the base model. | |
| **The comparison clips** β `compare-prop.mp4`, `compare-costume.mp4`, `compare-style.mp4` β are a | |
| different matter. Their driving videos are | |
| real filmed footage that we hold the rights to, and they are among the clips this model was | |
| evaluated against. Each file is a four-panel stack: painted reference, driving video, this model, | |
| Wan2.2-Animate-14B. The Wan2.2-Animate panels were rendered by us from the official | |
| [`Wan-AI/Wan2.2-Animate-14B`](https://huggingface.co/Wan-AI/Wan2.2-Animate-14B) weights and code, | |
| unmodified, at the replacement-mode settings its own README documents (20 steps, `sample_shift 5.0`, | |
| `--refert_num 1 --replace_flag --use_relighting_lora`, preprocessing at `--w_len 1 --h_len 1`) β | |
| that panel is Wan's output, not ours. | |
| Wan's `generate.py` hardcodes a 30 fps output timebase regardless of the source, so its raw files | |
| claim 4.10 s for motion that is 24 fps. The panels are retimed (`setpts`), not resampled, so no | |
| frames are dropped and both models play at the same speed. Every panel is letterboxed into the | |
| driving clip's own geometry; nothing is stretched. | |
| ## Not redistributed here | |
| The VAE, audio VAE, schedulers, text encoder, tokenizer and processor are **not** | |
| included in this repository and are not modified. They are loaded at runtime from | |
| your own copy of `MiniMaxAI/MiniMax-H3`, which you must download separately and | |
| under its own license terms. | |