| --- |
| license: other |
| license_name: minimax-h3-community-license |
| license_link: LICENSE |
| base_model: MiniMaxAI/MiniMax-H3 |
| pipeline_tag: video-to-video |
| library_name: minimax-h3 |
| tags: |
| - video-editing |
| - character-replacement |
| - video-to-video |
| - lora |
| --- |
| |
| # Viggle Remake |
|
|
| ### Remake any video. Just ask. |
|
|
| <h2 align="center"><a href="https://viggle.ai/meme/app/home?ui=v2">👉 Try Viggle Remake in the Viggle app: no setup, no GPU</a></h2> |
|
|
| <p align="center"><a href="https://viggle.ai/meme/app/home?ui=v2"><img src="https://img.shields.io/badge/Open_the_Viggle_app-000000?style=for-the-badge" alt="Open the Viggle app"></a></p> |
|
|
| <video autoplay controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Remake/resolve/main/examples/media/lala-land.mp4"></video> |
|
|
| <sub><b>La La Land.</b> One shot from <i>La La Land</i> (2016), remade 22 ways: recast, new outfits, weather, things added and removed, |
| a new style. Unmute to hear it. Footage and music from <i>La La Land</i> (2016), © its owners. Not affiliated with or endorsed by them.</sub> |
|
|
| <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Remake/resolve/main/examples/media/white-house-funk.mp4"></video> |
|
|
| <sub><b>White House Funk.</b> A 116-second music video recast with <code>lora/swap</code>, 69 cuts, original |
| soundtrack (unmute). <b>AI-generated parody: the people shown did not take part in, approve or endorse it.</b> |
| Source: Mark Ronson ft. Bruno Mars, <i>Uptown Funk</i> (2014). To make one yourself, see [the MCP](#agent-the-mcp-server).</sub> |
|
|
| <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Remake/resolve/main/examples/media/matrix-burly-brawl.mp4"></video> |
|
|
| <sub><b>Burly Brawl.</b> <i>The Matrix Reloaded</i> (2003), Neo as a tuxedo cat and every Agent Smith as a |
| shiba inu, 101 seconds with <code>lora/swap</code> and the original soundtrack.</sub> |
|
|
| **Viggle Remake is built agent-first.** Give an agent (Claude Code, codex, anything that speaks MCP) a video and one |
| sentence: "recast the band as US politicians", "make it snow", "turn everyone into cats". It finds the cuts, paints |
| the keyframes, renders every window, checks its own work and hands back the finished film, with the original motion, |
| camera, timing and soundtrack. [Set up the agent](#agent-the-mcp-server). |
|
|
| Recast people · swap outfits · add or remove objects · change the weather, light, setting or style · whole films, |
| not just clips. |
|
|
| Prefer to drive it by hand? Write a prompt, paint one to four frames, and [`sample.py`](#quick-start) renders the shot. |
|
|
| Remake is the "Viggle-Animate v2" previewed on the Viggle-Animate card: Animate replaces a character; Remake also |
| swaps, removes and inserts objects, and changes the setting, the weather or the light. |
|
|
| ## Quick start |
|
|
| > **Just want to use it?** The [Viggle app](https://viggle.ai/meme/app/home?ui=v2) runs Remake for you. The rest of this section is for running it on your own GPUs. |
|
|
| You need the base model (about 144 GB of its files) and this repository's LoRAs and sampler: |
|
|
| ```bash |
| hf download MiniMaxAI/MiniMax-H3 --local-dir ./MiniMax-H3 \ |
| --include "modular_model_index.json" "transformer/*" "text_encoder/*" "tokenizer/*" \ |
| "processor/*" "vae/*" "audio_vae/*" "scheduler/*" "audio_scheduler/*" "docs/*" |
| |
| hf download Viggle/Viggle-Remake --local-dir ./Viggle-Remake \ |
| --include "lora/*" "inference/*" "examples/*" "requirements.txt" # 5.5 GB; the whole repo is ~210 GB |
| pip install -r Viggle-Remake/requirements.txt # plus an ffmpeg binary on PATH |
| |
| python Viggle-Remake/inference/sample.py --mode swap \ |
| --video source.mp4 --prompt prompt.txt \ |
| --ref ref1.png --ref ref2.png --pin 12 --pin 96 \ |
| --frames 124 --short-edge 480 --seed 0 \ |
| --base ./MiniMax-H3 --lora-repo ./Viggle-Remake --out out.mp4 |
| ``` |
|
|
| `sample.py` renders one window of 90–345 frames (`--frames` = 17n + 5, at 24 fps; we recommend up to 294) |
| with the source soundtrack; `--short-edge 736` renders 1280×736. Its defaults are the settings every clip |
| here used. [`examples/demo.sh`](examples/demo.sh) reruns one example of each mode from |
| [`examples/inputs/`](examples/inputs): if the turbines are still there, or the pizza maker is unchanged, |
| the LoRA did not load. |
|
|
| **Smaller LoRAs:** `--rank 128` or `--rank 64` loads `lora_lowrank/` (1.34 / 0.67 GB), near-identical output. |
|
|
| ## ComfyUI and smaller GPUs |
|
|
| Node pack, workflows, and int8/GGUF files down to 12 GB: [`COMFYUI.md`](COMFYUI.md). |
|
|
| ## Agent: the MCP server |
|
|
| [`mcp/`](mcp) lets a coding agent (Claude Code, codex) remake a whole film: it cuts the windows, paints |
| the keyframes, writes the prompts, renders on your GPUs and splices the result with the soundtrack. |
|
|
| ```bash |
| hf download Viggle/Viggle-Remake --include "mcp/*" --local-dir viggle-remake # the MCP only, not the weights |
| cd viggle-remake/mcp |
| pip install -e . # the server |
| pip install -e '.[worker]' # GPU machines: also the render worker |
| ``` |
|
|
| Do not `git clone` this repository: with git-lfs it downloads all ~210 GB of weights. |
|
|
| To remake White House Funk (source, eight characters and task in |
| [`mcp/examples/white-house-funk/`](mcp/examples/white-house-funk)): |
|
|
| ```bash |
| for g in 0 1; do viggle-remake-worker --gpu $g > worker_$g.log 2>&1 & done # one render worker per GPU |
| examples/white-house-funk/run.sh # codex (ChatGPT plan login) runs the whole film: ~1.5 h on two GPUs |
| ``` |
|
|
| The agent paints its own keyframes, so every run differs in detail. Setup, tools and paint backends: |
| [`mcp/README.md`](mcp/README.md). |
|
|
| ## Two modes |
|
|
| Each mode is a LoRA (rank 256) on MiniMax-H3's base `transformer`; `--mode` picks one. |
|
|
| - **`--mode swap`** (`lora/swap`): replace people and characters, one or several at once. |
| - **`--mode edit`** (`lora/edit`): everything else: swap, remove or insert objects, change the background, weather or light. |
|
|
| ## Inputs: painted frames and the prompt |
|
|
| Pass one to four painted frames (`--ref`, called `<Picture N>` in the prompt), optionally each pinned |
| to the source frame it was painted from (`--pin`), and a prompt in MiniMax-H3's six-section format. |
| How to paint them, when to pin, and how to write the prompt: [`PROMPTING.md`](PROMPTING.md). |
|
|
| ## Limitations |
|
|
| - **About 12 seconds per render.** Longer videos are split into windows, at cuts where possible (the MCP does this). |
| - **Heavy.** The 144 GB base and its text encoder are required. `sample.py` needs one 80 GB+ GPU at bf16; see [`COMFYUI.md`](COMFYUI.md) for smaller cards. |
| - **The six-section prompt is required** ([`PROMPTING.md`](PROMPTING.md)); timestamps must fit `--frames`. |
| - **The painted frames decide identity and look**; pinned frames also fix the composition at that frame. |
| - **Sound is carried, not made**: the output has the source soundtrack; a silent source gives a silent result. |
| - **Known failures:** a non-humanoid character in one long take; small artifacts in fine detail; shots drifting in time when a multi-shot clip is rendered without pins. |
|
|
| ## Intended use |
|
|
| This model exists to re-shoot footage you have the right to change: recasting a consenting |
| performer, or changing a set, a prop or the weather. It will just as readily put a real person's |
| likeness into footage they never agreed to appear in. Note where that decision is made: **the |
| identity comes from the frames you paint**. Any image editor's safeguards sit upstream of this |
| model, and none of them are in it. It cannot verify identity or consent. Do not run it on people who |
| have not agreed to it, label what you generate as AI-generated, and see Section V.5 of the Agreement |
| if you offer this as a service. |
|
|
| ## Example footage |
|
|
| The source footage in `examples/` and `mcp/examples/` is third-party, © its owners, and credited in |
| [`examples/CREDITS.md`](examples/CREDITS.md). |
|
|
| ## What this repository contains |
|
|
| | path | contents | |
| |---|---| |
| | `lora/swap/`, `lora/edit/` | the two modes, PEFT LoRA, rank 256 (2.67 GB each) | |
| | `lora_lowrank/` | the same at rank 128 and 64 | |
| | `comfyui/`, `int8/`, `gguf/` | ComfyUI LoRAs, node pack and workflows; merged int8 and GGUF transformers. See [`COMFYUI.md`](COMFYUI.md) | |
| | `inference/`, `requirements.txt` | `sample.py`, the reference sampler, and its package set | |
| | `examples/` | `demo.sh`, the example prompts and inputs, the clips on this card | |
| | `mcp/` | the MCP server and render worker, with a full example run | |
|
|
| ## License |
|
|
| The weights, including the merged `int8/` and `gguf/` files, are a Model Derivative of MiniMax H3, so the |
| [MiniMax H3 Community License](LICENSE) applies to them. Read it before you redistribute them or |
| ship a product on them. What we added is listed in [`MODIFICATIONS.md`](MODIFICATIONS.md). |
|
|
| The code in [`inference/`](inference), [`examples/`](examples), [`comfyui/`](comfyui) and [`mcp/`](mcp) is Apache 2.0 |
| ([`LICENSE-CODE`](LICENSE-CODE)). That licence covers the scripts and prompts only. Neither licence |
| grants any rights to the third-party footage in `examples/media/`, `examples/inputs/` and |
| `mcp/examples/`, or to the likenesses of the people shown. |
|
|
| ### Attribution |
|
|
| If you build a product or service on Viggle Remake, please credit it where your users will see it, for example |
| in the UI or the credits: |
|
|
| ```text |
| Powered by Viggle Remake (https://huggingface.co/Viggle/Viggle-Remake) |
| ``` |
|
|
| Separately, if you ship a product or service on these weights, the MiniMax H3 Community License requires you to: |
|
|
| - display "MiniMax H3" prominently in the user interface of a commercial product or service (Section IV.2); |
| - give a copy of the [License](LICENSE) to everyone you distribute the weights to, mark any files you modified, and |
| include the [`NOTICE`](NOTICE) file (Section III); |
| - get written authorization from MiniMax first if your products and services make more than US$20 million a year |
| (Section IV.1). |
|
|
| The Viggle Remake weights are Copyright © 2026 Viggle AI. "Viggle" is a name of Viggle AI. Do not use it in a way that |
| suggests your product is made or endorsed by Viggle. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{viggle2026remake, |
| title = {Viggle Remake: Remake Any Video. Just Ask.}, |
| author = {Viggle Research}, |
| year = {2026}, |
| url = {https://huggingface.co/Viggle/Viggle-Remake} |
| } |
| ``` |
|
|
| ## Links |
|
|
| - [Viggle app](https://viggle.ai/meme/app/home?ui=v2) |
|
|
