Instructions to use Viggle/Viggle-Animate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Viggle/Viggle-Animate with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Viggle/Viggle-Animate", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Comparison: three Wan head-to-heads on fast motion + pose accuracy; name that strength up top
Browse files- .gitattributes +2 -0
- README.md +19 -6
- examples/media/compare-fastmotion.mp4 +3 -0
- examples/media/compare-highkick.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -59,3 +59,5 @@ examples/media/compare-gamechar.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
| 59 |
examples/media/teaser.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 60 |
MiniMaxH3TransformerBlock/package.pt2 filter=lfs diff=lfs merge=lfs -text
|
| 61 |
MiniMaxH3TokenRefinerBlock/package.pt2 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 59 |
examples/media/teaser.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 60 |
MiniMaxH3TransformerBlock/package.pt2 filter=lfs diff=lfs merge=lfs -text
|
| 61 |
MiniMaxH3TokenRefinerBlock/package.pt2 filter=lfs diff=lfs merge=lfs -text
|
| 62 |
+
examples/media/compare-fastmotion.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 63 |
+
examples/media/compare-highkick.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -27,7 +27,9 @@ Built on **[MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3)**
|
|
| 27 |
frames.** The motion, camera and timing underneath are untouched.
|
| 28 |
|
| 29 |
It reads no pose skeleton, no mask and no prompt, and runs no model but itself. Two inputs, three
|
| 30 |
-
forward passes, one GPU, 26 seconds a shot.
|
|
|
|
|
|
|
| 31 |
|
| 32 |
## Abstract
|
| 33 |
|
|
@@ -166,15 +168,26 @@ its 160 s.
|
|
| 166 |
Four panels each: **painted reference · driving video · this model · Wan2.2-Animate-14B**, the last
|
| 167 |
rendered from the official unmodified weights at its documented replacement settings.
|
| 168 |
|
| 169 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
|
| 171 |
-
<video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/compare-
|
| 172 |
|
| 173 |
-
|
|
|
|
|
|
|
|
|
|
| 174 |
|
| 175 |
-
|
|
|
|
|
|
|
|
|
|
| 176 |
|
| 177 |
-
|
|
|
|
| 178 |
|
| 179 |
## Generalization
|
| 180 |
|
|
|
|
| 27 |
frames.** The motion, camera and timing underneath are untouched.
|
| 28 |
|
| 29 |
It reads no pose skeleton, no mask and no prompt, and runs no model but itself. Two inputs, three
|
| 30 |
+
forward passes, one GPU, 26 seconds a shot. It is strongest exactly where replacement is hardest:
|
| 31 |
+
**fast motion**, and **pose transfer accurate enough to follow it** — whipping heads, full kicks,
|
| 32 |
+
jumps, tracked frame for frame instead of smeared through.
|
| 33 |
|
| 34 |
## Abstract
|
| 35 |
|
|
|
|
| 168 |
Four panels each: **painted reference · driving video · this model · Wan2.2-Animate-14B**, the last
|
| 169 |
rendered from the official unmodified weights at its documented replacement settings.
|
| 170 |
|
| 171 |
+
**The gap is widest under fast motion.** Whipping heads, swinging limbs and airborne frames are
|
| 172 |
+
where a replacement model has the least stable evidence to work from, and it shows: through the
|
| 173 |
+
fast frames Wan's panel goes soft and smears while ours stays sharp. **Pose accuracy is the other
|
| 174 |
+
half.** This model reads no skeleton, and still lands closer to the driving body — stance width,
|
| 175 |
+
arm extension, how far a limb actually travels, and on which frame it gets there.
|
| 176 |
|
| 177 |
+
<video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/compare-fastmotion.mp4"></video>
|
| 178 |
|
| 179 |
+
*Head-whip dance at speed, hoodie repainted to a sweater. Ours tracks the motion frame for frame;
|
| 180 |
+
the comparison blurs through it.*
|
| 181 |
+
|
| 182 |
+
<video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/compare-highkick.mp4"></video>
|
| 183 |
|
| 184 |
+
*A suited man throwing full kicks from a standing start. The extreme extensions arrive on the right
|
| 185 |
+
frame and at the right reach, and the stance stays as wide as the driving one.*
|
| 186 |
+
|
| 187 |
+
<video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/compare-cosplay.mp4"></video>
|
| 188 |
|
| 189 |
+
*A costume with a silhouette of its own — horns, fur coat, sash — carried through a tucked jump.
|
| 190 |
+
Ours holds both the outline and the pose in mid-air.*
|
| 191 |
|
| 192 |
## Generalization
|
| 193 |
|
examples/media/compare-fastmotion.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b79b19f959b370431dc5e090a80068c9046b07504064977d28d47d81e3ef50b7
|
| 3 |
+
size 1183899
|
examples/media/compare-highkick.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a137d9d19701eb1c75aa9e6b67776cfbf069619d9c0668885a717098f42fe37
|
| 3 |
+
size 847770
|