yycc commited on
Commit
d29073f
Β·
verified Β·
1 Parent(s): 404f35e

docs: head-to-head vs Wan2.2-Animate, with clips

Browse files
.gitattributes CHANGED
@@ -42,3 +42,13 @@ examples/media/compare-horns.mp4 filter=lfs diff=lfs merge=lfs -text
42
  examples/media/compare-costume.mp4 filter=lfs diff=lfs merge=lfs -text
43
  examples/media/compare-prop.mp4 filter=lfs diff=lfs merge=lfs -text
44
  examples/media/compare-style.mp4 filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
42
  examples/media/compare-costume.mp4 filter=lfs diff=lfs merge=lfs -text
43
  examples/media/compare-prop.mp4 filter=lfs diff=lfs merge=lfs -text
44
  examples/media/compare-style.mp4 filter=lfs diff=lfs merge=lfs -text
45
+ examples/media/swap-airliner.mp4 filter=lfs diff=lfs merge=lfs -text
46
+ examples/media/swap-anime-duo.mp4 filter=lfs diff=lfs merge=lfs -text
47
+ examples/media/swap-claymation.mp4 filter=lfs diff=lfs merge=lfs -text
48
+ examples/media/swap-corgi.mp4 filter=lfs diff=lfs merge=lfs -text
49
+ examples/media/swap-panda.mp4 filter=lfs diff=lfs merge=lfs -text
50
+ examples/media/swap-penguin.mp4 filter=lfs diff=lfs merge=lfs -text
51
+ examples/media/swap-photoreal-duo.mp4 filter=lfs diff=lfs merge=lfs -text
52
+ examples/media/swap-robot.mp4 filter=lfs diff=lfs merge=lfs -text
53
+ examples/media/swap-wushu-animals.mp4 filter=lfs diff=lfs merge=lfs -text
54
+ examples/media/swap-wushu-robots.mp4 filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -101,6 +101,77 @@ These three are selected. Across the 24 clips we ran head to head, Wan came out
101
  tight talking-head shot, where its explicit face channel tracks expression that we render flat. That
102
  is the same weakness listed under Limitations below. Roughly six were a wash.
103
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  ## What this is
105
 
106
  Two parts, both derived from [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3)'s
@@ -169,9 +240,22 @@ in **40–60 s**. That quantization path is not shipped here.
169
  - **More steps is not better.** `--steps 4` is the operating point, not a degraded `--steps 12`. It
170
  renders sharper than its own teacher β€” Laplacian variance 16–50 % higher β€” which reads as detail
171
  sometimes and over-sharpening other times, and worsens with more steps.
 
 
 
 
 
 
 
 
 
 
172
  - **No facial-landmark channel**, so tight close-up dialogue is the weakest case β€” Wan drives the
173
- face from explicit landmarks and is visibly more expressive there. Trained on one
174
- subject filling a portrait frame: wide shots, crowds and landscape framing are out of distribution.
 
 
 
175
  - The driving clip's audio is dropped at input; the model emits its own track, and the fixed prompt
176
  asks for silence.
177
 
 
101
  tight talking-head shot, where its explicit face channel tracks expression that we render flat. That
102
  is the same weakness listed under Limitations below. Roughly six were a wash.
103
 
104
+ ## What you can paint in
105
+
106
+ The model is never told what the new character is. It sees one edited frame and has to carry it
107
+ through five seconds of motion it did not choreograph β€” so the question is how far from a person
108
+ that character can get. Three panels each: **painted reference Β· driving video Β· this model.** All
109
+ ten are single attempts at both stages: one paint, one render, `--steps 4 --seed 42`, no best-of-N.
110
+
111
+ ### Animals
112
+
113
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-corgi.mp4"></video>
114
+
115
+ A corgi in the driver's own tank top and trousers. The ears track head rotation independently β€”
116
+ nothing in the driving clip has ears to copy.
117
+
118
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-panda.mp4"></video>
119
+
120
+ A giant panda in a denim jacket and ripped jeans. Clothing deforms onto fur without arguing about
121
+ it, and the eye patches stay symmetric through the hands-over-face beat.
122
+
123
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-penguin.mp4"></video>
124
+
125
+ An emperor penguin in a red floral suit. The paint put the flippers where the arms were, and the
126
+ render animates them *as* arms β€” they sweep on the arms' trajectories while keeping a flipper's shape.
127
+
128
+ ### Not humanoid
129
+
130
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-airliner.mp4"></video>
131
+
132
+ The hardest case here. A man standing with his arms out becomes an airliner whose wings are his arms
133
+ and whose landing gear are his legs, and that binding survives the whole take. The geometry is done
134
+ in the paint β€” the wings placed exactly where the arms were β€” and the model only has to keep it.
135
+
136
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-robot.mp4"></video>
137
+
138
+ Specular metal has to relight itself as the body turns rather than just deform. The cable joints at
139
+ the elbows and knees stay put instead of migrating.
140
+
141
+ ### Stylised
142
+
143
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-claymation.mp4"></video>
144
+
145
+ A stop-motion plasticine character in front of a kitchen that stays photographic. The style boundary
146
+ holds for the whole clip β€” fingerprints and seams on the figure, real appliances behind it.
147
+
148
+ ### Two characters, one edit
149
+
150
+ The prompt has to bind each new character to a specific person β€” "the girl on the left", "the athlete
151
+ on the right" β€” or the image editor swaps one and leaves the other. All four of these held.
152
+
153
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-anime-duo.mp4"></video>
154
+
155
+ Two guitarists become a cat and a corgi, drawn in the clip's own cel-shaded style, so the swap has to
156
+ survive as line art rather than as texture. The fretting hands stay on the necks.
157
+
158
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-photoreal-duo.mp4"></video>
159
+
160
+ The same clip with photoreal fur in front of a hand-drawn night sky. Neither side pulls the other
161
+ across: the background stays anime and the characters stay photographic.
162
+
163
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-wushu-animals.mp4"></video>
164
+
165
+ A panda and a tiger through a synchronised *duilian* routine, in a wide arena shot where each figure
166
+ is a few dozen pixels tall β€” landscape framing and two subjects, both outside the training
167
+ distribution. They stay distinct from each other and neither swaps back.
168
+
169
+ <video controls muted loop playsinline width="100%" src="https://huggingface.co/Viggle/Viggle-Animate/resolve/main/examples/media/swap-wushu-robots.mp4"></video>
170
+
171
+ Same clip, two robots that differ only in metal β€” gold on the left, chrome on the right. They stay
172
+ told apart through the jumps and stance changes, which is what would fail if the pair were being
173
+ treated as one region.
174
+
175
  ## What this is
176
 
177
  Two parts, both derived from [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3)'s
 
240
  - **More steps is not better.** `--steps 4` is the operating point, not a degraded `--steps 12`. It
241
  renders sharper than its own teacher β€” Laplacian variance 16–50 % higher β€” which reads as detail
242
  sometimes and over-sharpening other times, and worsens with more steps.
243
+ - **The reference defines only the views it contains.** A character painted from behind has no
244
+ front, and when the driving clip rotates to reveal one the model fills it from its own prior
245
+ rather than from your character. On a figure skater who spins to face camera, a swapped-in cat is
246
+ a cat while her back is turned, a generic woman at the moment she faces us, and a cat again when
247
+ she turns away. Prefer driving clips that keep one side to camera, or paint a frame that shows the
248
+ view the clip will eventually reach.
249
+ - **A silhouette far from the driver's survives as colour, not as geometry.** A LEGO minifigure on
250
+ that same skating clip kept its palette and its yellow claw hands, but the moulded plastic body
251
+ reverted to human anatomy. The airliner above works because the paint bound its wings to the arms
252
+ β€” geometry the *driving pose* can still explain.
253
  - **No facial-landmark channel**, so tight close-up dialogue is the weakest case β€” Wan drives the
254
+ face from explicit landmarks and is visibly more expressive there.
255
+ - **Out of distribution is not the same as broken.** Training was one subject filling a portrait
256
+ frame. The last four gallery clips are two subjects in landscape framing, including a wide arena
257
+ shot, and they hold β€” but that is untuned territory with no evaluation behind it, and crowds
258
+ remain untested.
259
  - The driving clip's audio is dropped at input; the model emits its own track, and the fixed prompt
260
  asks for silence.
261
 
examples/media/swap-airliner.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a7a31fd225e258633e208f13c18ff8523517834a9ea2c67b97ed060972becf3
3
+ size 397277
examples/media/swap-anime-duo.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:205700e55841a5fd22f517c6eb08c447068d0417a8e902bc91c863dede743d86
3
+ size 108986
examples/media/swap-claymation.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd08b83d8a57e77408203710cf55eccdf8d1ee136c22123cc42ed8ac14434528
3
+ size 286267
examples/media/swap-corgi.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5954aeab0360ac8c1c5c62b5d35935520ac0feffdb8d11b8c55a6ff84f9d9239
3
+ size 272402
examples/media/swap-panda.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bec174f093bf553b39d1f022b6fedd46b9616a6ac30fa10a939753cf5e4a5150
3
+ size 390485
examples/media/swap-penguin.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea2ccf8ce410331d194403903e2d1e836079b2d8fbafebf113265e52aafb3cd0
3
+ size 346127
examples/media/swap-photoreal-duo.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2d7c7fb36c5b2779176484b0a2fcb1735578ea3cad2fa0a2c0b057218881554
3
+ size 108311
examples/media/swap-robot.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8147fb8f85eca14e372ecd26f6d7b4556c649f5c980e0dd514a582ac8619077
3
+ size 194541
examples/media/swap-wushu-animals.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ed178d0c88053df75803d904bef26c83c81da78c33d575ec8f5897701b622a7
3
+ size 204001
examples/media/swap-wushu-robots.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63cc5dfe297ca472b7c54a9ef32aaa575f84e6cc1088a42d01236e40888785b2
3
+ size 201465