sprited commited on
Commit
3b5f45c
·
verified ·
1 Parent(s): b08a630

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +11 -5
README.md CHANGED
@@ -28,12 +28,18 @@ optimizer state stripped. Load with the trainers' `--init` (warm-start) or `--re
28
  | `unet_img128.pt` | UNet, min-SNR-5 | 128² | 20k | batch 128 | .226 / .073 / .22 (.203 / .047 / .23) |
29
  | `dit_img128_p4.pt` | DiT-FM patch 4 | 128² | 40k | batch 128 | .251 / .065 / .23 (.209 / .048 / .21) |
30
 
31
- **Video (64² × 8 frames, UNet 46 M, batch 16, unconditional) — interim snapshots, replaced by the finished runs:**
 
32
 
33
- | file | init | steps | note |
34
- |---|---|---|---|
35
- | `unet_vid64_scratch.pt` | random | 46k / 85k | from scratch; `unet_vid64_scratch_samples.gif` |
36
- | `unet_vid64_from_img.pt` | `unet_img64.pt` | 22k / 61k | Seedance-style stage 2 (image model video); `unet_vid64_from_img_samples.gif` |
 
 
 
 
 
37
 
38
  **Autoregressive video (the route's step 4) — `unet_ar64.pt`**: same UNet, `--ar_ctx 8 --frames 8 --stride 2` (8 context + 8 new
39
  frames per chunk, 10 fps), initialised from `unet_img64.pt`, 60k steps on one H100 (batch 16, ~5.5 h). Rolls out to any length:
 
28
  | `unet_img128.pt` | UNet, min-SNR-5 | 128² | 20k | batch 128 | .226 / .073 / .22 (.203 / .047 / .23) |
29
  | `dit_img128_p4.pt` | DiT-FM patch 4 | 128² | 40k | batch 128 | .251 / .065 / .23 (.209 / .048 / .21) |
30
 
31
+ **Video (64² × 8 frames @ 20 fps, UNet 46 M, batch 16, unconditional) — finished runs.** Oracle on 64 samples × 2 seeds vs 64 real
32
+ 8-frame windows (floor); FVD with a frame-repeated I3D (real-vs-real ≈ 110–120 in this setup):
33
 
34
+ | file | init | steps | tvr↓ | lie↓ | head-jitter↓ | angle-jerk↓ | FVD↓ |
35
+ |---|---|---|---|---|---|---|---|
36
+ | `unet_vid64_scratch.pt` | random | 85k | .171 | .101 | .40 | .125 | 199 |
37
+ | `unet_vid64_from_img.pt` | `unet_img64.pt` | 61k | .154 | .115 | .41 | .129 | 213 |
38
+ | real windows (floor) | — | — | .13 | .09–.11 | .32–.36 | .077 | 110–120 |
39
+
40
+ Read: warm-starting from the image model gets there **~2.5× sooner** (loss at 4k ≈ scratch at 10k) but ends in the same place — a
41
+ convergence-speed win, not a quality win. Per-frame anatomy is within ~0.03 of real frames; temporal jitter ~1.2× real; FVD ~80–100
42
+ above the real-vs-real floor. Sample GIFs: `unet_vid64_scratch_samples.gif`, `unet_vid64_from_img_samples.gif`.
43
 
44
  **Autoregressive video (the route's step 4) — `unet_ar64.pt`**: same UNet, `--ar_ctx 8 --frames 8 --stride 2` (8 context + 8 new
45
  frames per chunk, 10 fps), initialised from `unet_img64.pt`, 60k steps on one H100 (batch 16, ~5.5 h). Rolls out to any length: