Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -28,12 +28,18 @@ optimizer state stripped. Load with the trainers' `--init` (warm-start) or `--re
|
|
| 28 |
| `unet_img128.pt` | UNet, min-SNR-5 | 128² | 20k | batch 128 | .226 / .073 / .22 (.203 / .047 / .23) |
|
| 29 |
| `dit_img128_p4.pt` | DiT-FM patch 4 | 128² | 40k | batch 128 | .251 / .065 / .23 (.209 / .048 / .21) |
|
| 30 |
|
| 31 |
-
**Video (64² × 8 frames, UNet 46 M, batch 16, unconditional) —
|
|
|
|
| 32 |
|
| 33 |
-
| file | init | steps |
|
| 34 |
-
|---|---|---|---|
|
| 35 |
-
| `unet_vid64_scratch.pt` | random |
|
| 36 |
-
| `unet_vid64_from_img.pt` | `unet_img64.pt` |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
**Autoregressive video (the route's step 4) — `unet_ar64.pt`**: same UNet, `--ar_ctx 8 --frames 8 --stride 2` (8 context + 8 new
|
| 39 |
frames per chunk, 10 fps), initialised from `unet_img64.pt`, 60k steps on one H100 (batch 16, ~5.5 h). Rolls out to any length:
|
|
|
|
| 28 |
| `unet_img128.pt` | UNet, min-SNR-5 | 128² | 20k | batch 128 | .226 / .073 / .22 (.203 / .047 / .23) |
|
| 29 |
| `dit_img128_p4.pt` | DiT-FM patch 4 | 128² | 40k | batch 128 | .251 / .065 / .23 (.209 / .048 / .21) |
|
| 30 |
|
| 31 |
+
**Video (64² × 8 frames @ 20 fps, UNet 46 M, batch 16, unconditional) — finished runs.** Oracle on 64 samples × 2 seeds vs 64 real
|
| 32 |
+
8-frame windows (floor); FVD with a frame-repeated I3D (real-vs-real ≈ 110–120 in this setup):
|
| 33 |
|
| 34 |
+
| file | init | steps | tvr↓ | lie↓ | head-jitter↓ | angle-jerk↓ | FVD↓ |
|
| 35 |
+
|---|---|---|---|---|---|---|---|
|
| 36 |
+
| `unet_vid64_scratch.pt` | random | 85k | .171 | .101 | .40 | .125 | 199 |
|
| 37 |
+
| `unet_vid64_from_img.pt` | `unet_img64.pt` | 61k | .154 | .115 | .41 | .129 | 213 |
|
| 38 |
+
| real windows (floor) | — | — | .13 | .09–.11 | .32–.36 | .077 | 110–120 |
|
| 39 |
+
|
| 40 |
+
Read: warm-starting from the image model gets there **~2.5× sooner** (loss at 4k ≈ scratch at 10k) but ends in the same place — a
|
| 41 |
+
convergence-speed win, not a quality win. Per-frame anatomy is within ~0.03 of real frames; temporal jitter ~1.2× real; FVD ~80–100
|
| 42 |
+
above the real-vs-real floor. Sample GIFs: `unet_vid64_scratch_samples.gif`, `unet_vid64_from_img_samples.gif`.
|
| 43 |
|
| 44 |
**Autoregressive video (the route's step 4) — `unet_ar64.pt`**: same UNet, `--ar_ctx 8 --frames 8 --stride 2` (8 context + 8 new
|
| 45 |
frames per chunk, 10 fps), initialised from `unet_img64.pt`, 60k steps on one H100 (batch 16, ~5.5 h). Rolls out to any length:
|