The 25-second clip, with the schedule that actually fixes it
Browse files- .gitattributes +2 -0
- samples/long/ghosting-fixed-vs-adaptive.png +3 -0
- samples/long/idol25-final.mp4 +3 -0
- samples/long/schedule.md +57 -0
.gitattributes
CHANGED
|
@@ -122,3 +122,5 @@ samples/long/ab.png filter=lfs diff=lfs merge=lfs -text
|
|
| 122 |
samples/long/crop_ab.png filter=lfs diff=lfs merge=lfs -text
|
| 123 |
samples/long/idol25-fit.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 124 |
samples/long/idol25-nofit.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 122 |
samples/long/crop_ab.png filter=lfs diff=lfs merge=lfs -text
|
| 123 |
samples/long/idol25-fit.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 124 |
samples/long/idol25-nofit.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 125 |
+
samples/long/idol25-final.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 126 |
+
samples/long/ghosting-fixed-vs-adaptive.png filter=lfs diff=lfs merge=lfs -text
|
samples/long/ghosting-fixed-vs-adaptive.png
ADDED
|
Git LFS Details
|
samples/long/idol25-final.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:714b6888de93aa37c46bdc55be14b4adc175312cb7d6e10733b07dccd8213a3c
|
| 3 |
+
size 19826732
|
samples/long/schedule.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# What actually breaks a 25-second clip
|
| 2 |
+
|
| 3 |
+
## Not the positions — the schedule
|
| 4 |
+
|
| 5 |
+
`ltx_render_clips` samples with `DISTILLED_SIGMAS`, nine fixed numbers solved
|
| 6 |
+
for the clip lengths the distillation used. `LTXVScheduler` instead derives its
|
| 7 |
+
shift from the **latent's own token count**. A 25-second clip carries about 4.4x
|
| 8 |
+
the tokens of a 2-second one, and the fixed schedule does not know that.
|
| 9 |
+
|
| 10 |
+
The result is a sample that has not converged. On screen it is ghosting —
|
| 11 |
+
furniture and curtains rendered semi-transparent, overlapping themselves — and
|
| 12 |
+
roughly half the colour.
|
| 13 |
+
|
| 14 |
+
| 25 s, 512x320, same prompt and seed | saturation | contrast | ghosting |
|
| 15 |
+
|---|---:|---:|---|
|
| 16 |
+
| fixed 8 (what this project shipped) | 0.094 | 0.108 | severe |
|
| 17 |
+
| adaptive 8 | 0.127 | 0.124 | gone |
|
| 18 |
+
| **adaptive 20** | 0.108 | **0.166** | gone, hands and fabric resolved |
|
| 19 |
+
| *15 s reference* | *0.208* | *0.161* | — |
|
| 20 |
+
|
| 21 |
+
Final render, 25 s at 1024x640 with the second pass: **saturation 0.161,
|
| 22 |
+
contrast 0.180** — colour close to the fifteen-second reference and contrast
|
| 23 |
+
above it.
|
| 24 |
+
|
| 25 |
+
Long clips now switch to the adaptive schedule automatically past 15 s, in
|
| 26 |
+
`ltx_av_generate`. A caller passing `steps` explicitly still overrides it.
|
| 27 |
+
|
| 28 |
+
## Three times a metric pointed the wrong way
|
| 29 |
+
|
| 30 |
+
Worth recording, because the same mistake kept recurring in different clothes.
|
| 31 |
+
|
| 32 |
+
1. **Laplacian variance said the broken clips had improved 2.5x.** They were
|
| 33 |
+
sharper *and* broken — fused fingers, blown highlights, collapsing
|
| 34 |
+
proportions. Sharpness cannot see any of that.
|
| 35 |
+
2. **"Nothing degrades over the clip"** came from the same measure: the
|
| 36 |
+
first-fifth to last-fifth sharpness ratio was flat. Saturation across the
|
| 37 |
+
whole clip had halved, which no sharpness statistic reports.
|
| 38 |
+
3. **Saturation then preferred adaptive-8 over adaptive-20** (0.127 against
|
| 39 |
+
0.108). Side by side, adaptive-20 is clearly better: cleaner hands, defined
|
| 40 |
+
dress, a legible star belt. Saturation rewards a colour cast as readily as
|
| 41 |
+
colour.
|
| 42 |
+
|
| 43 |
+
The common error was choosing a measure before enumerating how the thing could
|
| 44 |
+
fail. "It will get blurry" was assumed, so sharpness was measured, and the
|
| 45 |
+
actual failure was colour and anatomy. The user looking at the clips found it
|
| 46 |
+
in seconds.
|
| 47 |
+
|
| 48 |
+
## Not established
|
| 49 |
+
|
| 50 |
+
* One prompt, one seed, one duration. Where the fixed schedule starts failing
|
| 51 |
+
was not bracketed — 15 s is where the switch was placed, from one good clip at
|
| 52 |
+
15 s and one bad one at 25 s, not from a sweep.
|
| 53 |
+
* 20 steps was picked over 8 by eye. 12, 16, 30 were not tried.
|
| 54 |
+
* The two-pass at 25 s does not fit a 16 GB card (OOM at 601 frames); the final
|
| 55 |
+
render was done on the Space's H200. One pass at 25 s does fit, at 6.58 GiB.
|
| 56 |
+
* Position interpolation is still applied above 20 s and still measurably
|
| 57 |
+
sharpens. It is not what fixed this.
|