Re-render the samples through the full pipeline at 1024x640
Browse files- .gitattributes +8 -0
- samples/sharp/00-robot.mp4 +3 -0
- samples/sharp/01-dune.mp4 +3 -0
- samples/sharp/02-forge-unpadded-local.mp4 +3 -0
- samples/sharp/02-forge.mp4 +3 -0
- samples/sharp/03-road.mp4 +3 -0
- samples/sharp/04-smoke.mp4 +3 -0
- samples/sharp/05-idol-15s.mp4 +3 -0
- samples/sharp/README.md +75 -0
- samples/sharp/sheet.png +3 -0
.gitattributes
CHANGED
|
@@ -110,3 +110,11 @@ samples/idol/idol-10s-bf16.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
| 110 |
samples/idol/idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 111 |
samples/idol/idol-10s-1024x640-2pass.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 112 |
samples/idol/sharpness-1pass-vs-2pass.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
samples/idol/idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 111 |
samples/idol/idol-10s-1024x640-2pass.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 112 |
samples/idol/sharpness-1pass-vs-2pass.png filter=lfs diff=lfs merge=lfs -text
|
| 113 |
+
samples/sharp/00-robot.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 114 |
+
samples/sharp/01-dune.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 115 |
+
samples/sharp/02-forge-unpadded-local.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 116 |
+
samples/sharp/02-forge.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 117 |
+
samples/sharp/03-road.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 118 |
+
samples/sharp/04-smoke.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 119 |
+
samples/sharp/05-idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 120 |
+
samples/sharp/sheet.png filter=lfs diff=lfs merge=lfs -text
|
samples/sharp/00-robot.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:989a5203550b42ae616e258398a7f36b20896bc86f5ee09728c8f024d548cf1d
|
| 3 |
+
size 1562023
|
samples/sharp/01-dune.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:34a3b5d2c455d43b86254fd002b469ffef06e78438ef07c8033bf00345c72871
|
| 3 |
+
size 573208
|
samples/sharp/02-forge-unpadded-local.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95f5f59400a4fec548d0c10b436d656ff7eab16ae260ef149df10e23b4b15b63
|
| 3 |
+
size 462652
|
samples/sharp/02-forge.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9aa91b6bbbdc3af4dd324c0a9ba5449836c7a2ab94d337abf86dc639f8797c2f
|
| 3 |
+
size 1743460
|
samples/sharp/03-road.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7de20479a64e2b837338c77fd5f14f4db370013e1fb6b395510c81960357db04
|
| 3 |
+
size 934042
|
samples/sharp/04-smoke.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d23d14e8cfa9df2969e76f42fc023c21aaa7d2cffce311e287f92ab409cda86f
|
| 3 |
+
size 657191
|
samples/sharp/05-idol-15s.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b167fc290bde99935f1af16f0839cd25f60bb45c4132ef6b3781a076a9ac06a2
|
| 3 |
+
size 9985850
|
samples/sharp/README.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Full-pipeline renders — 1024x640
|
| 2 |
+
|
| 3 |
+
These are the samples as the vendor's graph actually produces them: sample at
|
| 4 |
+
512x320, upscale the latent 2x, sample again at 1024x640. Everything else in
|
| 5 |
+
`../` stops after the first pass.
|
| 6 |
+
|
| 7 |
+
That second pass was skipped in this project from the start, on a written
|
| 8 |
+
assumption that a 16 GB card could not afford four times the token count. It
|
| 9 |
+
was never measured. Measured, it peaks at **10.03 GiB** and adds about a minute
|
| 10 |
+
— and it is the difference between soft and sharp: **4.1x the Laplacian
|
| 11 |
+
variance** against the one-pass output enlarged afterwards, on a common grid.
|
| 12 |
+
The comparison crop is at `../idol/sharpness-1pass-vs-2pass.png`.
|
| 13 |
+
|
| 14 |
+
| file | seconds | prompt |
|
| 15 |
+
|---|---:|---|
|
| 16 |
+
| `00-robot.mp4` | 2.0 | humanoid robot, rain-slick loading bay |
|
| 17 |
+
| `01-dune.mp4` | 2.0 | lone figure walking a red dune at sunset |
|
| 18 |
+
| `02-forge.mp4` | 2.0 | blacksmith, anvil, sparks — **does not follow the prompt, see below** |
|
| 19 |
+
| `02-forge-unpadded-local.mp4` | 2.0 | the same prompt, encoded unpadded, which does |
|
| 20 |
+
| `03-road.mp4` | 2.0 | sports car through a wet mountain hairpin at night |
|
| 21 |
+
| `04-smoke.mp4` | 2.0 | rescue robot in a collapsed corridor |
|
| 22 |
+
| `05-idol-15s.mp4` | **15.0** | the anime idol prompt, 361 frames |
|
| 23 |
+
|
| 24 |
+
Prompts are the ones in `../README.md` and `../idol/prompt.txt`, unchanged.
|
| 25 |
+
|
| 26 |
+
## Rendered on the Space, not the workstation
|
| 27 |
+
|
| 28 |
+
All six came from
|
| 29 |
+
[the Space](https://huggingface.co/spaces/topabaem/LTX-2.5-Text-Encoder-4bit-8GB-Demo),
|
| 30 |
+
which runs this same 4-bit encoder on ZeroGPU. The 15-second two-pass render
|
| 31 |
+
takes 110 s there. The local V100 was lent to another job at the time, and this
|
| 32 |
+
did not need it.
|
| 33 |
+
|
| 34 |
+
One consequence worth stating: the Space encodes through ComfyUI's own path,
|
| 35 |
+
which left-pads every prompt to 1024 tokens. The clips in `../` were encoded
|
| 36 |
+
unpadded. Same encoder, same weights, slightly different conditioning — so
|
| 37 |
+
these are not the one-pass clips with a second pass bolted on, they are
|
| 38 |
+
different samples.
|
| 39 |
+
|
| 40 |
+
## One of these does not follow its prompt, and it is informative
|
| 41 |
+
|
| 42 |
+
`02-forge.mp4` is not a forge. It is a man beside a tree, and three seeds
|
| 43 |
+
(20260815, 7, 42) all landed somewhere unrelated.
|
| 44 |
+
|
| 45 |
+
That prompt renders correctly in `../` — anvil, glowing steel, hammer, sparks.
|
| 46 |
+
`02-forge-unpadded-local.mp4` is that render, kept here for the comparison.
|
| 47 |
+
|
| 48 |
+
The difference is not the second pass: re-running on the Space at `refine=0`,
|
| 49 |
+
same seed, still gives the man and the tree. It is **the conditioning path**.
|
| 50 |
+
The Space encodes the way ComfyUI does, left-padding every prompt to 1024
|
| 51 |
+
tokens; the clips in `../` were encoded unpadded. Same encoder, same weights,
|
| 52 |
+
same DiT, same seed, same resolution — and one finds the forge while the other
|
| 53 |
+
does not.
|
| 54 |
+
|
| 55 |
+
It does not run one way. On this batch the padded path produced a *better*
|
| 56 |
+
robot, dune and idol than the unpadded renders in `../` did, and the unpadded
|
| 57 |
+
path is the one that finds the forge. The honest statement is that padding
|
| 58 |
+
changes **which prompts land**, not how good the output is, and this project's
|
| 59 |
+
own note in `ltx_conditioning_dump.py` — that the padded path is simply the
|
| 60 |
+
right one for feeding a DiT — is not supported by that.
|
| 61 |
+
|
| 62 |
+
Which of the two is closer to what Lightricks intended has not been established
|
| 63 |
+
here, and neither has whether the padded implementation is faithful.
|
| 64 |
+
|
| 65 |
+
## What is deliberately not here
|
| 66 |
+
|
| 67 |
+
**The BF16 comparison in `../` was not regenerated.** Both of its sides are
|
| 68 |
+
one-pass and differ only in the encoder, which is the whole point of it.
|
| 69 |
+
Re-rendering one side at twice the resolution would have replaced a controlled
|
| 70 |
+
comparison with a pretty one.
|
| 71 |
+
|
| 72 |
+
**Image-to-video cannot use the second pass yet.** `LTXVLatentUpsampler`
|
| 73 |
+
discards the noise mask, so a guide would have to be rebuilt at the new scale,
|
| 74 |
+
and nothing does that. The Space hides the control on that tab rather than
|
| 75 |
+
offering something that would fail.
|
samples/sharp/sheet.png
ADDED
|
Git LFS Details
|