topabaem commited on
Commit
132a3de
·
verified ·
1 Parent(s): 40ae3c7

Re-render the samples through the full pipeline at 1024x640

Browse files
.gitattributes CHANGED
@@ -110,3 +110,11 @@ samples/idol/idol-10s-bf16.mp4 filter=lfs diff=lfs merge=lfs -text
110
  samples/idol/idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
111
  samples/idol/idol-10s-1024x640-2pass.mp4 filter=lfs diff=lfs merge=lfs -text
112
  samples/idol/sharpness-1pass-vs-2pass.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
110
  samples/idol/idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
111
  samples/idol/idol-10s-1024x640-2pass.mp4 filter=lfs diff=lfs merge=lfs -text
112
  samples/idol/sharpness-1pass-vs-2pass.png filter=lfs diff=lfs merge=lfs -text
113
+ samples/sharp/00-robot.mp4 filter=lfs diff=lfs merge=lfs -text
114
+ samples/sharp/01-dune.mp4 filter=lfs diff=lfs merge=lfs -text
115
+ samples/sharp/02-forge-unpadded-local.mp4 filter=lfs diff=lfs merge=lfs -text
116
+ samples/sharp/02-forge.mp4 filter=lfs diff=lfs merge=lfs -text
117
+ samples/sharp/03-road.mp4 filter=lfs diff=lfs merge=lfs -text
118
+ samples/sharp/04-smoke.mp4 filter=lfs diff=lfs merge=lfs -text
119
+ samples/sharp/05-idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
120
+ samples/sharp/sheet.png filter=lfs diff=lfs merge=lfs -text
samples/sharp/00-robot.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:989a5203550b42ae616e258398a7f36b20896bc86f5ee09728c8f024d548cf1d
3
+ size 1562023
samples/sharp/01-dune.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34a3b5d2c455d43b86254fd002b469ffef06e78438ef07c8033bf00345c72871
3
+ size 573208
samples/sharp/02-forge-unpadded-local.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f5f59400a4fec548d0c10b436d656ff7eab16ae260ef149df10e23b4b15b63
3
+ size 462652
samples/sharp/02-forge.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9aa91b6bbbdc3af4dd324c0a9ba5449836c7a2ab94d337abf86dc639f8797c2f
3
+ size 1743460
samples/sharp/03-road.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7de20479a64e2b837338c77fd5f14f4db370013e1fb6b395510c81960357db04
3
+ size 934042
samples/sharp/04-smoke.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d23d14e8cfa9df2969e76f42fc023c21aaa7d2cffce311e287f92ab409cda86f
3
+ size 657191
samples/sharp/05-idol-15s.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b167fc290bde99935f1af16f0839cd25f60bb45c4132ef6b3781a076a9ac06a2
3
+ size 9985850
samples/sharp/README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Full-pipeline renders — 1024x640
2
+
3
+ These are the samples as the vendor's graph actually produces them: sample at
4
+ 512x320, upscale the latent 2x, sample again at 1024x640. Everything else in
5
+ `../` stops after the first pass.
6
+
7
+ That second pass was skipped in this project from the start, on a written
8
+ assumption that a 16 GB card could not afford four times the token count. It
9
+ was never measured. Measured, it peaks at **10.03 GiB** and adds about a minute
10
+ — and it is the difference between soft and sharp: **4.1x the Laplacian
11
+ variance** against the one-pass output enlarged afterwards, on a common grid.
12
+ The comparison crop is at `../idol/sharpness-1pass-vs-2pass.png`.
13
+
14
+ | file | seconds | prompt |
15
+ |---|---:|---|
16
+ | `00-robot.mp4` | 2.0 | humanoid robot, rain-slick loading bay |
17
+ | `01-dune.mp4` | 2.0 | lone figure walking a red dune at sunset |
18
+ | `02-forge.mp4` | 2.0 | blacksmith, anvil, sparks — **does not follow the prompt, see below** |
19
+ | `02-forge-unpadded-local.mp4` | 2.0 | the same prompt, encoded unpadded, which does |
20
+ | `03-road.mp4` | 2.0 | sports car through a wet mountain hairpin at night |
21
+ | `04-smoke.mp4` | 2.0 | rescue robot in a collapsed corridor |
22
+ | `05-idol-15s.mp4` | **15.0** | the anime idol prompt, 361 frames |
23
+
24
+ Prompts are the ones in `../README.md` and `../idol/prompt.txt`, unchanged.
25
+
26
+ ## Rendered on the Space, not the workstation
27
+
28
+ All six came from
29
+ [the Space](https://huggingface.co/spaces/topabaem/LTX-2.5-Text-Encoder-4bit-8GB-Demo),
30
+ which runs this same 4-bit encoder on ZeroGPU. The 15-second two-pass render
31
+ takes 110 s there. The local V100 was lent to another job at the time, and this
32
+ did not need it.
33
+
34
+ One consequence worth stating: the Space encodes through ComfyUI's own path,
35
+ which left-pads every prompt to 1024 tokens. The clips in `../` were encoded
36
+ unpadded. Same encoder, same weights, slightly different conditioning — so
37
+ these are not the one-pass clips with a second pass bolted on, they are
38
+ different samples.
39
+
40
+ ## One of these does not follow its prompt, and it is informative
41
+
42
+ `02-forge.mp4` is not a forge. It is a man beside a tree, and three seeds
43
+ (20260815, 7, 42) all landed somewhere unrelated.
44
+
45
+ That prompt renders correctly in `../` — anvil, glowing steel, hammer, sparks.
46
+ `02-forge-unpadded-local.mp4` is that render, kept here for the comparison.
47
+
48
+ The difference is not the second pass: re-running on the Space at `refine=0`,
49
+ same seed, still gives the man and the tree. It is **the conditioning path**.
50
+ The Space encodes the way ComfyUI does, left-padding every prompt to 1024
51
+ tokens; the clips in `../` were encoded unpadded. Same encoder, same weights,
52
+ same DiT, same seed, same resolution — and one finds the forge while the other
53
+ does not.
54
+
55
+ It does not run one way. On this batch the padded path produced a *better*
56
+ robot, dune and idol than the unpadded renders in `../` did, and the unpadded
57
+ path is the one that finds the forge. The honest statement is that padding
58
+ changes **which prompts land**, not how good the output is, and this project's
59
+ own note in `ltx_conditioning_dump.py` — that the padded path is simply the
60
+ right one for feeding a DiT — is not supported by that.
61
+
62
+ Which of the two is closer to what Lightricks intended has not been established
63
+ here, and neither has whether the padded implementation is faithful.
64
+
65
+ ## What is deliberately not here
66
+
67
+ **The BF16 comparison in `../` was not regenerated.** Both of its sides are
68
+ one-pass and differ only in the encoder, which is the whole point of it.
69
+ Re-rendering one side at twice the resolution would have replaced a controlled
70
+ comparison with a pretty one.
71
+
72
+ **Image-to-video cannot use the second pass yet.** `LTXVLatentUpsampler`
73
+ discards the noise mask, so a guide would have to be rebuilt at the new scale,
74
+ and nothing does that. The Space hides the control on that tab rather than
75
+ offering something that would fail.
samples/sharp/sheet.png ADDED

Git LFS Details

  • SHA256: f38e9d6c1042050d7dd009ca5ca4d0f1380133c037aba5b503b1a7a145d7e91c
  • Pointer size: 131 Bytes
  • Size of remote file: 519 kB