topabaem's picture
Document corrected Space conditioning validation
03b5f64
|
Raw
History Blame Contribute Delete
4.31 kB

Full-pipeline renders — 1024x640

These are the samples as the vendor's graph actually produces them: sample at 512x320, upscale the latent 2x, sample again at 1024x640. Everything else in ../ stops after the first pass.

Historical preprocessing notice (2026-08-22): every clip in this folder predates the current gemma4-raw-intermediate-slots-v1 conditioning contract. The old path applied the encoder's learned final norm to all 49 hidden-state slots; the current path preserves raw intermediate slots and keeps only the model-returned final slot normalized. These clips therefore document the old runtime and must not be used as a current-runtime oracle.

That second pass was skipped in this project from the start, on a written assumption that a 16 GB card could not afford four times the token count. It was never measured. Measured, it peaks at 10.03 GiB and adds about a minute — and it is the difference between soft and sharp: 4.1x the Laplacian variance against the one-pass output enlarged afterwards, on a common grid. The comparison crop is at ../idol/sharpness-1pass-vs-2pass.png.

file seconds prompt
00-robot.mp4 2.0 humanoid robot, rain-slick loading bay
01-dune.mp4 2.0 lone figure walking a red dune at sunset
02-forge.mp4 2.0 blacksmith, anvil, sparks — does not follow the prompt, see below
02-forge-unpadded-local.mp4 2.0 the same prompt, encoded unpadded, which does
03-road.mp4 2.0 sports car through a wet mountain hairpin at night
04-smoke.mp4 2.0 rescue robot in a collapsed corridor
05-idol-15s.mp4 15.0 the anime idol prompt, 361 frames

Prompts are the ones in ../README.md and ../idol/prompt.txt, unchanged.

Rendered on an earlier Space runtime, not the workstation

All six came from the Space, using this same 4-bit weight file on ZeroGPU before the conditioning correction. The 15-second two-pass render took 110 s there. The local V100 was lent to another job at the time, and this did not need it.

One consequence worth stating: the Space encodes through ComfyUI's own path, which left-pads every prompt to 1024 tokens. The clips in ../ were encoded unpadded. Same encoder, same weights, slightly different conditioning — so these are not the one-pass clips with a second pass bolted on, they are different samples.

One of these did not follow its prompt; its cause is unresolved

02-forge.mp4 is not a forge. It is a man beside a tree, and three seeds (20260815, 7, 42) all landed somewhere unrelated.

That prompt renders correctly in ../ — anvil, glowing steel, hammer, sparks. 02-forge-unpadded-local.mp4 is that render, kept here for the comparison.

The difference was not the second pass: re-running on the old Space at refine=0, with the same seed, still gave the man and the tree. At the time this was attributed to left padding because the Space padded to 1024 tokens and the local comparison did not. That causal attribution is no longer valid: the old Space path also applied the learned final norm to every hidden-state slot, while the current contract preserves raw intermediate slots. Padding and hidden-state normalization changed together, so these files cannot isolate either variable.

On this historical batch, the old padded path produced a better robot, dune, and idol, while the old unpadded path found the forge. The defensible conclusion is only that the two legacy preprocessing paths produced different samples. A controlled rerender that changes one variable at a time—and a corrected-contract BF16 oracle—would be needed to assign the cause.

What is deliberately not here

The BF16 comparison in ../ was not regenerated. Both of its sides are one-pass and differ only in the encoder, which is the whole point of it. Re-rendering one side at twice the resolution would have replaced a controlled comparison with a pretty one.

Image-to-video cannot use the second pass yet. LTXVLatentUpsampler discards the noise mask, so a guide would have to be rebuilt at the new scale, and nothing does that. The Space hides the control on that tab rather than offering something that would fail.