twanghcmut's picture
|
download
raw
1.09 kB
# step 5000 — root cause: wrong Phi bootstrap latent
The rollout seeded its autoregressive Phi history with `zero_flow_latent.pt`, which is the
VAE encoding of a WHITE image (the legacy Middlebury colour wheel renders zero flow white).
causal_mode's Phi is raw (u,v,dz) clipped to [-1,1] — the VAE's own input range — so its
"no motion" point is the ZEROS image, i.e. mid-GREY.
| | norm | L2 to quietest real GT Phi |
|---|---|---|
| old `zero_flow_latent` (white) | 88.4 | **90.5** |
| new `zero_phi_latent` (grey) | 46.1 | **33.0** |
| quietest real GT Phi frame | 44.2 | 0 |
Effect (same checkpoint, same seed, only the bootstrap changed):
| | PSNR mean | min |
|---|---|---|
| white bootstrap | 14.14 dB | 10.55 |
| grey bootstrap | **16.08 dB** | **14.62** |
Video drift is gone. TRAINING WAS NEVER AFFECTED — it takes cond_f from the dataset's GT
Phi latent, never from the bootstrap — so no training compute was wasted.
Still open: decoded Phi is high-frequency noise rather than the sparse grey+patches of GT,
at 8% of training. Watch; do not conclude yet.

Xet Storage Details

Size:
1.09 kB
·
Xet hash:
36ebb042bfa68d69098e0a0c38f1efb1f97e73c36e1649c613265a1ad584eede

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.