topabaem commited on
Commit
7bb238c
·
verified ·
1 Parent(s): 930003a

25-second renders, with and without temporal position interpolation

Browse files
.gitattributes CHANGED
@@ -118,3 +118,7 @@ samples/sharp/03-road.mp4 filter=lfs diff=lfs merge=lfs -text
118
  samples/sharp/04-smoke.mp4 filter=lfs diff=lfs merge=lfs -text
119
  samples/sharp/05-idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
120
  samples/sharp/sheet.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
118
  samples/sharp/04-smoke.mp4 filter=lfs diff=lfs merge=lfs -text
119
  samples/sharp/05-idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
120
  samples/sharp/sheet.png filter=lfs diff=lfs merge=lfs -text
121
+ samples/long/ab.png filter=lfs diff=lfs merge=lfs -text
122
+ samples/long/crop_ab.png filter=lfs diff=lfs merge=lfs -text
123
+ samples/long/idol25-fit.mp4 filter=lfs diff=lfs merge=lfs -text
124
+ samples/long/idol25-nofit.mp4 filter=lfs diff=lfs merge=lfs -text
samples/long/README.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Past 20 seconds: what the temporal RoPE does, and what to do about it
2
+
3
+ ## There is a hard number, and it is in the config
4
+
5
+ `LTXAVModel` carries `positional_embedding_max_pos=[20, 2048, 2048]`, and
6
+ `get_fractional_positions` divides each coordinate by its entry. The temporal
7
+ one is **seconds, not frames** — both `_prepare_positional_embeddings`
8
+ (`model.py:841`) and the AV cross-attention path (`av_model.py:865`) multiply
9
+ the temporal coordinate by `1 / frame_rate` first.
10
+
11
+ So the temporal rotary embedding was built for **20 seconds**. Past that, the
12
+ fraction exceeds 1.0 and the frequencies extrapolate off the end of anything the
13
+ model saw in training. This is the failure
14
+ [Train Short, Inference Long (arXiv 2602.14027)](https://arxiv.org/html/2602.14027v1)
15
+ attributes long-horizon degradation to.
16
+
17
+ `fit_temporal_positions` raises `max_pos[0]` to the clip's own duration, mapping
18
+ it back onto [0, 1] — position interpolation, the same trick used to extend a
19
+ language model's context, and training-free. It is safe to patch at runtime
20
+ because `generate_freq_grid_np`'s cache is keyed on theta, dimension count and
21
+ device, never on `max_pos`.
22
+
23
+ ## Measured at 25 seconds
24
+
25
+ 601 frames, 1024x640, two-pass, same prompt and seed, on the Space's H200.
26
+
27
+ | | segment sharpness (5 equal parts) | last/first |
28
+ |---|---|---:|
29
+ | untreated | 0.00086 0.00130 0.00143 0.00139 0.00096 | 1.12 |
30
+ | **interpolated** | **0.00247 0.00302 0.00286 0.00318 0.00198** | 0.80 |
31
+
32
+ The interpolated render is **2.2-2.9x sharper in every segment**.
33
+ `crop_ab.png` at 1:1: hair strands, eyelashes and background props are resolved
34
+ on the treated side and waxy on the untreated one.
35
+
36
+ ## What this did not find
37
+
38
+ **No progressive collapse.** The premise being tested was that quality decays as
39
+ the clip runs on. It does not, in either version — the last-fifth to first-fifth
40
+ sharpness ratio is 1.12 untreated and 0.80 treated, and neither drifts in
41
+ character, wardrobe or room across 25 seconds. Most of the long-video literature
42
+ found in this search — compounding error, history forgetting, attribute drift —
43
+ describes **autoregressive or sliding-window** generation, where each window
44
+ conditions on the last. LTX-2.5 as driven here samples the whole clip at once,
45
+ so that class of failure does not apply and searching for it would have been
46
+ looking in the wrong place.
47
+
48
+ What extrapolation costs here is **detail across the whole clip**, not stability
49
+ at the end.
50
+
51
+ ## Limits
52
+
53
+ * The two renders are different trajectories, not the same trajectory rendered
54
+ twice, so the sharpness gap is what changed and not a controlled isolation of
55
+ cause. Same prompt, seed, size, sampler and refine strength; only `max_pos[0]`
56
+ differs.
57
+ * One duration (25 s), one prompt, one seed. 30 s, 40 s and beyond were not
58
+ tried, and the point where interpolation itself starts to hurt — it compresses
59
+ temporal detail by construction — was not located.
60
+ * Below 20 s this changes nothing and is not applied.
61
+ * Laplacian variance tracks contrast as well as sharpness. It is used here
62
+ between two renders of the same scene, which is the only place it is fair.
63
+
64
+ ## Files
65
+
66
+ `idol25-fit.mp4`, `idol25-nofit.mp4`, `ab.png` (timeline), `crop_ab.png` (1:1).
samples/long/ab.png ADDED

Git LFS Details

  • SHA256: c4bb5497efc4a3b80d206013a2a430e6d8e77dce9d24428468ba229e47a7145b
  • Pointer size: 131 Bytes
  • Size of remote file: 628 kB
samples/long/crop_ab.png ADDED

Git LFS Details

  • SHA256: 1c4a3dfca387e901ae6421c148573fc19210bb7ee1a84bda84d1aee0876b6bf5
  • Pointer size: 131 Bytes
  • Size of remote file: 273 kB
samples/long/idol25-fit.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb69e83909f4ef5b5308d13c4b431b001579d206b4dae902996d8931e605a18c
3
+ size 16806661
samples/long/idol25-nofit.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a31a7b8539f709ef18cac15d518e90dc3de29cc3a8f908bc88c6cb380b550b3a
3
+ size 16186038