topabaem commited on
Commit
b9babe8
·
verified ·
1 Parent(s): 38f5ab7

Add a 15-second long-form sample, with its BF16 control

Browse files
.gitattributes CHANGED
@@ -104,3 +104,7 @@ samples/compare-det-03.mp4 filter=lfs diff=lfs merge=lfs -text
104
  samples/compare-det-04.mp4 filter=lfs diff=lfs merge=lfs -text
105
  samples/comparison-sheet.png filter=lfs diff=lfs merge=lfs -text
106
  banner.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
104
  samples/compare-det-04.mp4 filter=lfs diff=lfs merge=lfs -text
105
  samples/comparison-sheet.png filter=lfs diff=lfs merge=lfs -text
106
  banner.png filter=lfs diff=lfs merge=lfs -text
107
+ samples/idol/idol-10s-4bit.mp4 filter=lfs diff=lfs merge=lfs -text
108
+ samples/idol/idol-10s-768x512.mp4 filter=lfs diff=lfs merge=lfs -text
109
+ samples/idol/idol-10s-bf16.mp4 filter=lfs diff=lfs merge=lfs -text
110
+ samples/idol/idol-15s.mp4 filter=lfs diff=lfs merge=lfs -text
samples/idol/README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Long-form sample — an anime idol, 15 seconds
2
+
3
+ `idol-15s.mp4` is the one to watch: **14.7 s, 353 frames, 512x320, 24 fps**,
4
+ video and audio generated together, rendered on a **16 GB Tesla V100** from this
5
+ 4-bit encoder. `prompt.txt` is the prompt, unedited.
6
+
7
+ It follows a lot of what was asked. Blonde hair, large blue eyes, star hair
8
+ ornament, the white and light-blue frilly dress with its navy ribbon, the bright
9
+ pastel bedroom with white furniture, blue curtains and a mirrored dresser. The
10
+ choreography lands in order too — it opens on a hand over the lens, peeking
11
+ through, then the presenting gesture, the shy eye-rub, the point at the camera.
12
+
13
+ ## What it does not do
14
+
15
+ **It is not speaking Japanese, and probably not speaking at all.** The prompt
16
+ asked for it. There is audio — 14.7 s, 71% of its energy in the 300-3400 Hz band
17
+ a voice occupies — but its envelope peaks at **0.20 Hz**, and articulated speech
18
+ modulates at 3-8 Hz, once per syllable. The structure is closer to music or a
19
+ sustained vocalisation than to sentences. Judge it by ear; the measurement only
20
+ says do not expect dialogue.
21
+
22
+ **Style is not reliably controllable.** The same prompt, same encoder, same
23
+ seed, rendered at a different length or resolution, comes out photoreal instead
24
+ of animated:
25
+
26
+ | file | length | size | result |
27
+ |---|---|---|---|
28
+ | `idol-15s.mp4` | 14.7 s | 512x320 | **animated, on-prompt** |
29
+ | `idol-10s-4bit.mp4` | 9.7 s | 512x320 | photoreal, right scene |
30
+ | `idol-10s-768x512.mp4` | 9.7 s | 768x512 | photoreal, wrong scene — a dark stage with an audience |
31
+
32
+ Resolution and length change the latent shape, so they change the trajectory:
33
+ these are different samples, not the same sample rendered better or worse.
34
+
35
+ ## That is the model, not the compression
36
+
37
+ `idol-10s-bf16.mp4` is the same prompt through the **BF16 original**, same DiT,
38
+ same seed, same size. It is photoreal too, and it tracks the 4-bit render
39
+ gesture for gesture at the same timestamps. So "2D anime animation" is being
40
+ ignored by LTX-2.5 itself at that length, and the encoder is not what dropped
41
+ it.
42
+
43
+ Frame-to-frame difference between the two, over all 233 frames: **MAE 0.0563**.
44
+ That is higher than the 0.0261 mean across the five benchmark prompts in
45
+ `../README.md` — a 317-token prompt with ten sequenced actions gives compression
46
+ more to lose than "a blacksmith hammers steel".
47
+
48
+ ## Cost
49
+
50
+ | clip | wall clock | peak VRAM |
51
+ |---|---:|---:|
52
+ | 15 s, 512x320 | 123 s | 6.26 GiB |
53
+ | 10 s, 512x320 | 165 s | 6.10 GiB |
54
+ | 10 s, 768x512 | 178 s | 6.53 GiB |
55
+
56
+ All on one 16 GB V100 with `--fp16-unet --novram`. Fifteen seconds costs no more
57
+ memory than ten; the length ceiling here was not reached.
samples/idol/idol-10s-4bit.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dda2a5186d6443e97c8d4e925c50d2f61dd8a082804c0673cc2e5c698e78140c
3
+ size 2390698
samples/idol/idol-10s-768x512.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f243b8db28705ae7db177e87f9a6532e32ce073215bab6fead2d94e1db54db7
3
+ size 3687439
samples/idol/idol-10s-bf16.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8581400cd17dd76aad723cfb055f2e7dd17c9895e82e79ae7636a291c6b9046
3
+ size 2404505
samples/idol/idol-15s.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e31f2d27c8b53f59560329c22d744360fbad1ce2cf33f0f53745f0ceae5ef0b
3
+ size 4201457
samples/idol/prompt.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ A cute anime girl with shoulder-length blonde hair, large sparkling blue eyes, golden star earrings, fair skin and rosy cheeks, wearing a white and light blue frilly idol dress with sheer puffy sleeves, navy ribbon with gold star brooch, light blue bustier top with white dots, and gold buckle belt. She is in a bright pastel blue girl's bedroom with soft natural lighting from the window, white furniture, blue curtains, and a dresser with mirror. Dynamic, energetic, and extremely expressive performance directed at the camera: starts with her hand covering the camera lens, peeking with a surprised open mouth; pulls her hand away and raises both arms high in a joyful celebration pose, big smile; lowers arms and clasps hands near her chest in a cute pose; opens both palms upward in a presenting gesture with an open-mouthed surprised expression; looks down shyly with half-closed eyes while fidgeting with her hands; covers her face with both hands in a shy laughing motion; rubs her eyes cutely; points her index finger strongly at the camera with a confident expression; changes to a peace sign while smiling; ends by extending one arm outward and smiling at the viewer. Smooth fluid 2D anime animation, high quality, detailed clothing frills and hair movement, soft pastel blue and white color palette, bright cheerful atmosphere, shallow depth of field, medium close-up shots, camera slightly follows her movements, cinematic lighting, highly expressive facial animations, idol-like performance. Audio: a young woman speaking cheerfully in Japanese, bright and energetic idol voice.