stale2000 commited on
Commit
6700728
·
verified ·
1 Parent(s): 2a320b3

Document tested variants and show per-prompt token layout masks

Browse files
Files changed (1) hide show
  1. README.md +52 -14
README.md CHANGED
@@ -16,6 +16,10 @@ Experimental LoT adapter and rank-16 DiT LoRA, with benchmarks and visual exampl
16
 
17
  **Research status:** the latest campaign found no branch that met the predeclared promotion threshold. Metrics show tradeoffs, not a confirmed quality win. See [benchmark results](benchmarks/README.md) and [campaign summary](benchmarks/campaign-r4-results.json).
18
 
 
 
 
 
19
  ## Weights
20
 
21
  - [`adapter.safetensors`](adapter.safetensors): LoT adapter and fitted extent bank.
@@ -33,27 +37,61 @@ Both grids show matched 30-step outputs from the untouched confirmation prompts.
33
 
34
  ![Preserve branch comparison grid](examples/preserve/comparison.png)
35
 
36
- ## Generated video examples
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
- Each link opens an MP4 for one confirmation prompt. Prompts and seeds are in the branch [manifests](examples/control/manifest.json) and [preserve manifest](examples/preserve/manifest.json).
39
 
40
- ### Control branch
 
 
41
 
42
- | Prompt | dense-LoRA | detail 50% | uniform 50% |
43
  |---|---|---|---|
44
- | p0 | [MP4](examples/control/p0_dense_lora.mp4) | [MP4](examples/control/p0_detail50_warmup.mp4) | [MP4](examples/control/p0_uniform50_1x2.mp4) |
45
- | p1 | [MP4](examples/control/p1_dense_lora.mp4) | [MP4](examples/control/p1_detail50_warmup.mp4) | [MP4](examples/control/p1_uniform50_1x2.mp4) |
46
- | p2 | [MP4](examples/control/p2_dense_lora.mp4) | [MP4](examples/control/p2_detail50_warmup.mp4) | [MP4](examples/control/p2_uniform50_1x2.mp4) |
47
- | p3 | [MP4](examples/control/p3_dense_lora.mp4) | [MP4](examples/control/p3_detail50_warmup.mp4) | [MP4](examples/control/p3_uniform50_1x2.mp4) |
 
 
48
 
49
- ### Preserve branch
 
 
50
 
51
- | Prompt | dense-LoRA | detail 50% | uniform 50% |
52
  |---|---|---|---|
53
- | p0 | [MP4](examples/preserve/p0_dense_lora.mp4) | [MP4](examples/preserve/p0_detail50_warmup.mp4) | [MP4](examples/preserve/p0_uniform50_1x2.mp4) |
54
- | p1 | [MP4](examples/preserve/p1_dense_lora.mp4) | [MP4](examples/preserve/p1_detail50_warmup.mp4) | [MP4](examples/preserve/p1_uniform50_1x2.mp4) |
55
- | p2 | [MP4](examples/preserve/p2_dense_lora.mp4) | [MP4](examples/preserve/p2_detail50_warmup.mp4) | [MP4](examples/preserve/p2_uniform50_1x2.mp4) |
56
- | p3 | [MP4](examples/preserve/p3_dense_lora.mp4) | [MP4](examples/preserve/p3_detail50_warmup.mp4) | [MP4](examples/preserve/p3_uniform50_1x2.mp4) |
57
 
58
  ## Attribution
59
 
 
16
 
17
  **Research status:** the latest campaign found no branch that met the predeclared promotion threshold. Metrics show tradeoffs, not a confirmed quality win. See [benchmark results](benchmarks/README.md) and [campaign summary](benchmarks/campaign-r4-results.json).
18
 
19
+ ## What was tried
20
+
21
+ We measured token-layout speed, screened sparse layouts, trained on still images and OpenVidHD clips, compared a control against dense-teacher preservation, and tested inference-time detail/entropy selectors. The best measured DiT-forward speedup was 2.64× on a 37-frame 768×1344 case; VAE decode was unchanged. The r4 500-update branches were effectively tied on the 24-source confirmation set and neither passed promotion criteria. Selector previews were small validation screens, not confirmation benchmarks. [Read the experiment history and values](benchmarks/experiment-history.md).
22
+
23
  ## Weights
24
 
25
  - [`adapter.safetensors`](adapter.safetensors): LoT adapter and fitted extent bank.
 
37
 
38
  ![Preserve branch comparison grid](examples/preserve/comparison.png)
39
 
40
+ ## Prompts, videos, and tile-layout masks
41
+
42
+ The mask sheets show all seven latent-time layouts for each prompt. Each sheet compares dense 1×1, uniform 1×2, and the adaptive detail50 pattern recorded at the warm-up selection step. These are token-layout maps, not semantic segmentation masks. Each prompt has separate masks for the control and preserve checkpoints.
43
+
44
+ ### Prompt p0
45
+
46
+ > In the video, a man wearing glasses and a blue shirt is seen interacting with a small dog. The dog, with its white and brown fur, is sitting on a pink surface. The man appears to be speaking to the dog, possibly giving it a command or a treat. The scene is set against a white wall, which provides a neutral backdrop for the interaction between the man and the dog. The overall style of the video is casual and intimate, capturing a moment of connection between the man and his pet.
47
+
48
+ | Control branch — step 500 | Preserve branch — step 500 |
49
+ |---|---|
50
+ | ![Control p0 tile masks](examples/control/masks/p0.png) | ![Preserve p0 tile masks](examples/preserve/masks/p0.png) |
51
+
52
+ | Branch | Dense LoRA video | Adaptive detail50 video | Uniform 1×2 video |
53
+ |---|---|---|---|
54
+ | Control | [MP4](examples/control/p0_dense_lora.mp4) | [MP4](examples/control/p0_detail50_warmup.mp4) | [MP4](examples/control/p0_uniform50_1x2.mp4) |
55
+ | Preserve | [MP4](examples/preserve/p0_dense_lora.mp4) | [MP4](examples/preserve/p0_detail50_warmup.mp4) | [MP4](examples/preserve/p0_uniform50_1x2.mp4) |
56
+
57
+ ### Prompt p1
58
+
59
+ > In the video, a man is seen interacting with a yellow and black device attached to a green door. The device appears to be a camera or a sensor, and the man is holding it up to the door, possibly taking a picture or recording a video. The man is dressed in a black jacket and seems to be focused on his task. The door is green and has a window, through which the man's reflection can be seen. The scene suggests that the man might be a security officer or a technician, inspecting or setting up the device for security purposes. The overall style of the video is realistic and straightforward, with no additional elements or distractions.
60
+
61
+ | Control branch — step 500 | Preserve branch — step 500 |
62
+ |---|---|
63
+ | ![Control p1 tile masks](examples/control/masks/p1.png) | ![Preserve p1 tile masks](examples/preserve/masks/p1.png) |
64
+
65
+ | Branch | Dense LoRA video | Adaptive detail50 video | Uniform 1×2 video |
66
+ |---|---|---|---|
67
+ | Control | [MP4](examples/control/p1_dense_lora.mp4) | [MP4](examples/control/p1_detail50_warmup.mp4) | [MP4](examples/control/p1_uniform50_1x2.mp4) |
68
+ | Preserve | [MP4](examples/preserve/p1_dense_lora.mp4) | [MP4](examples/preserve/p1_detail50_warmup.mp4) | [MP4](examples/preserve/p1_uniform50_1x2.mp4) |
69
+
70
+ ### Prompt p2
71
 
72
+ > The video is a simple yet elegant presentation of a green smoothie. The smoothie is served in a tall glass, which is placed on a white surface. The glass is adorned with a slice of orange and a sprig of mint, adding a touch of color and freshness to the scene. The background is a plain white wall, which contrasts with the green of the smoothie and the orange slice, making the drink the focal point of the image. The overall style of the video is minimalist and clean, with a focus on the smoothie and its ingredients. The video does not contain any text or additional elements, allowing the viewer to fully appreciate the smoothie and its presentation.
73
 
74
+ | Control branch — step 500 | Preserve branch — step 500 |
75
+ |---|---|
76
+ | ![Control p2 tile masks](examples/control/masks/p2.png) | ![Preserve p2 tile masks](examples/preserve/masks/p2.png) |
77
 
78
+ | Branch | Dense LoRA video | Adaptive detail50 video | Uniform 1×2 video |
79
  |---|---|---|---|
80
+ | Control | [MP4](examples/control/p2_dense_lora.mp4) | [MP4](examples/control/p2_detail50_warmup.mp4) | [MP4](examples/control/p2_uniform50_1x2.mp4) |
81
+ | Preserve | [MP4](examples/preserve/p2_dense_lora.mp4) | [MP4](examples/preserve/p2_detail50_warmup.mp4) | [MP4](examples/preserve/p2_uniform50_1x2.mp4) |
82
+
83
+ ### Prompt p3
84
+
85
+ > The video features a man in a suit, standing in a room with green walls. He is looking to his left, and his expression is serious. The room has a window in the background, and there is a blurred figure of a person in the foreground. The man is wearing a white shirt and a gray suit. The overall style of the video is realistic, with a focus on the man's expression and the room's interior.
86
 
87
+ | Control branch — step 500 | Preserve branch — step 500 |
88
+ |---|---|
89
+ | ![Control p3 tile masks](examples/control/masks/p3.png) | ![Preserve p3 tile masks](examples/preserve/masks/p3.png) |
90
 
91
+ | Branch | Dense LoRA video | Adaptive detail50 video | Uniform 1×2 video |
92
  |---|---|---|---|
93
+ | Control | [MP4](examples/control/p3_dense_lora.mp4) | [MP4](examples/control/p3_detail50_warmup.mp4) | [MP4](examples/control/p3_uniform50_1x2.mp4) |
94
+ | Preserve | [MP4](examples/preserve/p3_dense_lora.mp4) | [MP4](examples/preserve/p3_detail50_warmup.mp4) | [MP4](examples/preserve/p3_uniform50_1x2.mp4) |
 
 
95
 
96
  ## Attribution
97