Hugging Face's logo

SyFeee
/
Bernini-LTX-Research-Checkpoints

Text-to-Video
ltx-video
ltx-2.3
lora
video-generation
research
Model card Files
xet
 Community
SyFeee commited on
Commit
6743696
·
verified ·
1 Parent(s): 6f43c40

Add files using upload-large-folder tool

Browse files
Files changed (38) hide show
  1. README.md +51 -0
  2. SHA256SUMS +24 -0
  3. runs/bernini_av_phaseA/checkpoints/bernini_P_step_01000.safetensors +3 -0
  4. runs/bernini_av_phaseA/checkpoints/lora_weights_step_01000.safetensors +3 -0
  5. runs/bernini_av_phaseA/training_config.yaml +28 -0
  6. runs/bernini_av_phaseA_704/checkpoints/bernini_P_step_01000.safetensors +3 -0
  7. runs/bernini_av_phaseA_704/checkpoints/lora_weights_step_01000.safetensors +3 -0
  8. runs/bernini_av_phaseA_704/training_config.yaml +28 -0
  9. runs/bernini_av_phaseB_704/checkpoints/bernini_P_step_01000.safetensors +3 -0
  10. runs/bernini_av_phaseB_704/checkpoints/lora_weights_step_01000.safetensors +3 -0
  11. runs/bernini_av_phaseB_704/training_config.yaml +31 -0
  12. runs/bernini_combined_pilot/checkpoints/bernini_P_step_01800.safetensors +3 -0
  13. runs/bernini_combined_pilot/checkpoints/lora_weights_step_01800.safetensors +3 -0
  14. runs/bernini_combined_pilot/training_config.yaml +33 -0
  15. runs/bernini_duo_pilot_inline/checkpoints/bernini_P_step_01000.safetensors +3 -0
  16. runs/bernini_duo_pilot_inline/checkpoints/lora_weights_step_01000.safetensors +3 -0
  17. runs/bernini_duo_pilot_inline/training_config.yaml +29 -0
  18. runs/bernini_fight_motion_pilot/checkpoints/bernini_P_step_01500.safetensors +3 -0
  19. runs/bernini_fight_motion_pilot/checkpoints/lora_weights_step_01500.safetensors +3 -0
  20. runs/bernini_fight_motion_pilot/training_config.yaml +33 -0
  21. runs/bernini_multishot_704/checkpoints/bernini_P_step_01000.safetensors +3 -0
  22. runs/bernini_multishot_704/checkpoints/lora_weights_step_01000.safetensors +3 -0
  23. runs/bernini_multishot_704/training_config.yaml +30 -0
  24. runs/bernini_p2_baseline/checkpoints/bernini_P_step_01500.safetensors +3 -0
  25. runs/bernini_p2_baseline/checkpoints/lora_weights_step_01500.safetensors +3 -0
  26. runs/bernini_p2_baseline/training_config.yaml +32 -0
  27. runs/bernini_p2_plan/checkpoints/bernini_P_step_01500.safetensors +3 -0
  28. runs/bernini_p2_plan/checkpoints/lora_weights_step_01500.safetensors +3 -0
  29. runs/bernini_p2_plan/training_config.yaml +34 -0
  30. runs/bernini_r2v_msr_baseline/checkpoints/bernini_P_step_01500.safetensors +3 -0
  31. runs/bernini_r2v_msr_baseline/checkpoints/lora_weights_step_01500.safetensors +3 -0
  32. runs/bernini_r2v_msr_baseline/training_config.yaml +32 -0
  33. runs/bernini_r2v_plan_pure/checkpoints/bernini_P_step_01500.safetensors +3 -0
  34. runs/bernini_r2v_plan_pure/checkpoints/lora_weights_step_01500.safetensors +3 -0
  35. runs/bernini_r2v_plan_pure/training_config.yaml +35 -0
  36. runs/bernini_r2v_plan_ref/checkpoints/bernini_P_step_01500.safetensors +3 -0
  37. runs/bernini_r2v_plan_ref/checkpoints/lora_weights_step_01500.safetensors +3 -0
  38. runs/bernini_r2v_plan_ref/training_config.yaml +31 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: ltx-2-community-license
4
+ license_link: https://huggingface.co/Lightricks/LTX-2.3/blob/main/LICENSE
5
+ base_model: Lightricks/LTX-2.3
6
+ pipeline_tag: text-to-video
7
+ tags:
8
+ - ltx-video
9
+ - ltx-2.3
10
+ - lora
11
+ - video-generation
12
+ - research
13
+ ---
14
+
15
+ # Bernini LTX Research Checkpoints
16
+
17
+ This repository contains 12 experimental Bernini adaptation runs trained against the LTX-2.3 22B development checkpoint. These are early research artifacts, not production-validated releases.
18
+
19
+ ## Important Loading Requirement
20
+
21
+ Each run contains two files that must be loaded together:
22
+
23
+ - `lora_weights_step_*.safetensors`: the rank-64 LoRA adapter.
24
+ - `bernini_P_step_*.safetensors`: the matching Bernini projection weights.
25
+
26
+ The LoRA alone is incomplete. Keep both files from the same run and step; projection weights are not interchangeable across runs.
27
+
28
+ ## Included Runs
29
+
30
+ | Run | Step | Research focus |
31
+ | --- | ---: | --- |
32
+ | `bernini_av_phaseA` | 1000 | Reference-to-video plus joint audio |
33
+ | `bernini_av_phaseA_704` | 1000 | Phase A at the 704 training setup |
34
+ | `bernini_av_phaseB_704` | 1000 | Reference-voice conditioning at 704 |
35
+ | `bernini_combined_pilot` | 1800 | Combined conditioning pilot |
36
+ | `bernini_duo_pilot_inline` | 1000 | Inline two-subject conditioning |
37
+ | `bernini_fight_motion_pilot` | 1500 | High-motion pose-control pilot |
38
+ | `bernini_multishot_704` | 1000 | Multi-shot continuity at 704 |
39
+ | `bernini_p2_baseline` | 1500 | P2 baseline |
40
+ | `bernini_p2_plan` | 1500 | P2 planning-conditioned run |
41
+ | `bernini_r2v_msr_baseline` | 1500 | R2V MSR baseline |
42
+ | `bernini_r2v_plan_pure` | 1500 | R2V plan-only variant |
43
+ | `bernini_r2v_plan_ref` | 1500 | R2V planning with references |
44
+
45
+ Each `runs/<run>/training_config.yaml` records the corresponding experiment configuration. Paths in these archival configs reflect the original training machine and must be adapted locally.
46
+
47
+ ## Scope and Use
48
+
49
+ The checkpoints explore identity/reference conditioning, planning, multi-subject control, motion control, multi-shot behavior, and audio generation. Quality varies by run. Validate identity, motion, audio, and temporal consistency on representative prompts before deployment.
50
+
51
+ Use of these derivatives remains subject to the LTX-2 Community License and the terms of the base model.
SHA256SUMS ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ b9ea8a9634a2779d60f955d8436b8a9eb456c05c1145147d2276743f109dca8e runs/bernini_av_phaseA/checkpoints/bernini_P_step_01000.safetensors
2
+ 1f586ffebb1e2f0f4a7319cda0e4e586803e1e326db73cbbfb825c9ea0d1c6a1 runs/bernini_av_phaseA/checkpoints/lora_weights_step_01000.safetensors
3
+ 4e0621ea822786bd194c94f96840e7c93c6928a3aa75dd596b52ffb7bedbe66a runs/bernini_av_phaseA_704/checkpoints/bernini_P_step_01000.safetensors
4
+ 6e5a9e678ac59792d38a0943e350db5518054c3b6b73a5c511a8b6cff0776ca6 runs/bernini_av_phaseA_704/checkpoints/lora_weights_step_01000.safetensors
5
+ 9a48d18e2dc481c4c35c52b021d13c28b94c3906f9f25b2eae7862e658c72895 runs/bernini_av_phaseB_704/checkpoints/bernini_P_step_01000.safetensors
6
+ c7a8264df84ef25e3291f784584d9cec17247e75e3af91ecd1eca0ed8e6702b2 runs/bernini_av_phaseB_704/checkpoints/lora_weights_step_01000.safetensors
7
+ 704c4b2f6f465e0bbfbb4212c0cf141f387aff167485b19b7261133d3b12dfa5 runs/bernini_combined_pilot/checkpoints/bernini_P_step_01800.safetensors
8
+ 67b4d92837a1df640d641a13508b4399e7ac680565b7c667aea66891ab8da5b4 runs/bernini_combined_pilot/checkpoints/lora_weights_step_01800.safetensors
9
+ e07376c5123b98fec0eaef99ad2dd4ef575f4efa020499bf48d2983327e11f93 runs/bernini_duo_pilot_inline/checkpoints/bernini_P_step_01000.safetensors
10
+ 87c14b74f0351592505993bc79ba4f74271636953ba15ea2505dae9586e45934 runs/bernini_duo_pilot_inline/checkpoints/lora_weights_step_01000.safetensors
11
+ 677ed327c5fc4215593c6a30d526c530871357947c67e81378cf52233575a8d0 runs/bernini_fight_motion_pilot/checkpoints/bernini_P_step_01500.safetensors
12
+ a0c23507c4317aad133d7ed38d0c82b5d179b3d31965d9c3822f17f94c72fab7 runs/bernini_fight_motion_pilot/checkpoints/lora_weights_step_01500.safetensors
13
+ 61729a5290d3e3667eb32a6a7c41683549ef3844d0963953dee9f9a38b481e19 runs/bernini_multishot_704/checkpoints/bernini_P_step_01000.safetensors
14
+ a4b537924e4a89ece4cfc62709db3c6412807baa3ba4fee8a40068685c1fa5da runs/bernini_multishot_704/checkpoints/lora_weights_step_01000.safetensors
15
+ a93b9727ebbdcc833986f927136874c39b80f5af68bd0e143a92f953ce27416d runs/bernini_p2_baseline/checkpoints/bernini_P_step_01500.safetensors
16
+ 9fdfef9a90094e74ae75644893b3ea28503c217a0731c3b41b1c44c4b23ca79a runs/bernini_p2_baseline/checkpoints/lora_weights_step_01500.safetensors
17
+ 20fbd75fe9daba71cc89075bc904cb5ffe3194173cc6f886429ed5af87139ae6 runs/bernini_p2_plan/checkpoints/bernini_P_step_01500.safetensors
18
+ d5357b464740d6f56dfb7b8c98700df204087ca0d5afa9c134437e4a3b454b6e runs/bernini_p2_plan/checkpoints/lora_weights_step_01500.safetensors
19
+ a93b9727ebbdcc833986f927136874c39b80f5af68bd0e143a92f953ce27416d runs/bernini_r2v_msr_baseline/checkpoints/bernini_P_step_01500.safetensors
20
+ 570d59bc4fcbb9057b919592a47fd3381cca53230c43d1bed0ba8f617ad4f033 runs/bernini_r2v_msr_baseline/checkpoints/lora_weights_step_01500.safetensors
21
+ 182735c6e2505ee71292f33142cc108a6456c0f374bbcc1645a46a3d7283f5fd runs/bernini_r2v_plan_pure/checkpoints/bernini_P_step_01500.safetensors
22
+ 7e7f12892e8762184ed8400b7637efec63b24b07c6a9f26d4c78d3650c491996 runs/bernini_r2v_plan_pure/checkpoints/lora_weights_step_01500.safetensors
23
+ 03c4580f801049875b0202ba3c7610ced8659d75a00f32ccbd48c899937dd589 runs/bernini_r2v_plan_ref/checkpoints/bernini_P_step_01500.safetensors
24
+ 6cf95d7290c0a859ed8c7e337446b4a7f2a9b2d2da588ba7643efa2ec5d95d70 runs/bernini_r2v_plan_ref/checkpoints/lora_weights_step_01500.safetensors
runs/bernini_av_phaseA/checkpoints/bernini_P_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9ea8a9634a2779d60f955d8436b8a9eb456c05c1145147d2276743f109dca8e
3
+ size 33562808
runs/bernini_av_phaseA/checkpoints/lora_weights_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f586ffebb1e2f0f4a7319cda0e4e586803e1e326db73cbbfb825c9ea0d1c6a1
3
+ size 981826968
runs/bernini_av_phaseA/training_config.yaml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AV Phase-A full run — the R2V stack (plan + refs, the identity setup that gave FaceSim 0.170) PLUS the
2
+ # joint audio arm (enable_audio). Trains one LoRA that keeps identity AND co-generates coherent audio.
3
+ # No ref-voice yet (that's Phase B). enable_audio=false would reproduce the video-only run byte-identically.
4
+ run_name: bernini_av_phaseA
5
+ enable_plan: true
6
+ context_key: vit
7
+ enable_ref_latents: true
8
+ enable_audio: true
9
+ ref_time_gap: 0.5
10
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_w0
11
+ audio_latents_root: /data0/syfe/bernini_ltx/audio_latents_w0
12
+ max_steps: 1000
13
+ save_every: 250
14
+ val_every: 250
15
+ log_every: 20
16
+ data_seed: 1234
17
+ noise_seed: 5678
18
+ val_seed: 99
19
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
20
+ lr_lora: 1.0e-4
21
+ lr_P: 2.0e-4
22
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
23
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_train.jsonl
24
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_val.jsonl
25
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_killswitch/shards
26
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
27
+ num_workers: 4
28
+ out_dir: /data0/syfe/lora_output/bernini_av_phaseA
runs/bernini_av_phaseA_704/checkpoints/bernini_P_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e0621ea822786bd194c94f96840e7c93c6928a3aa75dd596b52ffb7bedbe66a
3
+ size 33562808
runs/bernini_av_phaseA_704/checkpoints/lora_weights_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e5a9e678ac59792d38a0943e350db5518054c3b6b73a5c511a8b6cff0776ca6
3
+ size 981826968
runs/bernini_av_phaseA_704/training_config.yaml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AV Phase-A full run — the R2V stack (plan + refs, the identity setup that gave FaceSim 0.170) PLUS the
2
+ # joint audio arm (enable_audio). Trains one LoRA that keeps identity AND co-generates coherent audio.
3
+ # No ref-voice yet (that's Phase B). enable_audio=false would reproduce the video-only run byte-identically.
4
+ run_name: bernini_av_phaseA_704
5
+ enable_plan: true
6
+ context_key: vit
7
+ enable_ref_latents: true
8
+ enable_audio: true
9
+ ref_time_gap: 0.5
10
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_704
11
+ audio_latents_root: /data0/syfe/bernini_ltx/audio_latents_704
12
+ max_steps: 1000
13
+ save_every: 250
14
+ val_every: 250
15
+ log_every: 20
16
+ data_seed: 1234
17
+ noise_seed: 5678
18
+ val_seed: 99
19
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
20
+ lr_lora: 1.0e-4
21
+ lr_P: 2.0e-4
22
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_704
23
+ join_path: /data0/syfe/bernini_ltx/preprocess_704/w0_join_r2v_train.jsonl
24
+ val_join: /data0/syfe/bernini_ltx/preprocess_704/w0_join_r2v_val.jsonl
25
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_704/shards
26
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
27
+ num_workers: 4
28
+ out_dir: /data0/syfe/lora_output/bernini_av_phaseA_704
runs/bernini_av_phaseB_704/checkpoints/bernini_P_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a48d18e2dc481c4c35c52b021d13c28b94c3906f9f25b2eae7862e658c72895
3
+ size 33562808
runs/bernini_av_phaseB_704/checkpoints/lora_weights_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7a8264df84ef25e3291f784584d9cec17247e75e3af91ecd1eca0ed8e6702b2
3
+ size 981826968
runs/bernini_av_phaseB_704/training_config.yaml ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AV Phase-B — LIP-SYNC. R2V identity stack (plan + refs) + speech captions + the audio FROZEN as clean
2
+ # conditioning (freeze_audio) so audio_to_video_attn drives the MOUTH to the dialogue. Adapted to our LTX-2
3
+ # fork (no cross-modal masks -> audio-driven lip-sync, not voice-clone ref-audio-IC). Trains on 209 speech
4
+ # clips. The a2v + video LoRA learn to sync the mouth to the clip's own (frozen) audio.
5
+ run_name: bernini_av_phaseB_704
6
+ enable_plan: true
7
+ context_key: vit
8
+ enable_ref_latents: true
9
+ enable_audio: true
10
+ freeze_audio: true # <-- audio = clean conditioning (lip-sync)
11
+ ref_time_gap: 0.5
12
+ speech_conditions_root: /data0/syfe/bernini_ltx/conditions_speech_704 # [VISUAL][SPEECH][SOUNDS]
13
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_704
14
+ audio_latents_root: /data0/syfe/bernini_ltx/audio_latents_704 # the FROZEN target voice per clip
15
+ max_steps: 1000
16
+ save_every: 250
17
+ val_every: 250
18
+ log_every: 20
19
+ data_seed: 1234
20
+ noise_seed: 5678
21
+ val_seed: 99
22
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
23
+ lr_lora: 1.0e-4
24
+ lr_P: 2.0e-4
25
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_704
26
+ join_path: /data0/syfe/bernini_ltx/preprocess_704/w0_join_avB_train.jsonl
27
+ val_join: /data0/syfe/bernini_ltx/preprocess_704/w0_join_avB_val.jsonl
28
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_704/shards
29
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
30
+ num_workers: 4
31
+ out_dir: /data0/syfe/lora_output/bernini_av_phaseB_704
runs/bernini_combined_pilot/checkpoints/bernini_P_step_01800.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:704c4b2f6f465e0bbfbb4212c0cf141f387aff167485b19b7261133d3b12dfa5
3
+ size 33562808
runs/bernini_combined_pilot/checkpoints/lora_weights_step_01800.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:67b4d92837a1df640d641a13508b4399e7ac680565b7c667aea66891ab8da5b4
3
+ size 402758624
runs/bernini_combined_pilot/training_config.yaml ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Combined pilot — ANY-N (1/2/3/4 subjects) + PROPS capability test. Same region-placed-ref mechanism as the
2
+ # duo pilot, now N-agnostic + a PROP is just another region-placed ref (product-still + drama-crop). Trains on
3
+ # 204 clips (N=1:71 / N=2:48 / N=3:44 / N=4:41) incl. 52 prop clips (drama + product prop refs grafted into
4
+ # ref_latents). LTM/STM is DEFERRED here (chain-aware selection needed for a dense signal; mechanism already
5
+ # validated in bernini_multishot_704). Trains from scratch (no warm-start support); more steps than duo for the
6
+ # richer task. Video-only.
7
+ run_name: bernini_combined_pilot
8
+ enable_plan: true
9
+ context_key: vit
10
+ enable_ref_latents: true
11
+ enable_region_refs: true # spatial routing — each ref (char OR prop) placed at ITS region
12
+ enable_stm: false # LTM deferred (thin chains in this selection; validated separately)
13
+ enable_audio: false
14
+ ref_time_gap: 0.5
15
+ ref_latents_root: /data0/syfe/bernini_ltx/combined_pilot/ref_latents
16
+ precomp_root: /data0/syfe/bernini_ltx/combined_pilot/precomputed
17
+ speech_conditions_root: /data0/syfe/bernini_ltx/combined_pilot/conditions_inline
18
+ join_path: /data0/syfe/bernini_ltx/combined_pilot/w0_join_r2v_train.jsonl
19
+ val_join: /data0/syfe/bernini_ltx/combined_pilot/w0_join_r2v_val.jsonl
20
+ bernini_dir: /data0/syfe/bernini_ltx/combined_pilot/contexts_r2v/shards
21
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
22
+ max_steps: 1800
23
+ save_every: 300
24
+ val_every: 300
25
+ log_every: 20
26
+ data_seed: 1234
27
+ noise_seed: 5678
28
+ val_seed: 99
29
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
30
+ lr_lora: 1.0e-4
31
+ lr_P: 2.0e-4
32
+ num_workers: 4
33
+ out_dir: /data0/syfe/lora_output/bernini_combined_pilot
runs/bernini_duo_pilot_inline/checkpoints/bernini_P_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e07376c5123b98fec0eaef99ad2dd4ef575f4efa020499bf48d2983327e11f93
3
+ size 33562808
runs/bernini_duo_pilot_inline/checkpoints/lora_weights_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87c14b74f0351592505993bc79ba4f74271636953ba15ea2505dae9586e45934
3
+ size 402758624
runs/bernini_duo_pilot_inline/training_config.yaml ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Duo pilot (P1) — multi-character SPATIAL ROUTING test. R2V (plan + refs) + enable_region_refs: each subject's
2
+ # body-crop ref latent is placed at ITS region on the target grid (not the shared corner) so the model routes
3
+ # 2 identities to 2 regions. Trains on 578 clean left/right duo clips. Video-only. The one new flag vs R2V.
4
+ run_name: bernini_duo_pilot_inline
5
+ enable_plan: true
6
+ context_key: vit
7
+ enable_ref_latents: true
8
+ enable_region_refs: true # <-- the spatial-routing mechanism
9
+ enable_audio: false
10
+ ref_time_gap: 0.5
11
+ ref_latents_root: /data0/syfe/bernini_ltx/duo_pilot/ref_latents
12
+ precomp_root: /data0/syfe/bernini_ltx/duo_pilot/precomputed
13
+ speech_conditions_root: /data0/syfe/bernini_ltx/duo_pilot/conditions_inline
14
+ join_path: /data0/syfe/bernini_ltx/duo_pilot/preprocess/w0_join_r2v_train.jsonl
15
+ val_join: /data0/syfe/bernini_ltx/duo_pilot/preprocess/w0_join_r2v_val.jsonl
16
+ bernini_dir: /data0/syfe/bernini_ltx/duo_pilot/contexts_r2v/shards
17
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
18
+ max_steps: 1000
19
+ save_every: 250
20
+ val_every: 250
21
+ log_every: 20
22
+ data_seed: 1234
23
+ noise_seed: 5678
24
+ val_seed: 99
25
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
26
+ lr_lora: 1.0e-4
27
+ lr_P: 2.0e-4
28
+ num_workers: 4
29
+ out_dir: /data0/syfe/lora_output/bernini_duo_pilot_inline
runs/bernini_fight_motion_pilot/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:677ed327c5fc4215593c6a30d526c530871357947c67e81378cf52233575a8d0
3
+ size 33562808
runs/bernini_fight_motion_pilot/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0c23507c4317aad133d7ed38d0c82b5d179b3d31965d9c3822f17f94c72fab7
3
+ size 402758624
runs/bernini_fight_motion_pilot/training_config.yaml ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Fight/motion pilot (Bug-3 validation) — the LTX-native MOTION CHANNEL on high-velocity clips.
2
+ # Trains the pose control-video conditioning with CONTROL DROPOUT so image+text ALONE produces rich motion
3
+ # (the prior), and a control video is OPTIONAL at inference. No refs/plan-refs — this isolates the motion fix.
4
+ # Data = 140 clips selected by DWPose LIMB VELOCITY >=30 (real body motion, not camera/VFX) + dense pose.
5
+ run_name: bernini_fight_motion_pilot
6
+ enable_plan: true
7
+ context_key: vit
8
+ enable_ref_latents: false # motion-isolation run — no character/prop refs
9
+ enable_region_refs: false
10
+ enable_stm: false
11
+ enable_audio: false
12
+ enable_control: true # <-- Bug-3 motion channel (pose control-video)
13
+ control_dropout: 0.4 # <-- image+text alone still learns motion (prior); control optional at infer
14
+ control_attn_strength: 1.0
15
+ control_latents_root: /data0/syfe/bernini_ltx/fight_pilot/control_latents
16
+ ref_attn_mask: true # Bug-2 block mask (isolates the control group)
17
+ precomp_root: /data0/syfe/bernini_ltx/fight_pilot/precomputed
18
+ join_path: /data0/syfe/bernini_ltx/fight_pilot/join_train.jsonl
19
+ val_join: /data0/syfe/bernini_ltx/fight_pilot/join_val.jsonl
20
+ bernini_dir: /data0/syfe/bernini_ltx/fight_pilot/contexts/shards
21
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
22
+ max_steps: 1500
23
+ save_every: 300
24
+ val_every: 300
25
+ log_every: 20
26
+ data_seed: 1234
27
+ noise_seed: 5678
28
+ val_seed: 99
29
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
30
+ lr_lora: 1.0e-4
31
+ lr_P: 2.0e-4
32
+ num_workers: 4
33
+ out_dir: /data0/syfe/lora_output/bernini_fight_motion_pilot
runs/bernini_multishot_704/checkpoints/bernini_P_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61729a5290d3e3667eb32a6a7c41683549ef3844d0963953dee9f9a38b481e19
3
+ size 33562808
runs/bernini_multishot_704/checkpoints/lora_weights_step_01000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4b537924e4a89ece4cfc62709db3c6412807baa3ba4fee8a40068685c1fa5da
3
+ size 402758624
runs/bernini_multishot_704/training_config.yaml ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Multi-shot MINIMAL — STM-only cross-shot continuity. R2V identity stack (plan + refs, video-only) PLUS the
2
+ # STM memory slot: the PREVIOUS shot's tail Pv=4 clean latent frames prepended in the Strata-RoPE band
3
+ # [124,127]. gstm=1.0 (no boundary), no MLP/LTM/AdaLN/audio — isolates whether prev-shot memory improves
4
+ # cross-cut identity/continuity. Trains on 354 same-face 704 "continue" pairs (val 52, clean cross-clip).
5
+ run_name: bernini_multishot_704
6
+ enable_plan: true
7
+ context_key: vit
8
+ enable_ref_latents: true
9
+ enable_audio: false
10
+ enable_stm: true
11
+ stm_frames: 4
12
+ ref_time_gap: 0.5
13
+ ref_latents_root: /data0/syfe/bernini_ltx/multishot_704_continue/ref_latents
14
+ precomp_root: /data0/syfe/bernini_ltx/multishot_704_continue/precomputed
15
+ join_path: /data0/syfe/bernini_ltx/multishot_704_continue/preprocess/ms_join_train.jsonl
16
+ val_join: /data0/syfe/bernini_ltx/multishot_704_continue/preprocess/ms_join_val.jsonl
17
+ bernini_dir: /data0/syfe/bernini_ltx/multishot_704_continue/contexts_r2v/shards
18
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
19
+ max_steps: 1000
20
+ save_every: 250
21
+ val_every: 250
22
+ log_every: 20
23
+ data_seed: 1234
24
+ noise_seed: 5678
25
+ val_seed: 99
26
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
27
+ lr_lora: 1.0e-4
28
+ lr_P: 2.0e-4
29
+ num_workers: 4
30
+ out_dir: /data0/syfe/lora_output/bernini_multishot_704
runs/bernini_p2_baseline/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a93b9727ebbdcc833986f927136874c39b80f5af68bd0e143a92f953ce27416d
3
+ size 33562808
runs/bernini_p2_baseline/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9fdfef9a90094e74ae75644893b3ea28503c217a0731c3b41b1c44c4b23ca79a
3
+ size 402758624
runs/bernini_p2_baseline/training_config.yaml ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bernini Phase-2 — BASELINE run (caption-only full-attn LoRA; P never used).
2
+ # IDENTICAL to plan.yaml except enable_plan:false + out_dir. Seeds UNCHANGED (1234 / 5678 / 99) so both
3
+ # runs see byte-identical noise + data order; the trained-adapter delta = the Bernini-P path alone.
4
+ run_name: bernini_p2_baseline
5
+ enable_plan: false
6
+
7
+ max_steps: 1500 # MUST match plan.yaml
8
+ save_every: 250
9
+ val_every: 250
10
+ log_every: 20
11
+
12
+ data_seed: 1234
13
+ noise_seed: 5678
14
+ val_seed: 99
15
+
16
+ lora:
17
+ rank: 64
18
+ alpha: 64
19
+ dropout: 0.0
20
+ lr_lora: 1.0e-4
21
+ lr_P: 2.0e-4
22
+
23
+ context_key: bare
24
+
25
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
26
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_train.jsonl
27
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_val.jsonl
28
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_killswitch/shards
29
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
30
+
31
+ num_workers: 4
32
+ out_dir: /data0/syfe/lora_output/bernini_p2_baseline
runs/bernini_p2_plan/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20fbd75fe9daba71cc89075bc904cb5ffe3194173cc6f886429ed5af87139ae6
3
+ size 33562808
runs/bernini_p2_plan/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5357b464740d6f56dfb7b8c98700df204087ca0d5afa9c134437e4a3b454b6e
3
+ size 402758624
runs/bernini_p2_plan/training_config.yaml ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bernini Phase-2 — PLAN run ([Gemma ; P(diff_mllm_contexts)] + full-attn LoRA).
2
+ # Baseline (configs/baseline.yaml) is IDENTICAL except enable_plan:false + out_dir. Same seeds => the only
3
+ # difference between the two trained adapters is the Bernini-P branch (clean R1 A/B).
4
+ run_name: bernini_p2_plan
5
+ enable_plan: true
6
+
7
+ max_steps: 1500 # kill-switch (~3.4 epochs over 441 clips, ~12h at ~25-40 s/step)
8
+ save_every: 250
9
+ val_every: 250
10
+ log_every: 20
11
+
12
+ # shared RNG (baseline MUST match) — identical noise + data order across the two runs
13
+ data_seed: 1234
14
+ noise_seed: 5678
15
+ val_seed: 99
16
+
17
+ lora:
18
+ rank: 64
19
+ alpha: 64 # alpha==rank -> merge scale 1.0 (exact reload)
20
+ dropout: 0.0
21
+ lr_lora: 1.0e-4
22
+ lr_P: 2.0e-4
23
+
24
+ context_key: bare # "bare" = diff_mllm_contexts (txt+vit); "vit" = plan-only slice (ablation)
25
+
26
+ # data (on 2220)
27
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
28
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_train.jsonl
29
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_val.jsonl
30
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_killswitch/shards
31
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
32
+
33
+ num_workers: 4
34
+ out_dir: /data0/syfe/lora_output/bernini_p2_plan
runs/bernini_r2v_msr_baseline/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a93b9727ebbdcc833986f927136874c39b80f5af68bd0e143a92f953ce27416d
3
+ size 33562808
runs/bernini_r2v_msr_baseline/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:570d59bc4fcbb9057b919592a47fd3381cca53230c43d1bed0ba8f617ad4f033
3
+ size 402758624
runs/bernini_r2v_msr_baseline/training_config.yaml ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # R2V leg: msr_baseline — ref VAE latents ONLY, NO plan (C3 only) = the controlled MSR analogue.
2
+ # The incumbent Bernini's reasoning must beat. Uses the SAME ref-latent injection code as plan_pure/plan_ref
3
+ # (only enable_plan differs) so the plan_pure - msr_baseline delta is attributable to reasoning alone.
4
+ # NOTE: this is a CONTROLLED analogue, NOT the production MSR (run_msr_dev.py); M4 also scores the real MSR.
5
+ run_name: bernini_r2v_msr_baseline
6
+ enable_plan: false # NO reasoning plan (P never used)
7
+ context_key: bare # irrelevant when enable_plan=false
8
+ enable_ref_latents: true # ref VAE latents only (C3)
9
+
10
+ max_steps: 1500
11
+ save_every: 250
12
+ val_every: 250
13
+ log_every: 20
14
+ data_seed: 1234
15
+ noise_seed: 5678
16
+ val_seed: 99
17
+
18
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
19
+ lr_lora: 1.0e-4
20
+ lr_P: 2.0e-4
21
+
22
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_w0
23
+ ref_time_gap: 0.5 # per-REF pre-roll (LTX port of src-id sep) — refs at -(k+1)*0.5 before target [0,7.04s]
24
+
25
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
26
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_train.jsonl
27
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_val.jsonl
28
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_killswitch/shards
29
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
30
+
31
+ num_workers: 4
32
+ out_dir: /data0/syfe/lora_output/bernini_r2v_msr_baseline
runs/bernini_r2v_plan_pure/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:182735c6e2505ee71292f33142cc108a6456c0f374bbcc1645a46a3d7283f5fd
3
+ size 33562808
runs/bernini_r2v_plan_pure/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e7f12892e8762184ed8400b7637efec63b24b07c6a9f26d4c78d3650c491996
3
+ size 402758624
runs/bernini_r2v_plan_pure/training_config.yaml ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # R2V leg: plan_pure — the reasoning plan (vit slice, NO raw ref-ViT) + ref latents.
2
+ # vs msr_baseline this is THE "does Bernini's reasoning beat MSR on identity" test (refs held constant via
3
+ # ref latents; the only difference is the reasoning plan). context_key=vit => C2+C3 only, no C1 raw ref-ViT.
4
+ run_name: bernini_r2v_plan_pure
5
+ enable_plan: true
6
+ context_key: vit # plan-only slice (cond_embeds_wotxt_wvit) — excludes raw ref-ViT (C1)
7
+ enable_ref_latents: true # ref VAE latents in the self-attn sequence (C3)
8
+
9
+ max_steps: 1500
10
+ save_every: 250
11
+ val_every: 250
12
+ log_every: 20
13
+ data_seed: 1234 # shared across ALL legs (identical noise + data order)
14
+ noise_seed: 5678
15
+ val_seed: 99
16
+
17
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
18
+ lr_lora: 1.0e-4
19
+ lr_P: 2.0e-4
20
+
21
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_w0
22
+ # per-REF pre-roll = LTX-native port of Bernini's per-source-id RoPE separation (LTX has no src-id axis).
23
+ # 0.5s/ref places ref k at t=-(k+1)*0.5, just before the target [0,7.04s] range (~0.33s/frame), each distinct
24
+ # -> no positional collision, refs disambiguable. TUNE on the smoke (verify ref positions vs target range).
25
+ ref_time_gap: 0.5
26
+
27
+ # data (r2v join = latents ∩ conditions ∩ r2v-contexts, built post-extraction; val = clean_crossclip only)
28
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
29
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_train.jsonl
30
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_val.jsonl
31
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_killswitch/shards
32
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
33
+
34
+ num_workers: 4
35
+ out_dir: /data0/syfe/lora_output/bernini_r2v_plan_pure
runs/bernini_r2v_plan_ref/checkpoints/bernini_P_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03c4580f801049875b0202ba3c7610ced8659d75a00f32ccbd48c899937dd589
3
+ size 33562808
runs/bernini_r2v_plan_ref/checkpoints/lora_weights_step_01500.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6cf95d7290c0a859ed8c7e337446b4a7f2a9b2d2da588ba7643efa2ec5d95d70
3
+ size 402758624
runs/bernini_r2v_plan_ref/training_config.yaml ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # R2V leg: plan_ref — full context (caption + raw ref-ViT + plan) + ref latents = C1+C2+C3.
2
+ # vs plan_pure = the C1 (raw ref-ViT in cross-attn) contribution / double-count check.
3
+ # IDENTICAL to plan_pure except context_key=bare. Same seeds.
4
+ run_name: bernini_r2v_plan_ref
5
+ enable_plan: true
6
+ context_key: bare # full diff_mllm_contexts (caption + ref-ViT + plan)
7
+ enable_ref_latents: true
8
+
9
+ max_steps: 1500
10
+ save_every: 250
11
+ val_every: 250
12
+ log_every: 20
13
+ data_seed: 1234
14
+ noise_seed: 5678
15
+ val_seed: 99
16
+
17
+ lora: {rank: 64, alpha: 64, dropout: 0.0}
18
+ lr_lora: 1.0e-4
19
+ lr_P: 2.0e-4
20
+
21
+ ref_latents_root: /data0/syfe/bernini_ltx/ref_latents_w0
22
+ ref_time_gap: 0.5 # per-REF pre-roll (LTX port of src-id sep) — refs at -(k+1)*0.5 before target [0,7.04s]
23
+
24
+ precomp_root: /data0/syfe/bernini_ltx/precomputed_w0
25
+ join_path: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_train.jsonl
26
+ val_join: /data0/syfe/bernini_ltx/preprocess_w0/w0_join_r2v_val.jsonl
27
+ bernini_dir: /data0/syfe/bernini_ltx/contexts_r2v_killswitch/shards
28
+ ckpt: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
29
+
30
+ num_workers: 4
31
+ out_dir: /data0/syfe/lora_output/bernini_r2v_plan_ref