Hugging Face's logo

SyFeee
/
LTX-2.3-SyFe-MSR-LoRA

Image-to-Video
LTX-2
ltx-video
ltx-2.3
lora
ic-lora
multi-reference
msr
chinese-drama
Model card Files
xet
 Community
SyFeee commited on
Commit
2c4d060
·
verified ·
1 Parent(s): 3531ffa

Add files using upload-large-folder tool

Browse files
Files changed (10) hide show
  1. README.md +35 -0
  2. SHA256SUMS +4 -0
  3. runs/msr_corpus36_run01/checkpoints/lora_weights_step_06000.safetensors +3 -0
  4. runs/msr_corpus36_run01/training_config.yaml +86 -0
  5. runs/msr_plain_01/checkpoints/lora_weights_step_05000.safetensors +3 -0
  6. runs/msr_plain_01/training_config.yaml +91 -0
  7. runs/msr_plain_02/checkpoints/lora_weights_step_05000.safetensors +3 -0
  8. runs/msr_plain_02/training_config.yaml +91 -0
  9. runs/msr_plain_rebuilt01/checkpoints/lora_weights_step_05000.safetensors +3 -0
  10. runs/msr_plain_rebuilt01/training_config.yaml +86 -0
README.md ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Lightricks/LTX-2.3
3
+ library_name: ltx
4
+ license: other
5
+ license_name: ltx-2-community-license
6
+ license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
7
+ pipeline_tag: image-to-video
8
+ tags:
9
+ - ltx-video
10
+ - ltx-2.3
11
+ - lora
12
+ - ic-lora
13
+ - multi-reference
14
+ - msr
15
+ - chinese-drama
16
+ ---
17
+
18
+ # SyFe LTX-2.3 MSR LoRA Checkpoints
19
+
20
+ Multiple Subject Reference LoRAs trained by SyFe on LTX-2.3 22B-dev. Multiple subject and scene images are encoded as reference-video latents so target tokens can retrieve them through native self-attention.
21
+
22
+ ## Checkpoints
23
+
24
+ | Run | Data / construction | Rank | Steps | Status |
25
+ |---|---|---:|---:|---|
26
+ | `msr_plain_01` | Initial crop-fill references | 128 | 5,000 | Archived: crop-fill can crop full-body subjects |
27
+ | `msr_plain_02` | Correct white-canvas, never-crop subjects | 128 | 5,000 | Validated in the combined talking stack |
28
+ | `msr_plain_rebuilt01` | Rebuilt intermediate corpus | 128 | 5,000 | Superseded experiment |
29
+ | `msr_corpus36_run01` | 35 shows, 31,500 true-bilingual samples | 128 | 6,000 | Recommended; deployed final |
30
+
31
+ For `msr_corpus36_run01`, reference subjects must be contain-fit without cropping on a white canvas; scene references may be cover-fit. Prompts should begin with the ordered subject markers, for example `[VISUAL]: char_1_person, char_2_person...`, because marker order binds prompt subjects to reference slots.
32
+
33
+ The recommended checkpoint materially improves out-of-show costume, prop, hair, and scene adherence and removes memorized-cast substitution. Exact facial identity can still drift, especially when faces occupy few pixels. Two-subject reliability is not perfect.
34
+
35
+ The third-party `Licon-MSR-V1` checkpoint is intentionally not included. Use is subject to the LTX-2 community license.
SHA256SUMS ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ 979db91020b6e5fb22ee2a33f5c97ca5dfc6eab00950efed05fc7009157da3e1 runs/msr_corpus36_run01/checkpoints/lora_weights_step_06000.safetensors
2
+ 73dc6de8f76721b26a14d14868d85cbb7fcb0036dca917487760c5fa45c28d9f runs/msr_plain_rebuilt01/checkpoints/lora_weights_step_05000.safetensors
3
+ 86b923de2f9bee0ed9f23607b32d2db6ccc6163e985d55d67566447f972660c1 runs/msr_plain_01/checkpoints/lora_weights_step_05000.safetensors
4
+ a85a44b62f7929c0e7978bf7e17a65639b996de4d2e957db3f9692e92be40c9d runs/msr_plain_02/checkpoints/lora_weights_step_05000.safetensors
runs/msr_corpus36_run01/checkpoints/lora_weights_step_06000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:979db91020b6e5fb22ee2a33f5c97ca5dfc6eab00950efed05fc7009157da3e1
3
+ size 1711612856
runs/msr_corpus36_run01/training_config.yaml ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ acceleration:
2
+ load_text_encoder_in_8bit: false
3
+ mixed_precision_mode: bf16
4
+ quantization: null
5
+ checkpoints:
6
+ interval: 500
7
+ keep_last_n: 4
8
+ no_resume: false
9
+ precision: bfloat16
10
+ save_training_state: minimal
11
+ data:
12
+ num_dataloader_workers: 2
13
+ preprocessed_data_root: /data0/syfe/lora_output/msr_corpus36_run01/preprocessed
14
+ flow_matching:
15
+ timestep_sampling_mode: shifted_logit_normal
16
+ timestep_sampling_params: {}
17
+ hub:
18
+ hub_model_id: null
19
+ push_to_hub: false
20
+ lora:
21
+ alpha: 128
22
+ dropout: 0.0
23
+ rank: 128
24
+ target_modules:
25
+ - to_k
26
+ - to_q
27
+ - to_v
28
+ - to_out.0
29
+ model:
30
+ load_checkpoint: null
31
+ model_path: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
32
+ text_encoder_path: /data0/pretrain_models/for_text/google/gemma-3-12b-it-qat-q4_0-unquantized
33
+ training_mode: lora
34
+ optimization:
35
+ batch_size: 1
36
+ enable_gradient_checkpointing: true
37
+ gradient_accumulation_steps: 1
38
+ learning_rate: 0.0001
39
+ max_grad_norm: 1.0
40
+ optimizer_type: adamw
41
+ scheduler_params: {}
42
+ scheduler_type: linear
43
+ steps: 6000
44
+ output_dir: /data0/syfe/lora_output/msr_corpus36_run01
45
+ seed: 42
46
+ training_strategy:
47
+ first_frame_conditioning_p: 0.2
48
+ name: video_to_video
49
+ reference_latents_dir: reference_latents
50
+ validation:
51
+ frame_rate: 24.0
52
+ generate_audio: false
53
+ guidance_scale: 4.0
54
+ images: null
55
+ include_reference_in_output: false
56
+ inference_steps: 20
57
+ interval: 500
58
+ negative_prompt: no CGI, no animation, no illustration, no painterly style, no anime,
59
+ blurry, distorted, inconsistent appearance
60
+ prompts:
61
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
62
+ \u666F\u4E2D\u5E76\u80A9\u800C\u7ACB\u3001\u4F4E\u58F0\u4EA4\u8C08\uFF0C\u53C2\
63
+ \u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\u7535\u5F71\u611F\
64
+ \uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
65
+ reference_downscale_factor: 1
66
+ reference_videos:
67
+ - /data0/dataset/v1/tv-306954e6d5ca5f49b89424a3e6847232/001/refs/001_shot_0362_win_00/msr_refvideo.mp4
68
+ seed: 42
69
+ skip_initial_validation: true
70
+ stg_blocks:
71
+ - 29
72
+ stg_mode: stg_v
73
+ stg_scale: 1.0
74
+ video_dims: !!python/tuple
75
+ - 1280
76
+ - 704
77
+ - 65
78
+ videos_per_prompt: 1
79
+ wandb:
80
+ enabled: false
81
+ entity: null
82
+ log_validation_videos: true
83
+ project: ltx-2-trainer
84
+ tags:
85
+ - ltx2
86
+ - lora
runs/msr_plain_01/checkpoints/lora_weights_step_05000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86b923de2f9bee0ed9f23607b32d2db6ccc6163e985d55d67566447f972660c1
3
+ size 1711612856
runs/msr_plain_01/training_config.yaml ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ acceleration:
2
+ load_text_encoder_in_8bit: false
3
+ mixed_precision_mode: bf16
4
+ quantization: null
5
+ checkpoints:
6
+ interval: 500
7
+ keep_last_n: 4
8
+ no_resume: false
9
+ precision: bfloat16
10
+ save_training_state: minimal
11
+ data:
12
+ num_dataloader_workers: 2
13
+ preprocessed_data_root: /data0/syfe/lora_output/msr_plain_01/preprocessed
14
+ flow_matching:
15
+ timestep_sampling_mode: shifted_logit_normal
16
+ timestep_sampling_params: {}
17
+ hub:
18
+ hub_model_id: null
19
+ push_to_hub: false
20
+ lora:
21
+ alpha: 128
22
+ dropout: 0.0
23
+ rank: 128
24
+ target_modules:
25
+ - to_k
26
+ - to_q
27
+ - to_v
28
+ - to_out.0
29
+ model:
30
+ load_checkpoint: null
31
+ model_path: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
32
+ text_encoder_path: /data0/pretrain_models/for_text/google/gemma-3-12b-it-qat-q4_0-unquantized
33
+ training_mode: lora
34
+ optimization:
35
+ batch_size: 1
36
+ enable_gradient_checkpointing: true
37
+ gradient_accumulation_steps: 1
38
+ learning_rate: 0.0001
39
+ max_grad_norm: 1.0
40
+ optimizer_type: adamw
41
+ scheduler_params: {}
42
+ scheduler_type: linear
43
+ steps: 5000
44
+ output_dir: /data0/syfe/lora_output/msr_plain_01
45
+ seed: 42
46
+ training_strategy:
47
+ first_frame_conditioning_p: 0.2
48
+ name: video_to_video
49
+ reference_latents_dir: reference_latents
50
+ validation:
51
+ frame_rate: 24.0
52
+ generate_audio: false
53
+ guidance_scale: 4.0
54
+ images: null
55
+ include_reference_in_output: false
56
+ inference_steps: 20
57
+ interval: 500
58
+ negative_prompt: no CGI, no animation, no illustration, no painterly style, no anime,
59
+ blurry, distorted, inconsistent appearance
60
+ prompts:
61
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
62
+ \u666F\u4E2D\u5E76\u80A9\u800C\u7ACB\u3001\u4F4E\u58F0\u4EA4\u8C08\uFF0C\u53C2\
63
+ \u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\u7535\u5F71\u611F\
64
+ \uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
65
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
66
+ \u666F\u4E2D\u9762\u5BF9\u9762\u4EA4\u8C08\uFF0C\u4E00\u4EBA\u5F00\u53E3\u8BF4\
67
+ \u8BDD\uFF0C\u53C2\u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\
68
+ \u7535\u5F71\u611F\uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
69
+ reference_downscale_factor: 1
70
+ reference_videos:
71
+ - /data0/syfe/lora_output/msr_plain_v1/val_refs/val_pair_a.mp4
72
+ - /data0/syfe/lora_output/msr_plain_v1/val_refs/val_pair_b.mp4
73
+ seed: 42
74
+ skip_initial_validation: true
75
+ stg_blocks:
76
+ - 29
77
+ stg_mode: stg_v
78
+ stg_scale: 1.0
79
+ video_dims: !!python/tuple
80
+ - 1280
81
+ - 704
82
+ - 145
83
+ videos_per_prompt: 1
84
+ wandb:
85
+ enabled: false
86
+ entity: null
87
+ log_validation_videos: true
88
+ project: ltx-2-trainer
89
+ tags:
90
+ - ltx2
91
+ - lora
runs/msr_plain_02/checkpoints/lora_weights_step_05000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a85a44b62f7929c0e7978bf7e17a65639b996de4d2e957db3f9692e92be40c9d
3
+ size 1711612856
runs/msr_plain_02/training_config.yaml ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ acceleration:
2
+ load_text_encoder_in_8bit: false
3
+ mixed_precision_mode: bf16
4
+ quantization: null
5
+ checkpoints:
6
+ interval: 500
7
+ keep_last_n: 4
8
+ no_resume: false
9
+ precision: bfloat16
10
+ save_training_state: minimal
11
+ data:
12
+ num_dataloader_workers: 2
13
+ preprocessed_data_root: /data0/syfe/lora_output/msr_plain_02/preprocessed
14
+ flow_matching:
15
+ timestep_sampling_mode: shifted_logit_normal
16
+ timestep_sampling_params: {}
17
+ hub:
18
+ hub_model_id: null
19
+ push_to_hub: false
20
+ lora:
21
+ alpha: 128
22
+ dropout: 0.0
23
+ rank: 128
24
+ target_modules:
25
+ - to_k
26
+ - to_q
27
+ - to_v
28
+ - to_out.0
29
+ model:
30
+ load_checkpoint: null
31
+ model_path: /data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
32
+ text_encoder_path: /data0/pretrain_models/for_text/google/gemma-3-12b-it-qat-q4_0-unquantized
33
+ training_mode: lora
34
+ optimization:
35
+ batch_size: 1
36
+ enable_gradient_checkpointing: true
37
+ gradient_accumulation_steps: 1
38
+ learning_rate: 0.0001
39
+ max_grad_norm: 1.0
40
+ optimizer_type: adamw
41
+ scheduler_params: {}
42
+ scheduler_type: linear
43
+ steps: 5000
44
+ output_dir: /data0/syfe/lora_output/msr_plain_02
45
+ seed: 42
46
+ training_strategy:
47
+ first_frame_conditioning_p: 0.2
48
+ name: video_to_video
49
+ reference_latents_dir: reference_latents
50
+ validation:
51
+ frame_rate: 24.0
52
+ generate_audio: false
53
+ guidance_scale: 4.0
54
+ images: null
55
+ include_reference_in_output: false
56
+ inference_steps: 20
57
+ interval: 500
58
+ negative_prompt: no CGI, no animation, no illustration, no painterly style, no anime,
59
+ blurry, distorted, inconsistent appearance
60
+ prompts:
61
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
62
+ \u666F\u4E2D\u5E76\u80A9\u800C\u7ACB\u3001\u4F4E\u58F0\u4EA4\u8C08\uFF0C\u53C2\
63
+ \u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\u7535\u5F71\u611F\
64
+ \uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
65
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
66
+ \u666F\u4E2D\u9762\u5BF9\u9762\u4EA4\u8C08\uFF0C\u4E00\u4EBA\u5F00\u53E3\u8BF4\
67
+ \u8BDD\uFF0C\u53C2\u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\
68
+ \u7535\u5F71\u611F\uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
69
+ reference_downscale_factor: 1
70
+ reference_videos:
71
+ - /data0/syfe/lora_output/msr_plain_v1/val_refs/val_pair_a.mp4
72
+ - /data0/syfe/lora_output/msr_plain_v1/val_refs/val_pair_b.mp4
73
+ seed: 42
74
+ skip_initial_validation: true
75
+ stg_blocks:
76
+ - 29
77
+ stg_mode: stg_v
78
+ stg_scale: 1.0
79
+ video_dims: !!python/tuple
80
+ - 1280
81
+ - 704
82
+ - 145
83
+ videos_per_prompt: 1
84
+ wandb:
85
+ enabled: false
86
+ entity: null
87
+ log_validation_videos: true
88
+ project: ltx-2-trainer
89
+ tags:
90
+ - ltx2
91
+ - lora
runs/msr_plain_rebuilt01/checkpoints/lora_weights_step_05000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73dc6de8f76721b26a14d14868d85cbb7fcb0036dca917487760c5fa45c28d9f
3
+ size 1711612856
runs/msr_plain_rebuilt01/training_config.yaml ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ acceleration:
2
+ load_text_encoder_in_8bit: false
3
+ mixed_precision_mode: bf16
4
+ quantization: null
5
+ checkpoints:
6
+ interval: 500
7
+ keep_last_n: 4
8
+ no_resume: false
9
+ precision: bfloat16
10
+ save_training_state: minimal
11
+ data:
12
+ num_dataloader_workers: 2
13
+ preprocessed_data_root: /mnt/20.5-data0/syfe/lora_output/msr_plain_rebuilt01/preprocessed
14
+ flow_matching:
15
+ timestep_sampling_mode: shifted_logit_normal
16
+ timestep_sampling_params: {}
17
+ hub:
18
+ hub_model_id: null
19
+ push_to_hub: false
20
+ lora:
21
+ alpha: 128
22
+ dropout: 0.0
23
+ rank: 128
24
+ target_modules:
25
+ - to_k
26
+ - to_q
27
+ - to_v
28
+ - to_out.0
29
+ model:
30
+ load_checkpoint: null
31
+ model_path: /mnt/20.5-data0/pretrain_models/for_video/Lightricks/LTX-2.3/ltx-2.3-22b-dev.safetensors
32
+ text_encoder_path: /mnt/20.5-data0/pretrain_models/for_text/google/gemma-3-12b-it-qat-q4_0-unquantized
33
+ training_mode: lora
34
+ optimization:
35
+ batch_size: 1
36
+ enable_gradient_checkpointing: true
37
+ gradient_accumulation_steps: 1
38
+ learning_rate: 0.0001
39
+ max_grad_norm: 1.0
40
+ optimizer_type: adamw
41
+ scheduler_params: {}
42
+ scheduler_type: linear
43
+ steps: 5000
44
+ output_dir: /mnt/20.5-data0/syfe/lora_output/msr_plain_rebuilt01
45
+ seed: 42
46
+ training_strategy:
47
+ first_frame_conditioning_p: 0.2
48
+ name: video_to_video
49
+ reference_latents_dir: reference_latents
50
+ validation:
51
+ frame_rate: 24.0
52
+ generate_audio: false
53
+ guidance_scale: 4.0
54
+ images: null
55
+ include_reference_in_output: false
56
+ inference_steps: 20
57
+ interval: 500
58
+ negative_prompt: no CGI, no animation, no illustration, no painterly style, no anime,
59
+ blurry, distorted, inconsistent appearance
60
+ prompts:
61
+ - "\u53C2\u8003\u56FE1\u3001\u53C2\u8003\u56FE2 \u4E24\u4F4D\u4EBA\u7269\u5728\u573A\
62
+ \u666F\u4E2D\u5E76\u80A9\u800C\u7ACB\u3001\u4F4E\u58F0\u4EA4\u8C08\uFF0C\u53C2\
63
+ \u8003\u56FE3\u4E3A\u80CC\u666F\u573A\u666F\u3002\u771F\u4EBA\u7535\u5F71\u611F\
64
+ \uFF0C\u5199\u5B9E\u4E2D\u5F0F\u53E4\u88C5\u5267"
65
+ reference_downscale_factor: 1
66
+ reference_videos:
67
+ - /data0/dataset/v1/tv-306954e6d5ca5f49b89424a3e6847232/001/refs/001_shot_0362_win_00/msr_refvideo.mp4
68
+ seed: 42
69
+ skip_initial_validation: true
70
+ stg_blocks:
71
+ - 29
72
+ stg_mode: stg_v
73
+ stg_scale: 1.0
74
+ video_dims: !!python/tuple
75
+ - 1280
76
+ - 704
77
+ - 65
78
+ videos_per_prompt: 1
79
+ wandb:
80
+ enabled: false
81
+ entity: null
82
+ log_validation_videos: true
83
+ project: ltx-2-trainer
84
+ tags:
85
+ - ltx2
86
+ - lora