Alissonerdx commited on
Commit
ae6647b
Β·
verified Β·
1 Parent(s): 967ae5e

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -42,3 +42,14 @@ examples/example_6_reference.mp4 filter=lfs diff=lfs merge=lfs -text
42
  examples/example_7_reference.mp4 filter=lfs diff=lfs merge=lfs -text
43
  examples/example_8_reference.mp4 filter=lfs diff=lfs merge=lfs -text
44
  examples/example_9_multi_task.mp4 filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
42
  examples/example_7_reference.mp4 filter=lfs diff=lfs merge=lfs -text
43
  examples/example_8_reference.mp4 filter=lfs diff=lfs merge=lfs -text
44
  examples/example_9_multi_task.mp4 filter=lfs diff=lfs merge=lfs -text
45
+ examples/refs/084b67fac2342669a7802a282dfb397d86279f8854a683f6ea183544561e1a7b.png filter=lfs diff=lfs merge=lfs -text
46
+ examples/refs/10cac154a98bf7c4b6320787c59563d63dcbb74a8f07f5e8d4229c8a2bb953c3.png filter=lfs diff=lfs merge=lfs -text
47
+ examples/refs/35e764bd898b6556d59bf91c14f8634db64432034b8a35e099ad1645e6e75ab6.png filter=lfs diff=lfs merge=lfs -text
48
+ examples/refs/3e57d239e1a327c569e518b748d84761c7301bf79fb82ef1d266040bc122bb62.png filter=lfs diff=lfs merge=lfs -text
49
+ examples/refs/3e5806092fab2571046bca535026065b852e8708463b224dac363f8992a0eba2.png filter=lfs diff=lfs merge=lfs -text
50
+ examples/refs/3ea5f738336d6afa6aca8b47d0d0288ccf53d7c334f623896b91d4c9bc47d07f.png filter=lfs diff=lfs merge=lfs -text
51
+ examples/refs/4a1c6d4c36e7b442890ca061cffc9e2c1bb2aca8ec4ae1128750752cf8ddabff.png filter=lfs diff=lfs merge=lfs -text
52
+ examples/refs/64ada2c934c988feea57f0597366684332e9ea4f25c8b2a96b779ab0d17664e4.png filter=lfs diff=lfs merge=lfs -text
53
+ examples/refs/7dce66ebc96daa0dc0ee9eea16606406b745d83ff0bc6bce3eff32865603dd13.png filter=lfs diff=lfs merge=lfs -text
54
+ examples/refs/7e43a6c8b433fed0f1382feb50a60482cdc159448b3191edfbad51301e588e31.png filter=lfs diff=lfs merge=lfs -text
55
+ examples/refs/80eb8745da114b62ae825a681332159792402dca88fb3e5a57ad7ed340572ed8.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -6,7 +6,6 @@ tags:
6
  - lora
7
  - video
8
  - video-editing
9
- - ltxv
10
  - ltx-2.3
11
  ---
12
 
@@ -134,18 +133,25 @@ conditioning β€” the model is driven entirely by the text prompt and the
134
  guide video. Trained on a balanced mix of **Add, Remove, Replace, Style**
135
  edits.
136
 
137
- ### What it's good for
138
 
139
- - Multi-task edits without dealing with reference images.
140
- - **Remove** and **Style** specifically β€” those aren't on v0.1's menu.
141
- - Cases where preparing an edited first frame (the motion-transfer route)
142
- isn't an option.
 
 
 
 
 
 
 
143
 
144
  ### How to use it
145
 
146
- This LoRA was fine-tuned on top of v0.1's lineage, so it expects v0.1 to be
147
- loaded with it. **Stack v0.1 + this LoRA together** at inference. If
148
- results look over-saturated, drop v0.1's strength to ~0.7 first.
149
 
150
  ### Limitations
151
 
@@ -159,9 +165,9 @@ Same imperative shape as v0.1, but the training set split into four very
159
  distinct caption styles. Match the one that fits the edit you want β€” the
160
  distribution is narrow and the model expects the right shape.
161
 
162
- The dataset has **~6000 Add, 6000 Remove, 6000 Replace, ~3840 Style**
163
- samples (~22k total). Captions below are real examples drawn from those
164
- buckets.
165
 
166
  #### Add β€” 15 to 30+ words, describe what to add and where
167
 
@@ -219,11 +225,10 @@ These are honest limits of the training distribution β€” don't expect them
219
  to work just because the model is multitask:
220
 
221
  - **No compositional prompts.** *"Add X and remove Y"*, *"Replace A with B
222
- and add C"*, etc. are **not** in the training set. Out of ~22k captions,
223
- only 13 combine two action verbs at all, and almost all of those are
224
- the "Remove X and replace with Y" idiom (which is really a single
225
- Replace). Pure multi-action edits will fall apart β€” split them into
226
- separate runs.
227
  - **No "change background" as a task.** Background is only used as a
228
  *positional reference* ("in the background", "on the wall in the
229
  background"). To swap the entire backdrop, phrase it as a **Replace**
@@ -240,14 +245,13 @@ to work just because the model is multitask:
240
  | If you want… | Use |
241
  |---|---|
242
  | Motion transfer (edit first frame externally, model copies motion) | **v0.1 motion transfer** |
243
- | Multi-task edits (add / remove / replace / style) driven only by prompt | **no-ref multitask r256** (stacked with v0.1) |
244
  | Strong identity transfer from a reference image (Add / Replace) | **Ref V2V** |
245
 
246
  ### How to wire the LoopingSampler
247
 
248
- Two standard LoRAs stacked, no sidecar. Chain through two ComfyUI
249
- LoraLoaders before the LoopingSampler: load **v0.1** first, then the
250
- **no-ref multitask** LoRA on top. On the sampler:
251
 
252
  - `editanything_module` β†’ **leave disconnected**.
253
  - `ref_image` β†’ **leave disconnected**. This LoRA has no reference-image
@@ -463,6 +467,44 @@ Tips for Ref V2V:
463
 
464
  ---
465
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
466
  ## ComfyUI nodes
467
 
468
  All recommended inference paths run through the **BFSnodes** custom node
@@ -471,6 +513,7 @@ stabilise they may move elsewhere.
471
 
472
  Specific nodes used by these LoRAs:
473
 
 
474
  - `πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping Sampler` β€” sampler that injects role /
475
  AdaLN / visual cross-attention and handles long videos in chunks.
476
  - `LTXVEditAnythingModuleLoader` β€” load the `*.module.safetensors` sidecar.
 
6
  - lora
7
  - video
8
  - video-editing
 
9
  - ltx-2.3
10
  ---
11
 
 
133
  guide video. Trained on a balanced mix of **Add, Remove, Replace, Style**
134
  edits.
135
 
136
+ ### What it's different about it (vs v0.1)
137
 
138
+ The task vocabulary overlaps heavily with v0.1 β€” both can do Add, Remove,
139
+ Replace, Change, Convert. What changes here:
140
+
141
+ - **Two-stage training continuation**: the first stage gave the model its
142
+ edit vocabulary; the second stage refined it on a larger, more balanced
143
+ video pair set covering Add / Remove / Replace / Style.
144
+ - **Rank 256** (vs v0.1's effective rank from the merge), giving more
145
+ capacity for the broader task mix.
146
+ - Trained directly on video pairs, so the temporal behaviour on these
147
+ tasks tends to be steadier than on a model whose first stage was on
148
+ still images.
149
 
150
  ### How to use it
151
 
152
+ **Standalone** β€” load it as a regular LoRA on vanilla LTX-2.3 through any
153
+ ComfyUI LoRA loader. The file already carries everything it needs; no
154
+ stacking with v0.1, no companion module.
155
 
156
  ### Limitations
157
 
 
165
  distinct caption styles. Match the one that fits the edit you want β€” the
166
  distribution is narrow and the model expects the right shape.
167
 
168
+ The training set is roughly balanced across **Add, Remove, Replace and
169
+ Style** buckets, with Style being the smallest of the four. Captions
170
+ below are real examples drawn from those buckets.
171
 
172
  #### Add β€” 15 to 30+ words, describe what to add and where
173
 
 
225
  to work just because the model is multitask:
226
 
227
  - **No compositional prompts.** *"Add X and remove Y"*, *"Replace A with B
228
+ and add C"*, etc. are **not** in the training set. Captions combining
229
+ two action verbs are essentially absent (the only ones present are the
230
+ "Remove X and replace with Y" idiom, which is really a single Replace).
231
+ Pure multi-action edits will fall apart β€” split them into separate runs.
 
232
  - **No "change background" as a task.** Background is only used as a
233
  *positional reference* ("in the background", "on the wall in the
234
  background"). To swap the entire backdrop, phrase it as a **Replace**
 
245
  | If you want… | Use |
246
  |---|---|
247
  | Motion transfer (edit first frame externally, model copies motion) | **v0.1 motion transfer** |
248
+ | Multi-task edits (add / remove / replace / style) driven only by prompt | **no-ref multitask r256** (standalone) |
249
  | Strong identity transfer from a reference image (Add / Replace) | **Ref V2V** |
250
 
251
  ### How to wire the LoopingSampler
252
 
253
+ A single **standard LoRA**, no sidecar, no stacking. Load through one
254
+ ComfyUI LoraLoader before the LoopingSampler. On the sampler:
 
255
 
256
  - `editanything_module` β†’ **leave disconnected**.
257
  - `ref_image` β†’ **leave disconnected**. This LoRA has no reference-image
 
467
 
468
  ---
469
 
470
+ ## Inference tips (applies to all models)
471
+
472
+ **CFG matters a lot here.** The default workflow runs with the LTX-2.3
473
+ **distilled / acceleration LoRAs** for fast 4–8 step sampling, which
474
+ locks **CFG = 1.0**. That's fine for casual runs, but at CFG 1 the model
475
+ follows the prompt loosely β€” you get the reference image to "show up"
476
+ but the edit instruction itself is only weakly enforced.
477
+
478
+ **For harder prompts, raise CFG above 1.0.** This means dropping (or
479
+ weakening) the distilled / acceleration LoRAs and going back to a normal
480
+ sampler with more steps β€” significantly slower, but the model follows
481
+ the prompt much more closely. Trade-off:
482
+
483
+ | Setup | CFG | Steps | Speed | Prompt following |
484
+ |---|---|---|---|---|
485
+ | Distilled / accel default | **1.0** | 4–8 | fast | loose |
486
+ | Non-distilled, mild | **3–4** | 20–30 | medium | good baseline |
487
+ | Non-distilled, aggressive | **5–7** | 25–40 | slow | strong, may saturate |
488
+
489
+ Other knobs:
490
+
491
+ - If the model is **ignoring the prompt** (edit isn't being applied, the
492
+ reference is barely showing up, the style transfer is faint), raising
493
+ CFG is the single most common fix. Go up to 6–8 if needed.
494
+ - If results look **over-saturated, plasticky, or motion is freezing**,
495
+ CFG is too high β€” pull back toward 3–4 or re-enable the distilled LoRA
496
+ for CFG 1 if you don't actually need stronger prompt adherence.
497
+ - Ref V2V in particular benefits from being more aggressive with CFG when
498
+ the reference identity isn't transferring cleanly.
499
+ - Combine CFG tuning with the LoRA-specific knobs from each section
500
+ (`adaln_scale`, `ref_context_scale`, `ref_token_scale` for Ref V2V;
501
+ prompt rewriting for v0.1 / no-ref).
502
+
503
+ Treat CFG as a real knob, not a constant β€” and be ready to give up some
504
+ speed when you actually need the edit to land.
505
+
506
+ ---
507
+
508
  ## ComfyUI nodes
509
 
510
  All recommended inference paths run through the **BFSnodes** custom node
 
513
 
514
  Specific nodes used by these LoRAs:
515
 
516
+ - `LTXVEditAnythingApply` β€” load the LoRA + extras and patch the model.
517
  - `πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping Sampler` β€” sampler that injects role /
518
  AdaLN / visual cross-attention and handles long videos in chunks.
519
  - `LTXVEditAnythingModuleLoader` β€” load the `*.module.safetensors` sidecar.
examples/refs/084b67fac2342669a7802a282dfb397d86279f8854a683f6ea183544561e1a7b.png ADDED

Git LFS Details

  • SHA256: 17d5e317cd8b4a83fcaeccf4b87a4e305d2e21362a4266d43861670358d61256
  • Pointer size: 131 Bytes
  • Size of remote file: 376 kB
examples/refs/10cac154a98bf7c4b6320787c59563d63dcbb74a8f07f5e8d4229c8a2bb953c3.png ADDED

Git LFS Details

  • SHA256: 3bbc8c905eed63a420a2536e14de8a25b408cdb343e79e86f5e0191d85a4ca50
  • Pointer size: 132 Bytes
  • Size of remote file: 1.31 MB
examples/refs/35e764bd898b6556d59bf91c14f8634db64432034b8a35e099ad1645e6e75ab6.png ADDED

Git LFS Details

  • SHA256: cba294fe842d1d53cdf005f35a5275636a0e5113ed2f82b5603fb61af8440266
  • Pointer size: 131 Bytes
  • Size of remote file: 790 kB
examples/refs/3e57d239e1a327c569e518b748d84761c7301bf79fb82ef1d266040bc122bb62.png ADDED

Git LFS Details

  • SHA256: 47cbaf507db255d7a3f086ec4bd085ccc9510ec69399cac133e0a5cbed54077e
  • Pointer size: 131 Bytes
  • Size of remote file: 593 kB
examples/refs/3e5806092fab2571046bca535026065b852e8708463b224dac363f8992a0eba2.png ADDED

Git LFS Details

  • SHA256: e7f2c2996944f35b36ab339dbafa537d2f4d464664f47870bbd7f7c84d858b42
  • Pointer size: 131 Bytes
  • Size of remote file: 980 kB
examples/refs/3ea5f738336d6afa6aca8b47d0d0288ccf53d7c334f623896b91d4c9bc47d07f.png ADDED

Git LFS Details

  • SHA256: 98690f62a1af3ff05043f799e6f9907439924954cda849c4d5598045c325cdd7
  • Pointer size: 131 Bytes
  • Size of remote file: 953 kB
examples/refs/4a1c6d4c36e7b442890ca061cffc9e2c1bb2aca8ec4ae1128750752cf8ddabff.png ADDED

Git LFS Details

  • SHA256: e5b84f139b93ffa14a7273dc2b6b2840e48f9c07e720cd6a18e77ba5d1484d30
  • Pointer size: 132 Bytes
  • Size of remote file: 1.24 MB
examples/refs/64ada2c934c988feea57f0597366684332e9ea4f25c8b2a96b779ab0d17664e4.png ADDED

Git LFS Details

  • SHA256: 1c881f082831082b34aae6994f45d9cd9e31e9ad66c758fbce5bf3744bece2ec
  • Pointer size: 132 Bytes
  • Size of remote file: 1.03 MB
examples/refs/7dce66ebc96daa0dc0ee9eea16606406b745d83ff0bc6bce3eff32865603dd13.png ADDED

Git LFS Details

  • SHA256: db193a1f7b4939b64eeb6fe01171659223d4ac3f5cfae796e2df03c297a70a78
  • Pointer size: 131 Bytes
  • Size of remote file: 893 kB
examples/refs/7e43a6c8b433fed0f1382feb50a60482cdc159448b3191edfbad51301e588e31.png ADDED

Git LFS Details

  • SHA256: 05e38fe1fbeacef2efe7704d36885a5ebbe573de023306516472a523b96e9340
  • Pointer size: 131 Bytes
  • Size of remote file: 434 kB
examples/refs/80eb8745da114b62ae825a681332159792402dca88fb3e5a57ad7ed340572ed8.png ADDED

Git LFS Details

  • SHA256: a4e44f772a795c5fa668be4f4d07964cf2c9e3867a0fbecf4635d8d387459c48
  • Pointer size: 131 Bytes
  • Size of remote file: 459 kB