Alissonerdx commited on
Commit
e85a55d
Β·
verified Β·
1 Parent(s): cf11885

Add Edit Anything v2 (LTX-2.5) section: usage, prompt format, training

Browse files
Files changed (1) hide show
  1. README.md +660 -539
README.md CHANGED
@@ -1,539 +1,660 @@
1
- ---
2
- license: apache-2.0
3
- library_name: diffusers
4
- base_model: Lightricks/LTX-2.3
5
- tags:
6
- - lora
7
- - video
8
- - video-editing
9
- - ltx-2.3
10
- ---
11
-
12
- # Edit Anything β€” Experimental LTX-2 Video Editing LoRAs
13
-
14
- > **Heads up.** These LoRAs are research experiments. They are far from
15
- > production-ready and will fail on many inputs. They are released for the
16
- > community to play with and break, not as a finished tool.
17
-
18
- This repository hosts three unrelated training tracks built on top of
19
- **LTX-2.3 (22B)** for video editing:
20
-
21
- 1. **Edit Anything v0.1 β€” motion transfer LoRA** (two ranks).
22
- 2. **Edit Anything β€” no-reference multitask LoRA** (rank 256, prompt-driven only).
23
- 3. **Reference video-to-video (Ref V2V) β€” experimental IC-LoRA + sidecar modules** (two builds).
24
-
25
- Inference is meant to run through the **BFSnodes** ComfyUI custom nodes β€”
26
- the Ref V2V build in particular needs them to load the sidecar modules and
27
- install the custom branches into the transformer.
28
-
29
- ---
30
-
31
- ## 1. Edit Anything v0.1 (motion transfer)
32
-
33
- Files:
34
-
35
- - `edit_anything_30k_v0.1_motion_transfer_r128.safetensors`
36
- - `edit_anything_30k_v0.1_motion_transfer_r256.safetensors`
37
-
38
- ### What it is
39
-
40
- **v0.1 is not a direct continuation of v1.0.** It was trained from scratch
41
- in two stages:
42
-
43
- 1. **Stage 1 β€” image-only pretraining.** ~30 000 image edit pairs. Training
44
- a *video* model on still images is admittedly not ideal, but it was a way
45
- to push the editing vocabulary beyond what a small video-only dataset can
46
- teach.
47
- 2. **Stage 2 β€” video fine-tune with `first_frame_conditioning > 0`.** This
48
- restored the temporal prior and unlocked the motion-transfer behaviour
49
- described below.
50
-
51
- In theory v0.1 can do the same edits as v1.0, but **temporal consistency may
52
- be weaker than v1.0** because so much of stage 1 happened on still images.
53
- Test against v1.0 case-by-case before assuming v0.1 wins on your task.
54
-
55
- ### Motion transfer
56
-
57
- Because stage 2 included first-frame conditioning, you can drive the LoRA
58
- into a motion-transfer mode:
59
-
60
- 1. Take a guide video.
61
- 2. **Replace its first frame** with an edited still (insert a new subject,
62
- swap an object, etc.). Use a strong image-editing model β€” **Flux Klein**
63
- or similar β€” to prepare it; the quality of this single frame propagates
64
- through the whole clip.
65
- 3. Feed the edited frame as the first frame of the input, and the original
66
- guide video as the motion source.
67
-
68
- The model uses the new first frame as the appearance anchor and copies the
69
- motion from the rest of the guide.
70
-
71
- Limitations (these are real, not theoretical β€” expect them to bite):
72
-
73
- - **Hard scene cuts break it.** The model assumes continuous motion from
74
- the first frame onwards. A cut to a different camera angle or location
75
- mid-clip will produce smearing, ghosting, or the inserted subject jumping
76
- to the wrong position. Use clips without cuts, or split at the cuts and
77
- process each segment separately.
78
- - **Very fast motion fails.** Quick pans, fast subject movement, or
79
- high-velocity action confuse the motion-copy mechanism. Outputs degrade
80
- to blur or to the model "freezing" on the first-frame appearance and
81
- losing the motion entirely. Stick to moderate-speed clips.
82
- - Poor blending / artefacts in the first frame propagate everywhere.
83
- - Works best when the inserted subject roughly occupies the same region as
84
- whatever it replaces.
85
-
86
- ### Prompting
87
-
88
- Prompt is just as critical as in v1.0. **Describe both the object being
89
- replaced and the new one in detail**. Example: *"Replace the bronze statue on
90
- the left with a tall man wearing a navy raincoat and brown boots."* Vague
91
- prompts produce bad edits.
92
-
93
- ### Which rank to use
94
-
95
- The same training produced both files. v0.1 is actually the merge of the
96
- two-stage training (one LoRA per stage), re-extracted at two different ranks
97
- via Frobenius-optimal truncated SVD:
98
-
99
- | File | Rank | Size | Frobenius retention |
100
- |---|---|---|---|
101
- | `edit_anything_30k_v0.1_motion_transfer_r128.safetensors` | 128 | 1.31 GB | ~99.4% |
102
- | `edit_anything_30k_v0.1_motion_transfer_r256.safetensors` | 256 | 2.62 GB | ~99.9% |
103
-
104
- r256 is closer to the merged source. r128 is normally indistinguishable in
105
- practice. Pick whichever fits your workflow.
106
-
107
- ### How to wire the LoopingSampler
108
-
109
- This is a **standard LoRA**, not a sidecar. Load it through the regular
110
- ComfyUI LoraLoader **before** the LoopingSampler. On the sampler itself:
111
-
112
- - `editanything_module` β†’ **leave disconnected**.
113
- - `ref_image` β†’ the edited first frame (for motion transfer) **or** the
114
- source frame you want preserved (for plain editing).
115
- - `guide_frames` β†’ the guide video.
116
- - `enable_role_embedding`, `enable_adaln`, `enable_visual_crossattn` β†’
117
- all **off**. None of those branches were trained for v0.1; turning them
118
- on with no module connected does nothing anyway, but keeping them off
119
- silences the WARN logs.
120
-
121
- ---
122
-
123
- ## 2. Edit Anything β€” no-reference multitask LoRA
124
-
125
- File:
126
-
127
- - `edit_anything_v1.1_r256.safetensors`
128
-
129
- ### What it is
130
-
131
- A **prompt-only** multitask editing LoRA. No reference image, no first-frame
132
- conditioning β€” the model is driven entirely by the text prompt and the
133
- guide video. Trained on a balanced mix of **Add, Remove, Replace, Style**
134
- edits.
135
-
136
- ### What it's different about it (vs v0.1)
137
-
138
- The task vocabulary overlaps heavily with v0.1 β€” both can do Add, Remove,
139
- Replace, Change, Convert. What changes here:
140
-
141
- - **Two-stage training continuation**: the first stage gave the model its
142
- edit vocabulary; the second stage refined it on a larger, more balanced
143
- video pair set covering Add / Remove / Replace / Style.
144
- - **Rank 256** (vs v0.1's effective rank from the merge), giving more
145
- capacity for the broader task mix.
146
- - Trained directly on video pairs, so the temporal behaviour on these
147
- tasks tends to be steadier than on a model whose first stage was on
148
- still images.
149
-
150
- ### How to use it
151
-
152
- **Standalone** β€” load it as a regular LoRA on vanilla LTX-2.3 through any
153
- ComfyUI LoRA loader. The file already carries everything it needs; no
154
- stacking with v0.1, no companion module.
155
-
156
- ### Limitations
157
-
158
- - No reference image β†’ identity is not anchored, so Add / Replace of a
159
- specific person or object will be wobblier than the Ref V2V build.
160
- - No motion transfer (that's v0.1 only).
161
-
162
- ### Prompting
163
-
164
- Same imperative shape as v0.1, but the training set split into four very
165
- distinct caption styles. Match the one that fits the edit you want β€” the
166
- distribution is narrow and the model expects the right shape.
167
-
168
- The training set is roughly balanced across **Add, Remove, Replace and
169
- Style** buckets, with Style being the smallest of the four. Captions
170
- below are real examples drawn from those buckets.
171
-
172
- #### Add β€” 15 to 30+ words, describe what to add and where
173
-
174
- * `Add a smiling woman with brown hair, wearing a pink sleeveless top, sitting to the right of the man at the news desk.`
175
- * `Add a person wearing a blue denim shirt over a white t-shirt to the right side of the frame, behind the person cooking.`
176
- * `Add a decorated Christmas tree with red and white ornaments and lights to the right of the man.`
177
- * `Add a blonde boy wearing a black t-shirt with a blue collar and blue patterned pants, sitting behind the other children in the upper center of the frame.`
178
- * `Add two horizontal wooden strips to the front of the white range hood.`
179
-
180
- Pattern: `Add <detailed subject description>, <position in frame>, <surrounding context>.`
181
-
182
- #### Remove β€” very short, 4 to 10 words
183
-
184
- * `Remove the man drinking from a glass.`
185
- * `Remove the disco ball.`
186
- * `Remove the large tree on the right.`
187
- * `Remove the squirrel in the foreground.`
188
- * `Remove the man on the left.`
189
-
190
- Pattern: `Remove the <object>` (+ optional position). Resist the urge to
191
- over-describe β€” long Remove prompts drift outside the training shape and
192
- often fail.
193
-
194
- #### Replace β€” 20 to 35 words, describe both old and new
195
-
196
- * `Replace the white panel door on the right side of the frame with a dark brown grandfather clock.`
197
- * `Replace the light-colored cat lying on the mat on the floor with a young woman sitting on the mat.`
198
- * `Replace the dark grey knitted sweater on the man's torso with a black and white patterned Christmas sweater.`
199
- * `Replace the blue robot with a glowing blue face on the left with a smiling man wearing sunglasses and a blue shirt.`
200
- * `Replace the sitting person wearing a black cape on the left with a black fabric draped over an object.`
201
-
202
- Pattern: `Replace <description of the original subject and its location> with <description of the new subject>.`
203
-
204
- #### Style β€” fixed template, the style name is what changes
205
-
206
- * `Convert the video into a Pencil Sketch style.`
207
- * `Convert the video into a Watercolor Painting style.`
208
- * `Convert the video into a Van Gogh style.`
209
- * `Convert the video into a Play-Doh style.`
210
- * `Convert the video into a Claymation style.`
211
- * `Convert the video into a 3D Chibi style.`
212
- * `Convert the video into a Ghibli style.`
213
- * `Convert the video into a Pop Art style.`
214
- * `Convert the video into an American Cartoon style.`
215
- * `Convert the video into a Flat Vector Cartoon style.`
216
-
217
- The training set covers **300+ distinct style names**. Many work; many do
218
- not. The list above is heavily represented in training. Use the exact
219
- phrase `Convert the video into a <STYLE> style` β€” deviations from this
220
- template degrade quality noticeably.
221
-
222
- #### What it does *not* do
223
-
224
- These are honest limits of the training distribution β€” don't expect them
225
- to work just because the model is multitask:
226
-
227
- - **No compositional prompts.** *"Add X and remove Y"*, *"Replace A with B
228
- and add C"*, etc. are **not** in the training set. Captions combining
229
- two action verbs are essentially absent (the only ones present are the
230
- "Remove X and replace with Y" idiom, which is really a single Replace).
231
- Pure multi-action edits will fall apart β€” split them into separate runs.
232
- - **No "change background" as a task.** Background is only used as a
233
- *positional reference* ("in the background", "on the wall in the
234
- background"). To swap the entire backdrop, phrase it as a **Replace**
235
- on a concrete background element, e.g.
236
- *"Replace the brick wall in the background with a forest at sunset"*.
237
- Vague prompts like *"Change the background to a beach"* are off-
238
- distribution and rarely work.
239
- - **No global colour grade / lighting change.** Only the Style template
240
- is trained as a global transform. Anything else global (LUT-style
241
- adjustments, time-of-day swaps without a concrete object) is unreliable.
242
-
243
- ### Which LoRA should I use?
244
-
245
- | If you want… | Use |
246
- |---|---|
247
- | Motion transfer (edit first frame externally, model copies motion) | **v0.1 motion transfer** |
248
- | Multi-task edits (add / remove / replace / style) driven only by prompt | **no-ref multitask r256** (standalone) |
249
- | Strong identity transfer from a reference image (Add / Replace) | **Ref V2V** |
250
-
251
- ### How to wire the LoopingSampler
252
-
253
- A single **standard LoRA**, no sidecar, no stacking. Load through one
254
- ComfyUI LoraLoader before the LoopingSampler. On the sampler:
255
-
256
- - `editanything_module` β†’ **leave disconnected**.
257
- - `ref_image` β†’ **leave disconnected**. This LoRA has no reference-image
258
- path; passing one will just pre-encode tokens that you do not want.
259
- - `guide_frames` β†’ the guide video.
260
- - `enable_role_embedding`, `enable_adaln`, `enable_visual_crossattn` β†’
261
- all **off** (no module = nothing to inject anyway).
262
-
263
- ---
264
-
265
- ## 3. Reference video-to-video (Ref V2V) β€” experimental
266
-
267
- Files (two builds of the same LoRA family β€” each ships as a `(.standard, .module)` pair):
268
-
269
- - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.standard.safetensors`
270
- - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.module.safetensors`
271
- - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.standard.safetensors`
272
- - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.module.safetensors`
273
-
274
- ### What it is
275
-
276
- The goal is **add / replace using a reference image** β€” same vibe as Edit
277
- Anything v1.0, but with an explicit image as the appearance source instead
278
- of relying only on the prompt.
279
-
280
- Trained on **~1600** Add / Replace video pairs. Reference-paired video
281
- datasets are basically nonexistent, so the dataset had to be built from
282
- scratch β€” that is why the sample count is small. **It often fails.** This
283
- is fully experimental; thousands of training runs went into landing on this
284
- LoRA layout, and it is still unclear how much it actually helps.
285
-
286
- ### Architecture β€” why this LoRA has "modules"
287
-
288
- Trained as a conventional IC-LoRA, plus extra projection branches that try
289
- to make the reference signal survive across layers:
290
-
291
- - **`ref_visual_proj`** β€” projects the reference VAE latent into 32 visual
292
- memory tokens.
293
- - **`ref_attn`** β€” a dedicated cross-attention branch inside each
294
- transformer block, reading those tokens.
295
- - **`ref_adaln_proj`** β€” a global AdaLN bias derived from the reference
296
- (palette / overall look).
297
- - **`role_embedding`** β€” an experimental token bias inspired by some of
298
- Kijai's tests; whether it actually helps is still unclear.
299
-
300
- These extra weights are saved alongside the LoRA in a `.module.safetensors`
301
- sidecar because they are **not standard LoRA adapters** β€” the regular
302
- ComfyUI LoRA loader can't consume them, so they need a dedicated node.
303
-
304
- ### How to load
305
-
306
- | File | What it is | Where it goes |
307
- |---|---|---|
308
- | `*.standard.safetensors` | LoRA on `attn1` / `attn2` / `ff` only | Standard ComfyUI LoRA loader |
309
- | `*.module.safetensors` | `role_embedding`, `ref_adaln_proj`, `ref_visual_proj`, `ref_attn` LoRA adapters | `LTXVEditAnythingModuleLoader` (BFSnodes) |
310
-
311
- Both files of a pair must be loaded **together** β€” the LoRA was trained
312
- against the sidecar adapters and they only make sense as a unit. Do not mix
313
- `.standard` from one build with `.module` from another.
314
-
315
- The module file is consumed by the **`πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping
316
- Sampler`** node, which was written specifically to:
317
-
318
- 1. Install the `ref_attn` cross-attention branch on every transformer block.
319
- 2. Inject the AdaLN / role / visual cross-attention conditioning at the
320
- correct points in the model.
321
- 3. Sample long videos in overlapping chunks with the conditioning re-applied
322
- per chunk.
323
-
324
- ### How to wire the LoopingSampler
325
-
326
- - Load the `*.standard.safetensors` through a normal ComfyUI LoraLoader
327
- before the sampler.
328
- - Load the `*.module.safetensors` through `LTXVEditAnythingModuleLoader`
329
- and connect its `editanything_module` output to the sampler.
330
- - On the sampler:
331
- - `editanything_module` β†’ **the module loader output** (required).
332
- - `ref_image` β†’ **the reference image** (required β€” this is what
333
- `Add` / `Replace` will insert).
334
- - `guide_frames` β†’ the source video to edit.
335
- - `enable_adaln` β†’ **on** (defaults match training).
336
- - `enable_visual_crossattn` β†’ **on** for the 4-extras build; off (or
337
- will be a no-op) for the 2-extras build.
338
- - `enable_role_embedding` β†’ **off** for the 4-extras build (training
339
- config disabled it). On if you're loading the 2-extras build alone.
340
-
341
- Missing `ref_image` here silently disables AdaLN and the visual
342
- cross-attention β€” the sampler will warn in the log.
343
-
344
- ### Which build to use
345
-
346
- - **`ref_adaln_proj-role_embedding`** β€” the original training. Only ships
347
- the two side-channel modules.
348
- - **`ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj`** β€” the
349
- continuation. Adds the visual cross-attention branch and its projector on
350
- top.
351
-
352
- It is genuinely **not clear yet** whether the extra branches help over the
353
- plain LoRA. Both builds are honest experiments. Try both, decide for your
354
- own use case, and please share findings.
355
-
356
- ### Reading the layers
357
-
358
- For anyone who wants to understand what each layer in the Ref V2V
359
- checkpoint does:
360
-
361
- - [`lora_layers_reference.md`](./lora_layers_reference.md) β€” full tensor
362
- inventory of both builds.
363
- - [`lora_layers_impact.md`](./lora_layers_impact.md) β€” what each branch
364
- contributes at inference and which inference knob (`adaln_scale`,
365
- `ref_context_scale`, `ref_token_scale`, `ref_start_block`,
366
- `ref_end_block`, etc.) maps back to which training default.
367
-
368
- ---
369
-
370
- ## Prompt examples
371
-
372
- The two LoRAs were trained on very different caption styles. Match the
373
- style of whichever LoRA you're using β€” straying outside the training
374
- distribution is the fastest way to get garbage out.
375
-
376
- ### Edit Anything v0.1 β€” standard editing
377
-
378
- The stage-1 dataset uses short imperative captions describing one or two
379
- edits. Use the same shape at inference. Examples drawn from the training
380
- distribution:
381
-
382
- - *"Replace the stone statue of a man on the left with a young woman in a
383
- green dress."*
384
- - *"Add a black labrador retriever sitting beside the woman on the bench."*
385
- - *"Remove the teacher from the classroom."*
386
- - *"Alter the cap's colour from modern black to deep maroon."*
387
- - *"Replace the fresh citrus-green background with a wooden desk."*
388
- - *"Add faint tire tracks across the snow behind the car."*
389
- - *"Add a black statue, a blue camera, a cyan towel, a red guitar and a
390
- pink backpack to the lakeside pier."*
391
-
392
- Tips:
393
-
394
- - Imperative verbs: **Add / Replace / Remove / Alter / Change**.
395
- - When replacing, **describe both** the original and the new subject so the
396
- model can localise the edit.
397
- - Keep captions short and concrete. Long flowery prose hurts.
398
-
399
- ### Edit Anything v0.1 β€” motion transfer
400
-
401
- Workflow:
402
-
403
- 1. Pick a guide video.
404
- 2. Edit **only the first frame** externally (Flux Klein or any
405
- capable image-edit model) to introduce the new subject in the desired
406
- pose and position.
407
- 3. Feed the edited frame as the first frame of the input and the original
408
- guide as motion source.
409
- 4. The prompt should describe **the inserted subject and the action being
410
- preserved**.
411
-
412
- Examples:
413
-
414
- - *"Replace the standing man holding the umbrella with a woman in a red
415
- coat holding the same umbrella, walking across the puddles."*
416
- - *"Add a tabby cat curled up in the armchair while the man in the
417
- background keeps reading."*
418
- - *"Replace the runner in the blue jersey with a man wearing a white shirt
419
- and grey shorts running along the same path."*
420
-
421
- Limits: fast or chaotic motion will fail; the inserted subject should
422
- occupy roughly the same region/scale as what it replaces.
423
-
424
- ### Reference V2V (Ref V2V) β€” Add and Replace
425
-
426
- These captions are real samples from the ~1600-pair training set. They
427
- describe the **target scene after the edit** in detail. The reference
428
- image carries the *appearance* of the inserted subject; the caption
429
- carries *position, pose, action, and surrounding context*.
430
-
431
- **Add task** (the reference image holds the new subject):
432
-
433
- - *"Add a middle-aged man with curly grey hair, a beard and glasses,
434
- wearing a blue quarter-zip sweater, on the right side of the frame,
435
- standing in front of a raw cut of meat on a tray."*
436
- - *"Add a light-coloured small boat with dark seats and an outboard motor
437
- floating in the water."*
438
- - *"Add an open book filled with colourful pencils in the woman's hands."*
439
- - *"Add a silver metallic bucket on the table in front of the blonde
440
- character, with her hands stirring a mixture inside."*
441
- - *"Add two miniature dolls, one blonde and one brunette, dressed in
442
- patterned clothing, sitting at a small table with teacups and small
443
- white vases on the countertop."*
444
-
445
- **Replace task** (the reference image holds the new subject; the caption
446
- also describes what is being replaced):
447
-
448
- - *"Replace the standing kangaroo holding the bicycle handlebars with a
449
- man wearing a white t-shirt, light brown shorts and a yellow cap,
450
- holding the bicycle handlebars."*
451
- - *"Replace the stone statue of a man on the left side with a young woman
452
- in a green dress."*
453
- - *"Replace the wooden barrel near the entrance with a large brown leather
454
- suitcase."*
455
-
456
- Tips for Ref V2V:
457
-
458
- - **Describe the inserted subject in full**, even though the reference
459
- image is the source of truth β€” the text path drives placement and pose.
460
- - For *Replace*, **also describe what is being replaced** so the model can
461
- match the spatial region.
462
- - Keep the inserted subject roughly in the same scale and region as what
463
- it replaces.
464
- - The captions in the training set average ~25–40 words β€” aim for that
465
- range. Single-sentence captions like *"Add a man"* are far too sparse
466
- and will fail.
467
-
468
- ---
469
-
470
- ## Inference tips (applies to all models)
471
-
472
- **CFG matters a lot here.** The default workflow runs with the LTX-2.3
473
- **distilled / acceleration LoRAs** for fast 4–8 step sampling, which
474
- locks **CFG = 1.0**. That's fine for casual runs, but at CFG 1 the model
475
- follows the prompt loosely β€” you get the reference image to "show up"
476
- but the edit instruction itself is only weakly enforced.
477
-
478
- **For harder prompts, raise CFG above 1.0.** This means dropping (or
479
- weakening) the distilled / acceleration LoRAs and going back to a normal
480
- sampler with more steps β€” significantly slower, but the model follows
481
- the prompt much more closely. Trade-off:
482
-
483
- Other knobs:
484
-
485
- - If the model is **ignoring the prompt** (edit isn't being applied, the
486
- reference is barely showing up, the style transfer is faint), raising
487
- CFG is the single most common fix. Go up to 6–8 if needed.
488
- - If results look **over-saturated, plasticky, or motion is freezing**,
489
- CFG is too high β€” pull back toward 3–4 or re-enable the distilled LoRA
490
- for CFG 1 if you don't actually need stronger prompt adherence.
491
- - Ref V2V in particular benefits from being more aggressive with CFG when
492
- the reference identity isn't transferring cleanly.
493
- - Combine CFG tuning with the LoRA-specific knobs from each section
494
- (`adaln_scale`, `ref_context_scale`, `ref_token_scale` for Ref V2V;
495
- prompt rewriting for v0.1 / no-ref).
496
-
497
- Treat CFG as a real knob, not a constant β€” and be ready to give up some
498
- speed when you actually need the edit to land.
499
-
500
- ---
501
-
502
- ## ComfyUI nodes
503
-
504
- All recommended inference paths run through the **BFSnodes** custom node
505
- set. For now BFSnodes is the only place these nodes live; once they
506
- stabilise they may move elsewhere.
507
-
508
- Specific nodes used by these LoRAs:
509
-
510
- - `LTXVEditAnythingApply` β€” load the LoRA + extras and patch the model.
511
- - `πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping Sampler` β€” sampler that injects role /
512
- AdaLN / visual cross-attention and handles long videos in chunks.
513
- - `LTXVEditAnythingModuleLoader` β€” load the `*.module.safetensors` sidecar.
514
-
515
- ---
516
-
517
- ## Status
518
-
519
- Released as experimental research artefacts. Expect failures, do not
520
- deploy, and please report what works and what doesn't.
521
-
522
- ---
523
-
524
- ## Credits
525
-
526
- If you use these models β€” in a project, a demo, a paper, a video, a tweet,
527
- a workflow, anything β€” **please credit my work**. These checkpoints are the
528
- result of weeks of research, dataset building, and training runs, and that
529
- effort is what makes any of it usable. Crediting the source is the bare
530
- minimum that keeps open research like this sustainable.
531
-
532
- **Author:** Alisson Pereira dos Anjos ([@Alissonerdx](https://huggingface.co/Alissonerdx))
533
-
534
- Suggested attribution:
535
-
536
- > Edit Anything LoRAs by Alisson Pereira dos Anjos
537
- > ([huggingface.co/Alissonerdx/EditAnything](https://huggingface.co/Alissonerdx/EditAnything)).
538
-
539
- Links back to this repository are appreciated wherever you publish results.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: diffusers
4
+ base_model:
5
+ - Lightricks/LTX-2.3
6
+ - Lightricks/LTX-2.5
7
+ tags:
8
+ - lora
9
+ - video
10
+ - video-editing
11
+ - ltx-2.3
12
+ - ltx-2.5
13
+ ---
14
+
15
+ # Edit Anything β€” Experimental LTX-2 Video Editing LoRAs
16
+
17
+ > **Heads up.** These LoRAs are research experiments. They are far from
18
+ > production-ready and will fail on many inputs. They are released for the
19
+ > community to play with and break, not as a finished tool.
20
+
21
+ This repository hosts four unrelated training tracks:
22
+
23
+ 0. **Edit Anything v2 for LTX-2.5** β€” the current one, a plain instruction-editing
24
+ IC-LoRA. Start here. *(built on LTX-2.5 22B)*
25
+ 1. **Edit Anything v0.1 β€” motion transfer LoRA** (two ranks). *(LTX-2.3)*
26
+ 2. **Edit Anything β€” no-reference multitask LoRA** (rank 256, prompt-driven only). *(LTX-2.3)*
27
+ 3. **Reference video-to-video (Ref V2V) β€” experimental IC-LoRA + sidecar modules** (two builds). *(LTX-2.3)*
28
+
29
+ Tracks 1-3 are **LTX-2.3** and are kept for reference. Track 0 is **LTX-2.5** and is
30
+ the one being worked on.
31
+
32
+ Inference is meant to run through the **BFSnodes** ComfyUI custom nodes β€”
33
+ the Ref V2V build in particular needs them to load the sidecar modules and
34
+ install the custom branches into the transformer.
35
+
36
+ ---
37
+
38
+ ## 0. Edit Anything v2 β€” LTX-2.5
39
+
40
+ File: `edit_anything_v2_ltx2.5.safetensors` (rank 128)
41
+ Workflow: `workflows/EditAnythingLTX2.5.json`
42
+ Base model: **LTX-2.5 22B dev**
43
+
44
+ ### What it is
45
+
46
+ In this experimental version it is possible to perform several operations together in a
47
+ single prompt β€” **Replace, Remove, Add, Restyle** β€” and everything becomes easier with the
48
+ help of the prompt enhancer, so you can write in natural language and it converts it to the
49
+ appropriate prompt.
50
+
51
+ Unlike track 3, this LoRA takes **no reference image**. It is a plain IC-LoRA: your source
52
+ video goes in as the guide, the instruction goes in as text, and the edited video comes out.
53
+ No RoPE tricks, no sidecar modules, no loss masks β€” the simplest thing that works.
54
+
55
+ ### How to use it in ComfyUI
56
+
57
+ Load `workflows/EditAnythingLTX2.5.json`. If you are wiring it by hand with the **BFSnodes**
58
+ `LTX Multiple Controls` node, the settings that match training are:
59
+
60
+ | input | value |
61
+ |---|---|
62
+ | `guide_video` | your source video |
63
+ | `guide_source_id` | `0` |
64
+ | `guide_layout` | `overlap` |
65
+ | `guide_ref_resize_mode` | `match_target` |
66
+ | `guide_downscale_factor` | `1` |
67
+
68
+ Leave `identity_image`, `mask_video` and `identity_mask_image` **disconnected** β€” this LoRA
69
+ was trained without any of them.
70
+
71
+ Trained resolution is **704x384, 73 frames**. Other sizes and lengths are extrapolation and
72
+ degrade first in temporal consistency. Suggested start: LoRA strength `1.0`, CFG `3-5`,
73
+ 30 steps, no LightX2V.
74
+
75
+ ### Prompting
76
+
77
+ Every prompt starts with the trigger:
78
+
79
+ ```
80
+ edit_anything:
81
+ ```
82
+
83
+ Then one sentence per edit. The four verbs below are the only ones the model was trained on;
84
+ the percentage is that verb's share of the 6829 training instructions:
85
+
86
+ ```
87
+ Replace <the existing thing> with <the new thing, described>. 50%
88
+ Add a <object with attributes> <where>. 26%
89
+ Remove <the thing, identified precisely>. 18%
90
+ Restyle <the region, identified> into <style> style. 5%
91
+ ```
92
+
93
+ Rules that come straight from the training distribution β€” breaking them is the usual reason
94
+ a prompt does nothing:
95
+
96
+ - **10 to 20 words per instruction** (median 15). Shorter loses the description the model
97
+ needs; longer drifts off-distribution.
98
+ - **One object per `Add`.** 95% of trained Add instructions begin `Add a ...` / `Add an ...`.
99
+ For a group or a crowd use `Replace` on an existing region instead of `Add several ...`.
100
+ - **Anchor each instruction to something visible**, the way the data does: "to the left of
101
+ the man", "on the desk on the right side of the frame", "behind the speaker". 63% of the
102
+ trained Add instructions carry an anchor like this.
103
+ - **Describe state, not manner.** "standing in the green grass field", "sitting upright on
104
+ the desk" β€” not "swimming gracefully".
105
+ - **`Restyle` targets a region, not the whole clip.** 93% of trained Restyle instructions
106
+ name a wall, a screen, a background or a face. A whole-video restyle is off-distribution;
107
+ express it as two or three Restyle instructions over the regions that carry the look.
108
+ - **`Make it <style>` and `Turn it into <style>` do not exist** in this model's training
109
+ data β€” zero occurrences. Use `Restyle`.
110
+
111
+ Real examples, in the exact shape the model learned:
112
+
113
+ ```
114
+ edit_anything: Add a medium-sized brown dog sitting on the grass to the right of the
115
+ motorcycles. Remove the black jacket worn by the smiling man in the left foreground.
116
+ Replace the man's graying hair with bright red hair.
117
+
118
+ edit_anything: Replace the gray and black hoodie worn by the man with a bright red zip-up
119
+ jacket with white stripes on the sleeves.
120
+
121
+ edit_anything: Remove the smartphone held in the man's hand.
122
+
123
+ edit_anything: Restyle the large blue digital screen behind the man into flat vector style.
124
+ ```
125
+
126
+ Write one instruction per edit you actually want. Do not pad the prompt with edits you did
127
+ not ask for β€” an unrequested sky or background change will show up in the output.
128
+
129
+ ### Training
130
+
131
+ - **Base**: LTX-2.5 22B dev, trained from scratch (no continuation from the 2.3 LoRAs).
132
+ - **Data**: 2750 aligned source/edited video pairs from **CoinVE-200K**, at 704x384x73.
133
+ - **Recipe**: IC-LoRA with a single conditioning stream (the source video, `overlap`,
134
+ source id 0). Rank 128 / alpha 128, AdamW `1e-4` constant, batch 1, 10 000 steps.
135
+ - **This release is a merge**: the weights are the 50/50 average of the step-7000 and
136
+ step-9500 checkpoints, reduced back to rank 128 by truncated SVD (Frobenius-optimal;
137
+ 99.99% of the energy preserved). The earlier checkpoint generalises better outside the
138
+ training distribution, the later one follows instructions better inside it β€” the average
139
+ beat both parents on 4 of the 5 metrics measured on held-out data.
140
+
141
+ ### Limitations
142
+
143
+ - Trained only on CoinVE-style edits. Requests far from that vocabulary get weaker results.
144
+ - Edits that change very little of the frame (subtle removals) are the least reliable.
145
+ - No reference image support β€” you cannot say "add *this* object" with a picture. That is
146
+ what track 3 tried, and it is still unsolved.
147
+ - 704x384x73 is the trained box; longer clips lose temporal consistency first.
148
+
149
+
150
+ ---
151
+
152
+ ## 1. Edit Anything v0.1 (motion transfer)
153
+
154
+ Files:
155
+
156
+ - `edit_anything_30k_v0.1_motion_transfer_r128.safetensors`
157
+ - `edit_anything_30k_v0.1_motion_transfer_r256.safetensors`
158
+
159
+ ### What it is
160
+
161
+ **v0.1 is not a direct continuation of v1.0.** It was trained from scratch
162
+ in two stages:
163
+
164
+ 1. **Stage 1 β€” image-only pretraining.** ~30 000 image edit pairs. Training
165
+ a *video* model on still images is admittedly not ideal, but it was a way
166
+ to push the editing vocabulary beyond what a small video-only dataset can
167
+ teach.
168
+ 2. **Stage 2 β€” video fine-tune with `first_frame_conditioning > 0`.** This
169
+ restored the temporal prior and unlocked the motion-transfer behaviour
170
+ described below.
171
+
172
+ In theory v0.1 can do the same edits as v1.0, but **temporal consistency may
173
+ be weaker than v1.0** because so much of stage 1 happened on still images.
174
+ Test against v1.0 case-by-case before assuming v0.1 wins on your task.
175
+
176
+ ### Motion transfer
177
+
178
+ Because stage 2 included first-frame conditioning, you can drive the LoRA
179
+ into a motion-transfer mode:
180
+
181
+ 1. Take a guide video.
182
+ 2. **Replace its first frame** with an edited still (insert a new subject,
183
+ swap an object, etc.). Use a strong image-editing model β€” **Flux Klein**
184
+ or similar β€” to prepare it; the quality of this single frame propagates
185
+ through the whole clip.
186
+ 3. Feed the edited frame as the first frame of the input, and the original
187
+ guide video as the motion source.
188
+
189
+ The model uses the new first frame as the appearance anchor and copies the
190
+ motion from the rest of the guide.
191
+
192
+ Limitations (these are real, not theoretical β€” expect them to bite):
193
+
194
+ - **Hard scene cuts break it.** The model assumes continuous motion from
195
+ the first frame onwards. A cut to a different camera angle or location
196
+ mid-clip will produce smearing, ghosting, or the inserted subject jumping
197
+ to the wrong position. Use clips without cuts, or split at the cuts and
198
+ process each segment separately.
199
+ - **Very fast motion fails.** Quick pans, fast subject movement, or
200
+ high-velocity action confuse the motion-copy mechanism. Outputs degrade
201
+ to blur or to the model "freezing" on the first-frame appearance and
202
+ losing the motion entirely. Stick to moderate-speed clips.
203
+ - Poor blending / artefacts in the first frame propagate everywhere.
204
+ - Works best when the inserted subject roughly occupies the same region as
205
+ whatever it replaces.
206
+
207
+ ### Prompting
208
+
209
+ Prompt is just as critical as in v1.0. **Describe both the object being
210
+ replaced and the new one in detail**. Example: *"Replace the bronze statue on
211
+ the left with a tall man wearing a navy raincoat and brown boots."* Vague
212
+ prompts produce bad edits.
213
+
214
+ ### Which rank to use
215
+
216
+ The same training produced both files. v0.1 is actually the merge of the
217
+ two-stage training (one LoRA per stage), re-extracted at two different ranks
218
+ via Frobenius-optimal truncated SVD:
219
+
220
+ | File | Rank | Size | Frobenius retention |
221
+ |---|---|---|---|
222
+ | `edit_anything_30k_v0.1_motion_transfer_r128.safetensors` | 128 | 1.31 GB | ~99.4% |
223
+ | `edit_anything_30k_v0.1_motion_transfer_r256.safetensors` | 256 | 2.62 GB | ~99.9% |
224
+
225
+ r256 is closer to the merged source. r128 is normally indistinguishable in
226
+ practice. Pick whichever fits your workflow.
227
+
228
+ ### How to wire the LoopingSampler
229
+
230
+ This is a **standard LoRA**, not a sidecar. Load it through the regular
231
+ ComfyUI LoraLoader **before** the LoopingSampler. On the sampler itself:
232
+
233
+ - `editanything_module` β†’ **leave disconnected**.
234
+ - `ref_image` β†’ the edited first frame (for motion transfer) **or** the
235
+ source frame you want preserved (for plain editing).
236
+ - `guide_frames` β†’ the guide video.
237
+ - `enable_role_embedding`, `enable_adaln`, `enable_visual_crossattn` β†’
238
+ all **off**. None of those branches were trained for v0.1; turning them
239
+ on with no module connected does nothing anyway, but keeping them off
240
+ silences the WARN logs.
241
+
242
+ ---
243
+
244
+ ## 2. Edit Anything β€” no-reference multitask LoRA
245
+
246
+ File:
247
+
248
+ - `edit_anything_v1.1_r256.safetensors`
249
+
250
+ ### What it is
251
+
252
+ A **prompt-only** multitask editing LoRA. No reference image, no first-frame
253
+ conditioning β€” the model is driven entirely by the text prompt and the
254
+ guide video. Trained on a balanced mix of **Add, Remove, Replace, Style**
255
+ edits.
256
+
257
+ ### What it's different about it (vs v0.1)
258
+
259
+ The task vocabulary overlaps heavily with v0.1 β€” both can do Add, Remove,
260
+ Replace, Change, Convert. What changes here:
261
+
262
+ - **Two-stage training continuation**: the first stage gave the model its
263
+ edit vocabulary; the second stage refined it on a larger, more balanced
264
+ video pair set covering Add / Remove / Replace / Style.
265
+ - **Rank 256** (vs v0.1's effective rank from the merge), giving more
266
+ capacity for the broader task mix.
267
+ - Trained directly on video pairs, so the temporal behaviour on these
268
+ tasks tends to be steadier than on a model whose first stage was on
269
+ still images.
270
+
271
+ ### How to use it
272
+
273
+ **Standalone** β€” load it as a regular LoRA on vanilla LTX-2.3 through any
274
+ ComfyUI LoRA loader. The file already carries everything it needs; no
275
+ stacking with v0.1, no companion module.
276
+
277
+ ### Limitations
278
+
279
+ - No reference image β†’ identity is not anchored, so Add / Replace of a
280
+ specific person or object will be wobblier than the Ref V2V build.
281
+ - No motion transfer (that's v0.1 only).
282
+
283
+ ### Prompting
284
+
285
+ Same imperative shape as v0.1, but the training set split into four very
286
+ distinct caption styles. Match the one that fits the edit you want β€” the
287
+ distribution is narrow and the model expects the right shape.
288
+
289
+ The training set is roughly balanced across **Add, Remove, Replace and
290
+ Style** buckets, with Style being the smallest of the four. Captions
291
+ below are real examples drawn from those buckets.
292
+
293
+ #### Add β€” 15 to 30+ words, describe what to add and where
294
+
295
+ * `Add a smiling woman with brown hair, wearing a pink sleeveless top, sitting to the right of the man at the news desk.`
296
+ * `Add a person wearing a blue denim shirt over a white t-shirt to the right side of the frame, behind the person cooking.`
297
+ * `Add a decorated Christmas tree with red and white ornaments and lights to the right of the man.`
298
+ * `Add a blonde boy wearing a black t-shirt with a blue collar and blue patterned pants, sitting behind the other children in the upper center of the frame.`
299
+ * `Add two horizontal wooden strips to the front of the white range hood.`
300
+
301
+ Pattern: `Add <detailed subject description>, <position in frame>, <surrounding context>.`
302
+
303
+ #### Remove β€” very short, 4 to 10 words
304
+
305
+ * `Remove the man drinking from a glass.`
306
+ * `Remove the disco ball.`
307
+ * `Remove the large tree on the right.`
308
+ * `Remove the squirrel in the foreground.`
309
+ * `Remove the man on the left.`
310
+
311
+ Pattern: `Remove the <object>` (+ optional position). Resist the urge to
312
+ over-describe β€” long Remove prompts drift outside the training shape and
313
+ often fail.
314
+
315
+ #### Replace β€” 20 to 35 words, describe both old and new
316
+
317
+ * `Replace the white panel door on the right side of the frame with a dark brown grandfather clock.`
318
+ * `Replace the light-colored cat lying on the mat on the floor with a young woman sitting on the mat.`
319
+ * `Replace the dark grey knitted sweater on the man's torso with a black and white patterned Christmas sweater.`
320
+ * `Replace the blue robot with a glowing blue face on the left with a smiling man wearing sunglasses and a blue shirt.`
321
+ * `Replace the sitting person wearing a black cape on the left with a black fabric draped over an object.`
322
+
323
+ Pattern: `Replace <description of the original subject and its location> with <description of the new subject>.`
324
+
325
+ #### Style β€” fixed template, the style name is what changes
326
+
327
+ * `Convert the video into a Pencil Sketch style.`
328
+ * `Convert the video into a Watercolor Painting style.`
329
+ * `Convert the video into a Van Gogh style.`
330
+ * `Convert the video into a Play-Doh style.`
331
+ * `Convert the video into a Claymation style.`
332
+ * `Convert the video into a 3D Chibi style.`
333
+ * `Convert the video into a Ghibli style.`
334
+ * `Convert the video into a Pop Art style.`
335
+ * `Convert the video into an American Cartoon style.`
336
+ * `Convert the video into a Flat Vector Cartoon style.`
337
+
338
+ The training set covers **300+ distinct style names**. Many work; many do
339
+ not. The list above is heavily represented in training. Use the exact
340
+ phrase `Convert the video into a <STYLE> style` β€” deviations from this
341
+ template degrade quality noticeably.
342
+
343
+ #### What it does *not* do
344
+
345
+ These are honest limits of the training distribution β€” don't expect them
346
+ to work just because the model is multitask:
347
+
348
+ - **No compositional prompts.** *"Add X and remove Y"*, *"Replace A with B
349
+ and add C"*, etc. are **not** in the training set. Captions combining
350
+ two action verbs are essentially absent (the only ones present are the
351
+ "Remove X and replace with Y" idiom, which is really a single Replace).
352
+ Pure multi-action edits will fall apart β€” split them into separate runs.
353
+ - **No "change background" as a task.** Background is only used as a
354
+ *positional reference* ("in the background", "on the wall in the
355
+ background"). To swap the entire backdrop, phrase it as a **Replace**
356
+ on a concrete background element, e.g.
357
+ *"Replace the brick wall in the background with a forest at sunset"*.
358
+ Vague prompts like *"Change the background to a beach"* are off-
359
+ distribution and rarely work.
360
+ - **No global colour grade / lighting change.** Only the Style template
361
+ is trained as a global transform. Anything else global (LUT-style
362
+ adjustments, time-of-day swaps without a concrete object) is unreliable.
363
+
364
+ ### Which LoRA should I use?
365
+
366
+ | If you want… | Use |
367
+ |---|---|
368
+ | Motion transfer (edit first frame externally, model copies motion) | **v0.1 motion transfer** |
369
+ | Multi-task edits (add / remove / replace / style) driven only by prompt | **no-ref multitask r256** (standalone) |
370
+ | Strong identity transfer from a reference image (Add / Replace) | **Ref V2V** |
371
+
372
+ ### How to wire the LoopingSampler
373
+
374
+ A single **standard LoRA**, no sidecar, no stacking. Load through one
375
+ ComfyUI LoraLoader before the LoopingSampler. On the sampler:
376
+
377
+ - `editanything_module` β†’ **leave disconnected**.
378
+ - `ref_image` β†’ **leave disconnected**. This LoRA has no reference-image
379
+ path; passing one will just pre-encode tokens that you do not want.
380
+ - `guide_frames` β†’ the guide video.
381
+ - `enable_role_embedding`, `enable_adaln`, `enable_visual_crossattn` β†’
382
+ all **off** (no module = nothing to inject anyway).
383
+
384
+ ---
385
+
386
+ ## 3. Reference video-to-video (Ref V2V) β€” experimental
387
+
388
+ Files (two builds of the same LoRA family β€” each ships as a `(.standard, .module)` pair):
389
+
390
+ - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.standard.safetensors`
391
+ - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.module.safetensors`
392
+ - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.standard.safetensors`
393
+ - `edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.module.safetensors`
394
+
395
+ ### What it is
396
+
397
+ The goal is **add / replace using a reference image** β€” same vibe as Edit
398
+ Anything v1.0, but with an explicit image as the appearance source instead
399
+ of relying only on the prompt.
400
+
401
+ Trained on **~1600** Add / Replace video pairs. Reference-paired video
402
+ datasets are basically nonexistent, so the dataset had to be built from
403
+ scratch β€” that is why the sample count is small. **It often fails.** This
404
+ is fully experimental; thousands of training runs went into landing on this
405
+ LoRA layout, and it is still unclear how much it actually helps.
406
+
407
+ ### Architecture β€” why this LoRA has "modules"
408
+
409
+ Trained as a conventional IC-LoRA, plus extra projection branches that try
410
+ to make the reference signal survive across layers:
411
+
412
+ - **`ref_visual_proj`** β€” projects the reference VAE latent into 32 visual
413
+ memory tokens.
414
+ - **`ref_attn`** β€” a dedicated cross-attention branch inside each
415
+ transformer block, reading those tokens.
416
+ - **`ref_adaln_proj`** β€” a global AdaLN bias derived from the reference
417
+ (palette / overall look).
418
+ - **`role_embedding`** β€” an experimental token bias inspired by some of
419
+ Kijai's tests; whether it actually helps is still unclear.
420
+
421
+ These extra weights are saved alongside the LoRA in a `.module.safetensors`
422
+ sidecar because they are **not standard LoRA adapters** β€” the regular
423
+ ComfyUI LoRA loader can't consume them, so they need a dedicated node.
424
+
425
+ ### How to load
426
+
427
+ | File | What it is | Where it goes |
428
+ |---|---|---|
429
+ | `*.standard.safetensors` | LoRA on `attn1` / `attn2` / `ff` only | Standard ComfyUI LoRA loader |
430
+ | `*.module.safetensors` | `role_embedding`, `ref_adaln_proj`, `ref_visual_proj`, `ref_attn` LoRA adapters | `LTXVEditAnythingModuleLoader` (BFSnodes) |
431
+
432
+ Both files of a pair must be loaded **together** β€” the LoRA was trained
433
+ against the sidecar adapters and they only make sense as a unit. Do not mix
434
+ `.standard` from one build with `.module` from another.
435
+
436
+ The module file is consumed by the **`πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping
437
+ Sampler`** node, which was written specifically to:
438
+
439
+ 1. Install the `ref_attn` cross-attention branch on every transformer block.
440
+ 2. Inject the AdaLN / role / visual cross-attention conditioning at the
441
+ correct points in the model.
442
+ 3. Sample long videos in overlapping chunks with the conditioning re-applied
443
+ per chunk.
444
+
445
+ ### How to wire the LoopingSampler
446
+
447
+ - Load the `*.standard.safetensors` through a normal ComfyUI LoraLoader
448
+ before the sampler.
449
+ - Load the `*.module.safetensors` through `LTXVEditAnythingModuleLoader`
450
+ and connect its `editanything_module` output to the sampler.
451
+ - On the sampler:
452
+ - `editanything_module` β†’ **the module loader output** (required).
453
+ - `ref_image` β†’ **the reference image** (required β€” this is what
454
+ `Add` / `Replace` will insert).
455
+ - `guide_frames` β†’ the source video to edit.
456
+ - `enable_adaln` β†’ **on** (defaults match training).
457
+ - `enable_visual_crossattn` β†’ **on** for the 4-extras build; off (or
458
+ will be a no-op) for the 2-extras build.
459
+ - `enable_role_embedding` β†’ **off** for the 4-extras build (training
460
+ config disabled it). On if you're loading the 2-extras build alone.
461
+
462
+ Missing `ref_image` here silently disables AdaLN and the visual
463
+ cross-attention β€” the sampler will warn in the log.
464
+
465
+ ### Which build to use
466
+
467
+ - **`ref_adaln_proj-role_embedding`** β€” the original training. Only ships
468
+ the two side-channel modules.
469
+ - **`ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj`** β€” the
470
+ continuation. Adds the visual cross-attention branch and its projector on
471
+ top.
472
+
473
+ It is genuinely **not clear yet** whether the extra branches help over the
474
+ plain LoRA. Both builds are honest experiments. Try both, decide for your
475
+ own use case, and please share findings.
476
+
477
+ ### Reading the layers
478
+
479
+ For anyone who wants to understand what each layer in the Ref V2V
480
+ checkpoint does:
481
+
482
+ - [`lora_layers_reference.md`](./lora_layers_reference.md) β€” full tensor
483
+ inventory of both builds.
484
+ - [`lora_layers_impact.md`](./lora_layers_impact.md) β€” what each branch
485
+ contributes at inference and which inference knob (`adaln_scale`,
486
+ `ref_context_scale`, `ref_token_scale`, `ref_start_block`,
487
+ `ref_end_block`, etc.) maps back to which training default.
488
+
489
+ ---
490
+
491
+ ## Prompt examples
492
+
493
+ The two LoRAs were trained on very different caption styles. Match the
494
+ style of whichever LoRA you're using β€” straying outside the training
495
+ distribution is the fastest way to get garbage out.
496
+
497
+ ### Edit Anything v0.1 β€” standard editing
498
+
499
+ The stage-1 dataset uses short imperative captions describing one or two
500
+ edits. Use the same shape at inference. Examples drawn from the training
501
+ distribution:
502
+
503
+ - *"Replace the stone statue of a man on the left with a young woman in a
504
+ green dress."*
505
+ - *"Add a black labrador retriever sitting beside the woman on the bench."*
506
+ - *"Remove the teacher from the classroom."*
507
+ - *"Alter the cap's colour from modern black to deep maroon."*
508
+ - *"Replace the fresh citrus-green background with a wooden desk."*
509
+ - *"Add faint tire tracks across the snow behind the car."*
510
+ - *"Add a black statue, a blue camera, a cyan towel, a red guitar and a
511
+ pink backpack to the lakeside pier."*
512
+
513
+ Tips:
514
+
515
+ - Imperative verbs: **Add / Replace / Remove / Alter / Change**.
516
+ - When replacing, **describe both** the original and the new subject so the
517
+ model can localise the edit.
518
+ - Keep captions short and concrete. Long flowery prose hurts.
519
+
520
+ ### Edit Anything v0.1 β€” motion transfer
521
+
522
+ Workflow:
523
+
524
+ 1. Pick a guide video.
525
+ 2. Edit **only the first frame** externally (Flux Klein or any
526
+ capable image-edit model) to introduce the new subject in the desired
527
+ pose and position.
528
+ 3. Feed the edited frame as the first frame of the input and the original
529
+ guide as motion source.
530
+ 4. The prompt should describe **the inserted subject and the action being
531
+ preserved**.
532
+
533
+ Examples:
534
+
535
+ - *"Replace the standing man holding the umbrella with a woman in a red
536
+ coat holding the same umbrella, walking across the puddles."*
537
+ - *"Add a tabby cat curled up in the armchair while the man in the
538
+ background keeps reading."*
539
+ - *"Replace the runner in the blue jersey with a man wearing a white shirt
540
+ and grey shorts running along the same path."*
541
+
542
+ Limits: fast or chaotic motion will fail; the inserted subject should
543
+ occupy roughly the same region/scale as what it replaces.
544
+
545
+ ### Reference V2V (Ref V2V) β€” Add and Replace
546
+
547
+ These captions are real samples from the ~1600-pair training set. They
548
+ describe the **target scene after the edit** in detail. The reference
549
+ image carries the *appearance* of the inserted subject; the caption
550
+ carries *position, pose, action, and surrounding context*.
551
+
552
+ **Add task** (the reference image holds the new subject):
553
+
554
+ - *"Add a middle-aged man with curly grey hair, a beard and glasses,
555
+ wearing a blue quarter-zip sweater, on the right side of the frame,
556
+ standing in front of a raw cut of meat on a tray."*
557
+ - *"Add a light-coloured small boat with dark seats and an outboard motor
558
+ floating in the water."*
559
+ - *"Add an open book filled with colourful pencils in the woman's hands."*
560
+ - *"Add a silver metallic bucket on the table in front of the blonde
561
+ character, with her hands stirring a mixture inside."*
562
+ - *"Add two miniature dolls, one blonde and one brunette, dressed in
563
+ patterned clothing, sitting at a small table with teacups and small
564
+ white vases on the countertop."*
565
+
566
+ **Replace task** (the reference image holds the new subject; the caption
567
+ also describes what is being replaced):
568
+
569
+ - *"Replace the standing kangaroo holding the bicycle handlebars with a
570
+ man wearing a white t-shirt, light brown shorts and a yellow cap,
571
+ holding the bicycle handlebars."*
572
+ - *"Replace the stone statue of a man on the left side with a young woman
573
+ in a green dress."*
574
+ - *"Replace the wooden barrel near the entrance with a large brown leather
575
+ suitcase."*
576
+
577
+ Tips for Ref V2V:
578
+
579
+ - **Describe the inserted subject in full**, even though the reference
580
+ image is the source of truth β€” the text path drives placement and pose.
581
+ - For *Replace*, **also describe what is being replaced** so the model can
582
+ match the spatial region.
583
+ - Keep the inserted subject roughly in the same scale and region as what
584
+ it replaces.
585
+ - The captions in the training set average ~25–40 words β€” aim for that
586
+ range. Single-sentence captions like *"Add a man"* are far too sparse
587
+ and will fail.
588
+
589
+ ---
590
+
591
+ ## Inference tips (applies to all models)
592
+
593
+ **CFG matters a lot here.** The default workflow runs with the LTX-2.3
594
+ **distilled / acceleration LoRAs** for fast 4–8 step sampling, which
595
+ locks **CFG = 1.0**. That's fine for casual runs, but at CFG 1 the model
596
+ follows the prompt loosely β€” you get the reference image to "show up"
597
+ but the edit instruction itself is only weakly enforced.
598
+
599
+ **For harder prompts, raise CFG above 1.0.** This means dropping (or
600
+ weakening) the distilled / acceleration LoRAs and going back to a normal
601
+ sampler with more steps β€” significantly slower, but the model follows
602
+ the prompt much more closely. Trade-off:
603
+
604
+ Other knobs:
605
+
606
+ - If the model is **ignoring the prompt** (edit isn't being applied, the
607
+ reference is barely showing up, the style transfer is faint), raising
608
+ CFG is the single most common fix. Go up to 6–8 if needed.
609
+ - If results look **over-saturated, plasticky, or motion is freezing**,
610
+ CFG is too high β€” pull back toward 3–4 or re-enable the distilled LoRA
611
+ for CFG 1 if you don't actually need stronger prompt adherence.
612
+ - Ref V2V in particular benefits from being more aggressive with CFG when
613
+ the reference identity isn't transferring cleanly.
614
+ - Combine CFG tuning with the LoRA-specific knobs from each section
615
+ (`adaln_scale`, `ref_context_scale`, `ref_token_scale` for Ref V2V;
616
+ prompt rewriting for v0.1 / no-ref).
617
+
618
+ Treat CFG as a real knob, not a constant β€” and be ready to give up some
619
+ speed when you actually need the edit to land.
620
+
621
+ ---
622
+
623
+ ## ComfyUI nodes
624
+
625
+ All recommended inference paths run through the **BFSnodes** custom node
626
+ set. For now BFSnodes is the only place these nodes live; once they
627
+ stabilise they may move elsewhere.
628
+
629
+ Specific nodes used by these LoRAs:
630
+
631
+ - `LTXVEditAnythingApply` β€” load the LoRA + extras and patch the model.
632
+ - `πŸ…›πŸ…£πŸ…§ LTXV Edit Anything Looping Sampler` β€” sampler that injects role /
633
+ AdaLN / visual cross-attention and handles long videos in chunks.
634
+ - `LTXVEditAnythingModuleLoader` β€” load the `*.module.safetensors` sidecar.
635
+
636
+ ---
637
+
638
+ ## Status
639
+
640
+ Released as experimental research artefacts. Expect failures, do not
641
+ deploy, and please report what works and what doesn't.
642
+
643
+ ---
644
+
645
+ ## Credits
646
+
647
+ If you use these models β€” in a project, a demo, a paper, a video, a tweet,
648
+ a workflow, anything β€” **please credit my work**. These checkpoints are the
649
+ result of weeks of research, dataset building, and training runs, and that
650
+ effort is what makes any of it usable. Crediting the source is the bare
651
+ minimum that keeps open research like this sustainable.
652
+
653
+ **Author:** Alisson Pereira dos Anjos ([@Alissonerdx](https://huggingface.co/Alissonerdx))
654
+
655
+ Suggested attribution:
656
+
657
+ > Edit Anything LoRAs by Alisson Pereira dos Anjos
658
+ > ([huggingface.co/Alissonerdx/EditAnything](https://huggingface.co/Alissonerdx/EditAnything)).
659
+
660
+ Links back to this repository are appreciated wherever you publish results.