Alissonerdx/EditAnything

Edit Anything — Experimental LTX-2 Video Editing LoRAs

Heads up. These LoRAs are research experiments. They are far from production-ready and will fail on many inputs. They are released for the community to play with and break, not as a finished tool.

This repository hosts three unrelated training tracks built on top of LTX-2.3 (22B) for video editing:

1. Edit Anything v0.1 — motion transfer LoRA (two ranks).
2. Edit Anything — no-reference multitask LoRA (rank 256, prompt-driven only).
3. Reference video-to-video (Ref V2V) — experimental IC-LoRA + sidecar modules (two builds).

Inference is meant to run through the BFSnodes ComfyUI custom nodes — the Ref V2V build in particular needs them to load the sidecar modules and install the custom branches into the transformer.

---

1. Edit Anything v0.1 (motion transfer)

Files:

• edit_anything_30k_v0.1_motion_transfer_r128.safetensors
• edit_anything_30k_v0.1_motion_transfer_r256.safetensors

What it is

v0.1 is not a direct continuation of v1.0. It was trained from scratch in two stages:

1. Stage 1 — image-only pretraining. ~30 000 image edit pairs. Training a video model on still images is admittedly not ideal, but it was a way to push the editing vocabulary beyond what a small video-only dataset can teach.
2. Stage 2 — video fine-tune with first_frame_conditioning > 0. This restored the temporal prior and unlocked the motion-transfer behaviour described below.

In theory v0.1 can do the same edits as v1.0, but temporal consistency may be weaker than v1.0 because so much of stage 1 happened on still images. Test against v1.0 case-by-case before assuming v0.1 wins on your task.

Motion transfer

Because stage 2 included first-frame conditioning, you can drive the LoRA into a motion-transfer mode:

1. Take a guide video.
2. Replace its first frame with an edited still (insert a new subject, swap an object, etc.). Use a strong image-editing model — Flux Klein or similar — to prepare it; the quality of this single frame propagates through the whole clip.
3. Feed the edited frame as the first frame of the input, and the original guide video as the motion source.

The model uses the new first frame as the appearance anchor and copies the motion from the rest of the guide.

Limitations (these are real, not theoretical — expect them to bite):

• Hard scene cuts break it. The model assumes continuous motion from the first frame onwards. A cut to a different camera angle or location mid-clip will produce smearing, ghosting, or the inserted subject jumping to the wrong position. Use clips without cuts, or split at the cuts and process each segment separately.
• Very fast motion fails. Quick pans, fast subject movement, or high-velocity action confuse the motion-copy mechanism. Outputs degrade to blur or to the model "freezing" on the first-frame appearance and losing the motion entirely. Stick to moderate-speed clips.
• Poor blending / artefacts in the first frame propagate everywhere.
• Works best when the inserted subject roughly occupies the same region as whatever it replaces.

Prompting

Prompt is just as critical as in v1.0. Describe both the object being replaced and the new one in detail. Example: "Replace the bronze statue on the left with a tall man wearing a navy raincoat and brown boots." Vague prompts produce bad edits.

Which rank to use

The same training produced both files. v0.1 is actually the merge of the two-stage training (one LoRA per stage), re-extracted at two different ranks via Frobenius-optimal truncated SVD:

File	Rank	Size	Frobenius retention
edit_anything_30k_v0.1_motion_transfer_r128.safetensors	128	1.31 GB	~99.4%
edit_anything_30k_v0.1_motion_transfer_r256.safetensors	256	2.62 GB	~99.9%

r256 is closer to the merged source. r128 is normally indistinguishable in practice. Pick whichever fits your workflow.

How to wire the LoopingSampler

This is a standard LoRA, not a sidecar. Load it through the regular ComfyUI LoraLoader before the LoopingSampler. On the sampler itself:

• editanything_module → leave disconnected.
• ref_image → the edited first frame (for motion transfer) or the source frame you want preserved (for plain editing).
• guide_frames → the guide video.
• enable_role_embedding, enable_adaln, enable_visual_crossattn → all off. None of those branches were trained for v0.1; turning them on with no module connected does nothing anyway, but keeping them off silences the WARN logs.

---

2. Edit Anything — no-reference multitask LoRA

File:

• edit_anything_v1.1_r256.safetensors

What it is

A prompt-only multitask editing LoRA. No reference image, no first-frame conditioning — the model is driven entirely by the text prompt and the guide video. Trained on a balanced mix of Add, Remove, Replace, Style edits.

What it's different about it (vs v0.1)

The task vocabulary overlaps heavily with v0.1 — both can do Add, Remove, Replace, Change, Convert. What changes here:

• Two-stage training continuation: the first stage gave the model its edit vocabulary; the second stage refined it on a larger, more balanced video pair set covering Add / Remove / Replace / Style.
• Rank 256 (vs v0.1's effective rank from the merge), giving more capacity for the broader task mix.
• Trained directly on video pairs, so the temporal behaviour on these tasks tends to be steadier than on a model whose first stage was on still images.

How to use it

Standalone — load it as a regular LoRA on vanilla LTX-2.3 through any ComfyUI LoRA loader. The file already carries everything it needs; no stacking with v0.1, no companion module.

Limitations

• No reference image → identity is not anchored, so Add / Replace of a specific person or object will be wobblier than the Ref V2V build.
• No motion transfer (that's v0.1 only).

Prompting

Same imperative shape as v0.1, but the training set split into four very distinct caption styles. Match the one that fits the edit you want — the distribution is narrow and the model expects the right shape.

The training set is roughly balanced across Add, Remove, Replace and Style buckets, with Style being the smallest of the four. Captions below are real examples drawn from those buckets.

Add — 15 to 30+ words, describe what to add and where

• Add a smiling woman with brown hair, wearing a pink sleeveless top, sitting to the right of the man at the news desk.
• Add a person wearing a blue denim shirt over a white t-shirt to the right side of the frame, behind the person cooking.
• Add a decorated Christmas tree with red and white ornaments and lights to the right of the man.
• Add a blonde boy wearing a black t-shirt with a blue collar and blue patterned pants, sitting behind the other children in the upper center of the frame.
• Add two horizontal wooden strips to the front of the white range hood.

Pattern: Add <detailed subject description>, <position in frame>, <surrounding context>.

Remove — very short, 4 to 10 words

• Remove the man drinking from a glass.
• Remove the disco ball.
• Remove the large tree on the right.
• Remove the squirrel in the foreground.
• Remove the man on the left.

Pattern: Remove the <object> (+ optional position). Resist the urge to over-describe — long Remove prompts drift outside the training shape and often fail.

Replace — 20 to 35 words, describe both old and new

• Replace the white panel door on the right side of the frame with a dark brown grandfather clock.
• Replace the light-colored cat lying on the mat on the floor with a young woman sitting on the mat.
• Replace the dark grey knitted sweater on the man's torso with a black and white patterned Christmas sweater.
• Replace the blue robot with a glowing blue face on the left with a smiling man wearing sunglasses and a blue shirt.
• Replace the sitting person wearing a black cape on the left with a black fabric draped over an object.

Pattern: Replace <description of the original subject and its location> with <description of the new subject>.

Style — fixed template, the style name is what changes

• Convert the video into a Pencil Sketch style.
• Convert the video into a Watercolor Painting style.
• Convert the video into a Van Gogh style.
• Convert the video into a Play-Doh style.
• Convert the video into a Claymation style.
• Convert the video into a 3D Chibi style.
• Convert the video into a Ghibli style.
• Convert the video into a Pop Art style.
• Convert the video into an American Cartoon style.
• Convert the video into a Flat Vector Cartoon style.

The training set covers 300+ distinct style names. Many work; many do not. The list above is heavily represented in training. Use the exact phrase Convert the video into a <STYLE> style — deviations from this template degrade quality noticeably.

What it does not do

These are honest limits of the training distribution — don't expect them to work just because the model is multitask:

• No compositional prompts. "Add X and remove Y", "Replace A with B and add C", etc. are not in the training set. Captions combining two action verbs are essentially absent (the only ones present are the "Remove X and replace with Y" idiom, which is really a single Replace). Pure multi-action edits will fall apart — split them into separate runs.
• No "change background" as a task. Background is only used as a positional reference ("in the background", "on the wall in the background"). To swap the entire backdrop, phrase it as a Replace on a concrete background element, e.g. "Replace the brick wall in the background with a forest at sunset". Vague prompts like "Change the background to a beach" are off- distribution and rarely work.
• No global colour grade / lighting change. Only the Style template is trained as a global transform. Anything else global (LUT-style adjustments, time-of-day swaps without a concrete object) is unreliable.

Which LoRA should I use?

If you want…	Use
Motion transfer (edit first frame externally, model copies motion)	v0.1 motion transfer
Multi-task edits (add / remove / replace / style) driven only by prompt	no-ref multitask r256 (standalone)
Strong identity transfer from a reference image (Add / Replace)	Ref V2V

How to wire the LoopingSampler

A single standard LoRA, no sidecar, no stacking. Load through one ComfyUI LoraLoader before the LoopingSampler. On the sampler:

• editanything_module → leave disconnected.
• ref_image → leave disconnected. This LoRA has no reference-image path; passing one will just pre-encode tokens that you do not want.
• guide_frames → the guide video.
• enable_role_embedding, enable_adaln, enable_visual_crossattn → all off (no module = nothing to inject anyway).

---

3. Reference video-to-video (Ref V2V) — experimental

Files (two builds of the same LoRA family — each ships as a (.standard, .module) pair):

• edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.standard.safetensors
• edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding.module.safetensors
• edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.standard.safetensors
• edit_anything_reference_v0.1_r128_ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj.module.safetensors

What it is

The goal is add / replace using a reference image — same vibe as Edit Anything v1.0, but with an explicit image as the appearance source instead of relying only on the prompt.

Trained on ~1600 Add / Replace video pairs. Reference-paired video datasets are basically nonexistent, so the dataset had to be built from scratch — that is why the sample count is small. It often fails. This is fully experimental; thousands of training runs went into landing on this LoRA layout, and it is still unclear how much it actually helps.

Architecture — why this LoRA has "modules"

Trained as a conventional IC-LoRA, plus extra projection branches that try to make the reference signal survive across layers:

• ref_visual_proj — projects the reference VAE latent into 32 visual memory tokens.
• ref_attn — a dedicated cross-attention branch inside each transformer block, reading those tokens.
• ref_adaln_proj — a global AdaLN bias derived from the reference (palette / overall look).
• role_embedding — an experimental token bias inspired by some of Kijai's tests; whether it actually helps is still unclear.

These extra weights are saved alongside the LoRA in a .module.safetensors sidecar because they are not standard LoRA adapters — the regular ComfyUI LoRA loader can't consume them, so they need a dedicated node.

How to load

File	What it is	Where it goes
*.standard.safetensors	LoRA on attn1 / attn2 / ff only	Standard ComfyUI LoRA loader
*.module.safetensors	role_embedding, ref_adaln_proj, ref_visual_proj, ref_attn LoRA adapters	LTXVEditAnythingModuleLoader (BFSnodes)

Both files of a pair must be loaded together — the LoRA was trained against the sidecar adapters and they only make sense as a unit. Do not mix .standard from one build with .module from another.

The module file is consumed by the 🅛🅣🅧 LTXV Edit Anything Looping Sampler node, which was written specifically to:

1. Install the ref_attn cross-attention branch on every transformer block.
2. Inject the AdaLN / role / visual cross-attention conditioning at the correct points in the model.
3. Sample long videos in overlapping chunks with the conditioning re-applied per chunk.

How to wire the LoopingSampler

• Load the *.standard.safetensors through a normal ComfyUI LoraLoader before the sampler.
• Load the *.module.safetensors through LTXVEditAnythingModuleLoader and connect its editanything_module output to the sampler.
• On the sampler:
  • editanything_module → the module loader output (required).
  • ref_image → the reference image (required — this is what Add / Replace will insert).
  • guide_frames → the source video to edit.
  • enable_adaln → on (defaults match training).
  • enable_visual_crossattn → on for the 4-extras build; off (or will be a no-op) for the 2-extras build.
  • enable_role_embedding → off for the 4-extras build (training config disabled it). On if you're loading the 2-extras build alone.

Missing ref_image here silently disables AdaLN and the visual cross-attention — the sampler will warn in the log.

Which build to use

• ref_adaln_proj-role_embedding — the original training. Only ships the two side-channel modules.
• ref_adaln_proj-role_embedding-ref_attn-ref_visual_proj — the continuation. Adds the visual cross-attention branch and its projector on top.

It is genuinely not clear yet whether the extra branches help over the plain LoRA. Both builds are honest experiments. Try both, decide for your own use case, and please share findings.

Reading the layers

For anyone who wants to understand what each layer in the Ref V2V checkpoint does:

• lora_layers_reference.md — full tensor inventory of both builds.
• lora_layers_impact.md — what each branch contributes at inference and which inference knob (adaln_scale, ref_context_scale, ref_token_scale, ref_start_block, ref_end_block, etc.) maps back to which training default.

---

Prompt examples

The two LoRAs were trained on very different caption styles. Match the style of whichever LoRA you're using — straying outside the training distribution is the fastest way to get garbage out.

Edit Anything v0.1 — standard editing

The stage-1 dataset uses short imperative captions describing one or two edits. Use the same shape at inference. Examples drawn from the training distribution:

• "Replace the stone statue of a man on the left with a young woman in a green dress."
• "Add a black labrador retriever sitting beside the woman on the bench."
• "Remove the teacher from the classroom."
• "Alter the cap's colour from modern black to deep maroon."
• "Replace the fresh citrus-green background with a wooden desk."
• "Add faint tire tracks across the snow behind the car."
• "Add a black statue, a blue camera, a cyan towel, a red guitar and a pink backpack to the lakeside pier."

Tips:

• Imperative verbs: Add / Replace / Remove / Alter / Change.
• When replacing, describe both the original and the new subject so the model can localise the edit.
• Keep captions short and concrete. Long flowery prose hurts.

Edit Anything v0.1 — motion transfer

Workflow:

1. Pick a guide video.
2. Edit only the first frame externally (Flux Klein or any capable image-edit model) to introduce the new subject in the desired pose and position.
3. Feed the edited frame as the first frame of the input and the original guide as motion source.
4. The prompt should describe the inserted subject and the action being preserved.

Examples:

• "Replace the standing man holding the umbrella with a woman in a red coat holding the same umbrella, walking across the puddles."
• "Add a tabby cat curled up in the armchair while the man in the background keeps reading."
• "Replace the runner in the blue jersey with a man wearing a white shirt and grey shorts running along the same path."

Limits: fast or chaotic motion will fail; the inserted subject should occupy roughly the same region/scale as what it replaces.

Reference V2V (Ref V2V) — Add and Replace

These captions are real samples from the ~1600-pair training set. They describe the target scene after the edit in detail. The reference image carries the appearance of the inserted subject; the caption carries position, pose, action, and surrounding context.

Add task (the reference image holds the new subject):

• "Add a middle-aged man with curly grey hair, a beard and glasses, wearing a blue quarter-zip sweater, on the right side of the frame, standing in front of a raw cut of meat on a tray."
• "Add a light-coloured small boat with dark seats and an outboard motor floating in the water."
• "Add an open book filled with colourful pencils in the woman's hands."
• "Add a silver metallic bucket on the table in front of the blonde character, with her hands stirring a mixture inside."
• "Add two miniature dolls, one blonde and one brunette, dressed in patterned clothing, sitting at a small table with teacups and small white vases on the countertop."

Replace task (the reference image holds the new subject; the caption also describes what is being replaced):

• "Replace the standing kangaroo holding the bicycle handlebars with a man wearing a white t-shirt, light brown shorts and a yellow cap, holding the bicycle handlebars."
• "Replace the stone statue of a man on the left side with a young woman in a green dress."
• "Replace the wooden barrel near the entrance with a large brown leather suitcase."

Tips for Ref V2V:

• Describe the inserted subject in full, even though the reference image is the source of truth — the text path drives placement and pose.
• For Replace, also describe what is being replaced so the model can match the spatial region.
• Keep the inserted subject roughly in the same scale and region as what it replaces.
• The captions in the training set average ~25–40 words — aim for that range. Single-sentence captions like "Add a man" are far too sparse and will fail.

---

Inference tips (applies to all models)

CFG matters a lot here. The default workflow runs with the LTX-2.3 distilled / acceleration LoRAs for fast 4–8 step sampling, which locks CFG = 1.0. That's fine for casual runs, but at CFG 1 the model follows the prompt loosely — you get the reference image to "show up" but the edit instruction itself is only weakly enforced.

For harder prompts, raise CFG above 1.0. This means dropping (or weakening) the distilled / acceleration LoRAs and going back to a normal sampler with more steps — significantly slower, but the model follows the prompt much more closely. Trade-off:

Other knobs:

• If the model is ignoring the prompt (edit isn't being applied, the reference is barely showing up, the style transfer is faint), raising CFG is the single most common fix. Go up to 6–8 if needed.
• If results look over-saturated, plasticky, or motion is freezing, CFG is too high — pull back toward 3–4 or re-enable the distilled LoRA for CFG 1 if you don't actually need stronger prompt adherence.
• Ref V2V in particular benefits from being more aggressive with CFG when the reference identity isn't transferring cleanly.
• Combine CFG tuning with the LoRA-specific knobs from each section (adaln_scale, ref_context_scale, ref_token_scale for Ref V2V; prompt rewriting for v0.1 / no-ref).

Treat CFG as a real knob, not a constant — and be ready to give up some speed when you actually need the edit to land.

---

ComfyUI nodes

All recommended inference paths run through the BFSnodes custom node set. For now BFSnodes is the only place these nodes live; once they stabilise they may move elsewhere.

Specific nodes used by these LoRAs:

• LTXVEditAnythingApply — load the LoRA + extras and patch the model.
• 🅛🅣🅧 LTXV Edit Anything Looping Sampler — sampler that injects role / AdaLN / visual cross-attention and handles long videos in chunks.
• LTXVEditAnythingModuleLoader — load the *.module.safetensors sidecar.

---

Status

Released as experimental research artefacts. Expect failures, do not deploy, and please report what works and what doesn't.

---

Credits

If you use these models — in a project, a demo, a paper, a video, a tweet, a workflow, anything — please credit my work. These checkpoints are the result of weeks of research, dataset building, and training runs, and that effort is what makes any of it usable. Crediting the source is the bare minimum that keeps open research like this sustainable.

Author: Alisson Pereira dos Anjos (@Alissonerdx)

Suggested attribution:

Edit Anything LoRAs by Alisson Pereira dos Anjos (huggingface.co/Alissonerdx/EditAnything).

Links back to this repository are appreciated wherever you publish results.