Feature Request: Project-Based Character Memory with Smart Extend
Feature Request: Project-Based Character Memory with Smart Extend
Summary
Right now, tools like MSR LoRA need reference images every time you generate a clip — even for characters that already appeared in earlier clips. There's no "Project" concept that remembers characters as a story grows, and no way to extend an earlier clip while keeping every character consistent automatically.
This request asks for a Project mode that remembers characters across clips, plus a smarter Extend feature built on top of that memory.
The Problem
When making a multi-clip video with several characters (for example, a 10-clip story where new characters keep appearing), creators currently have to:
Upload the same reference images again and again, even for characters already established in earlier clips.
Keep track manually of which character appeared in which clip.
Generate clips strictly in order — there's no way to go back and extend an earlier clip once later ones already exist.
This is repetitive, error-prone, and gets harder the more characters a story has.
What's Being Proposed
- The user decides when to start a Project
Before generating the first clip, the user should have the option to create a Project. This should be the user's choice, not something the system decides on its own. Once a Project is created and the first clip is generated inside it, every character's identity (face, body, clothing) and the environment from that clip get saved and locked into the Project's memory.
- Characters are remembered as the story grows
For every new clip made inside the same Project:
Characters that already exist in memory — from any earlier clip, not just the last one — are applied automatically. No need to upload their reference again.
Only brand-new characters, ones that haven't appeared before, need a reference image.
Example: Clip 1 has characters A, B, and C. Clip 2 adds D and E. By Clip 3, all five characters are already known to the Project — nothing new needs to be uploaded unless a sixth character shows up.
- Extend should use that same memory
Instead of only generating the next clip in line, users should be able to pick any earlier clip and extend it directly.
When extending a clip:
Any character already in the Project's memory is pulled in automatically — no reference needed.
If the extension needs a character that hasn't appeared yet, the user should be able to add just that one reference, while the rest continue working automatically.
The reference upload shouldn't be mandatory. If a new character is implied by the prompt but no reference is given, the system should generate a sensible version of that character on its own, based on the prompt and the story's context, instead of producing something random or inconsistent.
Whether a character comes from a reference image or is generated automatically, it should be saved into the Project's memory right away. That way, if it shows up again later, it stays fully consistent without any extra work.
The end goal is simple: every clip and every extension inside a Project should keep every character's face, body, clothing, and voice (where audio-video LoRA is used) consistent — with as little repeated manual input as possible.
Why This Would Help
Cuts down repetitive work for anyone making multi-character, multi-clip content — especially long-form YouTube videos and episodic series.
Lowers the risk of characters drifting or looking different between clips, since their identity lives in one place instead of being re-entered each time.
Makes non-linear editing possible — going back to extend an earlier clip instead of only moving forward.
This goes further than the current native multishot generation, which only keeps characters consistent within a single generation. This feature would carry that consistency across separate clips and sessions within a whole project.
Related Work
MSR LoRA (LiconStudio) supports 2–5 reference images per generation, but has no memory across separate generations.
LTX-2.5's native multishot generation keeps characters consistent within one generation pass, not across multiple clips made separately.
A recent research paper, "VideoMemory" (arXiv 2601.03655), proposes a similar Dynamic Memory Bank for keeping characters, props, and backgrounds consistent across shots — supporting the idea that this approach is worth pursuing.
Ask
Please consider building a Project-based character memory system — either as a native feature in LTX-Desktop/ComfyUI or as an extension of the existing IC-LoRA/ID-LoRA/MSR tools — that remembers characters across clips within a project and supports extending any clip with that memory in place.
Based on where LTX-2.5 is today, I believe these features would fit well as part of the next LTX version.
A Personal Note
I'm sharing this after putting in a lot of thought, effort, and research into how character consistency actually works today and where the gaps are. I hope the team finds it useful.
+1 to this request, especially the voice part. Adding a concrete use case from a low-VRAM setup.
Voice consistency across clips (dubbing) – LTX-2.5 on 8 GB VRAM
Thank you for LTX-2.5. Native audio and the quality jump over 2.3 made it my main video model.
I run LTX-2.5 Distilled 22B (int8) locally through WanGP on an RTX 4060 with 8 GB VRAM and 40 GB RAM, using offloading. I make short ads, trailers and short films, many of them with dialogue in Polish. Visual consistency is now manageable with reference frames, but voice consistency is my biggest problem:
- Native audio: on 8 GB every shot has to be a separate generation of up to ~10 s. The same character gets a noticeably different timbre, pitch and delivery from clip to clip, even with the same prompt description of the voice.
- External audio (
audio_guide): to keep the voice fixed, I generate dialogue with an external TTS or voice clone and pass it as the audio guide. Lip-sync on calm close-ups is very good. However, the native audio track still contains the model's own rendition of the line in a different voice, so I have to drop the native audio completely and lose the ambience/foley it would otherwise give me. - Very short clips (1–1.5 s, e.g. camera bridges between shots) often produce random vocalizations in the native audio.
Example film: https://youtu.be/KBppWsWVfv8
My questions:
- Is there a recommended way to keep the same voice across separate generations, e.g. a speaker reference / audio conditioning, a fixed seed for the audio stream, or an audio-video LoRA for a character's voice?
- When dialogue comes from
audio_guide, can the model be told to generate only ambience/SFX in its own audio track, without re-voicing the line? - Are you planning a "voice identity" / speaker reference feature, similar to reference images for faces?
- How well is non-English speech (Polish) supported in terms of voice consistency, and are there prompting tips for it?
- Any tips for low-VRAM setups on extending clips while keeping the audio consistent?
I'm happy to share settings, test builds or more examples if that helps.
Best regards,
Marcin Wawrzyniak
DevPartner.pl / https://adfilmai.devpartner.pl
Follow-up / correction to my point 2 above, after digging into the WanGP code and measuring my clips.
The "double voice" with audio_guide was my mistake, not the model re-voicing the line. In WanGP (LTX-2.5, audio mode "A", Prompt Audio Strength 1.0) the guide audio is encoded with the audio VAE and frozen in both stages (AudioConditionByLatent, denoise mask 0) – the same "frozen audio latent" approach described in #44. The native audio track is just the guide line passed through the audio VAE/vocoder: envelope correlation 0.83–0.92 with the original, ~20 ms delay, ~1.5 dB quieter. Mixing it with the clean voice-over gave me a doubled / slapback voice.
The real catch: guide audio shorter than the clip. WanGP then switches to AudioConditionByLatentPrefix – only the beginning is frozen and the model freely generates the rest. In a 6 s clip with a 4.9 s line, the model added loud sound in the last 1.1 s (peak −8.6 dB).
Fix: pad the guide with silence to the exact clip length, e.g. ffmpeg -i line.mp3 -af apad=whole_dur=6.04 line_pad.wav. Same seed, same prompt: the tail is now clean (−91 dB) and the native track matches the guide (envelope correlation 1.00).
Side note on seeds (confirms #44): padding changes the result even with the same seed. With my original seed the man turned his head away from the camera mid-line; a second seed kept him talking to the camera. Worth generating 2–3 seeds per dialogue shot.
Tips for others:
- Pad the external audio to the full clip length.
- In the final mix use either the native track or the original line – never both.
- The frozen region carries no ambience, so add room tone / foley separately (or, untested, mix it into the guide before generating).
So my open questions are 1, 3, 4 and 5 – mainly a speaker-reference option for LTX-2.5. WanGP offers "Reference Voice (ID-LoRA)" for LTX-2.3 (id-lora-celebvhq-ltx2.3), but it is disabled for 2.5. Is an ID-LoRA (voice identity) for LTX-2.5 planned?
@Hassan139
Thank you for this detailed request and for the research behind it. The gaps you describe, especially around character consistency across clips, are real.
What you’re describing is an application layer built on top of the model rather than a model capability. LTX generates video and audio from the conditions it receives, such as prompts, start and end images, keyframes, and guidance videos. Grouping those conditions into persistent projects and reusing the same references consistently across separate generations is outside the model itself.
If you need this workflow today, products built on LTX already offer it. For example, LTX Studio lets you create reusable elements (characters, color palettes, styles, product references), build storyboards with them, and reuse them across generations to maintain consistency.
We recognize that character consistency and reference support are top requirements for the model, and we’re putting a lot of effort into making sure the next version excels at them. So stay tuned
@Don150
There is a fresh SepGen research project that does exactly what you request - open LoRA for LTX-2.5 that gives every sound in the video its own audio track (the code and the weights are provided)
I wish I could do +100 on this - I don't know why this is not a default setup for ANY director tool. Just trying to do a simple dance sequence turned into a nightmare for me. So YES YES YES. And I will research the suggestions others have made.