RuneXX

/

LTX-2.3-Workflows

like 92

Yes sometimes it may struggle. But things that help is prompting that "he is talking with perfect lip-sync to the attached audio".
Or at least include in the prompt that someone is talking.

And even better to transcribe what's in the audio (although often not needed).
Like for example: and then he talks and says "..... transcribe the dialog from your audio .... "

it's not the talking part but the effect sound part it usually muted only the dialog part is applied to generated videos

Ups sorry i misread. You said sound Fx.
I havent really played around much with that other than drummer and music (that the model understood automagically surprisingly well).
See drummer here: https://huggingface.co/RuneXX/LTX-2.3-Workflows/discussions/4

But taking a guess, it might help if your prompt includes the sound fx.
For example : a loud explosion can be seen and heard in the background... etc

I'll try some my end soon, see if i can figure it out, if that didnt help ;-)

ahh i see gonna try ur suggestion soon

Actually something came to mind after a coffee ;-) with a clear mind...
The default workflow has a vocal only extractor, ideal for lip-sync etc.
But this also means that the model never hears any sound fx ...

So try connect the audio directly (audio duration node) directly to the LTX audio encode node.
I will make an easy switch and update the workflow soon, makes sense to have the option ;-)

Updated the workflow, it now has a simple switch between vocals only (that can help if you have a busy audio and the focus is on lip-sync and dialog) or using full audio (including sound fx etc)

And if you get less lip-sync transcribe the dialog in the prompt

wow thank u gonna test the updated workflow now

it worked, thanks u

Good to hear ;-)

And maybe i should "expose" all the set and get nodes like you did ;-)
Will consider that for future workflows ..