Music Video Workflow

#117
by earthboundAI - opened

So im attempting to use the music video workflow, and i have each scene differentiated and seperated, but they all seem to merge into each other when i start up the workflow.

I also have this issue, the images from the latter scenes are used in scene two, for example.

Owner

Will check. I did see at some comfy builds it seemed to "cache" the image loaders, and got confused.

Owner

Try update comfyui so we are at same version.
I tried reproduce it my end, but everything ran smootly .

( i was able to reproduce it earlier, and was on my list of things to try other image loader, since it seemed like ComfyUI cached or did something strange with multiple image laoders, but when i am trying now, i cant reproduce it, everything seems ok again)

Try update comfyui so we are at same version.
I tried reproduce it my end, but everything ran smootly .

( i was able to reproduce it earlier, and was on my list of things to try other image loader, since it seemed like ComfyUI cached or did something strange with multiple image laoders, but when i am trying now, i cant reproduce it, everything seems ok again)

So I have updated my ComfyUI to the nightly update as i usually do. And the first scene is always merging with the fourth scene and so on. The images are merging together in the scenes.

Owner

Ok, will see if i can reproduce. I did have this happen once, that comfyUI got "too lazy" to load all the images.
But will try a better image loader than the stock one, should hopefully do the trick ;-)

(comfy gone a bit too extreme on some caching perhaps)

Owner
β€’
edited May 1

Try this one:
https://huggingface.co/RuneXX/LTX-2.3-Workflows/tree/main/Music-Video-Creator/Image-Test

Basically what it does, is forcing comfy to load all the images earlier by making a little image collage.
The theory being that the other images are loaded so late in the workflow run, that comfy gets "lazy" and just picks one from earlier loaders (some cache issue)

(if that doesnt fix it I will use other image loader instead.. actually maybe i can make one already quickly to test just in case)

Gonna load it up and test it right now :3

Owner

hehe ;-) thats wild haha ... entertaining for sure ;-)

Monkeys from outerspace.. haha

hehe ;-) thats wild haha ... entertaining for sure ;-)

Monkeys from outerspace.. haha

inspired by this with pretty much the same lyrics https://www.youtube.com/watch?v=iHvrwUUvLZk

with Vrgamedevgirl's T2V Wild West Lora, I had so much fun making this and doing retakes.

Owner

Oh wow, thats insanely good ;-)
extremely catchy and the music video is superb ;-)

And great humor ;-) well done

Owner

Was trying out some AceStep XL loras, so ended up as a little video ... may the force be with you ;-) haha...
(low res, was just testing some stuff)

Updated regular "music video" workflow
(this variant of the workflow does not save intermediate files, but rather generates it all in one go...)

  • Removed previous batch image nodes with overlap and forward (these are more typical for overlapping frames, and this workflow does not have that)
  • Each group's output frames stored in set variables, and merged at end with image batch multi for the full video (instead of accumulating frames and forward to next groups). Might be a little less RAM heavy.
  • ... plus all the small things from above (more accurate audio duration, better fps conditioning etc)<
  • A bit simpler alternative than the "save each segment" wf.

(not really sure both workflows are needed, since the save intermediate files wf might be the better option for many/most. But since it used to be 2 variants.. They are quite similar though)

Plus made an excuse for a little rock video when testing ;-) 🀘🎸🎢

Hey RuneXX, I am in desperate need for your help, I need the the prompt for this video please if you still have it (the music style and the lyrics and the seed πŸ˜€ ).. I love this music and I want to create a song with same music. please if you still have it.

Hey RuneXX, I am in desperate need for your help, I need the the prompt for this video please if you still have it (the music style and the lyrics and the seed πŸ˜€ ).. I love this music and I want to create a song with same music. please if you still have it.

Will try search for the audio, ref images, and prompt i used ... and share if I can find it ;-)
Can be a nice starting point to try out the workflow i guess..

Hey RuneXX, I am in desperate need for your help, I need the the prompt for this video please if you still have it (the music style and the lyrics and the seed πŸ˜€ ).. I love this music and I want to create a song with same music. please if you still have it.

Will try search for the audio, ref images, and prompt i used ... and share if I can find it ;-)
Can be a nice starting point to try out the workflow i guess..

Thank you so much, the most important thing is the music style prompt please, I know it is too much to ask but I uploaded your song to Grok and asked to give me a prompt for the music and it failed :))

Sign up or log in to comment