TenStrip

/

10Eros-Max

Like 815

Whats your prompt? Are you using the i2v prompt? Because im actually having the reverse effect. Its wickedly amazing at i2v but struggles with ref2v but im getting close.

I'm using the Ref2Va workflow but am not using any videos, just images. I find the other workflow always starts off with the picture before cutting to the scene which is annoying. The only way around it that I could find was to just stick to the ref2va workflow. I also use a few ref images at the same time which is only possble with this workflow. I am using the proper prompt format for this workflow that is recommended by the minimax team. Its strange that FL2va works really well for ref2v even though its not meant for it. It just looks better and sounds better then the ref2va model. I really like beta 6. Maybe I should try it with the other workflow. it would suck only being able to use one image though.

I also find beta 6 doesn't use my audio voice reference very well either. I can't seem to get it to match. FL2va model has no problems with this using the exact same prompt.

yeah no... you've got to use the image to video workflow and ensure the prompt is set up for i2v the beginning should be the standard - the target video at 00:00 blah blah. I've gotten amazing results with that.

but also... what's the prompt? if you don't want to share the actual prompt you're using, come up with a fake one and post the structure here. I can take a look at it.

Here is one prompt that I am messing with right now. there are several more in similar format using the default ref2v workflow. I just recently changed overall_soundscape: by what i read on the main page here hoping it would fix the audio ref. it does somewhat match sometimes but still not as good as the original model.

subject_definitions:

<Subject 1> is the red haired women fully_preserved in <Picture 1>.
<subject 2> is the blonde haired women fully_preserved in <Picture 2>.
<Audio 1> is the voice-timbre reference for <Subject 1> (S1).

summary:

[reference generation + audio reference] The target video uses <Audio 1> as the voice-timbre reference for <Subject 1>.

retention_analysis:

<Subject 1> (appears in [Shot 1]): fully_preserved - her face, hair, clothes, bone structure are exactly as in <picture 1>
<Subject 2> (appears in [Shot 1]): fully_preserved - her face, hair, clothes, bone structure are exactly as in <picture 2>
<Audio 1>: reference - its vocal timbre guides the dialogue delivery of <Subject 1> without copying the original signal.

detailed_description:

[Shot 1] <Subject 1> none of your business...

overall_soundscape:

<Subject 1> (S1) speaking in the timbre of <Audio 1>.

non_diegetic_music:

N/A

thanks by the way, for trying to help.

yup! That's reference to video, that's not i2v. Are you trying to do image to video or ref2vid? Your post is about the model being terrible at image to video. but to dissect what you have going on, you're summary needs to be a summary of the video clip, no assignments. and of course as you said it's none of my business... but if you want help I'll need to see how you have the detailed_description structured.

not sure why the description would really matter if this whole structure is no good anyway for this model but a quick example would be

[Shot 1] <Subject 2> turns to <subject 1> and says in a serious tone, [English] stop asking me for a detailed description. that's private.. [Shot 2] At 00:02.200, the camera cuts to a medium close-up of <subject 1> as she leans back, her face contorted in irritation, <Subject 1> (S1) responds angrily, using the voice from <Audio 1>: [English] If you want my help, I need more details dammit!.

for some reason, hugging face is removing the < / d > from my paragraph. When I go to edit it, it shows up but disappears after i post it. weird.

don't forget the tags for dialogue. cuts to and "as" are fighting each other. cuts to and then shows her lean back is the correct way.

what sampler/scheduler are you using?

This was just one quick example. I've made many other attempts with no dialogue but it still fails. Actually, come think of it, it seems to work ok with this format if I just use one image. Once I add another image of another person, that's were it is failing big time. Image 1 still looks decent enough(not as good as fl2va model) but image 2 is totally messed up. So I guess your right. I just need to stick to one image. Curious as to why tenstrip shows how to use an audio reference for his model when it was only meant for the image 2 video workflow.

As for the sampler/scheduler, I tried every single one suggested on the main page. As usual, Er_sde works the best and always does for every workflow prompt I've ever used. I constantly switch them around but always end up back to Er for better prompt adherence. Oh, and I switch around between simple and beta. they both produce similar results. I find ddim_uniform gives the clearest audio but it messes up the image contrast for some reason.

I just fixed my reference to video with my prompt. It was my summary that was jacking it all up, then on top of that it was how my detailed description was structured. So I'm getting good results with both now, i2v and ref2v. just needed to look at my prompt. lol I have prompts that I have saved and try to reuse instead of typing the whole scene out again, in this case, it just needed some tuning.

are you able to use a second image of another character and keep there likeness?

Indeed I am. I actually was able to get 4 characters in one video. I'm going to keep pushing the envelope to see how far I can take it before it breaks.

Something about the Loras or Reference deltas always makes my merges just not work on the FL node. But honestly the reference node does the same thing with much more control. If you want to use the reference node like the FL node, you can maybe try this string on one not-resized image input and just add a natural language i2v style prompt after it, or add a timestamp of when to reference the picture too:

<Picture 1> is the correct scene identity. All lighting, lighting hue, color palette, visual output style, camera angle, camera focal length, camera orbit, camera framing, background, background objects, and characters fully preserved from <Picture 1>.

Oh sheeett!! That's another thing I've been failing to mention this entire time too. Sorry ya'll. when my images are loaded I use pixaroma images, and I select the output to match what it will be when it hits my generation. so I crop my images to 9:16 if my output is going to be 9:16. 16:9 for 16:9 and so on. I do that for i2v. on my references I select max rather than match.

What is the FL node? I tried your suggestion Tenstrip with that prompt and it made no difference. Also made sure the aspect ratio was the same for the images and the video output but still nada. I just tried your model on a really good extension workflow for the first time and it is the only one that fails at keeping the likeness or using the audio reference for the characters.

I'm thinking maybe yours needs some kind of different prompting in order to get working properly. I also just tried the Dasiwa model which is probably very similar to yours and his has no issues keeping the likeness for extending or using multiple characters as a reference. the audio reference has no issues with his either so I really don't know where I'm messing up with your model here. Yours has way better audio then his and his tends to produce a distorted image a lot of the time.

The default models, Dasiwa and couple of other hybrid models I tried all work fine with my prompts. EllipsesMark claims your works great with no problems. So obviously it is something on my end screwing things up. I read a post of yours recently were you stated the default prompting doesn't work well with your model and that you were looking at posting a proper way to prompt for it. Will you be posting this info on your main page when you find a solution? Thanks.

Oh sheeett!! That's another thing I've been failing to mention this entire time too. Sorry ya'll. when my images are loaded I use pixaroma images, and I select the output to match what it will be when it hits my generation. so I crop my images to 9:16 if my output is going to be 9:16. 16:9 for 16:9 and so on. I do that for i2v. on my references I select max rather than match.

This. It seems like in beta 6, all reference images need to be the same aspect ratio as the output video. That's why whenever I generated a 3:4 video using 16:9 character model sheets, the output video had a lot of movement issues. Beta 5 handles reference images much better, though.