Is MSR an i2v model, or a scene composer? Experimenting different ways of using it
Hi, and thanks for releasing MSR. I'd like a clear explanation of the intended way to use it, because I'm not sure which paradigm it belongs to.
Is MSR an image-to-video model, where you give it a finished start frame and it animates that exact image? Or is it the other way round — you feed the elements separately (one image per subject, plus a background or a text description) and the model composes the scene itself from those pieces?
What I'm trying to do is the first one: define the composition first — framing, staging, lighting, with the characters already placed inside the image — and then have the clip animate that composition with no identity drift over the shot. The reference slots looked like the right tool to lock each person's identity so their face doesn't turn into a look-alike halfway through.
We did get part of the way there, so I'll describe what we tried in case it points at the right usage. First we pinned the start frame as a hard frame-0 keyframe alongside the reference slots: the reference frames leaked into the first second of the output, and the composition drifted (subjects re-centered or swapped sides relative to the start frame). Making the keyframe aware of the in-context reference parameters fixed the leak, but the composition was still being pulled around, and props or clothing from the subject references bled into the whole clip — a subject reference holding an instrument put that instrument in the shot, and a close-up face reference with no visible clothing produced an undressed character.
Then we tried the softer approach: no hard keyframe at all, and the start frame passed as one of the reference slots. That behaved much better — the clip does begin on our composition, no leak, wardrobe and scene context preserved, identities holding. The remaining problem is motion: the result comes out close to frozen, with very little camera or subject movement compared to a plain i2v pass on the same prompt, as if the in-context reference pins the shot in place.
So: is what I'm describing a supported use of MSR, or am I working against its intended paradigm? And if MSR is meant to compose the scene itself rather than animate a given frame, is there a supported way to pin a specific start composition and still get real motion?
Hi, and thanks for releasing MSR. I'd like a clear explanation of the intended way to use it, because I'm not sure which paradigm it belongs to.
Is MSR an image-to-video model, where you give it a finished start frame and it animates that exact image? Or is it the other way round — you feed the elements separately (one image per subject, plus a background or a text description) and the model composes the scene itself from those pieces?
What I'm trying to do is the first one: define the composition first — framing, staging, lighting, with the characters already placed inside the image — and then have the clip animate that composition with no identity drift over the shot. The reference slots looked like the right tool to lock each person's identity so their face doesn't turn into a look-alike halfway through.
We did get part of the way there, so I'll describe what we tried in case it points at the right usage. First we pinned the start frame as a hard frame-0 keyframe alongside the reference slots: the reference frames leaked into the first second of the output, and the composition drifted (subjects re-centered or swapped sides relative to the start frame). Making the keyframe aware of the in-context reference parameters fixed the leak, but the composition was still being pulled around, and props or clothing from the subject references bled into the whole clip — a subject reference holding an instrument put that instrument in the shot, and a close-up face reference with no visible clothing produced an undressed character.
Then we tried the softer approach: no hard keyframe at all, and the start frame passed as one of the reference slots. That behaved much better — the clip does begin on our composition, no leak, wardrobe and scene context preserved, identities holding. The remaining problem is motion: the result comes out close to frozen, with very little camera or subject movement compared to a plain i2v pass on the same prompt, as if the in-context reference pins the shot in place.
So: is what I'm describing a supported use of MSR, or am I working against its intended paradigm? And if MSR is meant to compose the scene itself rather than animate a given frame, is there a supported way to pin a specific start composition and still get real motion?
No, MSR references are not hard constraints, and they do not participate in the denoising process during training. However, you could try passing the start frame as the only reference image through the MSR node. It may give you slightly better consistency than a standard I2V workflow.