Is it possible to include 2.3-22b-IC-LoRA-Union-Control?

#49
by Erhart2024 - opened

Would it be possible to include the LoRa Union control to the LTX-2.3_-_I2V_T2V_Basic.json workflow? That workflow produces high quality video (1920x1088 in two stages) on the RTX5090 and works very fast and reliable. I tried to add the IC UNION control to it, but so far did not get it to work. (pose annotation, depth annotation, pose annotation)

https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Union-Control/blob/main/ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors

Owner

Yes, I been fighting a bit with that lora last few days, trying to make it best possible.
But I think it is what it is. It works great for slow motions etc. But for fast tik-tok alike dances it struggles a bit (it does a little better if you add more steps)

But i'll upload one soon to play around with ;-)

Thanks! I think LTX2.3 has the potential to come close to what Kling 2.X can do in terms of character animation. If the 22b-IC-Lora-Union-Control could be combined with the two-pass video generation workflows you already made (they produce high 1080p quality, much better than the single pass) then by first applying the character animation and then applying the new workflow; LTX-2.3_-_V2V_Just_Talk_add_lipsynced-voice_to_any_video.json it means local AI users can produce impressive videos with that, approaching Kling quality (animating character and adding voice), which would be fantastic. Or else - perhaps the 22b-IC-Lora-Union-Control and custom audio (two pass) workflows could be combined in a single workflow, that would even be more efficient.

Sign up or log in to comment