Instructions to use ZhengmingYu/DMAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ZhengmingYu/DMAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ZhengmingYu/DMAD") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
It works wonderfully and doesn't alter the sound, thank you.
Thanks for the LoRa, it worked very well for me, and with the H3-Optimizations nodes, I was able to make 25-second videos in a single take.
Thanks for the LoRa, it worked very well for me, and with the H3-Optimizations nodes, I was able to make 25-second videos in a single take.
Thanks for sharing! The result looks great! I am really happy to see this. Really appreciate it!
Thanks for the LoRa, it worked very well for me, and with the H3-Optimizations nodes, I was able to make 25-second videos in a single take.
Thanks for sharing! The result looks great! I am really happy to see this. Really appreciate it!
The video was made with an RTX 4070 Super with 12 GB of VRAM and 128 GB of RAM. It took me 20 minutes to make the video, and there's no problem with the sound; this Lora doesn't degrade it.
It's great but some longer clips (around 15 sec) lose consistency. Eg multi-shot clip in Shot 1 woman is holding bouquet of flowers while on Shot 2 she has empty hands. It would be best lora so far it this was fixed. Happens on single shots as well.
It's great but some longer clips (around 15 sec) lose consistency. Eg multi-shot clip in Shot 1 woman is holding bouquet of flowers while on Shot 2 she has empty hands. It would be best lora so far it this was fixed. Happens on single shots as well.
LoRa speeds up the video rendering process. For the issue you mentioned between shots, you should use a system like Chanondevs's; their clips save the latency of each previous shot. I also recommend creating good prompts and using refmods.
No I did A/B testing and not only on single clip. Only this lora has this issue
I use it even in 30-second videos, with consecutive takes, and it doesn't give me any problems. What does give me problems are some broadcast models. There's another LoRa version out there, I tried it and that one is bad; it burns the images and causes image glitches.