Instructions to use ZhengmingYu/DMAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ZhengmingYu/DMAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ZhengmingYu/DMAD") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
Emotional issues in voice-over ~
#3
by sunnyboxs - opened
The LoRA performs quite well; however, in my comparative tests—for instance, when using a prompt like (in Chinese):
男人的画外音说:"测试角色音色,这个效果真不错",
—the voice-over generated by DMAD sounds quite strange.
It lacks emotion, feels mechanical, and carries a very distinct "text-to-speech" (TTS) quality.
I also tested other LoRAs; using the same prompts, the emotional nuances differed significantly.
Could the use of TTS during training have caused the emotional expressiveness of the voice-over to weaken?
Thanks~
It might have been an issue with the model; changing the random seed resolved it.
sunnyboxs changed discussion status to closed