Text-to-Video
Diffusers
diffusers-training
lora
mochi-1-preview
mochi-1-preview-diffusers
template:sd-lora
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("genmo/mochi-1-preview", dtype=torch.bfloat16, device_map="cuda")
pipe.load_lora_weights("soumildatta/mochi-lora")
prompt = "A man with short gray hair plays a red electric guitar."
output = pipe(prompt=prompt).frames[0]
export_to_video(output, "output.mp4")Mochi-1 Preview LoRA Finetune
Model description
This is a lora finetune of the Mochi-1 preview model genmo/mochi-1-preview.
The model was trained using CogVideoX Factory - a repository containing memory-optimized training scripts for the CogVideoX and Mochi family of models using TorchAO and DeepSpeed. The scripts were adopted from CogVideoX Diffusers trainer.
Download model
Download LoRA in the Files & Versions tab.
Usage
Requires the 🧨 Diffusers library installed.
from diffusers import MochiPipeline
from diffusers.utils import export_to_video
import torch
pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview")
pipe.load_lora_weights("CHANGE_ME")
pipe.enable_model_cpu_offload()
with torch.autocast("cuda", torch.bfloat16):
video = pipe(
prompt="CHANGE_ME",
guidance_scale=6.0,
num_inference_steps=64,
height=480,
width=848,
max_sequence_length=256,
output_type="np"
).frames[0]
export_to_video(video)
For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers.
Intended uses & limitations
How to use
# TODO: add an example code snippet for running this diffusion pipeline
Limitations and bias
[TODO: provide examples of latent issues and potential remediations]
Training details
[TODO: describe the data used to train the model]
- Downloads last month
- -
Model tree for soumildatta/mochi-lora
Base model
genmo/mochi-1-preview