import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("jiagaoxiang/stable-video-diffusion-img2vid", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
All the model components are saved from fp16 format of stabilityai/stable-video-diffusion-img2vid except the vae folder is replaced with the fp32 format of stabilityai/stable-video-diffusion-img2vid. This may help solve the black image issue caused by the vae.
More context: For SDXL, converting vae to fp16 will cause NaNs which results in black images. This is because of overflow numbers inside vae weights. Link: https://huggingface.co/madebyollin/sdxl-vae-fp16-fix
- Downloads last month
- 2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support