Instructions to use NimVideo/cogvideox-2b-img2vid with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use NimVideo/cogvideox-2b-img2vid with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("NimVideo/cogvideox-2b-img2vid", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
CogVideoXImageToVideoPipeline
#3
by Luke100000 - opened
Hi,
When using the CogVideoXImageToVideoPipeline with the example code as shown here: https://huggingface.co/THUDM/CogVideoX-5b-I2V I get raw noise.
What changes are required?
Hi, the CogVideoX 5b img2vid pipeline is different from the 2b img2vid. Without padding but repeat images before it stack with input noise. You can see an example of pipeline here: https://github.com/Nim-Video/cogvideox-2b-img2vid
Ahh, nows it works, thanks!
Luke100000 changed discussion status to closed