Instructions to use maxin-cn/Cinemo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use maxin-cn/Cinemo with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("maxin-cn/Cinemo", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Link dataset to paper
Browse filesThis PR ensures the model can be discovered from the paper page: https://huggingface.co/papers/2407.15642
README.md
CHANGED
|
@@ -48,4 +48,18 @@ bash pipelines/video_editing.sh
|
|
| 48 |
```
|
| 49 |
|
| 50 |
## Acknowledgments
|
| 51 |
-
Cinemo has been greatly inspired by the following amazing works and teams: [LaVie](https://github.com/Vchitect/LaVie) and [SEINE](https://github.com/Vchitect/SEINE), we thank all the contributors for open-sourcing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
```
|
| 49 |
|
| 50 |
## Acknowledgments
|
| 51 |
+
Cinemo has been greatly inspired by the following amazing works and teams: [LaVie](https://github.com/Vchitect/LaVie) and [SEINE](https://github.com/Vchitect/SEINE), we thank all the contributors for open-sourcing.
|
| 52 |
+
|
| 53 |
+
## Bibtex citation
|
| 54 |
+
|
| 55 |
+
```bibtex
|
| 56 |
+
@misc{ma2024cinemoconsistentcontrollableimage,
|
| 57 |
+
title={Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Models},
|
| 58 |
+
author={Xin Ma and Yaohui Wang and Gengyun Jia and Xinyuan Chen and Yuan-Fang Li and Cunjian Chen and Yu Qiao},
|
| 59 |
+
year={2024},
|
| 60 |
+
eprint={2407.15642},
|
| 61 |
+
archivePrefix={arXiv},
|
| 62 |
+
primaryClass={cs.CV},
|
| 63 |
+
url={https://arxiv.org/abs/2407.15642},
|
| 64 |
+
}
|
| 65 |
+
```
|