Instructions to use suimu/VIRES_VAE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use suimu/VIRES_VAE with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("suimu/VIRES_VAE", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Improve model card with metadata and links
Browse filesThis PR improves the model card by adding crucial metadata, including the `pipeline_tag`, `library_name`, and `license`, making the model more discoverable on the Hugging Face Hub. It also ensures clear links to the paper and the GitHub repository are present for improved accessibility. A brief description of the model's purpose is included as well.
README.md
CHANGED
|
@@ -1,12 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# VIRES_VAE model card
|
| 2 |
|
| 3 |
**Model Page**: [VIRES](https://hjzheng.net/projects/VIRES/)
|
| 4 |
|
| 5 |
-
This is a component of VIRES model.
|
|
|
|
| 6 |
|
| 7 |
### Usage
|
| 8 |
|
| 9 |
-
Ref to our GitHub page: https://github.com/suimuc/VIRES/
|
| 10 |
|
| 11 |
## Citation
|
| 12 |
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-to-video
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
license: mit
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
# VIRES_VAE model card
|
| 8 |
|
| 9 |
**Model Page**: [VIRES](https://hjzheng.net/projects/VIRES/)
|
| 10 |
|
| 11 |
+
This is a component of the VIRES model, as presented in [VIRES: Video Instance Repainting with Sketch and Text Guidance](https://huggingface.co/papers/2411.16199). VIRES is a video instance repainting method with sketch and text guidance, enabling video instance repainting, replacement, generation, and removal. It leverages the generative priors of text-to-video models to maintain temporal consistency and produce visually pleasing results.
|
| 12 |
+
|
| 13 |
|
| 14 |
### Usage
|
| 15 |
|
| 16 |
+
Ref to our GitHub page: [https://github.com/suimuc/VIRES/](https://github.com/suimuc/VIRES/)
|
| 17 |
|
| 18 |
## Citation
|
| 19 |
|