Update model card: pipeline_tag, library_name, paper link, and content from GitHub

README.md

@@ -1,13 +1,16 @@
 ---
-license: apache-2.0
 language:
 - en
-pipeline_tag: image-to-video
+license: apache-2.0
+pipeline_tag: any-to-any
+library_name: diffusers
 ---
+
 <div align="center">
 <h1> Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation </h1>
 
-<a href="https://arxiv.org/abs/2510.01284"><img src="https://img.shields.io/badge/arXiv%20paper-2509.08519-b31b1b.svg"></a>
+<a href="https://arxiv.org/abs/2510.01284"><img src="https://img.shields.io/badge/arXiv%20paper-2510.01284-b31b1b.svg"></a>
+<a href="https://github.com/character-ai/Ovi"><img src="https://img.shields.io/badge/Code-GitHub-181717.svg?logo=github"></a>
 <a href="https://aaxwaz.github.io/Ovi/"><img src="https://img.shields.io/badge/Project_page-More_visualizations-green"></a>
 <a href="https://huggingface.co/chetwinlow1/Ovi"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Model&color=orange"></a>
 <a href="https://huggingface.co/spaces/akhaliq/Ovi"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue"></a>
@@ -36,6 +39,8 @@ Ovi is a veo-3 like, **video+audio generation model** that simultaneously genera
 - **🎬 Video+Audio Generation**: Generate synchronized video and audio content simultaneously
 - **📝 Flexible Input**: Supports text-only or text+image conditioning
 - **⏱️ 5-second Videos**: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)
+- **🎬 Create videos now on wavespeed.ai**: https://wavespeed.ai/models/character-ai/ovi/image-to-video & https://wavespeed.ai/models/character-ai/ovi/text-to-video
+- **🎬 Create videos now on HuggingFace**: https://huggingface.co/spaces/akhaliq/Ovi
 
 ---
 ## 📋 Todo List
@@ -46,6 +51,9 @@ Ovi is a veo-3 like, **video+audio generation model** that simultaneously genera
   - [x] Text or Text+Image as input
   - [x] Gradio application code
   - [x] Multi-GPU inference with or without the support of sequence parallel
+  - [x] fp8 weights and improved memory efficiency (credits to [@rkfg](https://github.com/rkfg))
+  - [ ] Improve efficiency of Sequence Parallel implementation
+  - [ ] Implement Sharded inference with FSDP
 - [x] Video creation example prompts and format
 - [ ] Finetuned model with higher resolution
 - [ ] Longer video generation
@@ -129,6 +137,9 @@ OR
 # Optional can specific --output-dir to download to a specific directory
 # but if a custom directory is used, the inference yaml has to be updated with the custom directory
 python3 download_weights.py --output-dir <custom_dir>
+
+# Additionally, if you only have ~ 24Gb of GPU vram, please download the fp8 quantized version of the model, and follow the following instructions in sections below to run with fp8
+wget -O "./ckpts/Ovi/model_fp8_e4m3fn.safetensors" "https://huggingface.co/rkfg/Ovi-fp8_quantized/resolve/main/model_fp8_e4m3fn.safetensors"
 ```
 
 ## 🚀 Run Examples
@@ -156,6 +167,8 @@ slg_layer: 11                            # Layer for applying SLG (Skip Layer Gu
 
 # Multi-GPU and Performance
 sp_size: 1                               # Sequence parallelism size. Set equal to number of GPUs used
+cpu_offload: False                       # CPU offload, will largely reduce peak GPU VRAM but increase end to end runtime by ~20 seconds
+fp8: False                               # load fp8 version of model, will have quality degradation and will not have speed up in inference time as it still uses bf16 matmuls, but can be paired with cpu_offload=True, to run model with 24Gb of GPU vram
 
 # Input Configuration
 text_prompt: "/path/to/csv" or "your prompt here"          # Text prompt OR path to CSV/TSV file with prompts
@@ -182,7 +195,19 @@ torchrun --nnodes 1 --nproc_per_node 8 inference.py --config-file ovi/configs/in
 ```
 *Use this to run samples in parallel across multiple GPUs for faster processing.*
 
-
+### Memory & Performance Requirements
+Below are approximate GPU memory requirements for different configurations. Sequence parallel implementation will be optimized in the future.
+All End-to-End time calculated based on a 121 frame, 720x720 video, using 50 denoising steps. Minimum GPU vram requirement to run our model is **32Gb**, fp8 parameters is currently supported, reducing peak VRAM usage to **24Gb** with slight quality degradation.
+
+| Sequence Parallel Size | FlashAttention-3 Enabled | CPU Offload | With Image Gen Model | Peak VRAM Required | End-to-End Time |
+|-------------------------|---------------------------|-------------|-----------------------|---------------|-----------------|
+| 1                       | Yes                        | No          | No                    | ~80 GB        | ~83s         |
+| 1                       | No                        | No          | No                    | ~80 GB        | ~96s         |
+| 1                       | Yes                        | Yes          | No                    | ~80 GB        | ~105s         |
+| 1                       | No                        | Yes          | No                    | ~32 GB        | ~118s         |
+| **1**                       | **Yes**                        | **Yes**          | **Yes**                    | **~32 GB**        | **~140s**         |
+| 4                       | Yes                        | No          | No                    | ~80 GB        | ~55s         |
+| 8                       | Yes                        | No          | No                    | ~80 GB        | ~40s         |
 ### Gradio
 We provide a simple script to run our model in a gradio UI. It uses the `ckpt_dir` in `ovi/configs/inference/inference_fusion.yaml` to initialize the model
 ```bash
@@ -197,6 +222,12 @@ OR
 
 # To enable an additional image generation model to generate first frames for I2V, cpu_offload is automatically enabled if image generation model is enabled
 python3 gradio_app.py --use_image_gen
+
+OR
+
+# To run model with 24Gb GPU vram
+python3 gradio_app.py --cpu_offload --fp8
+
 ```
 ---
 
@@ -209,6 +240,30 @@ We would like to thank the following projects:
 
 ---
 
+## 🤝 Collaboration
+
+We welcome all types of collaboration! Whether you have feedback, want to contribute, or have any questions, please feel free to reach out.
+
+**Contact**: [Weimin Wang](https://linkedin.com/in/weimin-wang-will) for any issues or feedback.
+
+
+## 🤝 Contributors
+
+We thank all contributors who have helped improve Ovi!
+
+<div align="center">
+  <a href="https://github.com/character-ai/Ovi/graphs/contributors">
+    <img src="https://contrib.rocks/image?repo=character-ai/Ovi" />
+  </a>
+</div>
+
+<br>
+
+If you’ve contributed to this repository (code, documentation, issues, etc.), you’re automatically included in the [contributors list](https://github.com/character-ai/Ovi/graphs/contributors).  
+
+We deeply appreciate your support in advancing open multimodal generation research!
+---
+
 ## ⭐ Citation
 
 If Ovi is helpful, please help to ⭐ the repo.
@@ -227,4 +282,4 @@ If you find this project useful for your research, please consider citing our [p
       primaryClass={cs.MM},
       url={https://arxiv.org/abs/2510.01284}, 
 }
-```
+```