Update README.md

README.md

@@ -1,20 +1,114 @@
 ---
 license: apache-2.0
-base_model:
-- alibaba-pai/CogVideoX-Fun-V1.5-5b-InP
+tags:
+  - video-inpainting
+  - video-editing
+  - object-removal
+  - cogvideox
+  - diffusion
+  - video-generation
+pipeline_tag: video-to-video
 ---
 
 # VOID: Video Object and Interaction Deletion
-<div style="line-height: 1;">
-  <a href="https://void-model.github.io/" target="_blank" style="margin: 2px;">
-    <img alt="Website" src="https://img.shields.io/badge/Website-VOID-4285F4" style="display: inline-block; vertical-align: middle;"/>
-  </a>
-  <a href="https://arxiv.org/abs/XXXX.XXXXX" target="_blank" style="margin: 2px;">
-    <img alt="arXiv" src="https://img.shields.io/badge/arXiv-VOID-FBBC06" style="display: inline-block; vertical-align: middle;"/>
-  </a>
-
-</div>
-
-This is the model card for VOID: Video Object Interaction Deletion. 
-It is built on top of [CogVideoX](https://github.com/THUDM/CogVideo) and fine-tuned for video inpainting with interaction-aware mask conditioning.
-Pass_1 and Pass_2 models correspond to the initial and optional refinement pass for object and associated interaction removal.
+
+<video src="https://github.com/user-attachments/assets/ad174ca0-2feb-45f9-9405-83167037d9be" width="100%" controls autoplay loop muted></video>
+
+VOID removes objects from videos along with all interactions they induce on the scene — not just secondary effects like shadows and reflections, but **physical interactions** like objects falling when a person is removed.
+
+**[Project Page](https://void-model.github.io/)** | **[Paper](https://arxiv.org/abs/XXXX.XXXXX)** | **[GitHub](https://github.com/netflix/void-model)** | **[Demo](https://huggingface.co/spaces/sam-motamed/VOID)**
+
+## Quick Start
+
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/netflix/void-model/blob/main/notebook.ipynb)
+
+The included notebook handles setup, downloads models, runs inference on a sample video, and displays the result. Requires a GPU with **40GB+ VRAM** (e.g., A100).
+
+## Model Details
+
+VOID is built on [CogVideoX-Fun-V1.5-5b-InP](https://huggingface.co/alibaba-pai/CogVideoX-Fun-V1.5-5b-InP) and fine-tuned for video inpainting with interaction-aware **quadmask** conditioning — a 4-value mask that encodes the primary object (remove), overlap regions, affected regions (falling objects, displaced items), and background (keep).
+
+### Checkpoints
+
+| File | Description | Required? |
+|------|-------------|-----------|
+| `void_pass1.safetensors` | Base inpainting model | Yes |
+| `void_pass2.safetensors` | Warped-noise refinement for temporal consistency | Optional |
+
+Pass 1 is sufficient for most videos. Pass 2 adds optical flow-warped latent initialization for improved temporal consistency on longer clips.
+
+### Architecture
+
+- **Base:** CogVideoX 3D Transformer (5B parameters)
+- **Input:** Video + quadmask + text prompt describing the scene after removal
+- **Resolution:** 384x672 (default)
+- **Max frames:** 197
+- **Scheduler:** DDIM
+- **Precision:** BF16 with FP8 quantization for memory efficiency
+
+## Usage
+
+### From the Notebook
+
+The easiest way — clone the repo and run [`notebook.ipynb`](https://github.com/netflix/void-model/blob/main/notebook.ipynb):
+
+```bash
+git clone https://github.com/netflix/void-model.git
+cd void-model
+```
+
+### From the CLI
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+
+# Download the base model
+huggingface-cli download alibaba-pai/CogVideoX-Fun-V1.5-5b-InP \
+    --local-dir ./CogVideoX-Fun-V1.5-5b-InP
+
+# Download VOID checkpoints
+huggingface-cli download netflix/void-model \
+    --local-dir .
+
+# Run Pass 1 inference on a sample
+python inference/cogvideox_fun/predict_v2v.py \
+    --config config/quadmask_cogvideox.py \
+    --config.data.data_rootdir="./sample" \
+    --config.experiment.run_seqs="lime" \
+    --config.experiment.save_path="./outputs" \
+    --config.video_model.transformer_path="./void_pass1.safetensors"
+```
+
+### Input Format
+
+Each video needs three files in a folder:
+
+```
+my-video/
+  input_video.mp4      # source video
+  quadmask_0.mp4       # 4-value mask (0=remove, 63=overlap, 127=affected, 255=keep)
+  prompt.json          # {"bg": "description of scene after removal"}
+```
+
+The repo includes a mask generation pipeline (`VLM-MASK-REASONER/`) that creates quadmasks from raw videos using SAM2 + Gemini.
+
+## Training
+
+Trained on paired counterfactual videos generated from two sources:
+
+- **HUMOTO** — human-object interactions rendered in Blender with physics simulation
+- **Kubric** — object-only interactions using Google Scanned Objects
+
+Training was run on **8x A100 80GB GPUs** using DeepSpeed ZeRO Stage 2. See the [GitHub repo](https://github.com/netflix/void-model#%EF%B8%8F-training) for full training instructions and data generation code.
+
+## Citation
+
+```bibtex
+@article{motamed2026void,
+  author    = {Motamed, Saman and Harvey, William and Klein, Benjamin and Van Gool, Luc and Yuan, Zhuoning and Cheng, Ta-ying},
+  title     = {VOID: Video Object and Interaction Deletion},
+  journal   = {arXiv preprint},
+  year      = {2026},
+}
+```