BianYx commited on
Commit
1a7670f
·
verified ·
1 Parent(s): cd2940e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -5
README.md CHANGED
@@ -26,7 +26,7 @@ Keywords: Video Inpainting, Video Editing, Video Generation
26
  <p align="center">
27
  <a href="https://yxbian23.github.io/project/video-painter">🌐Project Page</a> |
28
  <a href="https://arxiv.org/abs/2503.05639">📜Arxiv</a> |
29
- <a href="https://huggingface.co/datasets/TencentARC/VPBench">🗄️Data</a> |
30
  <a href="https://youtu.be/HYzNfsD3A0s">📹Video</a> |
31
  <a href="https://huggingface.co/TencentARC/VideoPainter">🤗Hugging Face Model</a> |
32
  </p>
@@ -53,14 +53,14 @@ Keywords: Video Inpainting, Video Editing, Video Generation
53
 
54
  ## 🔥 Update Log
55
  - [2025/3/09] 📢 📢 [VideoPainter](https://huggingface.co/TencentARC/VideoPainter) are released, an efficient, any-length video inpainting & editing framework with plug-and-play context control.
56
- - [2025/3/09] 📢 📢 [VPData and VPBench](https://huggingface.co/datasets/TencentARC/VPBench) are released, the largest video inpainting dataset with precise segmentation masks and dense video captions (>390K clips).
57
 
58
  ## TODO
59
 
60
  - [x] Release trainig and inference code
61
  - [x] Release evluation code
62
  - [x] Release [VideoPainter checkpoints](https://huggingface.co/TencentARC/VideoPainter) (based on CogVideoX-5B)
63
- - [x] Release [VPData and VPBench](https://huggingface.co/datasets/TencentARC/VPBench) for large-scale training and evaluation.
64
  - [x] Release gradio demo
65
  - [ ] Data preprocessing code
66
  ## 🛠️ Method Overview
@@ -115,7 +115,7 @@ pip install -e .
115
 
116
  **VPBench and VPData**
117
 
118
- You can download the VPBench [here](https://huggingface.co/datasets/TencentARC/VPBench) (as well as the Davis we re-processed), which are used for training and testing the BrushNet. By downloading the data, you are agreeing to the terms and conditions of the license. The data structure should be like:
119
 
120
  ```
121
  |-- data
@@ -170,7 +170,16 @@ unzip davis.zip
170
  unzip video_inpainting.zip
171
  ```
172
 
173
- Noted: *Due to the space limit, you need to run the following script to download the complete VPData. The format should be consistent with VPBench above (After download the VPBench, the script will automatically place the video and mask sequences from VPData into the corresponding dataset directories that have been created by VPBench).*
 
 
 
 
 
 
 
 
 
174
 
175
  ```
176
  cd data_utils
@@ -224,6 +233,11 @@ The ckpt structure should be like:
224
  |-- transformer
225
  |-- vae
226
  |-- ...
 
 
 
 
 
227
  ```
228
 
229
 
 
26
  <p align="center">
27
  <a href="https://yxbian23.github.io/project/video-painter">🌐Project Page</a> |
28
  <a href="https://arxiv.org/abs/2503.05639">📜Arxiv</a> |
29
+ <a href="https://huggingface.co/collections/TencentARC/videopainter-67cc49c6146a48a2ba93d159">🗄️Data</a> |
30
  <a href="https://youtu.be/HYzNfsD3A0s">📹Video</a> |
31
  <a href="https://huggingface.co/TencentARC/VideoPainter">🤗Hugging Face Model</a> |
32
  </p>
 
53
 
54
  ## 🔥 Update Log
55
  - [2025/3/09] 📢 📢 [VideoPainter](https://huggingface.co/TencentARC/VideoPainter) are released, an efficient, any-length video inpainting & editing framework with plug-and-play context control.
56
+ - [2025/3/09] 📢 📢 [VPData](https://huggingface.co/datasets/TencentARC/VPData) and [VPBench](https://huggingface.co/datasets/TencentARC/VPBench) are released, the largest video inpainting dataset with precise segmentation masks and dense video captions (>390K clips).
57
 
58
  ## TODO
59
 
60
  - [x] Release trainig and inference code
61
  - [x] Release evluation code
62
  - [x] Release [VideoPainter checkpoints](https://huggingface.co/TencentARC/VideoPainter) (based on CogVideoX-5B)
63
+ - [x] Release [VPData and VPBench](https://huggingface.co/collections/TencentARC/videopainter-67cc49c6146a48a2ba93d159) for large-scale training and evaluation.
64
  - [x] Release gradio demo
65
  - [ ] Data preprocessing code
66
  ## 🛠️ Method Overview
 
115
 
116
  **VPBench and VPData**
117
 
118
+ You can download the VPBench [here](https://huggingface.co/datasets/TencentARC/VPBench), and the VPData [here](https://huggingface.co/datasets/TencentARC/VPData) (as well as the Davis we re-processed), which are used for training and testing the BrushNet. By downloading the data, you are agreeing to the terms and conditions of the license. The data structure should be like:
119
 
120
  ```
121
  |-- data
 
170
  unzip video_inpainting.zip
171
  ```
172
 
173
+ You can download the VPData (only mask and text annotations due to the space limit), and put the dataset to the `data` folder by:
174
+ ```
175
+ git lfs install
176
+ git clone https://huggingface.co/datasets/TencentARC/VPData
177
+ mv VPBench data
178
+ cd data
179
+ unzip video_inpainting.zip
180
+ ```
181
+
182
+ Noted: *Due to the space limit, you need to run the following script to download the raw videos of the complete VPData. The format should be consistent with VPData/VPBench above (After download the VPData/VPBench, the script will automatically place the raw videos of VPData into the corresponding dataset directories that have been created by VPBench).*
183
 
184
  ```
185
  cd data_utils
 
233
  |-- transformer
234
  |-- vae
235
  |-- ...
236
+ |-- flux_inp
237
+ |-- scheduler
238
+ |-- transformer
239
+ |-- vae
240
+ |-- ...
241
  ```
242
 
243