Spaces:
Configuration error
Configuration error
Commit
·
b5e0a87
1
Parent(s):
69073d3
update intro
Browse files
README.md
CHANGED
|
@@ -8,6 +8,17 @@
|
|
| 8 |
[](https://youtu.be/JKDLet618hU)
|
| 9 |
|
| 10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
<table class="center" border="1" cellspacing="0" cellpadding="5">
|
| 13 |
<tr>
|
|
@@ -92,7 +103,8 @@ VideoGrain is a training-free framework. To run the inference script, use the fo
|
|
| 92 |
|
| 93 |
```bash
|
| 94 |
bash test.sh
|
| 95 |
-
or
|
|
|
|
| 96 |
```
|
| 97 |
|
| 98 |
<details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
|
|
|
|
| 8 |
[](https://youtu.be/JKDLet618hU)
|
| 9 |
|
| 10 |
|
| 11 |
+
## Introduction
|
| 12 |
+
VideoGrain is a zero-shot method for class-level, instance-level, and part-level video editing.
|
| 13 |
+
- **Multi-grained Video Editing**
|
| 14 |
+
- class-level: Editing objects within the same class (previous SOTA limited to this level)
|
| 15 |
+
- instance-level: Editing each individual instance to distinct object
|
| 16 |
+
- part-level: Adding new objects or modifying existing attributes at the part-level
|
| 17 |
+
- **Training-Free**
|
| 18 |
+
- Does not require any training/fine-tuning
|
| 19 |
+
- **One prompt to control multi-regions, deep investigations about cross/self attn**
|
| 20 |
+
- modulating cross attn weight for multi-regions control, visualization are available
|
| 21 |
+
- modulating self attn weight for feature decoupling, clustering function are available
|
| 22 |
|
| 23 |
<table class="center" border="1" cellspacing="0" cellpadding="5">
|
| 24 |
<tr>
|
|
|
|
| 103 |
|
| 104 |
```bash
|
| 105 |
bash test.sh
|
| 106 |
+
#or
|
| 107 |
+
CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
|
| 108 |
```
|
| 109 |
|
| 110 |
<details><summary>The result is saved at `./result` . (Click for directory structure) </summary>
|
annotator/dwpose/__pycache__/wholebody.cpython-310.pyc
CHANGED
|
Binary files a/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc and b/annotator/dwpose/__pycache__/wholebody.cpython-310.pyc differ
|
|
|
config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
pretrained_model_path: "
|
| 2 |
logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
|
| 3 |
|
| 4 |
dataset_config:
|
|
|
|
| 1 |
+
pretrained_model_path: "ckpt/stable-diffusion-v1-5"
|
| 2 |
logdir: ./result/part_level/run_two_man/left2spider_right2polar-sunglasses
|
| 3 |
|
| 4 |
dataset_config:
|
test.sh
CHANGED
|
@@ -1,2 +1,2 @@
|
|
| 1 |
export CUDA_VISIBLE_DEVICES=0
|
| 2 |
-
accelerate launch test.py --config config/
|
|
|
|
| 1 |
export CUDA_VISIBLE_DEVICES=0
|
| 2 |
+
accelerate launch test.py --config config/part_level/adding_new_object/run_two_man/running_spider_polar_sunglass.yaml
|
video_diffusion/common/__pycache__/image_util.cpython-310.pyc
CHANGED
|
Binary files a/video_diffusion/common/__pycache__/image_util.cpython-310.pyc and b/video_diffusion/common/__pycache__/image_util.cpython-310.pyc differ
|
|
|