Update README.md
Browse files
README.md
CHANGED
|
@@ -21,15 +21,21 @@ Terminal Technology Department, Alipay, Ant Group.
|
|
| 21 |
<a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a>
|
| 22 |
<a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a>
|
| 23 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## 🚀 EchoMimic Series
|
| 26 |
* EchoMimicV1: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning. [GitHub](https://github.com/antgroup/echomimic)
|
| 27 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
| 28 |
|
| 29 |
## 📣 Updates
|
|
|
|
|
|
|
| 30 |
* [2024.11.22] ๐ฅ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
| 31 |
-
* [2024.11.
|
| 32 |
-
* [2024.11.
|
| 33 |
* [2024.11.15] ๐ฅ Our [paper](https://arxiv.org/abs/2411.10061) is in public on arxiv.
|
| 34 |
|
| 35 |
## 🌅 Gallery
|
|
@@ -149,7 +155,11 @@ Create conda environment (Recommended):
|
|
| 149 |
|
| 150 |
Install packages with `pip`
|
| 151 |
```bash
|
|
|
|
|
|
|
|
|
|
| 152 |
pip install -r requirements.txt
|
|
|
|
| 153 |
```
|
| 154 |
|
| 155 |
### Download ffmpeg-static
|
|
@@ -178,7 +188,7 @@ The **pretrained_weights** is organized as follows.
|
|
| 178 |
โโโ sd-image-variations-diffusers
|
| 179 |
โ โโโ ...
|
| 180 |
โโโ audio_processor
|
| 181 |
-
โโโ
|
| 182 |
```
|
| 183 |
|
| 184 |
In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth** / **pose_encoder.pth** are the main checkpoints of **EchoMimic**. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:
|
|
@@ -187,6 +197,10 @@ In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth**
|
|
| 187 |
- [audio_processor(whisper)](https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt)
|
| 188 |
|
| 189 |
### Inference on Demo
|
|
|
|
|
|
|
|
|
|
|
|
|
| 190 |
Run the python inference script:
|
| 191 |
```bash
|
| 192 |
python infer.py --config='./configs/prompts/infer.yaml'
|
|
@@ -245,4 +259,4 @@ If you find our work useful for your research, please consider citing the paper
|
|
| 245 |
```
|
| 246 |
|
| 247 |
## 🌟 Star History
|
| 248 |
-
[](https://star-history.com/#antgroup/echomimic_v2&Date)
|
|
|
|
| 21 |
<a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a>
|
| 22 |
<a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a>
|
| 23 |
</div>
|
| 24 |
+
<div align='center'>
|
| 25 |
+
<a href='https://github.com/antgroup/echomimic_v2/discussions/53'><img src='https://img.shields.io/badge/English-Common Problems-orange'></a>
|
| 26 |
+
<a href='https://github.com/antgroup/echomimic_v2/discussions/40'><img src='https://img.shields.io/badge/ไธญๆ็-ๅธธ่ง้ฎ้ขๆฑๆป-orange'></a>
|
| 27 |
+
</div>
|
| 28 |
|
| 29 |
## 🚀 EchoMimic Series
|
| 30 |
* EchoMimicV1: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning. [GitHub](https://github.com/antgroup/echomimic)
|
| 31 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
| 32 |
|
| 33 |
## 📣 Updates
|
| 34 |
+
* [2024.11.27] ๐ฅ Thanks [AiMotionStudio](https://www.youtube.com/@AiMotionStudio) for the [installation tutorial](https://www.youtube.com/watch?v=2ab6U1-nVTQ).
|
| 35 |
+
* [2024.11.22] ๐ฅ [GradioUI](https://github.com/antgroup/echomimic_v2/blob/main/app.py) is now available. Thanks @gluttony-10 for the contribution.
|
| 36 |
* [2024.11.22] ๐ฅ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
| 37 |
+
* [2024.11.21] ๐ฅ We release the EMTD dataset list and processing scripts.
|
| 38 |
+
* [2024.11.21] ๐ฅ We release our [EchoMimicV2](https://github.com/antgroup/echomimic_v2) codes and models.
|
| 39 |
* [2024.11.15] ๐ฅ Our [paper](https://arxiv.org/abs/2411.10061) is in public on arxiv.
|
| 40 |
|
| 41 |
## 🌅 Gallery
|
|
|
|
| 155 |
|
| 156 |
Install packages with `pip`
|
| 157 |
```bash
|
| 158 |
+
pip install pip -U
|
| 159 |
+
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 xformers==0.0.28.post3 --index-url https://download.pytorch.org/whl/cu124
|
| 160 |
+
pip install torchao --index-url https://download.pytorch.org/whl/nightly/cu124
|
| 161 |
pip install -r requirements.txt
|
| 162 |
+
pip install --no-deps facenet_pytorch==2.6.0
|
| 163 |
```
|
| 164 |
|
| 165 |
### Download ffmpeg-static
|
|
|
|
| 188 |
โโโ sd-image-variations-diffusers
|
| 189 |
โ โโโ ...
|
| 190 |
โโโ audio_processor
|
| 191 |
+
โโโ tiny.pt
|
| 192 |
```
|
| 193 |
|
| 194 |
In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth** / **pose_encoder.pth** are the main checkpoints of **EchoMimic**. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:
|
|
|
|
| 197 |
- [audio_processor(whisper)](https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt)
|
| 198 |
|
| 199 |
### Inference on Demo
|
| 200 |
+
Run the gradio:
|
| 201 |
+
```bash
|
| 202 |
+
python app.py
|
| 203 |
+
```
|
| 204 |
Run the python inference script:
|
| 205 |
```bash
|
| 206 |
python infer.py --config='./configs/prompts/infer.yaml'
|
|
|
|
| 259 |
```
|
| 260 |
|
| 261 |
## 🌟 Star History
|
| 262 |
+
[](https://star-history.com/#antgroup/echomimic_v2&Date)
|