Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,7 @@ language:
|
|
| 6 |
- en
|
| 7 |
---
|
| 8 |
|
|
|
|
| 9 |
## Using HunyuanDiT ControlNet
|
| 10 |
|
| 11 |
|
|
@@ -20,7 +21,9 @@ language:
|
|
| 20 |
```bash
|
| 21 |
cd HunyuanDiT
|
| 22 |
# Use the huggingface-cli tool to download the model.
|
|
|
|
| 23 |
huggingface-cli download Tencent-Hunyuan/HYDiT-ControlNet --local-dir ./ckpts/t2i/controlnet
|
|
|
|
| 24 |
|
| 25 |
# Quick start
|
| 26 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type canny --prompt "在夜晚的酒店门前,一座古老的中国风格的狮子雕像矗立着,它的眼睛闪烁着光芒,仿佛在守护着这座建筑。背景是夜晚的酒店前,构图方式是特写,平视,居中构图。这张照片呈现了真实摄影风格,蕴含了中国雕塑文化,同时展现了神秘氛围" --condition_image_path controlnet/asset/input/canny.jpg --control_weight 1.0
|
|
@@ -67,10 +70,17 @@ Examples of condition input and ControlNet results are as follows:
|
|
| 67 |
|
| 68 |
### Training
|
| 69 |
|
| 70 |
-
We utilize [**DWPose**](https://github.com/IDEA-Research/DWPose) for pose extraction. Please follow their guidelines to download the checkpoints and save them to `hydit/annotator/ckpts` directory.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
```bash
|
| 72 |
-
pip install matplotlib
|
| 73 |
-
pip install onnxruntime_gpu
|
|
|
|
| 74 |
```
|
| 75 |
|
| 76 |
|
|
@@ -162,3 +172,4 @@ c. Using depth ControlNet during inference
|
|
| 162 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type pose --prompt "一位亚洲女性,身穿绿色上衣,戴着紫色头巾和紫色围巾,站在黑板前。背景是黑板。照片采用近景、平视和居中构图的方式呈现真实摄影风格" --condition_image_path controlnet/asset/input/pose.jpg --control_weight 1.0
|
| 163 |
```
|
| 164 |
|
|
|
|
|
|
| 6 |
- en
|
| 7 |
---
|
| 8 |
|
| 9 |
+
|
| 10 |
## Using HunyuanDiT ControlNet
|
| 11 |
|
| 12 |
|
|
|
|
| 21 |
```bash
|
| 22 |
cd HunyuanDiT
|
| 23 |
# Use the huggingface-cli tool to download the model.
|
| 24 |
+
# We recommend using distilled weights as the base model for ControlNet inference, as our provided pretrained weights are trained on them.
|
| 25 |
huggingface-cli download Tencent-Hunyuan/HYDiT-ControlNet --local-dir ./ckpts/t2i/controlnet
|
| 26 |
+
huggingface-cli download Tencent-Hunyuan/Distillation-v1.1 ./pytorch_model_distill.pt --local-dir ./ckpts/t2i/model
|
| 27 |
|
| 28 |
# Quick start
|
| 29 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type canny --prompt "在夜晚的酒店门前,一座古老的中国风格的狮子雕像矗立着,它的眼睛闪烁着光芒,仿佛在守护着这座建筑。背景是夜晚的酒店前,构图方式是特写,平视,居中构图。这张照片呈现了真实摄影风格,蕴含了中国雕塑文化,同时展现了神秘氛围" --condition_image_path controlnet/asset/input/canny.jpg --control_weight 1.0
|
|
|
|
| 70 |
|
| 71 |
### Training
|
| 72 |
|
| 73 |
+
We utilize [**DWPose**](https://github.com/IDEA-Research/DWPose) for pose extraction. Please follow their guidelines to download the checkpoints and save them to `hydit/annotator/ckpts` directory. We provide serveral commands to quick install:
|
| 74 |
+
```bash
|
| 75 |
+
mkdir ./hydit/annotator/ckpts
|
| 76 |
+
wget -O ./hydit/annotator/ckpts/dwpose.zip https://dit.hunyuan.tencent.com/download/HunyuanDiT/dwpose.zip
|
| 77 |
+
unzip ./hydit/annotator/ckpts/dwpose.zip -d ./hydit/annotator/ckpts/
|
| 78 |
+
```
|
| 79 |
+
Additionally, ensure that you install the related dependencies.
|
| 80 |
```bash
|
| 81 |
+
pip install matplotlib==3.7.5
|
| 82 |
+
pip install onnxruntime_gpu==1.16.3
|
| 83 |
+
pip install opencv-python==4.8.1.78
|
| 84 |
```
|
| 85 |
|
| 86 |
|
|
|
|
| 172 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type pose --prompt "一位亚洲女性,身穿绿色上衣,戴着紫色头巾和紫色围巾,站在黑板前。背景是黑板。照片采用近景、平视和居中构图的方式呈现真实摄影风格" --condition_image_path controlnet/asset/input/pose.jpg --control_weight 1.0
|
| 173 |
```
|
| 174 |
|
| 175 |
+
|