alibaba-pai
/

Z-Image-Fun-Controlnet-Union-2.1

VideoX Fun

Model card Files Files and versions

xet

Community

bubbliiiing commited on Feb 2

Commit

bfd61e2

verified ·

1 Parent(s): 4b9ff82

Update README.md

Browse files

Files changed (1) hide show

README.md +150 -149

README.md CHANGED Viewed

@@ -1,150 +1,151 @@
----
-license: apache-2.0
----
-# Z-Image-Fun-Controlnet-Union-2.1
-[![Github](https://img.shields.io/badge/🎬%20Code-VideoX_Fun-blue)](https://github.com/aigc-apps/VideoX-Fun)
-## Model Card
-| Name | Description |
-|--|--|
-| Z-Image-Fun-Controlnet-Union-2.1.safetensors | ControlNet weights for Z-Image. The model supports multiple control conditions such as Canny, Depth, Pose, MLSD, Scribble, Hed and Gray. This ControlNet is added on 15 layer blocks and 2 refiner layer blocks. |
-| Z-Image-Fun-Controlnet-Union-2.1-lite.safetensors | Uses the same training scheme as the 2601 version, but compared to the large version of the model, fewer layers have control added, resulting in weaker control conditions. This makes it suitable for larger control_context_scale values, and the generation results appear more natural. It is also suitable for lower-spec machines. |
-## Model Features
-- This ControlNet is added on 15 layer blocks and 2 refiner layer blocks (Lite models are added on 3 layer blocks and 2 refiner blocks). It supports multiple control conditions—including Canny, Depth, Pose, MLSD, Scribble, Hed and Gray can be used like a standard ControlNet.
-- Inpainting mode is also supported.
-- You can adjust control_context_scale for stronger control and better detail preservation. For better stability, we highly recommend using a detailed prompt. The optimal range for control_context_scale is from 0.65 to 0.90.
-## Results
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Pose + Inpaint</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/inpaint.jpg" width="100%" /><img src="asset/mask.jpg" width="100%" /></td>
-    <td><img src="results/inpaint.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Pose + Inpaint</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/inpaint.jpg" width="100%" /><img src="asset/mask.jpg" width="100%" /><img src="asset/pose.jpg" width="100%" /></td>
-    <td><img src="results/pose_inpaint.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Pose</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/pose2.jpg" width="100%" /></td>
-    <td><img src="results/pose2.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Pose</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/pose.jpg" width="100%" /></td>
-    <td><img src="results/pose.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Pose</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/pose3.jpg" width="100%" /></td>
-    <td><img src="results/pose3.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Canny</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/canny.jpg" width="100%" /></td>
-    <td><img src="results/canny.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>HED</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/hed.jpg" width="100%" /></td>
-    <td><img src="results/hed.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Depth</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/depth.jpg" width="100%" /></td>
-    <td><img src="results/depth.png" width="100%" /></td>
-  </tr>
-</table>
-<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
-  <tr>
-    <td>Depth</td>
-    <td>Output</td>
-  </tr>
-  <tr>
-    <td><img src="asset/gray.jpg" width="100%" /></td>
-    <td><img src="results/gray.png" width="100%" /></td>
-  </tr>
-</table>
-## Inference
-Go to the VideoX-Fun repository for more details.
-Please clone the VideoX-Fun repository and create the required directories:
-```sh
-# Clone the code
-git clone https://github.com/aigc-apps/VideoX-Fun.git
-# Enter VideoX-Fun's directory
-cd VideoX-Fun
-# Create model directories
-mkdir -p models/Diffusion_Transformer
-mkdir -p models/Personalized_Model
-```
-Then download the weights into models/Diffusion_Transformer and models/Personalized_Model.
-```
-📦 models/
-├── 📂 Diffusion_Transformer/
-│   └── 📂 Z-Image/
-├── 📂 Personalized_Model/
-│   ├── 📦 Z-Image-Fun-Controlnet-Union-2.1.safetensors
-│   └── 📦 Z-Image-Fun-Controlnet-Union-2.1-lite.safetensors
-```
 Then run the file `examples/z_image_fun/predict_t2i_control_2.1.py` and `examples/z_image_fun/predict_i2i_inpaint_2.1.py`.

+---
+license: apache-2.0
+library_name: videox_fun
+---
+# Z-Image-Fun-Controlnet-Union-2.1
+[![Github](https://img.shields.io/badge/🎬%20Code-VideoX_Fun-blue)](https://github.com/aigc-apps/VideoX-Fun)
+## Model Card
+| Name | Description |
+|--|--|
+| Z-Image-Fun-Controlnet-Union-2.1.safetensors | ControlNet weights for Z-Image. The model supports multiple control conditions such as Canny, Depth, Pose, MLSD, Scribble, Hed and Gray. This ControlNet is added on 15 layer blocks and 2 refiner layer blocks. |
+| Z-Image-Fun-Controlnet-Union-2.1-lite.safetensors | Uses the same training scheme as the 2601 version, but compared to the large version of the model, fewer layers have control added, resulting in weaker control conditions. This makes it suitable for larger control_context_scale values, and the generation results appear more natural. It is also suitable for lower-spec machines. |
+## Model Features
+- This ControlNet is added on 15 layer blocks and 2 refiner layer blocks (Lite models are added on 3 layer blocks and 2 refiner blocks). It supports multiple control conditions—including Canny, Depth, Pose, MLSD, Scribble, Hed and Gray can be used like a standard ControlNet.
+- Inpainting mode is also supported.
+- You can adjust control_context_scale for stronger control and better detail preservation. For better stability, we highly recommend using a detailed prompt. The optimal range for control_context_scale is from 0.65 to 0.90.
+## Results
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Pose + Inpaint</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/inpaint.jpg" width="100%" /><img src="asset/mask.jpg" width="100%" /></td>
+    <td><img src="results/inpaint.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Pose + Inpaint</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/inpaint.jpg" width="100%" /><img src="asset/mask.jpg" width="100%" /><img src="asset/pose.jpg" width="100%" /></td>
+    <td><img src="results/pose_inpaint.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Pose</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/pose2.jpg" width="100%" /></td>
+    <td><img src="results/pose2.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Pose</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/pose.jpg" width="100%" /></td>
+    <td><img src="results/pose.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Pose</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/pose3.jpg" width="100%" /></td>
+    <td><img src="results/pose3.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Canny</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/canny.jpg" width="100%" /></td>
+    <td><img src="results/canny.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>HED</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/hed.jpg" width="100%" /></td>
+    <td><img src="results/hed.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Depth</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/depth.jpg" width="100%" /></td>
+    <td><img src="results/depth.png" width="100%" /></td>
+  </tr>
+</table>
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Depth</td>
+    <td>Output</td>
+  </tr>
+  <tr>
+    <td><img src="asset/gray.jpg" width="100%" /></td>
+    <td><img src="results/gray.png" width="100%" /></td>
+  </tr>
+</table>
+## Inference
+Go to the VideoX-Fun repository for more details.
+Please clone the VideoX-Fun repository and create the required directories:
+```sh
+# Clone the code
+git clone https://github.com/aigc-apps/VideoX-Fun.git
+# Enter VideoX-Fun's directory
+cd VideoX-Fun
+# Create model directories
+mkdir -p models/Diffusion_Transformer
+mkdir -p models/Personalized_Model
+```
+Then download the weights into models/Diffusion_Transformer and models/Personalized_Model.
+```
+📦 models/
+├── 📂 Diffusion_Transformer/
+│   └── 📂 Z-Image/
+├── 📂 Personalized_Model/
+│   ├── 📦 Z-Image-Fun-Controlnet-Union-2.1.safetensors
+│   └── 📦 Z-Image-Fun-Controlnet-Union-2.1-lite.safetensors
+```
 Then run the file `examples/z_image_fun/predict_t2i_control_2.1.py` and `examples/z_image_fun/predict_i2i_inpaint_2.1.py`.