Update Weights

README.md

@@ -3,28 +3,54 @@ license: apache-2.0
 library_name: videox_fun
 ---
 
-# Z-Image-Turbo-Fun-Controlnet-Union-2.0
+# Z-Image-Turbo-Fun-Controlnet-Union-2.1
 
 [![Github](https://img.shields.io/badge/🎬%20Code-VideoX_Fun-blue)](https://github.com/aigc-apps/VideoX-Fun)
 
-
 ## Update
-- Due to a typo in the code, `control_layers` was used instead of `control_noise_refiner` to process refiner latents during training. Although the model converged normally, the model inference speed was slow because `control_layers` forward pass was performed twice. In version 2.1, we made an urgent fix and the speed has returned to normal. [2025.12.17]
+- During testing, we found that applying ControlNet to Z-Image-Turbo caused the model to lose its acceleration capability and become blurry. We performed 8-step distillation on the version 2.1 model, and the distilled model demonstrates better performance when using 8-step prediction. Additionally, we have uploaded a tile model that can be used for super-resolution generation. [2025.12.22]
+- Due to a typo in version 2.0, `control_layers` was used instead of `control_noise_refiner` to process refiner latents during training. Although the model converged normally, the model inference speed was slow because `control_layers` forward pass was performed twice. In version 2.1, we made an urgent fix and the speed has returned to normal. [2025.12.17]
+
+## Model Card
+| Name | Description |
+|--|--|
+| Z-Image-Turbo-Fun-Controlnet-Union-2.1-8steps.safetensors | Based on version 2.1, the model was distilled using an 8-step distillation algorithm. 8-step prediction is recommended. Compared to version 2.1, when using 8-step prediction, the images are clearer and the composition is more reasonable. |
+| Z-Image-Turbo-Fun-Controlnet-Tile-2.1-8steps.safetensors | A Tile model trained on high-definition datasets that can be used for super-resolution, with a maximum training resolution of 2048x2048. The model was distilled using an 8-step distillation algorithm, and 8-step prediction is recommended. |
+| Z-Image-Turbo-Fun-Controlnet-Union-2.1.safetensors | A retrained model after fixing the typo in version 2.0, with faster single-step speed. Similar to version 2.0, the model lost some of its acceleration capability after training, thus requiring more steps. |
+| Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors | ControlNet weights for Z-Image-Turbo. Compared to version 1.0, it adds modifications to more layers and was trained for a longer time. However, due to a typo in the code, the layer blocks were forwarded twice, resulting in slower speed. The model supports multiple control conditions such as Canny, Depth, Pose, MLSD, etc. Additionally, the model lost some of its acceleration capability after training, thus requiring more steps. |
 
 ## Model Features
-- This ControlNet is added on 15 layer blocks and 2 refiner layer blocks.
-- The model was trained from scratch for 70,000 steps on a dataset of 1 million high-quality images covering both general and human-centric content. Training was performed at 1328 resolution using BFloat16 precision, with a batch size of 64, a learning rate of 2e-5, and a text dropout ratio of 0.10.
-- It supports multiple control conditions—including Canny, HED, Depth, Pose and MLSD can be used like a standard ControlNet.
-- We found that under different strength levels, using different step numbers has a certain impact on the realism and clarity of the results. For strength and step testing, please refer to [Scale Test Results](#scale-test-results). 
-- You can adjust control_context_scale for stronger control and better detail preservation. For better stability, we highly recommend using a detailed prompt. The optimal range for control_context_scale is from 0.65 to 0.90. 
-  - **Note on Steps: As you increase the control strength (higher control_context_scale values), it's recommended to appropriately increase the number of inference steps to achieve better results and maintain generation quality. This is likely because the control model has not been distilled.**
+- This ControlNet is added on 15 layer blocks and 2 refiner layer blocks. It supports multiple control conditions—including Canny, HED, Depth, Pose and MLSD can be used like a standard ControlNet. 
 - Inpainting mode is also supported.
+- Training Process:
+  - 2.0: The model was trained from scratch for 70,000 steps on a dataset of 1 million high-quality images covering both general and human-centric content. Training was performed at 1328 resolution using BFloat16 precision, with a batch size of 64, a learning rate of 2e-5, and a text dropout ratio of 0.10.
+  - 2.1: Version 2.1 is based on the version 2.0 weights and continued training for an additional 11,000 steps after the typo fix, using the same parameters and dataset.
+  - 2.1-8-steps: Version 2.1-8-steps was obtained by training for 5,500 steps using an 8-step distillation algorithm based on version 2.1. 
+- Note on Steps: 
+  - 2.0 and 2.1: As you increase the control strength (higher control_context_scale values), it's recommended to appropriately increase the number of inference steps to achieve better results and maintain generation quality. This is likely because the control model has not been distilled. 
+  - 2.1-8-steps: Just use 8 steps in inference.
+- You can adjust control_context_scale for stronger control and better detail preservation. For better stability, we highly recommend using a detailed prompt. The optimal range for control_context_scale is from 0.65 to 0.90. 
+- During testing, in versions 2.0 and 2.1, we found that applying ControlNet to Z-Image-Turbo caused the model to lose its acceleration capability and produce blurry images. For detailed information on strength and step count testing, please refer to Scale Test Results. These results were generated using version 2.0. For strength and step testing, please refer to [Scale Test Results](#scale-test-results). This was obtained by generating with version 2.0. 
 
 ## TODO 
 - [ ] Train on better data.
 
 ## Results
+### Difference between 2.1 and 2.1-8steps.
 
+8 steps results:
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Z-Image-Turbo-Fun-Controlnet-Union-2.1-8steps</td>
+    <td>Z-Image-Turbo-Fun-Controlnet-Union-2.1</td>
+  </tr>
+  <tr>
+    <td><img src="results/8steps.png" width="100%" /></td>
+    <td><img src="results/nsteps.png" width="100%" /></td>
+  </tr>
+</table>
+
+### Generation Results
 <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
   <tr>
     <td>Pose + Inpaint</td>
@@ -102,6 +128,17 @@ library_name: videox_fun
   </tr>
 </table>
 
+<table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
+  <tr>
+    <td>Low Resolution</td>
+    <td>High Resolution</td>
+  </tr>
+  <tr>
+    <td><img src="asset/low_res.jpg" width="100%" /></td>
+    <td><img src="results/high_res.png" width="100%" /></td>
+  </tr>
+</table>
+
 ## Inference
 Go to the VideoX-Fun repository for more details.
 
@@ -126,10 +163,15 @@ Then download the weights into models/Diffusion_Transformer and models/Personali
 ├── 📂 Diffusion_Transformer/
 │   └── 📂 Z-Image-Turbo/
 ├── 📂 Personalized_Model/
-│   └── 📦 Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors
+│   ├── 📦 Z-Image-Turbo-Fun-Controlnet-Union-2.1.safetensors
+│   ├── 📦 Z-Image-Turbo-Fun-Controlnet-Union-2.1-8steps.safetensors
+│   └── 📦 Z-Image-Turbo-Fun-Controlnet-Tile-2.1-8steps.safetensors
 ```
 
-Then run the file `examples/z_image_fun/predict_t2i_control_2.0.py` and `examples/z_image_fun/predict_i2i_inpaint_2.0.py`.
+Then run the file `examples/z_image_fun/predict_t2i_control_2.1.py` and `examples/z_image_fun/predict_i2i_inpaint_2.1.py`.
+
+<details>
+  <summary>(Obsolete) Scale Test Results:</summary>
 
 ## Scale Test Results
 
@@ -147,3 +189,4 @@ Parameter Description:
 
 Diffusion Steps: Number of iteration steps for the diffusion model (9, 10, 20, 30, 40)
 Control Scale: Control strength coefficient (0.65 - 1.0)
+</details>

Z-Image-Turbo-Fun-Controlnet-Tile-2.1-8steps.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d34935adc2a71b2d74ab51ea6d48a5e494a5a2236498486ca005ace7da4c8054
+size 6712485600

Z-Image-Turbo-Fun-Controlnet-Union-2.1-8steps.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41a0947636dc69fff97926df50eb87b406fa8835348a37df65354e9f3a0fbe18
+size 6712485600