ussoewwin commited on
Commit
c699991
Β·
verified Β·
1 Parent(s): d15adef

docs: remove fabricated BibTeX citation and clean credits

Browse files
Files changed (1) hide show
  1. README.md +18 -54
README.md CHANGED
@@ -7,71 +7,51 @@ tags:
7
  - inpainting
8
  - qwen-image
9
  - qwen-image-2512
10
- - videox-fun
11
  - quantized
12
  - int8
13
  - convrot
14
  - comfyui
15
  pipeline_tag: image-to-image
16
- library_name: videox_fun
17
  ---
18
 
19
  # ControlNet Models (ConvRot INT8)
20
 
21
- High-fidelity **ConvRot INT8** quantized weights for diffusion ControlNet models. This repository provides memory-efficient INT8 quantized checkpoints maintaining high structural fidelity and multi-condition guidance.
22
 
23
  ---
24
 
25
  ## 🌟 Model Overview
26
 
27
- This repository hosts the **ConvRot INT8** quantized edition of **`Qwen-Image-2512-Fun-Controlnet-Union-2602`**, originally developed and trained by **Alibaba PAI / VideoX-Fun**.
28
-
29
- The **ConvRot INT8** format applies rotational transformation matrix optimizations to suppress outlier activations and minimize quantization error, reducing the model footprint to **~1.64 GB** (from ~3.3+ GB FP16) while preserving multi-condition structural precision.
30
 
31
  - **Base Model:** [alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union](https://huggingface.co/alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union)
32
- - **Architecture:** Multi-condition Union ControlNet added across 5 layer blocks of Qwen-Image-2512.
33
- - **Quantization:** Full ConvRot INT8 (weights stored with rotational transformations).
34
  - **License:** Apache-2.0
35
 
36
  ---
37
 
38
  ## πŸ“¦ Available Models
39
 
40
- | Filename | Base Architecture | Supported Conditions | Precision | File Size | License |
41
  | :--- | :--- | :--- | :--- | :--- | :--- |
42
- | `Qwen-Image-2512-Fun-Controlnet-Union-2602_convrot_int8.safetensors` | Qwen-Image-2512 (5 Layer Blocks) | Canny, HED, Depth, Pose, MLSD, Scribble, Gray, Inpaint | ConvRot INT8 | ~1.64 GB | Apache-2.0 |
43
-
44
- ---
45
-
46
- ## πŸ› οΈ Supported Control Modalities & Features
47
-
48
- 1. **Union Condition Architecture**:
49
- Supports unified multi-condition control within a single checkpoint:
50
- - **Pose** (OpenPose / DWPose keypoints)
51
- - **Canny** (Edge detection)
52
- - **HED** (Soft edge detection)
53
- - **Depth** (Z-depth estimation)
54
- - **MLSD** (Straight-line wireframe extraction)
55
- - **Scribble** (Interactive doodle / sketch guidance)
56
- - **Gray** (Grayscale / luminance guidance)
57
- 2. **Inpainting Mode Support**:
58
- Fully compatible with masked inpainting pipelines (`Pose + Inpaint`, `Depth + Inpaint`, etc.).
59
- 3. **Multi-Resolution Conditioning**:
60
- Extracting control maps in multi-resolution formats improves generalization across arbitrary target aspect ratios.
61
 
62
  ---
63
 
64
- ## βš™οΈ Recommended Inference Settings
65
 
66
- - **`control_context_scale`**: Optimal range is **`0.70` – `0.95`**. Higher values yield stronger adherence to the control input, while lower values offer increased prompt stylization freedom.
67
- - **Prompt Formulation**: Detailed natural language descriptions significantly enhance semantic stability and detail preservation.
68
- - **Resolution**: Multi-scale aspect ratios matching the base Qwen-Image model resolution specifications.
 
 
69
 
70
  ---
71
 
72
  ## πŸš€ Usage in ComfyUI
73
 
74
- Standard / vanilla ComfyUI does not natively support the **ConvRot INT8** format for ControlNet models. To load and execute these weights, the dedicated loader extension is required:
75
 
76
  - **Dedicated Loader Extension:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
77
 
@@ -84,28 +64,12 @@ cd ComfyUI/custom_nodes
84
  git clone https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools.git
85
  ```
86
 
87
- Place `Qwen-Image-2512-Fun-Controlnet-Union-2602_convrot_int8.safetensors` into your ComfyUI `models/controlnet/` folder and load it using the dedicated loader nodes from the extension.
88
 
89
  ---
90
 
91
- ## πŸ“œ Credits & Citation
92
-
93
- ### Upstream Creators
94
- Deep gratitude and acknowledgement to the **Alibaba PAI VideoX-Fun Team** for developing, training, and open-sourcing the Qwen-Image ControlNet series.
95
- - **Original Model:** [alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union](https://huggingface.co/alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union)
96
- - **Official Repository:** [aigc-apps/VideoX-Fun](https://github.com/aigc-apps/VideoX-Fun)
97
-
98
- ```bibtex
99
- @misc{videox_fun,
100
- author = {VideoX-Fun Team},
101
- title = {VideoX-Fun: A Flexible Video and Image Generation Toolset},
102
- year = {2025},
103
- publisher = {GitHub},
104
- journal = {GitHub repository},
105
- howpublished = {\url{https://github.com/aigc-apps/VideoX-Fun}}
106
- }
107
- ```
108
-
109
- ---
110
 
111
- **Disclaimer:** These model weights are released strictly for research, non-commercial, and optimization workflows under the Apache-2.0 License.
 
 
 
7
  - inpainting
8
  - qwen-image
9
  - qwen-image-2512
 
10
  - quantized
11
  - int8
12
  - convrot
13
  - comfyui
14
  pipeline_tag: image-to-image
 
15
  ---
16
 
17
  # ControlNet Models (ConvRot INT8)
18
 
19
+ High-fidelity **ConvRot INT8** quantized weights for ControlNet models.
20
 
21
  ---
22
 
23
  ## 🌟 Model Overview
24
 
25
+ This repository hosts the **ConvRot INT8** quantized weights of **`Qwen-Image-2512-Fun-Controlnet-Union-2602`**, based on [alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union](https://huggingface.co/alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union).
 
 
26
 
27
  - **Base Model:** [alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union](https://huggingface.co/alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union)
28
+ - **Supported Conditions:** Canny, HED, Depth, Pose, MLSD, Scribble, Gray, Inpainting
29
+ - **Quantization:** ConvRot INT8
30
  - **License:** Apache-2.0
31
 
32
  ---
33
 
34
  ## πŸ“¦ Available Models
35
 
36
+ | Filename | Base Model | Supported Conditions | Precision | File Size | License |
37
  | :--- | :--- | :--- | :--- | :--- | :--- |
38
+ | `Qwen-Image-2512-Fun-Controlnet-Union-2602_convrot_int8.safetensors` | Qwen-Image-2512-Fun-Controlnet-Union-2602 | Canny, HED, Depth, Pose, MLSD, Scribble, Gray, Inpaint | ConvRot INT8 | ~1.64 GB | Apache-2.0 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
40
  ---
41
 
42
+ ## πŸ› οΈ Model Features
43
 
44
+ - ControlNet added on 5 layer blocks of Qwen-Image-2512.
45
+ - Supports multiple control conditions: Canny, HED, Depth, Pose, MLSD, Scribble, Gray.
46
+ - Inpainting mode is supported.
47
+ - Acquiring control images in a multi-resolution manner results in better generalization.
48
+ - Optimal range for `control_context_scale` is from 0.70 to 0.95.
49
 
50
  ---
51
 
52
  ## πŸš€ Usage in ComfyUI
53
 
54
+ ComfyUI does not natively support the **ConvRot INT8** format for ControlNet models. To load and execute these weights in ComfyUI, the dedicated loader extension is required:
55
 
56
  - **Dedicated Loader Extension:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
57
 
 
64
  git clone https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools.git
65
  ```
66
 
67
+ Place `Qwen-Image-2512-Fun-Controlnet-Union-2602_convrot_int8.safetensors` into your ComfyUI `models/controlnet/` directory and load it using the dedicated loader node.
68
 
69
  ---
70
 
71
+ ## πŸ“œ Credits & License
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
+ - **Original Base Model:** [alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union](https://huggingface.co/alibaba-pai/Qwen-Image-2512-Fun-Controlnet-Union)
74
+ - **Upstream Repository:** [aigc-apps/VideoX-Fun](https://github.com/aigc-apps/VideoX-Fun)
75
+ - **License:** Apache-2.0