RunningHubAI commited on
Commit
1a3f617
·
verified ·
1 Parent(s): cb18c6d

Upload 2 files

Browse files
Files changed (2) hide show
  1. README.md +189 -0
  2. README_CN.md +168 -0
README.md CHANGED
@@ -1,3 +1,192 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: comfyui
4
+ base_model: GAIR/daVinci-MagiHuman
5
+ pipeline_tag: image-to-video
6
+ language:
7
+ - en
8
+ - zh
9
+ - ja
10
+ - ko
11
+ - de
12
+ - fr
13
+ - yue
14
+ tags:
15
+ - comfyui
16
+ - image-to-video
17
+ - image-text-to-audio-video
18
+ - multimodal
19
+ - int8
20
  ---
21
+
22
+ # ComfyUI-RH-daVinci-MagiHuman INT8
23
+
24
+ [![RunningHub China](https://img.shields.io/badge/RunningHub-China-2F80ED)](https://www.runninghub.cn/?inviteCode=rh-v1367)
25
+ [![RunningHub International](https://img.shields.io/badge/RunningHub-International-7B61FF)](https://www.runninghub.ai/?inviteCode=rh-v1367)
26
+ [![English](https://img.shields.io/badge/Language-English-2563EB)](./README.md)
27
+ [![简体中文](https://img.shields.io/badge/Language-%E7%AE%80%E4%BD%93%E4%B8%AD%E6%96%87-EF4444)](./README_CN.md)
28
+
29
+ INT8-quantised daVinci-MagiHuman DiT and super-resolution weights for
30
+ [ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman).
31
+ This repository contains only the four converted `.pt` files. TurboVAE, the
32
+ T5 text encoder, and the video/audio VAEs still come from the official
33
+ releases.
34
+
35
+ The ComfyUI nodes load these INT8 files directly. Official BF16 DiT shards and
36
+ official `540p_sr/` / `1080p_sr/` config directories are not required.
37
+
38
+ Full snapshot is about **57 GiB**. Download only the files you need.
39
+
40
+ Mirror: [Hugging Face](https://huggingface.co/Gluttony10/ComfyUI-RH-daVinci-MagiHuman) /
41
+ [ModelScope](https://www.modelscope.cn/models/Gluttony10/ComfyUI-RH-daVinci-MagiHuman)
42
+
43
+ ## Run Online and API Access
44
+
45
+ If you do not want to download and configure the full model assets locally, you can first try related MagiHuman workflows online on RunningHub, or integrate RunningHub AI app / workflow APIs into your own product.
46
+
47
+ - [RunningHub](https://www.runninghub.ai)
48
+ - [RunningHub China](https://www.runninghub.cn)
49
+ - [RunningHub API](https://www.runninghub.ai/call-api)
50
+ - [RunningHub API Documentation](https://www.runninghub.cn/runninghub-api-doc-en/)
51
+ - [RunningHub API Docs CN](https://www.runninghub.cn/runninghub-api-doc-cn/)
52
+ - [ComfyUI-RH-daVinci-MagiHuman Plugin](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
53
+
54
+ RunningHub is useful for quickly validating workflows such as reference-image-to-talking-head video, digital human video, and AI presenter / spokesperson video generation. You can first run and tune the workflow online, then choose local deployment if needed, or integrate similar capabilities into your product through RunningHub API.
55
+
56
+ > Want to try it quickly? Open [RunningHub](https://www.runninghub.ai) to explore runnable AI workflows. Want to integrate digital human / talking-head video generation into your product? Start with the [RunningHub API Documentation](https://www.runninghub.cn/runninghub-api-doc-en/).
57
+
58
+ RunningHub API supports model APIs, AI app APIs, workflow APIs, and LLM APIs. For MagiHuman-style video generation workflows, the recommended task flow is:
59
+
60
+ ```text
61
+ Submit task -> get taskId -> check task status -> retrieve generated result
62
+ ```
63
+
64
+ Recommended use cases:
65
+
66
+ - Run daVinci-MagiHuman / MagiHuman talking-head video generation workflows online
67
+ - Add reference-image-to-video, digital human presenter, and AI video generation capabilities to your product
68
+ - Provide creators, developers, and teams with no-local-setup generative AI workflows
69
+ - Build automated video generation, content production, or multimodal applications with RunningHub API
70
+
71
+ ## Contents
72
+
73
+ | File | Size | Role |
74
+ |---|---:|---|
75
+ | `base_int8.pt` | 14.25 GiB | INT8 Base DiT (32 steps, higher quality) |
76
+ | `distill_int8.pt` | 14.25 GiB | INT8 Distill DiT (8 steps, faster) |
77
+ | `sr_540p_sr_int8.pt` | 14.25 GiB | Optional INT8 540p super-resolution |
78
+ | `sr_1080p_sr_int8.pt` | 14.25 GiB | Optional INT8 1080p super-resolution |
79
+
80
+ You need **one** of `base_int8.pt` / `distill_int8.pt`. Add an SR file only
81
+ when the loader's `sr_model` is `540p_sr` or `1080p_sr`.
82
+
83
+ ## Install into ComfyUI
84
+
85
+ ```text
86
+ ComfyUI/models/MagiHuman/
87
+ ├── base_int8.pt
88
+ ├── distill_int8.pt
89
+ ├── sr_540p_sr_int8.pt # optional
90
+ └── sr_1080p_sr_int8.pt # optional
91
+ ```
92
+
93
+ ### Hugging Face
94
+
95
+ ```bash
96
+ cd /path/to/ComfyUI
97
+ python3 -m pip install -U huggingface_hub
98
+
99
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt \
100
+ --local-dir ./models/MagiHuman
101
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt \
102
+ --local-dir ./models/MagiHuman
103
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt \
104
+ --local-dir ./models/MagiHuman
105
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt \
106
+ --local-dir ./models/MagiHuman
107
+ ```
108
+
109
+ ### ModelScope
110
+
111
+ ```bash
112
+ pip install modelscope
113
+ cd /path/to/ComfyUI
114
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt \
115
+ --local_dir ./models/MagiHuman
116
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt \
117
+ --local_dir ./models/MagiHuman
118
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt \
119
+ --local_dir ./models/MagiHuman
120
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt \
121
+ --local_dir ./models/MagiHuman
122
+ ```
123
+
124
+ ## Other required assets
125
+
126
+ These files are **not** in this repository:
127
+
128
+ ```bash
129
+ # Official MagiHuman assets (TurboVAE, T5 text encoder)
130
+ hf download GAIR/daVinci-MagiHuman --local-dir ./models/MagiHuman
131
+ pip install modelscope
132
+ modelscope download --model GAIR/daVinci-MagiHuman --local_dir ./models/MagiHuman
133
+
134
+ # External VAEs
135
+ hf download stabilityai/stable-audio-open-1.0 \
136
+ --local-dir ./models/audio_checkpoints/stable-audio-open-1.0
137
+ hf download Wan-AI/Wan2.2-TI2V-5B \
138
+ --local-dir ./models/Ovi/Wan2.2-TI2V-5B
139
+ ```
140
+
141
+ Expected layout after everything is in place:
142
+
143
+ ```text
144
+ ComfyUI/models/
145
+ ├── MagiHuman/
146
+ │ ├── base_int8.pt
147
+ │ ├── distill_int8.pt
148
+ │ ├── sr_540p_sr_int8.pt
149
+ │ ├── sr_1080p_sr_int8.pt
150
+ │ ├── t5gemma-9b-9b-ul2/
151
+ │ └── turbo_vae/
152
+ ├── audio_checkpoints/stable-audio-open-1.0/
153
+ └── Ovi/Wan2.2-TI2V-5B/
154
+ ```
155
+
156
+ ## Plugin
157
+
158
+ Current plugin: [RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
159
+
160
+ | Node | Purpose |
161
+ |---|---|
162
+ | `RH MagiHuman Model Loader` (`RH_MagiHumanModelLoader`) | Load INT8 DiT, VAEs, and optional SR |
163
+ | `RH MagiHuman Generate` (`RH_MagiHumanGenerate`) | Talking-head video from a reference image and text prompt |
164
+
165
+ Loader options:
166
+
167
+ - `model_type`: `base` (32 steps) or `distill` (8 steps)
168
+ - `vram_mode`: `mid_vram` (~16 GB) or `low_vram` (~6.5 GB)
169
+ - `sr_model`: `none` / `540p_sr` / `1080p_sr`
170
+
171
+ `mid_vram` + 540p SR needs about 24 GB. `mid_vram` + 1080p SR needs about 48 GB.
172
+
173
+ ## License
174
+
175
+ Converted weights follow the upstream
176
+ [Apache-2.0](https://opensource.org/licenses/Apache-2.0) license of
177
+ [daVinci-MagiHuman](https://huggingface.co/GAIR/daVinci-MagiHuman).
178
+
179
+ ## Links
180
+
181
+ [![RunningHub China](https://img.shields.io/badge/RunningHub-China-2F80ED)](https://www.runninghub.cn/?inviteCode=rh-v1367)
182
+ [![RunningHub International](https://img.shields.io/badge/RunningHub-International-7B61FF)](https://www.runninghub.ai/?inviteCode=rh-v1367)
183
+
184
+ - [RunningHub](https://www.runninghub.ai)
185
+ - [RunningHub China](https://www.runninghub.cn)
186
+ - [RunningHub API](https://www.runninghub.ai/call-api)
187
+ - [RunningHub API Documentation](https://www.runninghub.cn/runninghub-api-doc-en/)
188
+ - [TokenPlus](https://tokenplus.net/)
189
+ - [Official daVinci-MagiHuman](https://github.com/GAIR-NLP/daVinci-MagiHuman)
190
+ - [Official weights](https://huggingface.co/GAIR/daVinci-MagiHuman)
191
+ - [Official assets on ModelScope](https://www.modelscope.cn/models/GAIR/daVinci-MagiHuman)
192
+ - [ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
README_CN.md ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ComfyUI-RH-daVinci-MagiHuman INT8
2
+
3
+ [![RunningHub China](https://img.shields.io/badge/RunningHub-China-2F80ED)](https://www.runninghub.cn/?inviteCode=rh-v1367)
4
+ [![RunningHub International](https://img.shields.io/badge/RunningHub-International-7B61FF)](https://www.runninghub.ai/?inviteCode=rh-v1367)
5
+ [![English](https://img.shields.io/badge/Language-English-2563EB)](./README.md)
6
+ [![简体中文](https://img.shields.io/badge/Language-%E7%AE%80%E4%BD%93%E4%B8%AD%E6%96%87-EF4444)](./README_CN.md)
7
+
8
+ 面向 [ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
9
+ 的 daVinci-MagiHuman INT8 量化 DiT 与超分权重。本仓库只包含 4 个转换后的
10
+ `.pt` 文件。TurboVAE、T5 文本编码器以及视频/音频 VAE 仍需从官方仓库另行下载。
11
+
12
+ ComfyUI 节点直接加载这些 INT8 文件,不需要官方 BF16 DiT shard,也不需要官方
13
+ `540p_sr/`、`1080p_sr/` 配置目录。
14
+
15
+ 完整快照约 **57 GiB**。按需下载即可。
16
+
17
+ 镜像:[Hugging Face](https://huggingface.co/Gluttony10/ComfyUI-RH-daVinci-MagiHuman) /
18
+ [ModelScope](https://www.modelscope.cn/models/Gluttony10/ComfyUI-RH-daVinci-MagiHuman)
19
+
20
+ ## 在线运行与 API 接入
21
+
22
+ 如果你不想在本地下载和配置完整模型资源,可以先通过 RunningHub 在线体验相关 MagiHuman 工作流,或将 RunningHub 的 AI 应用 / 工作流 API 集成到自己的产品中。
23
+
24
+ - [RunningHub 在线体验](https://www.runninghub.ai)
25
+ - [RunningHub 中国站](https://www.runninghub.cn)
26
+ - [RunningHub API 文档](https://www.runninghub.cn/runninghub-api-doc-cn/)
27
+ - [RunningHub API Documentation](https://www.runninghub.cn/runninghub-api-doc-en/)
28
+ - [ComfyUI-RH-daVinci-MagiHuman 插件](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
29
+
30
+ RunningHub 适合用于快速验证参考图生成说话人视频、数字人视频、AI 口播视频等工作流效果。用户可以先在线运行和调试工作流,再根据需要选择本地部署,或通过 RunningHub API 将同类能力接入自己的产品。
31
+
32
+ > 想快速体验效果?点击 [RunningHub 在线体验](https://www.runninghub.ai) 查看可运行的 AI 工作流。想把数字人 / 说话人视频生成能力接入产品?查看 [RunningHub API 文档](https://www.runninghub.cn/runninghub-api-doc-cn/) 开始集成。
33
+
34
+ RunningHub API 支持模型 API、AI 应用 API、工作流 API 和 LLM API。对于 MagiHuman 这类视频生成工作流,推荐使用任务型调用流程:
35
+
36
+ ```text
37
+ 提交任务 → 获取 taskId → 查询任务状态 → 获取生成结果
38
+ ```
39
+
40
+ 适合的使用场景:
41
+
42
+ - 在线运行 daVinci-MagiHuman / MagiHuman 说话人视频生成工作流
43
+ - 将参考图生成视频、数字人口播、AI 视频生成能力接入自己的产品
44
+ - 为创作者、开发者和团队提供免本地部署的生成式 AI 工作流
45
+ - 基于 RunningHub API 构建自动化视频生成、内容生产或多模态应用
46
+
47
+ ## 内容清单
48
+
49
+ | 文件 | 大小 | 用途 |
50
+ |---|---:|---|
51
+ | `base_int8.pt` | 14.25 GiB | INT8 Base DiT(32 步,画质更高) |
52
+ | `distill_int8.pt` | 14.25 GiB | INT8 Distill DiT(8 步,更快) |
53
+ | `sr_540p_sr_int8.pt` | 14.25 GiB | 可选 INT8 540p 超分 |
54
+ | `sr_1080p_sr_int8.pt` | 14.25 GiB | 可选 INT8 1080p 超分 |
55
+
56
+ `base_int8.pt` 与 `distill_int8.pt` **任选其一**即可运行。只有加载节点的
57
+ `sr_model` 设为 `540p_sr` 或 `1080p_sr` 时,才需要对应的 SR 文件。
58
+
59
+ ## 安装到 ComfyUI
60
+
61
+ ```text
62
+ ComfyUI/models/MagiHuman/
63
+ ├── base_int8.pt
64
+ ├── distill_int8.pt
65
+ ├── sr_540p_sr_int8.pt # 可选
66
+ └── sr_1080p_sr_int8.pt # 可选
67
+ ```
68
+
69
+ ### Hugging Face
70
+
71
+ ```bash
72
+ cd /path/to/ComfyUI
73
+ python3 -m pip install -U huggingface_hub
74
+
75
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt \
76
+ --local-dir ./models/MagiHuman
77
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt \
78
+ --local-dir ./models/MagiHuman
79
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt \
80
+ --local-dir ./models/MagiHuman
81
+ hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt \
82
+ --local-dir ./models/MagiHuman
83
+ ```
84
+
85
+ ### ModelScope
86
+
87
+ ```bash
88
+ pip install modelscope
89
+ cd /path/to/ComfyUI
90
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt \
91
+ --local_dir ./models/MagiHuman
92
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt \
93
+ --local_dir ./models/MagiHuman
94
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt \
95
+ --local_dir ./models/MagiHuman
96
+ modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt \
97
+ --local_dir ./models/MagiHuman
98
+ ```
99
+
100
+ ## 其它必需资源
101
+
102
+ 以下文件**不在**本仓库中:
103
+
104
+ ```bash
105
+ # 官方 MagiHuman 资源(TurboVAE、T5 文本编码器)
106
+ hf download GAIR/daVinci-MagiHuman --local-dir ./models/MagiHuman
107
+ pip install modelscope
108
+ modelscope download --model GAIR/daVinci-MagiHuman --local_dir ./models/MagiHuman
109
+
110
+ # 外部 VAE
111
+ hf download stabilityai/stable-audio-open-1.0 \
112
+ --local-dir ./models/audio_checkpoints/stable-audio-open-1.0
113
+ hf download Wan-AI/Wan2.2-TI2V-5B \
114
+ --local-dir ./models/Ovi/Wan2.2-TI2V-5B
115
+ ```
116
+
117
+ 全部就位后的目录:
118
+
119
+ ```text
120
+ ComfyUI/models/
121
+ ├── MagiHuman/
122
+ │ ├── base_int8.pt
123
+ │ ├── distill_int8.pt
124
+ │ ├── sr_540p_sr_int8.pt
125
+ │ ├── sr_1080p_sr_int8.pt
126
+ │ ├── t5gemma-9b-9b-ul2/
127
+ │ └── turbo_vae/
128
+ ├── audio_checkpoints/stable-audio-open-1.0/
129
+ └── Ovi/Wan2.2-TI2V-5B/
130
+ ```
131
+
132
+ ## 插件
133
+
134
+ 当前插件仓库:[RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)
135
+
136
+ | 节点 | 用途 |
137
+ |---|---|
138
+ | `RH MagiHuman Model Loader` (`RH_MagiHumanModelLoader`) | 加载 INT8 DiT、VAE 和可选 SR |
139
+ | `RH MagiHuman Generate` (`RH_MagiHumanGenerate`) | 根据参考图和文本生成说话人视频 |
140
+
141
+ 加载节点选项:
142
+
143
+ - `model_type`:`base`(32 步)或 `distill`(8 步)
144
+ - `vram_mode`:`mid_vram`(约 16 GB)或 `low_vram`(约 6.5 GB)
145
+ - `sr_model`:`none` / `540p_sr` / `1080p_sr`
146
+
147
+ `mid_vram` + 540p SR 大约需要 24 GB;`mid_vram` + 1080p SR 大约需要 48 GB。
148
+
149
+ ## 许可证
150
+
151
+ 转换权重遵循上游
152
+ [daVinci-MagiHuman](https://huggingface.co/GAIR/daVinci-MagiHuman) 的
153
+ [Apache-2.0](https://opensource.org/licenses/Apache-2.0) 许可证。
154
+
155
+ ## 相关链接
156
+
157
+ [![RunningHub China](https://img.shields.io/badge/RunningHub-China-2F80ED)](https://www.runninghub.cn/?inviteCode=rh-v1367)
158
+ [![RunningHub International](https://img.shields.io/badge/RunningHub-International-7B61FF)](https://www.runninghub.ai/?inviteCode=rh-v1367)
159
+
160
+ - [RunningHub](https://www.runninghub.ai)
161
+ - [RunningHub 中国站](https://www.runninghub.cn)
162
+ - [RunningHub API](https://www.runninghub.ai/call-api)
163
+ - [RunningHub API Documentation](https://www.runninghub.cn/runninghub-api-doc-en/)
164
+ - [TokenPlus](https://tokenplus.net/)
165
+ - [daVinci-MagiHuman 官方项目](https://github.com/GAIR-NLP/daVinci-MagiHuman)
166
+ - [官方权重](https://huggingface.co/GAIR/daVinci-MagiHuman)
167
+ - [官方资源 ModelScope](https://www.modelscope.cn/models/GAIR/daVinci-MagiHuman)
168
+ - [ComfyUI-RH-daVinci-MagiHuman](https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman)