Update README.md
Browse files
README.md
CHANGED
|
@@ -38,7 +38,7 @@ consumes:
|
|
| 38 |
|
| 39 |
and outputs the teacher-equivalent representation `[1, S_T, 5376]`. Because the
|
| 40 |
output lives in the same space as the teacher's post-refiner embeddings, the DiT
|
| 41 |
-
consumes it directly
|
| 42 |
|
| 43 |
## Architecture
|
| 44 |
|
|
@@ -86,8 +86,6 @@ Held-out 1,002 prompts (disjoint from train):
|
|
| 86 |
| **MSE** | 0.8162 |
|
| 87 |
| **norm_ratio** (pred/target scale) | 1.013 |
|
| 88 |
|
| 89 |
-
Verified end-to-end: ComfyUI T2V generation with the adapter matches the prompt
|
| 90 |
-
content (no black frames, no semantic mismatch).
|
| 91 |
|
| 92 |
## Usage
|
| 93 |
|
|
@@ -121,7 +119,6 @@ Use with the [ComfyUI-MiniMaxH3-Adapter] node: `MiniMaxH3AdapterLoader`
|
|
| 121 |
or long cinematic prompts may drift from the 33B teacher.
|
| 122 |
- Trained on generated prompt combinations (9-skill template), not in-the-wild
|
| 123 |
caption distributions.
|
| 124 |
-
- Unconditional/vision-input (image-to-video) conditioning is out of scope.
|
| 125 |
- Intended for research / local use; check MiniMax-H3 terms of service.
|
| 126 |
|
| 127 |
## Training your own CondBridge
|
|
@@ -149,7 +146,7 @@ MiniMax-H3 视频生成用 33B 文本编码器(`condition_proj` + 2 层 `token
|
|
| 149 |
| `student_hidden` | `[S_S, 2560]` | Qwen3.5-4B `hidden_states[-1]`(final-norm 后) |
|
| 150 |
|
| 151 |
并输出与教师等价的表示 `[1, S_T, 5376]`。由于输出与教师的 post-refiner 嵌入
|
| 152 |
-
处于同一空间,DiT 可直接消费
|
| 153 |
|
| 154 |
## 模型架构
|
| 155 |
|
|
@@ -197,7 +194,6 @@ Held-out 1,002 条 prompt(与训练集不重叠):
|
|
| 197 |
| **MSE** | 0.8162 |
|
| 198 |
| **norm_ratio**(预测/目标尺度) | 1.013 |
|
| 199 |
|
| 200 |
-
端到端验证:ComfyUI T2V 生成内容与 prompt 对齐(无黑屏、无语义错位)。
|
| 201 |
|
| 202 |
## 使用方法
|
| 203 |
|
|
@@ -223,14 +219,12 @@ embeds = adapter(torch.tensor([h3_ids]), student_hidden.unsqueeze(0)) # [1, S_T
|
|
| 223 |
|
| 224 |
配合 [ComfyUI-MiniMaxH3-Adapter] 节点使用:`MiniMaxH3AdapterLoader`
|
| 225 |
(学生目录 + 适配器 .safetensors)→ 接入官方 `MiniMaxH3ImageToVideo` 节点。
|
| 226 |
-
节点把适配器封装成鸭子类型 CLIP。
|
| 227 |
|
| 228 |
## 局限性
|
| 229 |
|
| 230 |
- 蒸馏保真度受限于学生(Qwen3.5-4B):复杂多镜头或长电影级 prompt 可能
|
| 231 |
偏离 33B 教师
|
| 232 |
- 训练数据是生成的 prompt 组合(9 类技能模板),非真实世界字幕分布
|
| 233 |
-
- 不支持无条件 / 图像输入(图生视频)条件化
|
| 234 |
- 面向研究 / 本地使用;请遵守 MiniMax-H3 服务条款
|
| 235 |
|
| 236 |
# 我自己想训练一个CondBridge
|
|
|
|
| 38 |
|
| 39 |
and outputs the teacher-equivalent representation `[1, S_T, 5376]`. Because the
|
| 40 |
output lives in the same space as the teacher's post-refiner embeddings, the DiT
|
| 41 |
+
consumes it directly
|
| 42 |
|
| 43 |
## Architecture
|
| 44 |
|
|
|
|
| 86 |
| **MSE** | 0.8162 |
|
| 87 |
| **norm_ratio** (pred/target scale) | 1.013 |
|
| 88 |
|
|
|
|
|
|
|
| 89 |
|
| 90 |
## Usage
|
| 91 |
|
|
|
|
| 119 |
or long cinematic prompts may drift from the 33B teacher.
|
| 120 |
- Trained on generated prompt combinations (9-skill template), not in-the-wild
|
| 121 |
caption distributions.
|
|
|
|
| 122 |
- Intended for research / local use; check MiniMax-H3 terms of service.
|
| 123 |
|
| 124 |
## Training your own CondBridge
|
|
|
|
| 146 |
| `student_hidden` | `[S_S, 2560]` | Qwen3.5-4B `hidden_states[-1]`(final-norm 后) |
|
| 147 |
|
| 148 |
并输出与教师等价的表示 `[1, S_T, 5376]`。由于输出与教师的 post-refiner 嵌入
|
| 149 |
+
处于同一空间,DiT 可直接消费
|
| 150 |
|
| 151 |
## 模型架构
|
| 152 |
|
|
|
|
| 194 |
| **MSE** | 0.8162 |
|
| 195 |
| **norm_ratio**(预测/目标尺度) | 1.013 |
|
| 196 |
|
|
|
|
| 197 |
|
| 198 |
## 使用方法
|
| 199 |
|
|
|
|
| 219 |
|
| 220 |
配合 [ComfyUI-MiniMaxH3-Adapter] 节点使用:`MiniMaxH3AdapterLoader`
|
| 221 |
(学生目录 + 适配器 .safetensors)→ 接入官方 `MiniMaxH3ImageToVideo` 节点。
|
|
|
|
| 222 |
|
| 223 |
## 局限性
|
| 224 |
|
| 225 |
- 蒸馏保真度受限于学生(Qwen3.5-4B):复杂多镜头或长电影级 prompt 可能
|
| 226 |
偏离 33B 教师
|
| 227 |
- 训练数据是生成的 prompt 组合(9 类技能模板),非真实世界字幕分布
|
|
|
|
| 228 |
- 面向研究 / 本地使用;请遵守 MiniMax-H3 服务条款
|
| 229 |
|
| 230 |
# 我自己想训练一个CondBridge
|