akhaliq HF Staff commited on
Commit
082c471
·
verified ·
1 Parent(s): 88992dc

Card: reframe — video model works; image model tried, doesn't steer; fine-tune if demand

Browse files
Files changed (1) hide show
  1. README.md +18 -4
README.md CHANGED
@@ -19,11 +19,25 @@ tags:
19
 
20
  <div style="display:flex;flex-wrap:wrap;align-items:center;gap:8px;margin:0 0 2em"><a href="https://hf.co/chat?mode=ml-intern"><img src="https://hf.co/datasets/huggingface/badges/resolve/main/built-with-ml-intern-lg.svg" alt="Built with ML Intern" style="margin:0"></a><a href="https://hf.co/chat?mode=ml-intern&prompt=Fine-tune%20the%20control%20branch%20in%20akhaliq%2FMiniMax-H3-Image-Fun-Controlnet-Union-2.0%20so%20it%20steers%20H3-Image" style="display:inline-flex;align-items:center;gap:5px;padding:4px 12px;border:1.5px solid #eee;border-radius:16px;background:#f9fbfc;color:#c2410c;font-weight:600;font-size:14px;text-decoration:none">Adapt the control branch<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"><path d="M13.5 6H5a2 2 0 0 0-2 2v11a2 2 0 0 0 2 2h11a2 2 0 0 0 2-2v-8.5m-10.5 6L21 3m0 0h-5m5 0v5"/></svg></a><a href="https://hf.co/chat?mode=ml-intern&prompt=Test%20the%20control%20branch%20from%20akhaliq%2FMiniMax-H3-Image-Fun-Controlnet-Union-2.0%20on%20the%20native%20(non-distilled)%20H3-Image%20DiT%20at%2040%20steps" style="display:inline-flex;align-items:center;gap:5px;padding:4px 12px;border:1.5px solid #eee;border-radius:16px;background:#f9fbfc;color:#c2410c;font-weight:600;font-size:14px;text-decoration:none">Try the native DiT<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"><path d="M13.5 6H5a2 2 0 0 0-2 2v11a2 2 0 0 0 2 2h11a2 2 0 0 0 2-2v-8.5m-10.5 6L21 3m0 0h-5m5 0v5"/></svg></a></div>
21
 
22
- # MiniMax-H3-Image + Fun ControlNet-Union 2.0
23
 
24
- [MiniMaxAI/MiniMax-H3-Image](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image) packaged with the
25
- [alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0](https://huggingface.co/alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0)
26
- control branch (8 union conditions + inpainting) — plus verified, reproducible scripts and samples.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ## Official card demos, reproduced
29
 
 
19
 
20
  <div style="display:flex;flex-wrap:wrap;align-items:center;gap:8px;margin:0 0 2em"><a href="https://hf.co/chat?mode=ml-intern"><img src="https://hf.co/datasets/huggingface/badges/resolve/main/built-with-ml-intern-lg.svg" alt="Built with ML Intern" style="margin:0"></a><a href="https://hf.co/chat?mode=ml-intern&prompt=Fine-tune%20the%20control%20branch%20in%20akhaliq%2FMiniMax-H3-Image-Fun-Controlnet-Union-2.0%20so%20it%20steers%20H3-Image" style="display:inline-flex;align-items:center;gap:5px;padding:4px 12px;border:1.5px solid #eee;border-radius:16px;background:#f9fbfc;color:#c2410c;font-weight:600;font-size:14px;text-decoration:none">Adapt the control branch<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"><path d="M13.5 6H5a2 2 0 0 0-2 2v11a2 2 0 0 0 2 2h11a2 2 0 0 0 2-2v-8.5m-10.5 6L21 3m0 0h-5m5 0v5"/></svg></a><a href="https://hf.co/chat?mode=ml-intern&prompt=Test%20the%20control%20branch%20from%20akhaliq%2FMiniMax-H3-Image-Fun-Controlnet-Union-2.0%20on%20the%20native%20(non-distilled)%20H3-Image%20DiT%20at%2040%20steps" style="display:inline-flex;align-items:center;gap:5px;padding:4px 12px;border:1.5px solid #eee;border-radius:16px;background:#f9fbfc;color:#c2410c;font-weight:600;font-size:14px;text-decoration:none">Try the native DiT<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"><path d="M13.5 6H5a2 2 0 0 0-2 2v11a2 2 0 0 0 2 2h11a2 2 0 0 0 2-2v-8.5m-10.5 6L21 3m0 0h-5m5 0v5"/></svg></a></div>
21
 
22
+ # MiniMax-H3 + Fun ControlNet-Union 2.0 — video-model control, validated
23
 
24
+ **TL;DR: this repo is the ControlNet-Union-2.0 branch working with the MiniMax-H3 _video_ model** —
25
+ all 9 official demos reproduced, every condition validated, a ComfyUI workflow, and a
26
+ repeated-frame recipe that turns the video stack into a still-image controlnet (canny/gray/HED).
27
+
28
+ **What we tried that did NOT work:** driving the [MiniMax-H3-**Image**](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image)
29
+ model (the fast 6-step PDD-distilled image DiT) with this branch. The runtime wiring is verified
30
+ mathematically exact (scale-0 output is bit-identical to base), but the video-trained control
31
+ blocks collapse the spatial signal on the image model's distilled trajectory — it perturbs,
32
+ never steers (full evidence: [`assets/diag*`](assets), [`assets/image-dit-official/`](assets/image-dit-official)).
33
+
34
+ **Future work:** fine-tune the control branch on the H3-Image DiT (frozen DiT, zero-gated start —
35
+ all machinery in [`merge/`](merge)). **If there's enough demand we'll run it — watch/star this
36
+ repo or open a discussion to vote.**
37
+
38
+ Contents: [MiniMaxAI/MiniMax-H3-Image](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image) packaged
39
+ with the [alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0](https://huggingface.co/alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0)
40
+ control branch (8 union conditions + inpainting), plus verified, reproducible scripts and samples.
41
 
42
  ## Official card demos, reproduced
43