|
Download README.md from DiffSynth-Studio/MiniMax-H3-Image-EntroPack: direct link, hf CLI and curl.
- Browser
- Download file 15.7 kB
-
https://huggingface.co/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/resolve/main/README.md
- Command line
-
hf download hf://DiffSynth-Studio/MiniMax-H3-Image-EntroPack/README.md
-
curl -L -o README.md https://huggingface.co/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/resolve/main/README.md
15.7 kB
| base_model: MiniMax/MiniMax-H3-Image | |
| base_model_relation: quantized | |
| # MiniMax-H3-Image (AdaLN-pruned) — EntroPack Pre-Quantized Models | |
| We provide pre-quantized weight packages produced by [**EntroPack**](https://github.com/modelscope/entropack) for the AdaLN-pruned **MiniMax-H3-Image** transformer ([MiniMax/MiniMax-H3-Image](https://www.modelscope.cn/models/MiniMax/MiniMax-H3-Image)), which can be used with the quantization features of [**DiffSynth-Studio**](https://github.com/modelscope/DiffSynth-Studio). Use them with ComfyUI through [DiffSynth-ComfyUI](https://github.com/modelscope/DiffSynth-ComfyUI). A ready-to-run ComfyUI workflow is included in [`comfyui/featured_minimax_h3_image_entropack.json`](comfyui/featured_minimax_h3_image_entropack.json). | |
| The pruned transformer replaces every AdaLN projection and the timestep embedder with lookup tables for the six distilled sampling steps: 20.1B parameters instead of 33.1B, and 40.2 GB instead of 66.3 GB in bfloat16. **This replacement is completely lossless: every lookup-table row is precomputed from the original weights, so the pruned transformer produces exactly the same output as the unpruned checkpoint at the same prompt and seed. It is inference-only and applies only to the distilled six-step schedule.** | |
| The packages run with as little as 24 GB of VRAM. | |
| > **On a 24 GB GPU, use `dit_7bit` or `dit_7bit_int8`; on a 12 GB or smaller GPU, use `dit_4p5bit` or `dit_4p5bit_int8`.** | |
| The DiT is provided at five bitrate levels: | |
| * **`8bit` / `7bit` / `6bit`** — uniform entropy coding, with the quantization-sensitive layers held at 8.5 bpp; | |
| * **`4.5bit`** — the same mixed allocation, with the main blocks at the bitrate derived from the 4.5 bpp weighted target; | |
| * **`extreme_3bpp`** — extreme quantization: the same mixed allocation with the main blocks at 2.91 bpp, the two input projections at 8.5 bpp and the token-refiner blocks at 5.0 bpp; | |
| * **`int8_*`** — weights are first **INT8 W8A8-quantized** with convrot, then stored by EntroPack; the `int8_8bit` package stores the INT8 codes verbatim, the others entropy-code them to the level's bitrate. The forward pass uses **INT8 GEMM**, which delivers a faster inference speed than the BF16 GEMM used by the other levels. The INT8 GEMM path requires Triton and a CUDA GPU with compute capability 8.0 or newer. | |
| The text encoder is not re-quantized for this repository: reuse `models/MiniMax-H3/text_encoder_3bpp.safetensors` from [DiffSynth-Studio/EntroPackPreQuants](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants), the package shared by every MiniMax-H3 model (video, audio and image). | |
| The image VAE is provided as an INT8 package (`models/vae_int8.safetensors`): the ViT decoder's linear layers use INT8 GEMM with convrot, while the convolutional encoder and the normalization layers stay in bfloat16. | |
| Every `*.safetensors` ships with a same-named `*.json` that records the quantization config. `actual bpp = 8 × Σ storage bytes / Σ numel` is measured on the package artifacts. | |
| ## Preview | |
| All previews are generated with the same prompt, seed 42 and size 4096×4096; `bf16` is the released transformer and `bf16_pruned` the AdaLN-pruned transformer, both in full precision. | |
| <table width="100%"> | |
| <tr> | |
| <td width="50%" align="center">bf16 (reference)</td> | |
| <td width="50%" align="center">bf16 pruned</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/bf16.jpg"></td> | |
| <td align="center"><img width="100%" src="assets/bf16_pruned.jpg"></td> | |
| </tr> | |
| <tr> | |
| <td width="50%" align="center">dit_8bit</td> | |
| <td width="50%" align="center">dit_8bit_int8</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/dit_8bit.jpg"></td> | |
| <td align="center"><img width="100%" src="assets/dit_8bit_int8.jpg"></td> | |
| </tr> | |
| <tr> | |
| <td width="50%" align="center">dit_7bit</td> | |
| <td width="50%" align="center">dit_7bit_int8</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/dit_7bit.jpg"></td> | |
| <td align="center"><img width="100%" src="assets/dit_7bit_int8.jpg"></td> | |
| </tr> | |
| <tr> | |
| <td width="50%" align="center">dit_6bit</td> | |
| <td width="50%" align="center">dit_6bit_int8</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/dit_6bit.jpg"></td> | |
| <td align="center"><img width="100%" src="assets/dit_6bit_int8.jpg"></td> | |
| </tr> | |
| <tr> | |
| <td width="50%" align="center">dit_4p5bit</td> | |
| <td width="50%" align="center">dit_4p5bit_int8</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/dit_4p5bit.jpg"></td> | |
| <td align="center"><img width="100%" src="assets/dit_4p5bit_int8.jpg"></td> | |
| </tr> | |
| <tr> | |
| <td width="50%" align="center">dit_extreme_3bpp (extreme quantization)</td> | |
| <td width="50%"></td> | |
| </tr> | |
| <tr> | |
| <td align="center"><img width="100%" src="assets/dit_extreme_3bpp.jpg"></td> | |
| <td width="50%"></td> | |
| </tr> | |
| </table> | |
| The two full-precision rows are identical. | |
| ## DiT | |
| | name | file | actual bpp | size | | |
| |---|---|---|---| | |
| | bf16_pruned (full precision) | [`models/bf16_table_pruned.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fbf16_table_pruned.safetensors) | 16.028 | 40.20 GB | | |
| | dit_8bit | [`models/dit_8bit.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_8bit.safetensors) | 8.107 | 20.34 GB | | |
| | dit_7bit | [`models/dit_7bit.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_7bit.safetensors) | 7.134 | 17.89 GB | | |
| | dit_6bit | [`models/dit_6bit.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_6bit.safetensors) | 6.119 | 15.35 GB | | |
| | dit_4p5bit | [`models/dit_4p5bit.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_4p5bit.safetensors) | 4.587 | 11.51 GB | | |
| | dit_8bit_int8 (INT8 GEMM) | [`models/dit_8bit_int8.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_8bit_int8.safetensors) | 8.054 | 20.20 GB | | |
| | dit_7bit_int8 (INT8 GEMM) | [`models/dit_7bit_int8.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_7bit_int8.safetensors) | 7.076 | 17.75 GB | | |
| | dit_6bit_int8 (INT8 GEMM) | [`models/dit_6bit_int8.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_6bit_int8.safetensors) | 6.072 | 15.23 GB | | |
| | dit_4p5bit_int8 (INT8 GEMM) | [`models/dit_4p5bit_int8.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_4p5bit_int8.safetensors) | 4.568 | 11.46 GB | | |
| | dit_extreme_3bpp (extreme quantization) | [`models/dit_extreme_3bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fdit_extreme_3bpp.safetensors) | 3.057 | 7.67 GB | | |
| ## Text Encoder | |
| | name | file | actual bpp | size | | |
| |---|---|---|---| | |
| | text_encoder_3bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_3bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_3bpp.safetensors) | 3.531 | 11.02 GB | | |
| | text_encoder_4bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_4bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_4bpp.safetensors) | 4.541 | 14.17 GB | | |
| | text_encoder_5bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_5bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_5bpp.safetensors) | 5.575 | 17.40 GB | | |
| | text_encoder_6bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_6bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_6bpp.safetensors) | 6.604 | 20.61 GB | | |
| | text_encoder_7bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_7bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_7bpp.safetensors) | 7.629 | 23.81 GB | | |
| | text_encoder_8bpp | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_8bpp.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_8bpp.safetensors) | 8.597 | 26.83 GB | | |
| | text_encoder_int8_convrot (INT8 GEMM) | [`DiffSynth-Studio/EntroPackPreQuants:models/MiniMax-H3/text_encoder_int8_convrot.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants/file/view/master/models%2FMiniMax-H3%2Ftext_encoder_int8_convrot.safetensors) | 8.506 | 26.55 GB | | |
| ## VAE | |
| | name | file | actual bpp | size | | |
| |---|---|---|---| | |
| | vae_int8 (INT8 GEMM) | [`models/vae_int8.safetensors`](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-Image-EntroPack/file/view/master/models%2Fvae_int8.safetensors) | 8.035 | 0.77 GB | | |
| The 98 linear layers of the ViT decoder are INT8 with convrot; the convolutional encoder and the normalization layers stay in bfloat16. | |
| ## Usage | |
| ### Install | |
| ```bash | |
| # DiffSynth-Studio | |
| git clone https://github.com/modelscope/DiffSynth-Studio.git | |
| cd DiffSynth-Studio && pip install -e . | |
| # EntroPack (pick the extra matching your CUDA version: cuda13 / cuda12) | |
| git clone https://github.com/modelscope/entropack.git | |
| cd entropack && pip install -e '.[cuda13]' | |
| ``` | |
| If EntroPack is already installed but its CUDA backend cannot start, the error names the missing piece, for example: | |
| ``` | |
| entropack.registry.DispatchError: No backend can handle 'lattice_rans': cuda: cupy is not installed (No module named 'cupy'); | |
| install a matching build, e.g. `pip install cupy-cuda13x` for CUDA 13 or `pip install cupy-cuda12x` for CUDA 12 | |
| ``` | |
| Install the CuPy build that matches the CUDA major version of your PyTorch (`pip install cupy-cuda13x` for CUDA 13, `pip install cupy-cuda12x` for CUDA 12), or reinstall EntroPack with the matching extra (`pip install "entropack[cuda13]"` / `pip install "entropack[cuda12]"`). | |
| ### ComfyUI | |
| The packages are also available as ComfyUI nodes through [DiffSynth-ComfyUI](https://github.com/modelscope/DiffSynth-ComfyUI). A ready-to-run workflow is shipped as [`comfyui/featured_minimax_h3_image_entropack.json`](comfyui/featured_minimax_h3_image_entropack.json): it loads the 7-bit INT8 DiT, the 7bpp text encoder and the INT8 VAE, and generates a 4096×4096 image in six distilled steps. Load the file in ComfyUI and queue it; the models are downloaded and registered from their json sidecars automatically. | |
| ### Example scripts | |
| * Inference: [standard](https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/minimax_h3_image/model_inference/MiniMax-H3-Image-Pruned-EntroPack.py) / [low VRAM](https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/minimax_h3_image/model_inference_low_vram/MiniMax-H3-Image-Pruned-EntroPack.py) | |
| * LoRA training: [train](https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/minimax_h3_image/model_training/lora/MiniMax-H3-Image-EntroPack.sh) / [validation](https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/minimax_h3_image/model_training/validate_lora/MiniMax-H3-Image-EntroPack.py) | |
| `dit_package` and `text_encoder_package` are chosen independently: any combination works (e.g. `dit_int8_8bit` with `text_encoder_4bpp`). The 7-bit DiT and the 4-bit text encoder are pre-registered in DiffSynth-Studio, so they load with a plain `ModelConfig`; the helper below registers any package's json sidecar before loading: | |
| ```python | |
| import json | |
| import torch | |
| from PIL import Image | |
| from diffsynth.configs import MODEL_CONFIGS | |
| from diffsynth.pipelines.minimax_h3_image import MiniMaxH3ImagePipeline, ModelConfig | |
| dit_model_id = "DiffSynth-Studio/MiniMax-H3-Image-EntroPack" | |
| text_encoder_model_id = "DiffSynth-Studio/EntroPackPreQuants" | |
| dit_package = "models/dit_7bit.safetensors" | |
| text_encoder_package = "models/MiniMax-H3/text_encoder_3bpp.safetensors" | |
| def register_prequantized(model_id, package_pattern): | |
| package = ModelConfig(model_id=model_id, origin_file_pattern=package_pattern) | |
| package.download_if_necessary() | |
| sidecar = ModelConfig(model_id=model_id, origin_file_pattern=package_pattern.replace(".safetensors", ".json")) | |
| sidecar.download_if_necessary() | |
| with open(sidecar.path) as handle: | |
| entry = json.load(handle) | |
| if not any(config.get("model_hash") == entry["model_hash"] for config in MODEL_CONFIGS): | |
| MODEL_CONFIGS.append(entry) | |
| return package | |
| pipe = MiniMaxH3ImagePipeline.from_pretrained( | |
| torch_dtype=torch.bfloat16, | |
| device="cuda", | |
| model_configs=[ | |
| register_prequantized(dit_model_id, dit_package), | |
| register_prequantized(text_encoder_model_id, text_encoder_package), | |
| ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="pdd_fused_heads.safetensors"), | |
| ModelConfig(model_id="DiffSynth-Studio/MiniMax-H3-Image-EntroPack", origin_file_pattern="models/vae_int8.safetensors"), | |
| ], | |
| processor_config=ModelConfig(model_id="MiniMax/MiniMax-H3-Image", origin_file_pattern="processor/"), | |
| ) | |
| prompt = ( | |
| "Full-color cinematic outdoor night photograph of a stunning young Asian woman in her early twenties with a" | |
| "natural warm ivory complexion and a healthy glow, wearing a soft blush pink off-shoulder dress with a fitted" | |
| "bodice, framed from the waist up against softly illuminated historic architecture. She stands near the" | |
| "centre of the frame, facing the camera with a gentle closed-lip smile, her hands resting loosely clasped in" | |
| "front of her waist with relaxed elbows; she has a slim oval face with softly defined cheekbones and a" | |
| "delicate narrow jaw, dark expressive eyes, natural-shaped brows, and subtle natural makeup with soft red" | |
| "lips. Her long black hair falls in loose natural waves over her shoulders, with fine strands framing her" | |
| "face. The fabric of the dress falls in soft folds across her shoulders and chest. In the frame-left" | |
| "background, the word \"EntroPack\" glows as a hand-drawn bubbly yellow neon sign mounted on a softly floodlit" | |
| "dark stone building above the street; rows of warm-lit windows, stone cornices and shopfronts remain clearly" | |
| "readable around the sign in gentle amber light, while the thick rounded irregular letters ride a gently" | |
| "bouncing baseline with a bright core and a soft outer glow, kept sharp and unblurred against the night. The" | |
| "background recedes into deep blue tones: the Eiffel Tower rises above distant buildings on the frame-right" | |
| "as a softly silhouetted tapering lattice outline in the shallow depth of field, while small amber, muted" | |
| "turquoise and rose-coloured lights dissolve into rounded bokeh across the upper background, leaving a darker" | |
| "area behind her head. Soft neutral frontal illumination with gentle falloff reveals her face with natural" | |
| "warm skin tone and subtle texture, a faint cooler rim separates her hair from the night, and the skirt of" | |
| "her dress fills the lower edge of the frame." | |
| ) | |
| image = pipe(prompt=prompt, height=4096, width=4096, seed=42) | |
| image.save("image.jpg") | |
| ``` | |