--- license: other license_name: minimax-h3-community-license-agreement license_link: LICENSE base_model: MiniMaxAI/MiniMax-H3 library_name: comfyui pipeline_tag: image-text-to-video tags: - minimax-h3 - comfyui - quantization - int8 - video - audio - fl2va - ref2va - dynamic-time - separate-qkv - experimental --- # MiniMax-H3 DynTime sQKV Quants > **Experimental: a ComfyUI core patch is required.** These FL2VA and Ref2VA > checkpoints retain the original FP32 runtime time MLP and physically separate > Q, K, and V projections. They do not execute correctly in stock ComfyUI. Community mixed-precision INT8 conversions of [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3). All 50 transformer blocks are retained. The repository is separate from the [stock-compatible quants](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants) so patch-required files cannot be confused with ordinary ComfyUI checkpoints. These are community derivatives, not official MiniMax or ComfyOrg releases. ## Naming - `FL2VA` is text/first-frame/last-frame-to-audio-video generation. - `Ref2VA` is reference-image/video/audio-to-audio-video generation. - `DT-sQKV` means **dynamic-time conditioning with physically separate Q, K, and V projections**. - A filename without `DT-sQKV` belongs to the stock-compatible repository. - Exact INT8/BF16 inventories and GPU classes are documented here instead of being encoded in the filenames. ## Choose a checkpoint | Profile | Direct downloads | File size | GPU class and quant layout | |---|---|---:|---| | **DT-sQKV INT8 ConvRot** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV/resolve/main/FL2VA/MiniMax-H3_FL2VA-DT-sQKV-INT8-ConvRot.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV/resolve/main/Ref2VA/MiniMax-H3_Ref2VA-DT-sQKV-INT8-ConvRot.safetensors?download=true) | 20.999 GiB | **24 GB · RTX 30/40/50.** 170 INT8 + 30 BF16 main semantic matrices; 270 physical INT8 modules; BF16 token refiner. Patch required. | | **DT-sQKV INT8 ConvRot HQ** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV/resolve/main/FL2VA/MiniMax-H3_FL2VA-DT-sQKV-INT8-ConvRot-HQ.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV/resolve/main/Ref2VA/MiniMax-H3_Ref2VA-DT-sQKV-INT8-ConvRot-HQ.safetensors?download=true) | 27.994 GiB | **32 GB+ · RTX 30/40/50.** 74 INT8 + 126 BF16 main semantic matrices; 122 physical INT8 modules; BF16 token refiner. Patch required. | The 24/32 GB classes are capacity guidance, not full-workflow peak guarantees. Resolution, frame count, the text encoder, VAEs, attention backend, and offload settings also affect memory use. Moving a 21–28 GiB model across PCIe at every denoising step can make these editions impractical on 8–16 GB GPUs; use a stock-compatible W4/W8W4/NVFP4 checkpoint for those memory classes. ## Required ComfyUI patch Apply: ```text patches/ComfyUI-MiniMax-H3-DT-sQKV.patch ``` The patch changes MiniMax-H3 model detection, construction, loading, and forward execution. A loader merely accepting the checkpoint is not sufficient: the patched forward path must evaluate the original time MLP and the separate Q/K/V modules. ## What DT-sQKV preserves | Feature | Stock-compatible quants | These DT-sQKV quants | |---|---|---| | Transformer blocks | All 50 retained | All 50 retained | | Attention storage | Fused `qkv_proj = cat(Q,K,V)` | Physical `q_proj`, `k_proj`, `v_proj` tensors | | Attention execution | One fused projection call | Three projection calls | | Original `time_embedder` | Replaced by measured table | Retained in FP32 | | `time_embedder.proj_in` | Absent | FP32 weight `[5376,256]`, bias `[5376]` | | `time_embedder.proj_out` | Absent | FP32 weight `[2688,5376]`, bias `[2688]` | | `adaln_t_table` | FP32 `[4097,16]` | Absent | | `adaln_curve_basis` | Absent | FP32 `[2688,16]` | | `adaln_curve_mean` | Absent | FP32 `[2688]` | | Per-block AdaLN | 51 independent FP32 rank-16 projections | 51 independent FP32 rank-16 projections | | ComfyUI | Stock | Included core patch required | The original FP32 time path runs for every requested timestep: ```text 256 -> 5,376 -> 2,688 full_t = SiLU(original_time_embedder(t)) coords = (full_t - mean) @ basis[2,688 x 16] AdaLN_i(t) = independent_projection_i(coords) ``` The shared rank-16 basis removes redundant input width from the 51 large AdaLN projections. It does not replace the original time MLP and does not merge the per-block AdaLN layers. Full-time relative reconstruction error is about `3e-7`; measured basis orthogonality residual is below `6e-7`. ## Separate Q/K/V layout The original Diffusers checkpoints contain separate `to_q`, `to_k`, and `to_v` tensors. These files retain that layout through loading and execution: - 50 main transformer attention blocks; - 2 token-refiner attention blocks; - 156 physical Q/K/V weights; - three projection calls per attention block; - no fused `qkv_proj` modules. Released Q/K/V tensors were checked bit-for-bit against their corresponding contiguous slices in the stock-compatible fused checkpoint. ## Quantization profiles Both profiles use ConvRot/Hadamard group size 256, deterministic scale search, and per-row FP32 scales for INT8 weights. Norms, patch projections, output heads, the time MLP, rank-16 basis, and all AdaLN projections retain source precision. | Profile | Main semantic matrices | Physical INT8 modules | Token refiner | Time/AdaLN path | |---|---|---:|---|---| | `DT-sQKV-INT8-ConvRot` | 170 INT8 + 30 BF16 | 270 | BF16 | Original FP32 time MLP, basis, mean, and 51 FP32 AdaLN projections | | `DT-sQKV-INT8-ConvRot-HQ` | 74 INT8 + 126 BF16 | 122 | BF16 | Original FP32 time MLP, basis, mean, and 51 FP32 AdaLN projections | The standard profile keeps 30 high-risk main matrices in BF16. The HQ profile keeps every attention-output projection and every MLP `fc2` projection in BF16, together with the 26 highest-error QKV groups. All 50 HQ `fc1` projections remain INT8. ## Validation Every checkpoint passed: 1. exact key, shape, dtype, and quantization-inventory validation; 2. bitwise Q/K/V split verification; 3. dynamic-time reconstruction comparison; 4. complete CPU load through patched ComfyUI as `MiniMaxH3Model`; 5. remote LFS byte-size and SHA-256 verification. Reports under `reports/` retain their historical internal profile names so the validation provenance remains intact. Tests used clean ComfyUI commit `14b05228` plus the included patch. Future ComfyUI revisions may require the same small core changes to be forward-ported. No full prompt-to-decoded-video perceptual A/B score is claimed. The 32 GB profile was structurally validated but is not claimed to remain fully resident on a 24 GB GPU. ## Installation 1. Use a ComfyUI revision compatible with the included patch. 2. Apply `patches/ComfyUI-MiniMax-H3-DT-sQKV.patch` and restart ComfyUI. 3. Place one selected checkpoint in `ComfyUI/models/diffusion_models/`. 4. Use the matching FL2VA or Ref2VA workflow. A complete workflow also requires the MiniMax-H3 Qwen3-VL text encoder and the video/audio VAEs from the stock-compatible repository. They are not duplicated here. ## License and attribution Use is subject to the included MiniMax-H3 community license. The base model is by MiniMax. ComfyUI and its quantization runtimes are separate upstream projects. This community conversion is not endorsed by MiniMax or ComfyOrg.