Use FastH3 Omni name and add local AI recipes
Browse files
README.md
CHANGED
|
@@ -16,9 +16,9 @@ tags:
|
|
| 16 |
|
| 17 |
# FastH3 NVFP4 VAE Decoder
|
| 18 |
|
| 19 |
-
Blackwell NVFP4 video decoders for MiniMax-H3 and FastH3, including
|
| 20 |
|
| 21 |
-
[FastVideo](https://github.com/hao-ai-lab/FastVideo) 路 [
|
| 22 |
|
| 23 |
## Choose a decoder
|
| 24 |
|
|
@@ -38,15 +38,15 @@ hf download FastVideo/FastH3-NVFP4-VAE-Decoder \
|
|
| 38 |
quality-36L/decoder-nvfp4.safetensors --local-dir ./fasth3-vae
|
| 39 |
```
|
| 40 |
|
| 41 |
-
Follow the [FastVideo
|
| 42 |
|
| 43 |
-
The 36-layer decoder needs a full 36-layer MiniMax-H3 host VAE. The 8-layer tier also works with the light H3 VAE. The
|
| 44 |
|
| 45 |
-
For UniServe deployment of the complete
|
| 46 |
|
| 47 |
## Hardware
|
| 48 |
|
| 49 |
-
The FP4 decoder kernels require NVIDIA Blackwell: GB200/B200, RTX PRO 6000 Blackwell, RTX 50-series or DGX Spark (GB10). Follow the [local
|
| 50 |
|
| 51 |
## Files
|
| 52 |
|
|
@@ -63,7 +63,7 @@ Reconstruction against the original FP32 decoder on the same latents; lower LPIP
|
|
| 63 |
| Evaluation clips | Quality tier LPIPS / PSNR | Fast tier LPIPS / PSNR |
|
| 64 |
| --- | ---: | ---: |
|
| 65 |
| FastH3 V2 generations, 9 clips at 480p | 0.0104 / 37.9 dB | 0.0684 / 28.5 dB |
|
| 66 |
-
| FastH3
|
| 67 |
| Real videos, VAE-encoded, 6 held-out clips | 0.0038 / 45.0 dB | 0.0330 / 34.3 dB |
|
| 68 |
|
| 69 |
Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using FastVideo's fused NVFP4 kernels and batching all spatial tiles in one decoder call:
|
|
@@ -76,6 +76,12 @@ Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using Fast
|
|
| 76 |
|
| 77 |
These are decoder-only measurements. Full video/audio generation also includes encoding, denoising and media output.
|
| 78 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
## License
|
| 80 |
|
| 81 |
Use of these weights is governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE).
|
|
|
|
| 16 |
|
| 17 |
# FastH3 NVFP4 VAE Decoder
|
| 18 |
|
| 19 |
+
Blackwell NVFP4 video decoders for MiniMax-H3 and FastH3, including FastH3 Omni. They replace the video decoder while retaining the reference encoder, audio VAE and video latent space.
|
| 20 |
|
| 21 |
+
[FastVideo](https://github.com/hao-ai-lab/FastVideo) 路 [FastH3 Omni decoder setup](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) 路 [FastH3 Omni release collection](https://huggingface.co/collections/FastVideo/fasth3-omniref)
|
| 22 |
|
| 23 |
## Choose a decoder
|
| 24 |
|
|
|
|
| 38 |
quality-36L/decoder-nvfp4.safetensors --local-dir ./fasth3-vae
|
| 39 |
```
|
| 40 |
|
| 41 |
+
Follow the [FastVideo FastH3 Omni guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) and pass the downloaded file with `--decoder` to `examples/inference/basic/basic_fasth3_omniref_pdd.py`. Use `fast-8L/decoder-nvfp4.safetensors` to select the fast tier instead.
|
| 42 |
|
| 43 |
+
The 36-layer decoder needs a full 36-layer MiniMax-H3 host VAE. The 8-layer tier also works with the light H3 VAE. The FastH3 Omni guide explains model composition and device profiles.
|
| 44 |
|
| 45 |
+
For UniServe deployment of the complete FastH3 Omni model on GB200/B200, RTX PRO 6000 Blackwell or Hopper, follow the [UniServe guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md) with the [BF16 model](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1). These files are FastVideo decoder components, not complete diffusion checkpoints.
|
| 46 |
|
| 47 |
## Hardware
|
| 48 |
|
| 49 |
+
The FP4 decoder kernels require NVIDIA Blackwell: GB200/B200, RTX PRO 6000 Blackwell, RTX 50-series or DGX Spark (GB10). Follow the [local FastH3 Omni guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md#device-profiles) for Spark and RTX 5090 setup. Hopper and older GPUs should use the original H3 video decoder.
|
| 50 |
|
| 51 |
## Files
|
| 52 |
|
|
|
|
| 63 |
| Evaluation clips | Quality tier LPIPS / PSNR | Fast tier LPIPS / PSNR |
|
| 64 |
| --- | ---: | ---: |
|
| 65 |
| FastH3 V2 generations, 9 clips at 480p | 0.0104 / 37.9 dB | 0.0684 / 28.5 dB |
|
| 66 |
+
| FastH3 Omni generations, 20 clips | 0.0039 / 46.0 dB | 0.0276 / 36.1 dB |
|
| 67 |
| Real videos, VAE-encoded, 6 held-out clips | 0.0038 / 45.0 dB | 0.0330 / 34.3 dB |
|
| 68 |
|
| 69 |
Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using FastVideo's fused NVFP4 kernels and batching all spatial tiles in one decoder call:
|
|
|
|
| 76 |
|
| 77 |
These are decoder-only measurements. Full video/audio generation also includes encoding, denoising and media output.
|
| 78 |
|
| 79 |
+
## Local AI resources
|
| 80 |
+
|
| 81 |
+
Run FastH3 Omni on **DGX Spark or RTX 5090 through FastVideo** using the [FastVideo-format NVFP4 QAD weights](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD). Start with the [FastVideo cookbook](https://haoailab.com/FastVideo/cookbook/) and [FastH3 Omni recipe builder](https://haoailab.com/FastVideo/cookbook/minimax-h3/fasth3-omniref/); use the [Spark recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_spark.yaml) or [RTX 5090 recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx5090.yaml) and the [local inference guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) for the AdaLN table, offload and decoder setup.
|
| 82 |
+
|
| 83 |
+
[FP8 weights](https://huggingface.co/FastVideo/FastH3-OmniRef-FP8) are a separate FastVideo variant; the [FP8 recipe for RTX 4090](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx4090.yaml) illustrates the non-FP4 path with layerwise offload and compatible BF16 encoder components. Use each runtime's matching checkpoint format.
|
| 84 |
+
|
| 85 |
## License
|
| 86 |
|
| 87 |
Use of these weights is governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE).
|