wlsaidhi commited on
Commit
bdfc13f
路
verified 路
1 Parent(s): 96cd3df

Use FastH3 Omni name and add local AI recipes

Browse files
Files changed (1) hide show
  1. README.md +13 -7
README.md CHANGED
@@ -16,9 +16,9 @@ tags:
16
 
17
  # FastH3 NVFP4 VAE Decoder
18
 
19
- Blackwell NVFP4 video decoders for MiniMax-H3 and FastH3, including OmniRef. They replace the video decoder while retaining the reference encoder, audio VAE and video latent space.
20
 
21
- [FastVideo](https://github.com/hao-ai-lab/FastVideo) 路 [OmniRef decoder setup](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) 路 [OmniRef release collection](https://huggingface.co/collections/FastVideo/fasth3-omniref)
22
 
23
  ## Choose a decoder
24
 
@@ -38,15 +38,15 @@ hf download FastVideo/FastH3-NVFP4-VAE-Decoder \
38
  quality-36L/decoder-nvfp4.safetensors --local-dir ./fasth3-vae
39
  ```
40
 
41
- Follow the [FastVideo OmniRef guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) and pass the downloaded file with `--decoder` to `examples/inference/basic/basic_fasth3_omniref_pdd.py`. Use `fast-8L/decoder-nvfp4.safetensors` to select the fast tier instead.
42
 
43
- The 36-layer decoder needs a full 36-layer MiniMax-H3 host VAE. The 8-layer tier also works with the light H3 VAE. The OmniRef guide explains model composition and device profiles.
44
 
45
- For UniServe deployment of the complete OmniRef model on GB200/B200, RTX PRO 6000 Blackwell or Hopper, follow the [UniServe guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md) with the [BF16 model](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1). These files are FastVideo decoder components, not complete diffusion checkpoints.
46
 
47
  ## Hardware
48
 
49
- The FP4 decoder kernels require NVIDIA Blackwell: GB200/B200, RTX PRO 6000 Blackwell, RTX 50-series or DGX Spark (GB10). Follow the [local OmniRef guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md#device-profiles) for Spark and RTX 5090 setup. Hopper and older GPUs should use the original H3 video decoder.
50
 
51
  ## Files
52
 
@@ -63,7 +63,7 @@ Reconstruction against the original FP32 decoder on the same latents; lower LPIP
63
  | Evaluation clips | Quality tier LPIPS / PSNR | Fast tier LPIPS / PSNR |
64
  | --- | ---: | ---: |
65
  | FastH3 V2 generations, 9 clips at 480p | 0.0104 / 37.9 dB | 0.0684 / 28.5 dB |
66
- | FastH3 OmniRef generations, 20 clips | 0.0039 / 46.0 dB | 0.0276 / 36.1 dB |
67
  | Real videos, VAE-encoded, 6 held-out clips | 0.0038 / 45.0 dB | 0.0330 / 34.3 dB |
68
 
69
  Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using FastVideo's fused NVFP4 kernels and batching all spatial tiles in one decoder call:
@@ -76,6 +76,12 @@ Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using Fast
76
 
77
  These are decoder-only measurements. Full video/audio generation also includes encoding, denoising and media output.
78
 
 
 
 
 
 
 
79
  ## License
80
 
81
  Use of these weights is governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE).
 
16
 
17
  # FastH3 NVFP4 VAE Decoder
18
 
19
+ Blackwell NVFP4 video decoders for MiniMax-H3 and FastH3, including FastH3 Omni. They replace the video decoder while retaining the reference encoder, audio VAE and video latent space.
20
 
21
+ [FastVideo](https://github.com/hao-ai-lab/FastVideo) 路 [FastH3 Omni decoder setup](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) 路 [FastH3 Omni release collection](https://huggingface.co/collections/FastVideo/fasth3-omniref)
22
 
23
  ## Choose a decoder
24
 
 
38
  quality-36L/decoder-nvfp4.safetensors --local-dir ./fasth3-vae
39
  ```
40
 
41
+ Follow the [FastVideo FastH3 Omni guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) and pass the downloaded file with `--decoder` to `examples/inference/basic/basic_fasth3_omniref_pdd.py`. Use `fast-8L/decoder-nvfp4.safetensors` to select the fast tier instead.
42
 
43
+ The 36-layer decoder needs a full 36-layer MiniMax-H3 host VAE. The 8-layer tier also works with the light H3 VAE. The FastH3 Omni guide explains model composition and device profiles.
44
 
45
+ For UniServe deployment of the complete FastH3 Omni model on GB200/B200, RTX PRO 6000 Blackwell or Hopper, follow the [UniServe guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md) with the [BF16 model](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1). These files are FastVideo decoder components, not complete diffusion checkpoints.
46
 
47
  ## Hardware
48
 
49
+ The FP4 decoder kernels require NVIDIA Blackwell: GB200/B200, RTX PRO 6000 Blackwell, RTX 50-series or DGX Spark (GB10). Follow the [local FastH3 Omni guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md#device-profiles) for Spark and RTX 5090 setup. Hopper and older GPUs should use the original H3 video decoder.
50
 
51
  ## Files
52
 
 
63
  | Evaluation clips | Quality tier LPIPS / PSNR | Fast tier LPIPS / PSNR |
64
  | --- | ---: | ---: |
65
  | FastH3 V2 generations, 9 clips at 480p | 0.0104 / 37.9 dB | 0.0684 / 28.5 dB |
66
+ | FastH3 Omni generations, 20 clips | 0.0039 / 46.0 dB | 0.0276 / 36.1 dB |
67
  | Real videos, VAE-encoded, 6 held-out clips | 0.0038 / 45.0 dB | 0.0330 / 34.3 dB |
68
 
69
  Warm video-decode time for one 832脳480, 124-frame clip on one GB200, using FastVideo's fused NVFP4 kernels and batching all spatial tiles in one decoder call:
 
76
 
77
  These are decoder-only measurements. Full video/audio generation also includes encoding, denoising and media output.
78
 
79
+ ## Local AI resources
80
+
81
+ Run FastH3 Omni on **DGX Spark or RTX 5090 through FastVideo** using the [FastVideo-format NVFP4 QAD weights](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD). Start with the [FastVideo cookbook](https://haoailab.com/FastVideo/cookbook/) and [FastH3 Omni recipe builder](https://haoailab.com/FastVideo/cookbook/minimax-h3/fasth3-omniref/); use the [Spark recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_spark.yaml) or [RTX 5090 recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx5090.yaml) and the [local inference guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) for the AdaLN table, offload and decoder setup.
82
+
83
+ [FP8 weights](https://huggingface.co/FastVideo/FastH3-OmniRef-FP8) are a separate FastVideo variant; the [FP8 recipe for RTX 4090](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx4090.yaml) illustrates the non-FP4 path with layerwise offload and compatible BF16 encoder components. Use each runtime's matching checkpoint format.
84
+
85
  ## License
86
 
87
  Use of these weights is governed by the [MiniMax-H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE).