File size: 5,184 Bytes
9cb3b0c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cea9eef
 
 
 
9cb3b0c
 
cea9eef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9cb3b0c
cea9eef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9cb3b0c
 
 
 
 
 
 
 
 
 
 
cea9eef
9cb3b0c
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
---
license: other
license_name: minimax-h3-community-license
license_link: LICENSE
base_model: MiniMaxAI/MiniMax-H3
library_name: paiton
pipeline_tag: text-to-video
tags:
- video-generation
- audio-video-generation
- reference-to-video
- amd
- rocm
- rdna4
- gfx1201
- fp8
- paiton
---

# MiniMax H3 Ref2VA in FP8: video with sound on one R9700

Send a prompt and 1 to 9 reference pictures (or one reference video), and get back a 5.2-second 864×480 clip with
stereo sound in **30 seconds**, on one Radeon AI PRO R9700 (32 GB, 300 W).

These are the weights for the native MiniMax H3 Ref2VA runner in
[paiton-vllm-plugin](https://github.com/Eliovp-BV/paiton-vllm-plugin/tree/main/models/MiniMax-H3) (v1.2). The runner is
compiled HIP code for RDNA4, with no PyTorch, diffusers or ComfyUI inside. You call it over HTTP or from a ComfyUI node.

## Numbers

The original BF16 release is 144 GB, and its transformer alone (66.3 GB) is twice the memory of the card. This build
needs 50.9 GB on disk for one preset and peaks at 24.7 to 26.8 GiB of VRAM.

Time per request on one R9700 with the 4-step preset (Turbo4), 124 frames at 24 fps, from the prepared inputs to the
finished MP4. v1.1.0 is our earlier ComfyUI build with a W4A8 denoiser
([EliovpAI/MiniMax-H3-W4A8-Paiton-RDNA4](https://huggingface.co/EliovpAI/MiniMax-H3-W4A8-Paiton-RDNA4)) on the same
card.

| Request | v1.2 (these weights) | v1.1.0 | Faster by |
|---|---:|---:|---:|
| 864×480, 1 picture | **30.4 s** | 50.6 s | 1.7× |
| 864×480, 3 pictures | **33.4 s** | 85.3 s | 2.6× |
| 864×480, 9 pictures | **44.3 s** | 178.7 s | 4.0× |
| 1344×768, 9 pictures | **114.1 s** | 258.4 s | 2.3× |
| 864×480, 1 video with its soundtrack | **203.8 s** | not measured | |
| 864×480, 1 picture, 8 steps (Turbo8) | **51.4 s** | 109.2 s | 2.1× |

Peak VRAM for one picture at 864×480 went from 30.6 GiB (v1.1.0) to 24.8 GiB. The server loads the weights once when
it starts (76 s in our test, including a SHA-256 check of every file) and keeps them in memory between requests.

Quality, compared with the BF16 model in diffusers (MI355X, same prompts, references, seed and starting noise). CLIP-T
scores how well the clip matches its prompt:

| Request | CLIP-T, BF16 | CLIP-T, these weights |
|---|---:|---:|
| 864×480, 1 picture | 29.52 | 29.77 |
| 864×480, 3 pictures | 34.68 | 34.35 |
| 864×480, 9 pictures | 26.31 | 25.70 |
| 1344×768, 9 pictures | 25.80 | 25.71 |
| 864×480, 1 video | 29.87 | 29.86 |

Over 15 clips (two seeds, both presets) the frames have a mean similarity of 0.932 (CLIP-I) and 0.871 (DINOv2) to the
same frames of the BF16 clip, where 1.0 is the same image. With nine references the subjects are all there but can
stand in other places than in the BF16 clip. Method and limits:
[BENCHMARKS.md](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/main/models/MiniMax-H3/BENCHMARKS.md#quality-against-the-bf16-model).

## Run it

```sh
git clone --depth 1 https://github.com/Eliovp-BV/paiton-vllm-plugin.git
cd paiton-vllm-plugin
PAITON_DETACH=1 ./models/MiniMax-H3/native/launch.sh http
docker logs -f paiton-minimax-h3-v12        # wait for "READY http://..."; the first start downloads the weights
curl -o panda.mp4 -F image=@panda.png \
  -F prompt='The red panda from <Picture 1> walks through a sunlit forest. Audio: birds and a stream.' \
  http://127.0.0.1:30000/v1/videos/generations
```

The runner downloads these files on first use and checks each one against the SHA-256 in its `checkpoints.lock.json`,
so you don't need to download anything by hand. For ComfyUI, copy the `paiton_h3v_client` node from the plugin into
`custom_nodes`; it sends the job to the server, so ComfyUI itself can run on the CPU.

You need one R9700 with nothing else on it, Docker, 51 GB of disk for one preset, and about 41 GB of free host RAM for
the fastest text-encoder path (with less, the runner reads the text encoder from disk instead).

## Files

The files only work with this runner: they are packed for its kernels, not for diffusers or ComfyUI.

| Folder | Content | Size |
|---|---|---:|
| `weights/turbo4/` | transformer, 4 steps (Turbo LoRA merged, FP8) | 20.3 GB |
| `weights/turbo8/` | transformer, 8 steps (Turbo LoRA merged, FP8) | 20.5 GB |
| `dec8.bin`, `enc.bin`, `audio.bin` | video VAE decoder (INT8), video VAE encoder (BF16), audio VAE (FP32) | 3.4 GB |
| `model/tokenizer/` | tokenizer | 7 MB |

The text encoder (Qwen3-VL-32B, layers 0 to 49 in FP8, and the vision tower, 27.2 GB) is the same as the one of
MiniMax H3 Image, so it is not stored twice: the runner downloads it from the `te/` folder of
[EliovpAI/MiniMax-H3-Image-FP8-Paiton-RDNA4](https://huggingface.co/EliovpAI/MiniMax-H3-Image-FP8-Paiton-RDNA4/tree/b33b88f27c8919ca05632ccde801087796d603dd/te), pinned by revision and SHA-256.
The changes to each file are in `MODIFICATIONS.md`.

## License

MiniMax H3 Community License (`LICENSE`, `NOTICE`). The grant excludes the European Union, the United Kingdom, the
Republic of Korea and the United States, and commercial use above the agreement's revenue threshold needs MiniMax's
written authorization. Read the agreement before you download or use these files.