Instructions to use ModelsLab/MiniMax-H3-Turbo-RL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ModelsLab/MiniMax-H3-Turbo-RL with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ModelsLab/MiniMax-H3-Turbo-RL") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Add before/after sample videos (public CC BY sources) to the model card
Browse files- .gitattributes +8 -0
- README.md +18 -0
- samples/budapest_after_v4_rl_e4a.mp4 +3 -0
- samples/budapest_before_v4.mp4 +3 -0
- samples/flower_market_after_v4_rl_e4a.mp4 +3 -0
- samples/flower_market_before_v4.mp4 +3 -0
- samples/office_dialogue_after_v4_rl_e4a.mp4 +3 -0
- samples/office_dialogue_before_v4.mp4 +3 -0
- samples/voladores_after_v4_rl_e4a.mp4 +3 -0
- samples/voladores_before_v4.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
samples/budapest_after_v4_rl_e4a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
samples/budapest_before_v4.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
samples/flower_market_after_v4_rl_e4a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
samples/flower_market_before_v4.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
samples/office_dialogue_after_v4_rl_e4a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
samples/office_dialogue_before_v4.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
samples/voladores_after_v4_rl_e4a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
samples/voladores_before_v4.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -45,6 +45,24 @@ Load it with any loader that accepts Turbo V4's file (same keys, larger rank). S
|
|
| 45 |
|
| 46 |
Honest attribution: on this public set almost all of the gain over Turbo V4 comes from the E4a sampler; rl_s3 matches V4 + E4a within noise on these metrics (closer to the real clip than V4 + E4a on Lab b for 11/16 clips and on saturation for 8/16). Measured with a BF16 reference pipeline, not with the MXFP8 production engine.
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
## Training data and licences
|
| 49 |
|
| 50 |
The RL stages (rl_s0 → rl_s3) used short clips (only their cached latents and the prompts written for them) from:
|
|
|
|
| 45 |
|
| 46 |
Honest attribution: on this public set almost all of the gain over Turbo V4 comes from the E4a sampler; rl_s3 matches V4 + E4a within noise on these metrics (closer to the real clip than V4 + E4a on Lab b for 11/16 clips and on saturation for 8/16). Measured with a BF16 reference pipeline, not with the MXFP8 production engine.
|
| 47 |
|
| 48 |
+
## Samples: before / after
|
| 49 |
+
|
| 50 |
+
Same prompt, references and seed (11) in each row. **Before:** Turbo V4, 8-step Euler (the previous production setup). **After:** this file (Turbo V4 + rl_s3) with the E4a 10-NFE sampler. Rendered at 1344×768 with the BF16 reference pipeline (not the MXFP8 production engine). Prompts and reference images come from the public clips credited below; the videos and sound are generated.
|
| 51 |
+
|
| 52 |
+
| | Before: Turbo V4, 8 steps | After: Turbo V4 + RL, E4a |
|
| 53 |
+
| --- | --- | --- |
|
| 54 |
+
| Flower market (multi-subject, 5 s) | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/flower_market_before_v4.mp4"></video> | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/flower_market_after_v4_rl_e4a.mp4"></video> |
|
| 55 |
+
| Danza de los Voladores (portrait action, 10 s) | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/voladores_before_v4.mp4"></video> | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/voladores_after_v4_rl_e4a.mp4"></video> |
|
| 56 |
+
| View from Fisherman's Bastion (ambience, 10 s) | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/budapest_before_v4.mp4"></video> | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/budapest_after_v4_rl_e4a.mp4"></video> |
|
| 57 |
+
| Office dialogue (speech, 5 s) | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/office_dialogue_before_v4.mp4"></video> | <video controls width="100%" src="https://huggingface.co/ModelsLab/MiniMax-H3-Turbo-RL/resolve/main/samples/office_dialogue_after_v4_rl_e4a.mp4"></video> |
|
| 58 |
+
|
| 59 |
+
Source clips (used only to write the prompt and pick reference frames):
|
| 60 |
+
|
| 61 |
+
- Flower market: Netflix Open Content, source file `96x2160 60fps 10bit 420`, CC BY 4.0
|
| 62 |
+
- Danza de los Voladores: Wikimedia Commons, *Danza de los Voladores (Dance of the Flyers)* by Emma Matthies, CC BY 4.0
|
| 63 |
+
- View from Fisherman's Bastion: Wikimedia Commons, *View from Fisherman's Bastion (2)* by Elekes Andor, CC BY 4.0
|
| 64 |
+
- Office dialogue: Netflix Open Content, source file `3840x2160 59.94fps SDR`, CC BY 4.0
|
| 65 |
+
|
| 66 |
## Training data and licences
|
| 67 |
|
| 68 |
The RL stages (rl_s0 → rl_s3) used short clips (only their cached latents and the prompts written for them) from:
|
samples/budapest_after_v4_rl_e4a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:301d0957db6bf34f4276ad59020bfc1c05aa9e2744a607e84eeaf11dbdb7e51e
|
| 3 |
+
size 5862056
|
samples/budapest_before_v4.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b6e15e5cdd350f27db0394e5fb6d6a4cc234d6b1f15e4720ea7f6e65274d93f9
|
| 3 |
+
size 5911782
|
samples/flower_market_after_v4_rl_e4a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a03540ffdb28fcb0fdecd742347946b70a0353f6e20426346405959cfb15b842
|
| 3 |
+
size 7659468
|
samples/flower_market_before_v4.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c6b4e50e0b7d3e036b9167eebd9eb9e7e2935b3c75067179f8bdaa16d7310c4a
|
| 3 |
+
size 7605016
|
samples/office_dialogue_after_v4_rl_e4a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef4958dd31bb8af0c23698aa57ad2c3710de7a71a26ccb1241259149569d25cf
|
| 3 |
+
size 2219977
|
samples/office_dialogue_before_v4.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e489154bac567f826f6b227047525ae47ae89a8f7c81fc40a01c69a52d174094
|
| 3 |
+
size 2249861
|
samples/voladores_after_v4_rl_e4a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3987ea3a512f70d9a75d6b32cb39ef7a883dddbf9b54d7e806fce7baf3c40c7f
|
| 3 |
+
size 15335846
|
samples/voladores_before_v4.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0fd22e2cfbf3017873b31d9fd4be3fa65e7d1a8a24085d20013803ec72dfaa7e
|
| 3 |
+
size 15750715
|