Simplify the model card
Browse files
README.md
CHANGED
|
@@ -25,20 +25,16 @@ footage. The same first frame plus a different key sequence produces a different
|
|
| 25 |
| File | Description |
|
| 26 |
|---|---|
|
| 27 |
| `step-10000.safetensors` | LoRA checkpoint, step 10,000 of the released training run |
|
| 28 |
-
| `teaser.png` | Figure shown above |
|
| 29 |
|
| 30 |
## Method
|
| 31 |
|
|
|
|
|
|
|
| 32 |
Key presses are converted into one short English sentence per latent video frame (e.g. *"the man
|
| 33 |
walks forward, camera pans left sharply"*) and injected through MiniMax-H3's native text channel —
|
| 34 |
-
the same channel
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
rows recovers the base model exactly.
|
| 38 |
-
|
| 39 |
-
This checkpoint is a **rank-32 LoRA on `qkv_proj`/`out_proj`** across the 50 DiT blocks and 2
|
| 40 |
-
token-refiner layers (104 modules, 208 tensors, 65.6M parameters — 0.198% of the 33.1B-parameter
|
| 41 |
-
DiT). No action-specific module is added; the condition path itself carries zero extra weights.
|
| 42 |
|
| 43 |
Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Explorer-500h.
|
| 44 |
|
|
@@ -46,8 +42,7 @@ Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Expl
|
|
| 46 |
|
| 47 |
This checkpoint is a LoRA delta for MiniMax-H3 and requires the base model plus a directed-mask
|
| 48 |
attention patch to run (the mask is what binds each sentence to its frame; without it the LoRA
|
| 49 |
-
weights alone don't reproduce the reported behavior).
|
| 50 |
-
inference script are not published alongside this checkpoint yet.
|
| 51 |
|
| 52 |
```python
|
| 53 |
from safetensors.torch import load_file
|
|
|
|
| 25 |
| File | Description |
|
| 26 |
|---|---|
|
| 27 |
| `step-10000.safetensors` | LoRA checkpoint, step 10,000 of the released training run |
|
|
|
|
| 28 |
|
| 29 |
## Method
|
| 30 |
|
| 31 |
+
A pretrained video model already carries a language-to-motion prior from its training data, so
|
| 32 |
+
rather than learning action control from scratch, we ground keyboard input in that existing prior.
|
| 33 |
Key presses are converted into one short English sentence per latent video frame (e.g. *"the man
|
| 34 |
walks forward, camera pans left sharply"*) and injected through MiniMax-H3's native text channel —
|
| 35 |
+
the same channel it was already pretrained on. A directed attention mask binds each sentence to its
|
| 36 |
+
own video frame so the model knows which instruction applies when. The only thing trained is a
|
| 37 |
+
rank-32 LoRA on `qkv_proj`/`out_proj` (65.6M parameters, 0.198% of the 33.1B-parameter DiT).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Explorer-500h.
|
| 40 |
|
|
|
|
| 42 |
|
| 43 |
This checkpoint is a LoRA delta for MiniMax-H3 and requires the base model plus a directed-mask
|
| 44 |
attention patch to run (the mask is what binds each sentence to its frame; without it the LoRA
|
| 45 |
+
weights alone don't reproduce the reported behavior).
|
|
|
|
| 46 |
|
| 47 |
```python
|
| 48 |
from safetensors.torch import load_file
|