DANNY621 commited on
Commit
2f7bdcc
·
verified ·
1 Parent(s): 1f53b11

Simplify the model card

Browse files
Files changed (1) hide show
  1. README.md +6 -11
README.md CHANGED
@@ -25,20 +25,16 @@ footage. The same first frame plus a different key sequence produces a different
25
  | File | Description |
26
  |---|---|
27
  | `step-10000.safetensors` | LoRA checkpoint, step 10,000 of the released training run |
28
- | `teaser.png` | Figure shown above |
29
 
30
  ## Method
31
 
 
 
32
  Key presses are converted into one short English sentence per latent video frame (e.g. *"the man
33
  walks forward, camera pans left sharply"*) and injected through MiniMax-H3's native text channel —
34
- the same channel the model was already pretrained on. A **directed attention mask** binds annotation
35
- *k* to video frame *k* only: frame *k* can read annotation *k*, but annotation *k* cannot read any
36
- other annotation, which closes a bypass a symmetric mask would leave open. Removing the annotation
37
- rows recovers the base model exactly.
38
-
39
- This checkpoint is a **rank-32 LoRA on `qkv_proj`/`out_proj`** across the 50 DiT blocks and 2
40
- token-refiner layers (104 modules, 208 tensors, 65.6M parameters — 0.198% of the 33.1B-parameter
41
- DiT). No action-specific module is added; the condition path itself carries zero extra weights.
42
 
43
  Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Explorer-500h.
44
 
@@ -46,8 +42,7 @@ Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Expl
46
 
47
  This checkpoint is a LoRA delta for MiniMax-H3 and requires the base model plus a directed-mask
48
  attention patch to run (the mask is what binds each sentence to its frame; without it the LoRA
49
- weights alone don't reproduce the reported behavior). Code, the framework patch, and a minimal
50
- inference script are not published alongside this checkpoint yet.
51
 
52
  ```python
53
  from safetensors.torch import load_file
 
25
  | File | Description |
26
  |---|---|
27
  | `step-10000.safetensors` | LoRA checkpoint, step 10,000 of the released training run |
 
28
 
29
  ## Method
30
 
31
+ A pretrained video model already carries a language-to-motion prior from its training data, so
32
+ rather than learning action control from scratch, we ground keyboard input in that existing prior.
33
  Key presses are converted into one short English sentence per latent video frame (e.g. *"the man
34
  walks forward, camera pans left sharply"*) and injected through MiniMax-H3's native text channel —
35
+ the same channel it was already pretrained on. A directed attention mask binds each sentence to its
36
+ own video frame so the model knows which instruction applies when. The only thing trained is a
37
+ rank-32 LoRA on `qkv_proj`/`out_proj` (65.6M parameters, 0.198% of the 33.1B-parameter DiT).
 
 
 
 
 
38
 
39
  Trained on 7,872 clips (832x480, 124 frames / 5.2s @ 24fps) from ABot-World-Explorer-500h.
40
 
 
42
 
43
  This checkpoint is a LoRA delta for MiniMax-H3 and requires the base model plus a directed-mask
44
  attention patch to run (the mask is what binds each sentence to its frame; without it the LoRA
45
+ weights alone don't reproduce the reported behavior).
 
46
 
47
  ```python
48
  from safetensors.torch import load_file