HINT-WAM: Hindsight-Informed Training for World Action Models
HINT-WAM post-trains the world action model Fast-WAM. During training, the model acts as its own teacher: given hindsight (the future video of a demonstration), it supervises its own action expert on the denoising states that its sampler actually visits, using an energy-score objective. At deployment, only the student is used, so the policy has exactly the architecture and inference cost of Fast-WAM and never sees the future.
- Paper: link coming soon
- Code: link coming soon
Checkpoints
| File | Benchmark | Trained from | Notes |
|---|---|---|---|
libero90/hintwam_libero90.pt |
LIBERO-90 | Fast-WAM LIBERO checkpoint, after SFT (step 3k) | main result |
config.yaml |
Stage-B training config of the LIBERO checkpoint |
Results
Success rates (%) with 50 trials per task (LIBERO-90) or 50 episodes per task and setting (RoboTwin 2.0, hard subset of 25 tasks).
| Method | LIBERO-90 | RoboTwin clean | RoboTwin randomized |
|---|---|---|---|
| Fast-WAM (released) | 15.56 | 86.48 | 83.44 |
| SFT | 78.88 | โ | โ |
| Full SFT (video + action expert) | 81.24 | 84.12 | 83.24 |
| BC control (same extra steps, no teacher) | 71.56 | 85.36 | 82.08 |
| HINT-WAM | 85.18 | 87.04 | 83.68 |
Usage
Download the config together with a checkpoint:
from huggingface_hub import hf_hub_download
repo = "<hf-username>/HINT-WAM"
config = hf_hub_download(repo, "config.yaml")
ckpt = hf_hub_download(repo, "libero90/hintwam_libero90.pt") # or robotwin/hintwam_robotwin.pt
or download everything with hf download <hf-username>/HINT-WAM --local-dir checkpoints/hintwam.
The checkpoints load exactly like the released Fast-WAM checkpoints. Closed-loop evaluation uses the LIBERO and RoboTwin evaluators of Fast-WAM with the stock settings (10 Euler steps; on LIBERO, replanning every 10 control steps) and the normalization statistics of the released Fast-WAM checkpoints.
Training
| Stage 0 (SFT) | Stage A (teacher) | Stage B (HINT-WAM) | |
|---|---|---|---|
| Init. | released Fast-WAM | SFT step 3k, frozen | SFT step 3k |
| Trainable | action expert | hindsight encoder + adapters (13M) | action expert |
| Learning rate | 1e-4 | 3e-4 | 1e-4 |
| Steps | 5,000 | 2,000 | 2,000 |
All stages: AdamW, cosine schedule, effective batch size 192, 8 GPUs (24 GB), bfloat16, DeepSpeed ZeRO-2. The video expert is frozen throughout. Stage B uses the energy score with M = 4 student samples and a behavior-cloning anchor (both weights 1). No hyperparameter was tuned on RoboTwin.
License and acknowledgements
Released under the MIT license, as is Fast-WAM. HINT-WAM builds on Fast-WAM, LIBERO and RoboTwin 2.0.
Citation
Coming soon.
- Downloads last month
- -