| --- |
| library_name: pytorch |
| tags: |
| - robotics |
| - world-model |
| - vla |
| - fastwam |
| - lingbot-va |
| datasets: |
| - jokeru/pick_right_purple_box_and_place_it_in_the_middle |
| --- |
| |
| # FastWAM and LingBot-VA Purple Box Checkpoints |
|
|
| Public release of two checkpoints fine-tuned on |
| `jokeru/pick_right_purple_box_and_place_it_in_the_middle`. |
|
|
| ## Contents |
|
|
| | Path | Model | Training point | Notes | |
| | --- | --- | ---: | --- | |
| | `fastwam/step_000500.pt` | memory FastWAM | step 500 | Full model checkpoint used by the local FastWAM runtime | |
| | `lingbot-va/transformer/` | old `Robbyant/lingbot-va` | step 250 | Fine-tuned transformer weights and transformer config | |
|
|
| The LingBot-VA optimizer/FSDP state (`training_state.pt`, about 40.5 GB) is |
| intentionally excluded. The uploaded transformer is sufficient for inference |
| and as a fine-tuning initialization, but still requires the upstream |
| LingBot-VA code and base Wan/VAE/text-encoder components. |
|
|
| ## Data and split |
|
|
| - Source dataset: `jokeru/pick_right_purple_box_and_place_it_in_the_middle` |
| - Training episodes: 18 |
| - Held-out episodes: 4 and 15 |
| - Task: pick the right purple box and place it in the middle |
|
|
| ## FastWAM |
|
|
| - Architecture: project memory FastWAM variant |
| - Input views: `global_primary`, `left_wrist`, `right_wrist` |
| - Video size: 384 x 320 |
| - Action/proprio schema: canonical 80D with validity masks |
| - Context length: 128 |
| - Memory history: long 8, mid 2, short 1 |
| - Optimization: 8 GPUs, batch 8/GPU, global batch 64, bf16, 500 steps |
| - Initialization: official RoboTwin FastWAM MoT checkpoint plus the Helios |
| memory patchers from memory checkpoint step 35000 |
|
|
| The detailed transfer metadata is stored in |
| `fastwam/initialization_report.json`. Paths in that report have been reduced |
| to stable checkpoint identifiers. |
|
|
| ## LingBot-VA |
|
|
| - Architecture: old `Robbyant/lingbot-va` |
| - Optimization: 8 GPUs, batch 24/GPU, global batch 192, 250 steps |
| - Fixed latent window: 16 frames |
| - Action alignment: 16 source action steps per latent frame |
| - Training episodes: 18; held-out episodes: 4 and 15 |
| - Input views: `right_eye`, `left_wrist`, `right_wrist` |
| - Compact action: right-arm joints 7D plus right gripper 1D, mapped into the |
| model's 30D action schema |
|
|
| ## Held-out open-loop results |
|
|
| These numbers are diagnostics from eight generated cases, not closed-loop |
| robot success rates. |
|
|
| | Model | Generated frames | Video PSNR | Video MAE | Action MAE | |
| | --- | ---: | ---: | ---: | ---: | |
| | FastWAM | 101 | 16.727 | 0.0776 | 0.4830 | |
| | LingBot-VA | 141 | 16.540 | 0.0913 | 0.1146 | |
|
|
| ## Loading |
|
|
| Download the FastWAM checkpoint: |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| |
| checkpoint = hf_hub_download( |
| repo_id="PencilHu/FastWAM-LingBotVA-PurpleBox", |
| filename="fastwam/step_000500.pt", |
| ) |
| ``` |
|
|
| Download the LingBot-VA transformer snapshot: |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download( |
| repo_id="PencilHu/FastWAM-LingBotVA-PurpleBox", |
| allow_patterns="lingbot-va/transformer/*", |
| ) |
| ``` |
|
|
