File size: 3,152 Bytes
952993c b11372b 952993c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
tags:
- robotics
- video-generation
- vision-language-action
- fruit-picking
library_name: pytorch
---
# Fruit-picking Flashwam
Public archival bundle for the fruit-picking model trained by Amin.
## Status
This is a **partial checkpoint: step 12,600 of 18,900 (epoch 20 of 30)**. The job terminated when its shared `/dev/shm`
video staging disappeared; it did not finish the planned 30 epochs. The model
has not been evaluated on a robot, and the physical meaning of the `-1/+1`
gripper polarity still needs confirmation.
Architecture: FlashWAM M1 fused-KV, fixed-RoPE, 1-layer action expert.
Checkpoints follow the same layout as the existing WAM repositories:
- `checkpoints/weights/step_003150.pt` — SHA-256 `2dc3d4ce58a592944975fc6aa2214f3f22cf724b08fa3815588d4d83319e31b0`
- `checkpoints/weights/step_006300.pt` — SHA-256 `660268a99a6e298ad5e3948f4784e7d8be6b4e0ad0517d9404b52c1dc7797145`
- `checkpoints/weights/step_009450.pt` — SHA-256 `dc0497a96a8c2c7bcf072d00e700302707f260bf6be0c7fa77567520f101ca01`
- `checkpoints/weights/step_012600.pt` — SHA-256 `96c9587141286f2bef8f7c38bc31816e5f9ce689f3c66500b5fdf1d6f3b83a06`
## Conditioning
Exact task text:
> Lift the lid, put it aside, and pick the black plum.
`conditioning/text_embedding.pt` is the exact cached T5 embedding consumed
during training. It was generated with the Wan text stack, context length 128,
using `Wan-AI/Wan2.1-T2V-1.3B` as the tokenizer model reference. The resolved
training config sets `load_text_encoder: false`, so this cached tensor is part
of the required inference bundle.
## Input processing and normalization
- Two 256x256 RGB cameras (`agentview`, then `wrist`).
- Each camera is converted to a tensor and resized to 224x224.
- Cameras are concatenated horizontally to 224x448.
- Horizon: 33 observations; 32 action transitions at 10 Hz.
- Original 15-D state was converted to 8-D:
`eef_xyz(3) + quat-to-axis-angle(3) + [gripper_width/2, -gripper_width/2]`.
- Action is 7-D: delta XYZ, delta rotation XYZ, and gripper.
- Delta/padding mask is `[true, true, true, true, true, true, false]`; the
gripper channel is absolute rather than delta.
- `dataset_stats.json` contains the exact min/max normalization statistics
used by this run.
## Attention masks
The resolved model uses:
- `video_attention_mask_mode: first_frame_causal`: first-frame queries cannot
attend to later video frames; later-frame queries can attend to all video
tokens.
- `action_group_causal_mask_mode: group_diagonal`: each video temporal group
attends only to the corresponding action-token group.
- Text cross-attention is enabled for the action expert.
The exact implementations and preprocessing classes are included under
`training_code/`; the resolved config is `config.yaml`.
No license is asserted here for the bundled upstream code; its original terms
continue to apply.
## Base components
This weights-only checkpoint is not standalone. It references
`Wan-AI/Wan2.2-TI2V-5B` and requires the matching Wan VAE plus the included
FastWAM code/configuration. PyTorch `.pt` files may contain pickled objects;
load only in a trusted environment.
|