UR3-organize-lingbotva-1000step — LingBot-VA bimanual UR3 transformer

Fine-tuned transformer for LingBot-VA on a bimanual UR3 robot, task EmbodyX/UR3/organize_pi05: "sort the tools into their matching containers" (78 demos, 118,211 action rows, 15 fps source, 3 cameras).

This is LingBot-VA, not pi-zero. The pi05 in the source dataset folder name reflects that the demos were originally collected for the π0.5 project; we re-encoded them through our pipeline and fine-tuned LingBot-VA on them.

• Base: robbyant/lingbot-va-base
• Robot: bimanual UR3 (2× 6-DoF + 2 binary grippers; 14-dim raw action [L_arm(6), L_grip, R_arm(6), R_grip] mapped into 30-dim model space via used_action_channel_ids = [0..5, 28, 6..11, 29]).
• Cameras: camera_top, camera_wrist_left, camera_wrist_right (concatenated along width, 256×320 per cam after resize from 480×640).
• 4 GPUs × grad_accum=4 = effective batch 16, optimizer step 1000 of a 5000-step schedule.

Recipe — every G1 lesson applied upfront

• FDM v2 (lambda_fdm=1.0, fdm_prob=0.5, mutually-exclusive per-microstep rank-synced coin: FDM video-only L_fdm Eq.13 OR standard IDM L_dyn+L_inv; one forward, one backward) — randomized chunk_size ∈ {1..4} and window_size ∈ {4..64} per step (the trainer's hardcoded randomization).
• Action normalization: MIN/MAX, zero-inclusive bounds. Three combined fixes vs the put_away_tools v21 trap: (1) min/max instead of quantile q01/q99 — bounds real data to [-1, +1] (2) per-channel range EXPANDED to include 0 — the loader prepends frame_stride*4 = 16 zero action-rows per episode (conditioning context for the first model frame). UR3 joint angles often live entirely positive or negative (unlike G1 joints which span 0 at the home pose), so padded zeros would normalize to ±5 and crash FSDP backward without this expansion. (3) grippers pinned to [0, 1] (natural binary range). Verified: real-data absmax = 1.0, padded zero rows also normalize to exactly [-1, +1].
• FRAME_STRIDE=4 (not 2). UR3 organize episodes run up to 100 s @ 15 fps = 2830 raw frames; at stride=2 the max latent frame count (354) OOMs the H100 NVL during FSDP backward (~93 GB). Stride=4 caps it at 177 latent frames, fits cleanly. Trade-off: each latent frame spans ~1.07 s of motion (vs 0.53 s at stride=2). action_per_frame = 16, matches all G1 task configs.

Assemble an eval-ready checkpoint

hf download robbyant/lingbot-va-base                       --local-dir lingbot-va-base
hf download EmbodyX/UR3-organize-lingbotva-1000step        --local-dir ur3_org_1000_dl

mkdir -p ur3_org_1000
ln -sf $(realpath ur3_org_1000_dl/transformer)  ur3_org_1000/transformer
ln -sf $(realpath lingbot-va-base/vae)          ur3_org_1000/vae
ln -sf $(realpath lingbot-va-base/text_encoder) ur3_org_1000/text_encoder
ln -sf $(realpath lingbot-va-base/tokenizer)    ur3_org_1000/tokenizer

Serve with CONFIG_NAME=ur3_organize MODEL_PATH=ur3_org_1000. transformer/config.json has attn_mode: torch (inference-ready). The matching va_ur3_organize_cfg.py in the lingbot-va repo must have the min/max + zero-inclusion norm_stat (NOT quantile) — using a mismatched norm_stat at inference will denormalize actions incorrectly.