RoboColosseum/so101-lingbot-vla-v2

LingBot VLA v2 — MolmoAct2 SO100/101 Full Fine-Tuning

• Base: robbyant/lingbot-vla-v2-6b, LingBot VLA v2 with Qwen3-VL and its action expert/MoE architecture. Model parameter names and shapes are preserved; the local implementation changes camera preprocessing and valid-camera feature assembly.
• Data: Frozen selection of 1,215 / 1,220 repositories, 37,247 episodes, and 19,515,296 frames from the locally prepared MolmoAct2 SO100/101 collection. Five incomplete repositories were excluded. Community-recovered sources and revisions are recorded in the bundle. This is not the complete 1,220-repository collection.
• State/actions: Six raw dimensions [shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper], mapped to model indices [0,1,2,3,4,28] in a 55-dimensional representation. Absolute joint targets, action chunk length 50, and masking of unused dimensions and episode-tail action padding.
• Normalization: Pinned SO100 statistics, using LingBot's bounds_99_woclip for both arm and gripper. Statistics were not recomputed over the local selection. Provenance is included in finetune_assets/norm_stats.json.
• Cameras/language: Up to three views, randomly ordered per sample access; four available views are sampled down to three. Missing camera slots are removed before vision encoding; batch features can still be padded and masked. Current/future camera selection is paired. External episode instructions are used where available, otherwise original tasks; text is stripped, lowercased, and stripped of trailing .?! characters.
• Trained: Full fine-tuning, 8 GPUs, using the fsdp2 path with module_fsdp_enable=true, vlm_fsdp=true, and enable_full_shard=false; this is not described as ordinary DDP. Global batch 128 = 16/GPU × 8 × 1 accumulation. At step 90,000 and 100,000, nominal source-sample exposure is 11,520,000 and 12,800,000 samples, or approximately 0.5903 and 0.6559 passes over the selected frame count. This is an exposure calculation, not a claim of identical packed-sample exposure to MolmoAct2.
• Optim: Muon, base LR 1e-4, cosine decay to 5e-5 over the 200,000-step target, zero warmup, weight decay 0, expert LR scaling enabled. At step 90,000 the logged base/expert LRs were 7.89e-5 / 2.23e-4. Do not confuse this run with the separate AdamW recipe.
• Loss: Logged total training loss 0.5354 → 0.0948, and VLA_Loss 0.5022 → 0.0895, comparing the individual logs at steps 1 and 90,000. These are batch measurements, not validation averages. The action objective is L1_fm; depth, future-depth, future-video, and MoE regularization terms are also logged. At step 90,000: depth 0.4263, future depth 0.5524, future video 0.0523. Loss values are not directly comparable with another policy's MSE or token cross-entropy.
• W&B: lmh_proj/lingbot-vla2-so100/0621cf5bff421c2b. The run may continue beyond this uploaded step-90,000 snapshot; access depends on the run permissions.
• Checkpoint: Intermediate snapshots at steps 90,000 and 100,000, stored in checkpoint-90000/ and checkpoint-100000/, respectively. The configured target is 200,000 steps; neither snapshot represents a completed 200k run or a validation-selected best checkpoint.
• Files: Six FP32 Safetensors shards, model.safetensors.index.json, model configuration, tokenizer/processor files, finetune_assets/, and the expanded lingbotvla_cli.yaml. Tensor payload size in the index is 25,503,630,044 bytes (about 23.75 GiB); the HF export directory is about 23.77 GiB. This is a full model export, not a LoRA adapter.