LALABAB/VLA-Bind-adapters

VLA-Bind trained adapters

These are the policies trained for the paper Lost Among Look-Alikes: A Benchmark for Attribute Binding in Vision-Language-Action Policies (under review). The training data and the benchmark are in LALABAB/VLA-Bind.

Every saved training step is included. Optimizer states used to resume training are not.

Rows

Folders are <model>/<training recipe>/seed_<n>/step_<steps>/. Each recipe is one model row of the paper.

Recipe	Model	What it is
train_bind	OpenVLA-7B, LoRA	Trained on all of Train-Bind, 4,860 demonstrations, 20,000 steps
counterfactual_copies	OpenVLA-7B, LoRA	Train-Bind plus one targeted counterfactual copy of every demonstration: same start state and arm motion, with the look and the words changed so that the copy covers a gap of train_bind. 9,720 demonstrations
photometric_copies	OpenVLA-7B, LoRA	Train-Bind plus one photometric copy of every demonstration (brightness, contrast, crop; no hue change). The control for counterfactual_copies: same number of frames, no counterfactual
train_bind_continued	OpenVLA-7B, LoRA	train_bind continued from step 20,000 to 25,000 on the same data. The control for rewritten_instructions and pointer_head
rewritten_instructions	OpenVLA-7B, LoRA	train_bind continued from step 20,000 to 25,000 with the rewritten instructions of Train-Bind (train_bind/instruction_rewrites/ in the dataset)
pointer_head	OpenVLA-7B, LoRA	train_bind continued from step 20,000 to 25,000 with an auxiliary pointer head that marks the named object (pointer_head.pt beside each step)
train_bind_720_demos	OpenVLA-7B, LoRA	The train_bind recipe on 720 demonstrations of Train-Bind (train_bind/subsets/ in the dataset), 20,000 steps. Compared with train_bind, it shows what more data changes
train_bind, counterfactual_copies	pi0.5	The same two recipes on pi0.5; merged_step_020000 holds merged weights

Released checkpoints in this repository (each holds every saved step):

• openvla/counterfactual_copies/seed_1
• openvla/counterfactual_copies/seed_2
• openvla/photometric_copies/seed_1
• openvla/pointer_head/seed_1
• openvla/rewritten_instructions/seed_1
• openvla/rewritten_instructions/seed_2
• openvla/train_bind/seed_1
• openvla/train_bind/seed_2
• openvla/train_bind_720_demos/seed_8
• openvla/train_bind_continued/seed_1
• openvla/train_bind_continued/seed_2
• pi05/counterfactual_copies/seed_1
• pi05/train_bind/seed_1
• pi05/counterfactual_copies/seed_1/merged_step_020000
• pi05/train_bind/seed_1/merged_step_020000

Use

The OpenVLA adapters apply to openvla/openvla-7b (named in each adapter_config.json); load the base model, then the adapter with PEFT. dataset_statistics.json beside each step holds the action statistics used to un-normalise actions. Scenes and scoring are described in the dataset card.

Licence

The adapters follow the licences of their base models: the Llama 2 Community Licence for the OpenVLA rows and the Gemma terms of use for the pi0.5 rows. See LICENSE_NOTE.md.

Citation

Under review. A citation will be added after the review.