yashwanthgowdanm/abg-stage1-openvla

Model Card for OpenVLA-7B Stretch LoRA

This model is a Low-Rank Adaptation (LoRA) fine-tune of the openvla/openvla-7b Vision-Language-Action (VLA) model, specifically adapted for the Hello Robot Stretch platform. It translates natural language instructions and RGB image observations into continuous 7-Degree-of-Freedom (7-DoF) physical robot actions.

Model Details

Model Description

This model adapts the general-purpose OpenVLA-7B base model to the specific kinematics and workspace of a Hello Robot Stretch. It employs a parameter-efficient fine-tuning (PEFT) strategy, training only ~33.5 million parameters (0.44% of the base model) via LoRA. The model predicts normalized continuous actions in a 7-DoF space based on user instructions and visual input.

• Developed by: Yashwanth Gowda
• Model type: Vision-Language-Action (VLA) Model (LoRA Adapter)
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: openvla/openvla-7b

Uses

Direct Use

The model is designed for direct zero-shot inference on a physical Hello Robot Stretch. It accepts a 640x480 RGB image observation and a text instruction (e.g., "pick up the water bottle") and autoregressively generates action tokens. These tokens are detokenized into continuous physical actions for the robot's base, arm, and gripper.

Out-of-Scope Use

This model is strictly fine-tuned for the physical capabilities of the Hello Robot Stretch. It is not intended for use with high-payload industrial arms (e.g., UR5, Franka) or quadrupedal robots without further domain adaptation.

Training Details

Training Data

The model was fine-tuned on a custom behavioral cloning dataset collected via teleoperation on the Hello Robot Stretch platform[cite: 3].

• Total Episodes: 52
• Total State-Action Transitions: 10,652
• Action Space: Natively 6-DoF (arm_extend, lift_z, wrist_roll, wrist_pitch, wrist_yaw, gripper_pos), mathematically padded to 7-DoF with a static lateral Y-axis to match OpenVLA's pre-trained architecture[cite: 3, 2].
• Normalization: Continuous actions were min-max normalized to a [-1, 1] continuous space and discretized into 256 Llama tokenizer bins[cite: 3].

Training Procedure

Training Hyperparameters

• Training regime: bf16 mixed precision
• Optimizer: adamw_torch
• Learning Rate: 2e-4
• Epochs: 5
• Batch Size: 4 (per device)
• Gradient Accumulation Steps: 4
• LoRA Rank (r): 32
• LoRA Alpha: 64
• LoRA Dropout: 0.05
• Target Modules: q_proj, v_proj, k_proj, o_proj

Speeds, Sizes, Times

• Total Trainable Parameters: 33,554,432
• Total Model Parameters: 7,574,791,616
• Training Time: ~1 hour 57 minutes (3,325 optimization steps)
• Final Training Loss: 0.5316

Technical Specifications

Compute Infrastructure

The model was trained on the Sol Supercomputer (Arizona State University).

Hardware

• GPU: 1x NVIDIA A100 (64GB VRAM)
• CPU: 8 cores

Software

• PyTorch: 2.2.0+cu121
• Transformers: 4.40.1
• PEFT: 0.11.1
• Flash-Attention: 2.8.3

Framework versions

• PEFT 0.11.1