Beverage SmolVLA ? main action policy

Main SmolVLA action policy for a dual-arm SO-101 beverage setup. This repository contains the action policy and inference processors only. It does not contain the auxiliary progress head, optimizer state, training data, or training-state checkpoints.

Checkpoint

  • Selected by held-out action validation loss: 0.05627380684018135 at 30,000 steps.
  • Run: 20261116_052531_243914.
  • Pretrained frozen VLM: HuggingFaceTB/SmolVLM2-500M-Video-Instruct.
  • Action expert and state/action projections trained on the local beverage dataset.
  • 196 training episodes / 177,453 frames; 51 held-out episodes / 47,664 frames.
  • Validation loss is a flow-matching objective, not an action percentage error or measured robot success rate.

Inputs and outputs

  • Cameras: observation.images.left_wrist, observation.images.right_wrist, observation.images.top (RGB, 480 x 640).
  • observation.state: 12 joint/gripper values, left arm first, then right arm.
  • task: one of the English instructions below.
  • Action chunk: 50 steps, 12 joint/gripper values per step; recorded at 30 FPS.
  • State/action normalization and inverse normalization are included in the processor files.

Instructions

  1. Pick up the muddler and lower it into the cup.
  2. Lift the muddler out of the cup and return it to its holder.
  3. Pour green tea latte powder from the green-lidded container into the cup.
  4. Pour coffee powder from the red-lidded container into the cup.
  5. Pour water into the cup.

Load

Use LeRobot 0.6.1 with the SmolVLA and dataset dependencies. load_vlm_weights is false in this export because the full policy weights already contain the trained model's VLM; a separate initialization-weight download is unnecessary.

from lerobot.configs import PreTrainedConfig
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
from lerobot.policies import make_pre_post_processors

repo_id = "2chan/smolvla-beverage-main"
config = PreTrainedConfig.from_pretrained(repo_id)
config.device = "cuda"
policy = SmolVLAPolicy.from_pretrained(repo_id, config=config).eval()
preprocessor, postprocessor = make_pre_post_processors(
    policy_cfg=policy.config,
    pretrained_path=repo_id,
)
# observation includes the three RGB camera tensors, observation.state and task.
# RGB tensors must be float32 in [0, 1] before preprocessing.
batch = preprocessor(observation)
action = postprocessor(policy.select_action(batch))

Dataset sources

Downloads last month
28
Safetensors
Model size
0.5B params
Tensor type
F32
·
BF16
·
Video Preview
loading