Instructions to use 2chan/smolvla-beverage-main with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use 2chan/smolvla-beverage-main with LeRobot:
# See https://github.com/huggingface/lerobot?tab=readme-ov-file#installation for more details git clone https://github.com/huggingface/lerobot.git cd lerobot pip install -e .[smolvla]
# Launch finetuning on your dataset python lerobot/scripts/train.py \ --policy.path=2chan/smolvla-beverage-main \ --dataset.repo_id=lerobot/svla_so101_pickplace \ --batch_size=64 \ --steps=20000 \ --output_dir=outputs/train/my_smolvla \ --job_name=my_smolvla_training \ --policy.device=cuda \ --wandb.enable=true
# Run the policy using the record function python -m lerobot.record \ --robot.type=so101_follower \ --robot.port=/dev/ttyACM0 \ # <- Use your port --robot.id=my_blue_follower_arm \ # <- Use your robot id --robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras --dataset.single_task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording --dataset.repo_id=HF_USER/dataset_name \ # <- This will be the dataset name on HF Hub --dataset.episode_time_s=50 \ --dataset.num_episodes=10 \ --policy.path=2chan/smolvla-beverage-main - Notebooks
- Google Colab
- Kaggle
Beverage SmolVLA ? main action policy
Main SmolVLA action policy for a dual-arm SO-101 beverage setup. This repository contains the action policy and inference processors only. It does not contain the auxiliary progress head, optimizer state, training data, or training-state checkpoints.
Checkpoint
- Selected by held-out action validation loss: 0.05627380684018135 at 30,000 steps.
- Run:
20261116_052531_243914. - Pretrained frozen VLM: HuggingFaceTB/SmolVLM2-500M-Video-Instruct.
- Action expert and state/action projections trained on the local beverage dataset.
- 196 training episodes / 177,453 frames; 51 held-out episodes / 47,664 frames.
- Validation loss is a flow-matching objective, not an action percentage error or measured robot success rate.
Inputs and outputs
- Cameras:
observation.images.left_wrist,observation.images.right_wrist,observation.images.top(RGB, 480 x 640). observation.state: 12 joint/gripper values, left arm first, then right arm.task: one of the English instructions below.- Action chunk: 50 steps, 12 joint/gripper values per step; recorded at 30 FPS.
- State/action normalization and inverse normalization are included in the processor files.
Instructions
- Pick up the muddler and lower it into the cup.
- Lift the muddler out of the cup and return it to its holder.
- Pour green tea latte powder from the green-lidded container into the cup.
- Pour coffee powder from the red-lidded container into the cup.
- Pour water into the cup.
Load
Use LeRobot 0.6.1 with the SmolVLA and dataset dependencies. load_vlm_weights is false in this export because the full policy weights already contain the trained model's VLM; a separate initialization-weight download is unnecessary.
from lerobot.configs import PreTrainedConfig
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
from lerobot.policies import make_pre_post_processors
repo_id = "2chan/smolvla-beverage-main"
config = PreTrainedConfig.from_pretrained(repo_id)
config.device = "cuda"
policy = SmolVLAPolicy.from_pretrained(repo_id, config=config).eval()
preprocessor, postprocessor = make_pre_post_processors(
policy_cfg=policy.config,
pretrained_path=repo_id,
)
# observation includes the three RGB camera tensors, observation.state and task.
# RGB tensors must be float32 in [0, 1] before preprocessing.
batch = preprocessor(observation)
action = postprocessor(policy.select_action(batch))
Dataset sources
- lmsforAI/so101_muddler_insert_p1_merged50_20261109_masked @
b6736f38109b08b912dbed7642a5eb8df1f7b6a8 - lmsforAI/so101_muddler_retrieve_p1_merged50_20261109_masked @
ba4326800e64a54d1ac597edb0fbe58ac3350a06 - lmsforAI/so101_powder_greentea_v2_merged35_20261019 @
f63e0b1d70f93df3a5eea6ac4c18a62e643a5fcc - lmsforAI/so101_powder_coffee_v2_merged35_20261019 @
d044f10904c094370add1abdbc0eb5ae2b0ab502 - lmsforAI/so101_water_v2_merged54_20261020 @
fe4c0855ef99785b3b89567828b520b6833b3bb0 - lmsforAI/so101_muddler_p1_supplement_insert_20261115_masked @
1685ece7aafca39e4fee935cd4fd02f7d441e161 - lmsforAI/so101_muddler_p1_supplement_retrieve_20261115_masked @
71506b59d9ecc0de38aebb36bacf0b8171c47602 - lmsforAI/so101_muddler_p1_supplement_recovery_20261115_masked @
e775c55b8470b32e8f8484e98a771a2f0b8a93d3
- Downloads last month
- 28