🦾 MolmoAct2-SO101-PickPlace-LoRA

LoRA fine-tune of allenai/MolmoAct2-SO100_101 — Ai2's 5B vision-language-action model — on the public lerobot/svla_so101_pickplace dataset. Trains on a single 48 GB GPU.

Code & reproducible pipeline: github.com/AdalricP/molmoact2-so101-lora

Results

Flow-matching loss converged by ~step 840:

step	100	500	840	1000
loss	0.204	0.060	0.044	0.045

This checkpoint is step 1000 (fully converged). Weights are the full merged model (~11.5 GB).

Training

LoRA on the VLM (flow-matching action expert fully trained) · bfloat16 · batch 16 · mean/std normalization · 1× NVIDIA A40 (~24 GB peak) · LeRobot molmoact2 policy.

Usage

lerobot-rollout --policy.path=AdalricP/molmoact2-so101-pickplace-lora \
  --robot.type=so101_follower --robot.port=/dev/ttyACM0 \
  --task="pick up the cube" --duration=30

SO-100/101 joint calibration: on LeRobot ≥ 0.5.0 set joint_signs=[1,-1,1,1,1,1], joint_offsets=[0,90,90,0,0,0] or the arm moves the wrong way.

License

Apache 2.0, following the base model. Thanks to Ai2 and LeRobot. Fine-tune on a public demo dataset — a learning project, not a production policy.