README.md

metadata

license: apache-2.0
pipeline_tag: robotics
library_name: lerobot
tags:
  - act
  - so101
  - real-robot
  - manipulation
  - pick-and-place
  - lerobot
datasets:
  - wsagi/leisaac-real-pick-orange
base_model: []

ACT-Real-PickOrange — SO-101 真机抓橙子(sim-warm-start,ACT 真机 best)

真机 SO-101 pick-orange 的 ACT 策略 best checkpoint:先在仿真(Isaac Lab LeIsaac-SO101-PickOrange-v0)训到 sim leaderboard best,再用 wsagi/leisaac-real-pick-orange(30 集真机遥操演示)**warm-start 续训 5000 步(1.6 epoch)**。

ACT policy for real-world SO-101 pick-and-place: warm-started from our sim-best ACT checkpoint, then fine-tuned for 5000 steps (1.6 epochs) on 30 real teleoperation episodes. Sim-warm-start is confirmed effective on the real arm — nearly 2× the from-scratch baseline.

真机结果 / Real-arm results(2026-07-12)

每轮场景放 3 只橙子,记每轮放入盘中的橙子数。Each round has 3 oranges on the table; we count oranges placed on the plate per round.

策略 / Policy	轮次成绩 / Per-round	均值 / Mean
本模型(simwarm 5000)/ this model	2, 1, 1, 3, 2, 1, 0, 1, 2, 2(10 轮)	1.5 只/轮
从零 ACT 6000 / from-scratch baseline	2, 0, 1, 1, 0(5 轮)	0.8 只/轮

• simwarm 放入 ≥1 只的轮 9/10,含一轮 3 只满放;从零版仅 3/5。
• 结论:基于仿真 best 续训(sim-warm-start)有效 — 开环 MSE 低 ~40% 的增益在真机变现,均值近 2 倍。同数据的 FlowDP / GR00T-N1.7 头只能抓不能稳放,判别头(ACT)最强。

场景 / Scene

顶部相机视角(真机布置)/ top-camera view of the real setup:

配方 / Recipe

项	值
架构 / Arch	ACT(lerobot,chunk_size=100,n_action_steps=100)
init	sim-best ACT(Isaac Lab 仿真 leaderboard best,20k 步)
数据 / Data	wsagi/leisaac-real-pick-orange 30 集 / 25,091 帧 @30fps
续训 / Fine-tune	5000 步(1.6 ep),lr 1e-5,batch 8,MEAN_STD + ImageNet norm
归一化 / Norm stats	lerobot 0.5.2 warm-start 自动按真机数据集重建(sim stats 不残留)
观测 / Obs	front + wrist 相机 640×480 + 6 维关节位置

用法 / Usage

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("wsagi/ACT-Real-PickOrange")

相关项目 / Related projects

• https://github.com/vitorcen/isaaclab-experience — 伞仓:多 VLA 基线训练/评测全记录
• https://github.com/vitorcen/LeIsaac-Training — LeIsaac:SO-101 PickOrange 训练 / benchmark / 真机脚手架