| --- |
| license: apache-2.0 |
| pipeline_tag: robotics |
| library_name: lerobot |
| tags: |
| - act |
| - so101 |
| - real-robot |
| - manipulation |
| - pick-and-place |
| - lerobot |
| datasets: |
| - wsagi/leisaac-real-pick-orange |
| base_model: [] |
| --- |
| |
| # ACT-Real-PickOrange — SO-101 真机抓橙子(sim-warm-start,ACT 真机 best) |
|
|
| 真机 SO-101 pick-orange 的 **ACT 策略 best checkpoint**:先在仿真(Isaac Lab `LeIsaac-SO101-PickOrange-v0`)训到 sim leaderboard best,再用 [wsagi/leisaac-real-pick-orange](https://huggingface.co/datasets/wsagi/leisaac-real-pick-orange)(30 集真机遥操演示)**warm-start 续训 5000 步(1.6 epoch)**。 |
|
|
| ACT policy for real-world SO-101 pick-and-place: warm-started from our sim-best ACT checkpoint, then fine-tuned for 5000 steps (1.6 epochs) on 30 real teleoperation episodes. **Sim-warm-start is confirmed effective on the real arm** — nearly 2× the from-scratch baseline. |
|
|
| <video src="https://huggingface.co/wsagi/ACT-Real-PickOrange/resolve/main/ACT-Real-PickOrange.mp4" controls muted loop></video> |
|
|
| ## 真机结果 / Real-arm results(2026-07-12) |
|
|
| 每轮场景放 3 只橙子,记每轮**放入盘中的橙子数**。Each round has 3 oranges on the table; we count oranges placed on the plate per round. |
|
|
| | 策略 / Policy | 轮次成绩 / Per-round | 均值 / Mean | |
| |---|---|---| |
| | **本模型(simwarm 5000)/ this model** | 2, 1, 1, 3, 2, 1, 0, 1, 2, 2(10 轮) | **1.5 只/轮** | |
| | 从零 ACT 6000 / from-scratch baseline | 2, 0, 1, 1, 0(5 轮) | 0.8 只/轮 | |
|
|
| - simwarm 放入 ≥1 只的轮 9/10,含一轮 3 只满放;从零版仅 3/5。 |
| - **结论:基于仿真 best 续训(sim-warm-start)有效** — 开环 MSE 低 ~40% 的增益在真机变现,均值近 2 倍。同数据的 FlowDP / GR00T-N1.7 头只能抓不能稳放,判别头(ACT)最强。 |
|
|
| ## 场景 / Scene |
|
|
| 顶部相机视角(真机布置)/ top-camera view of the real setup: |
|
|
|  |
|
|
| ## 配方 / Recipe |
|
|
| | 项 | 值 | |
| |---|---| |
| | 架构 / Arch | ACT(lerobot,chunk_size=100,n_action_steps=100) | |
| | init | sim-best ACT(Isaac Lab 仿真 leaderboard best,20k 步) | |
| | 数据 / Data | [wsagi/leisaac-real-pick-orange](https://huggingface.co/datasets/wsagi/leisaac-real-pick-orange) 30 集 / 25,091 帧 @30fps | |
| | 续训 / Fine-tune | 5000 步(1.6 ep),lr 1e-5,batch 8,MEAN_STD + ImageNet norm | |
| | 归一化 / Norm stats | lerobot 0.5.2 warm-start 自动按真机数据集重建(sim stats 不残留) | |
| | 观测 / Obs | `front` + `wrist` 相机 640×480 + 6 维关节位置 | |
|
|
| ## 用法 / Usage |
|
|
| ```python |
| from lerobot.policies.act.modeling_act import ACTPolicy |
| |
| policy = ACTPolicy.from_pretrained("wsagi/ACT-Real-PickOrange") |
| ``` |
|
|
| ## 相关项目 / Related projects |
|
|
| - <https://github.com/vitorcen/isaaclab-experience> — 伞仓:多 VLA 基线训练/评测全记录 |
| - <https://github.com/vitorcen/LeIsaac-Training> — LeIsaac:SO-101 PickOrange 训练 / benchmark / 真机脚手架 |
|
|
