Full-Stacks commited on
Upload RoboTwin ACT model artifacts
Browse files- README.md +76 -0
- dataset_stats.pkl +3 -0
- policy_best.ckpt +3 -0
- policy_last.ckpt +3 -0
- results.json +27 -0
- train_val_kl_seed_0.png +0 -0
- train_val_l1_seed_0.png +0 -0
- train_val_loss_seed_0.png +0 -0
README.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- zh
|
| 4 |
+
tags:
|
| 5 |
+
- robotwin
|
| 6 |
+
- embodied-ai
|
| 7 |
+
- robotics
|
| 8 |
+
- bimanual-manipulation
|
| 9 |
+
- imitation-learning
|
| 10 |
+
- act
|
| 11 |
+
- pytorch
|
| 12 |
+
library_name: pytorch
|
| 13 |
+
license: other
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# RoboTwin ACT on beat_block_hammer demo_clean-50
|
| 17 |
+
|
| 18 |
+
这是一次基于 RoboTwin 官方代码仓库的单任务具身智能闭环实验模型产物。
|
| 19 |
+
|
| 20 |
+
- 平台版本:RoboTwin `main`
|
| 21 |
+
- 上游提交:`958a6d2910a0262f5531fcdeb7fffae4184bb586`
|
| 22 |
+
- 任务:`beat_block_hammer`
|
| 23 |
+
- 数据配置:`demo_clean`
|
| 24 |
+
- 专家轨迹数:`50`
|
| 25 |
+
- 策略:`ACT`
|
| 26 |
+
|
| 27 |
+
## 仓库内容
|
| 28 |
+
|
| 29 |
+
- `policy_best.ckpt`
|
| 30 |
+
- `policy_last.ckpt`
|
| 31 |
+
- `dataset_stats.pkl`
|
| 32 |
+
- 训练曲线图
|
| 33 |
+
- `results.json`
|
| 34 |
+
|
| 35 |
+
## 实验设置
|
| 36 |
+
|
| 37 |
+
- 数据来源:RoboTwin 官方 `collect_data.sh` 流程生成的 `demo_clean 50` 专家轨迹
|
| 38 |
+
- 预处理:RoboTwin 官方 ACT `process_data.sh`
|
| 39 |
+
- 训练脚本:RoboTwin 官方 ACT `train.sh`
|
| 40 |
+
- 训练参数:
|
| 41 |
+
- `batch_size=8`
|
| 42 |
+
- `num_epochs=6000`
|
| 43 |
+
- `chunk_size=50`
|
| 44 |
+
- `hidden_dim=512`
|
| 45 |
+
- `dim_feedforward=3200`
|
| 46 |
+
- `seed=0`
|
| 47 |
+
|
| 48 |
+
## 核心结果
|
| 49 |
+
|
| 50 |
+
- 同分布评测 `demo_clean -> demo_clean`:`0.64`
|
| 51 |
+
- 跨配置评测 `demo_clean -> demo_randomized`:`0.0`
|
| 52 |
+
|
| 53 |
+
## 工程心得
|
| 54 |
+
|
| 55 |
+
这次实验最有价值的地方,不是单独得到一个 checkpoint,而是完整跑通了具身智能中的环境、专家、数据、模型、部署、评测闭环:
|
| 56 |
+
|
| 57 |
+
1. RoboTwin 环境程序化定义任务与成功条件。
|
| 58 |
+
2. 官方专家程序自动生成并筛选成功轨迹。
|
| 59 |
+
3. 原始多模态数据落盘为 raw HDF5 / video / instruction / trajectory。
|
| 60 |
+
4. ACT 预处理把采集格式转换为训练格式。
|
| 61 |
+
5. 模型通过离线模仿学习吸收专家行为。
|
| 62 |
+
6. 学到的策略重新部署回环境接受正式评测。
|
| 63 |
+
|
| 64 |
+
从结果上看,ACT 已经学会了 clean 条件下的单任务执行模式,但对 randomized 环境几乎没有泛化能力。这正对应 RoboTwin 2.0 的研究动机:仅在干净分布上成功并不等于策略具有稳健性,强 domain randomization 和更高质量、更大规模的数据生成仍然是核心问题。
|
| 65 |
+
|
| 66 |
+
## 学术上下文
|
| 67 |
+
|
| 68 |
+
这个实验对应的是 RoboTwin 研究体系中的下游策略学习部分:
|
| 69 |
+
|
| 70 |
+
- RoboTwin 1.0 强调数字孪生 benchmark 对双臂操作的价值。
|
| 71 |
+
- RoboTwin 2.0 强调可扩展数据生成和强 domain randomization。
|
| 72 |
+
- 本仓库展示的是在 `beat_block_hammer` 上完成的单任务闭环复现。
|
| 73 |
+
|
| 74 |
+
## 引用
|
| 75 |
+
|
| 76 |
+
如果你使用了这些模型产物,请同时引用 RoboTwin 官方论文与代码仓库。
|
dataset_stats.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b26364dca716d4d9d2a61b078e114bb45c87d78afe787be912db41041599d72
|
| 3 |
+
size 7978
|
policy_best.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:227de6626335707f2b8e7f71c26265e809068e77bed64c223cc79ed6db286592
|
| 3 |
+
size 335907442
|
policy_last.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9685d4ecba5b895e3824bd1e1162287cc296b02c1a0b9a1ae280f606cb22e437
|
| 3 |
+
size 335907442
|
results.json
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"platform": "RoboTwin",
|
| 3 |
+
"upstream_commit": "958a6d2910a0262f5531fcdeb7fffae4184bb586",
|
| 4 |
+
"task_name": "beat_block_hammer",
|
| 5 |
+
"task_config": "demo_clean",
|
| 6 |
+
"policy": "ACT",
|
| 7 |
+
"expert_trajectories": 50,
|
| 8 |
+
"train_seed": 0,
|
| 9 |
+
"train_batch_size": 8,
|
| 10 |
+
"train_num_epochs": 6000,
|
| 11 |
+
"evaluation": {
|
| 12 |
+
"same_distribution": {
|
| 13 |
+
"task_config": "demo_clean",
|
| 14 |
+
"ckpt_setting": "demo_clean",
|
| 15 |
+
"success_rate": 0.64,
|
| 16 |
+
"instruction_type": "unseen",
|
| 17 |
+
"timestamp": "2026-04-22 18:14:41"
|
| 18 |
+
},
|
| 19 |
+
"cross_configuration": {
|
| 20 |
+
"task_config": "demo_randomized",
|
| 21 |
+
"ckpt_setting": "demo_clean",
|
| 22 |
+
"success_rate": 0.0,
|
| 23 |
+
"instruction_type": "unseen",
|
| 24 |
+
"timestamp": "2026-04-22 19:45:55"
|
| 25 |
+
}
|
| 26 |
+
}
|
| 27 |
+
}
|
train_val_kl_seed_0.png
ADDED
|
train_val_l1_seed_0.png
ADDED
|
train_val_loss_seed_0.png
ADDED
|