| --- |
| license: cc-by-nc-4.0 |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - lerobot |
| - act |
| - imitation-learning |
| - ur7e |
| - end-effector |
| --- |
| |
| # ACT · banana-in-pot · EEF (10-D) — checkpoint 40k |
|
|
| Action Chunking Transformer (ACT) trained on the **end-effector (EEF) action space** |
| for the task *"put the right banana in the pot"* (UR7e arm, GELLO teleoperation, |
| LeRobot v3.0). This is the **40k-step checkpoint**, selected as best by open-loop MAE. |
|
|
| This is the EEF counterpart of the joint-space model |
| [`Bigenlight/act_banana_in_pot`](https://huggingface.co/Bigenlight/act_banana_in_pot). |
|
|
| ## Action / observation space |
| - `observation.state` / `action`: **10-D** = `[x, y, z, r1..r6 (Zhou 6D rotation), gripper]` |
| — absolute next-frame TCP pose (xyz in metres) + gripper. (The joint model uses 7-D |
| `[q1..q6, gripper]`.) |
| - Cameras: `observation.images.cam1`, `observation.images.cam2` (RGB, resized 360×640). |
| - Backbone: ResNet18 + VAE, `chunk_size=100`, ~51.6M params. Normalization: MEAN_STD. |
| |
| ## Training |
| - Recipe identical to the joint baseline `train_act_valdiag.sh` except dataset + steps: |
| `--dataset.eval_split=0.117` (held-out episodes 45–50), batch 8, seed 1000, 50k steps. |
| - Dataset: `banana_in_pot_ee_action` (51 eps / 21,524 frames, 30 fps), built from the raw |
| [`Bigenlight/banana_in_pot_raw`](https://huggingface.co/datasets/Bigenlight/banana_in_pot_raw) |
| via recorded `tcp_pose` (no FK needed). |
| - Hardware: single RTX A4000, ~2h43m. No overfitting (held-out eval_loss monotone to 0.4594@50k). |
| |
| ## Held-out results (open-loop, eps 45–50) |
| | checkpoint | pose MAE (m + 6D) | gripper acc | |
| |---|---|---| |
| | **40k (this)** | **0.05564** | **0.914** | |
| | 50k | 0.05564 | 0.911 | |
| |
| Selected by open-loop MAE (repo convention), not by eval_loss. |
|
|
| > ⚠️ Note: EEF pose MAE mixes metres (xyz) and unitless 6D-rotation and is **not** |
| > directly comparable to the joint model's radian MAE. See the comparison writeup. |
|
|
| ## Usage |
| ```python |
| from lerobot.policies.act.modeling_act import ACTPolicy |
| policy = ACTPolicy.from_pretrained("Bigenlight/act_banana_in_pot_ee") |
| ``` |
|
|
| ## Links |
| - Experiments repo, full report, reproducibility (Docker): GitHub |
| [`Bigenlight/banana-in-pot-experiments`](https://github.com/Bigenlight/banana-in-pot-experiments) |
| — see `docs/ACT_EE_RESULTS.md` and `docs/JOINT_VS_EEF_ACT_COMPARISON.md`. |
|
|
| *License: CC-BY-NC-4.0 (trained on real-lab teleoperation video).* |
|
|
