File size: 2,432 Bytes
29b165c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
license: cc-by-nc-4.0
library_name: lerobot
pipeline_tag: robotics
tags:
- robotics
- lerobot
- act
- imitation-learning
- ur7e
- end-effector
---
# ACT · banana-in-pot · EEF (10-D) — checkpoint 40k
Action Chunking Transformer (ACT) trained on the **end-effector (EEF) action space**
for the task *"put the right banana in the pot"* (UR7e arm, GELLO teleoperation,
LeRobot v3.0). This is the **40k-step checkpoint**, selected as best by open-loop MAE.
This is the EEF counterpart of the joint-space model
[`Bigenlight/act_banana_in_pot`](https://huggingface.co/Bigenlight/act_banana_in_pot).
## Action / observation space
- `observation.state` / `action`: **10-D** = `[x, y, z, r1..r6 (Zhou 6D rotation), gripper]`
— absolute next-frame TCP pose (xyz in metres) + gripper. (The joint model uses 7-D
`[q1..q6, gripper]`.)
- Cameras: `observation.images.cam1`, `observation.images.cam2` (RGB, resized 360×640).
- Backbone: ResNet18 + VAE, `chunk_size=100`, ~51.6M params. Normalization: MEAN_STD.
## Training
- Recipe identical to the joint baseline `train_act_valdiag.sh` except dataset + steps:
`--dataset.eval_split=0.117` (held-out episodes 45–50), batch 8, seed 1000, 50k steps.
- Dataset: `banana_in_pot_ee_action` (51 eps / 21,524 frames, 30 fps), built from the raw
[`Bigenlight/banana_in_pot_raw`](https://huggingface.co/datasets/Bigenlight/banana_in_pot_raw)
via recorded `tcp_pose` (no FK needed).
- Hardware: single RTX A4000, ~2h43m. No overfitting (held-out eval_loss monotone to 0.4594@50k).
## Held-out results (open-loop, eps 45–50)
| checkpoint | pose MAE (m + 6D) | gripper acc |
|---|---|---|
| **40k (this)** | **0.05564** | **0.914** |
| 50k | 0.05564 | 0.911 |
Selected by open-loop MAE (repo convention), not by eval_loss.
> ⚠️ Note: EEF pose MAE mixes metres (xyz) and unitless 6D-rotation and is **not**
> directly comparable to the joint model's radian MAE. See the comparison writeup.
## Usage
```python
from lerobot.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("Bigenlight/act_banana_in_pot_ee")
```
## Links
- Experiments repo, full report, reproducibility (Docker): GitHub
[`Bigenlight/banana-in-pot-experiments`](https://github.com/Bigenlight/banana-in-pot-experiments)
— see `docs/ACT_EE_RESULTS.md` and `docs/JOINT_VS_EEF_ACT_COMPARISON.md`.
*License: CC-BY-NC-4.0 (trained on real-lab teleoperation video).*
|