Bigenlight commited on
Commit
29b165c
·
verified ·
1 Parent(s): edeb94a

Add model card

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ library_name: lerobot
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - robotics
7
+ - lerobot
8
+ - act
9
+ - imitation-learning
10
+ - ur7e
11
+ - end-effector
12
+ ---
13
+
14
+ # ACT · banana-in-pot · EEF (10-D) — checkpoint 40k
15
+
16
+ Action Chunking Transformer (ACT) trained on the **end-effector (EEF) action space**
17
+ for the task *"put the right banana in the pot"* (UR7e arm, GELLO teleoperation,
18
+ LeRobot v3.0). This is the **40k-step checkpoint**, selected as best by open-loop MAE.
19
+
20
+ This is the EEF counterpart of the joint-space model
21
+ [`Bigenlight/act_banana_in_pot`](https://huggingface.co/Bigenlight/act_banana_in_pot).
22
+
23
+ ## Action / observation space
24
+ - `observation.state` / `action`: **10-D** = `[x, y, z, r1..r6 (Zhou 6D rotation), gripper]`
25
+ — absolute next-frame TCP pose (xyz in metres) + gripper. (The joint model uses 7-D
26
+ `[q1..q6, gripper]`.)
27
+ - Cameras: `observation.images.cam1`, `observation.images.cam2` (RGB, resized 360×640).
28
+ - Backbone: ResNet18 + VAE, `chunk_size=100`, ~51.6M params. Normalization: MEAN_STD.
29
+
30
+ ## Training
31
+ - Recipe identical to the joint baseline `train_act_valdiag.sh` except dataset + steps:
32
+ `--dataset.eval_split=0.117` (held-out episodes 45–50), batch 8, seed 1000, 50k steps.
33
+ - Dataset: `banana_in_pot_ee_action` (51 eps / 21,524 frames, 30 fps), built from the raw
34
+ [`Bigenlight/banana_in_pot_raw`](https://huggingface.co/datasets/Bigenlight/banana_in_pot_raw)
35
+ via recorded `tcp_pose` (no FK needed).
36
+ - Hardware: single RTX A4000, ~2h43m. No overfitting (held-out eval_loss monotone to 0.4594@50k).
37
+
38
+ ## Held-out results (open-loop, eps 45–50)
39
+ | checkpoint | pose MAE (m + 6D) | gripper acc |
40
+ |---|---|---|
41
+ | **40k (this)** | **0.05564** | **0.914** |
42
+ | 50k | 0.05564 | 0.911 |
43
+
44
+ Selected by open-loop MAE (repo convention), not by eval_loss.
45
+
46
+ > ⚠️ Note: EEF pose MAE mixes metres (xyz) and unitless 6D-rotation and is **not**
47
+ > directly comparable to the joint model's radian MAE. See the comparison writeup.
48
+
49
+ ## Usage
50
+ ```python
51
+ from lerobot.policies.act.modeling_act import ACTPolicy
52
+ policy = ACTPolicy.from_pretrained("Bigenlight/act_banana_in_pot_ee")
53
+ ```
54
+
55
+ ## Links
56
+ - Experiments repo, full report, reproducibility (Docker): GitHub
57
+ [`Bigenlight/banana-in-pot-experiments`](https://github.com/Bigenlight/banana-in-pot-experiments)
58
+ — see `docs/ACT_EE_RESULTS.md` and `docs/JOINT_VS_EEF_ACT_COMPARISON.md`.
59
+
60
+ *License: CC-BY-NC-4.0 (trained on real-lab teleoperation video).*