Robotics
LeRobot
Safetensors
pi05
so101
imitation-learning
HyeonseokE commited on
Commit
e801777
ยท
verified ยท
1 Parent(s): 62db93e

Add model card

Browse files
Files changed (1) hide show
  1. README.md +184 -0
README.md ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: lerobot
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - lerobot
7
+ - robotics
8
+ - vla
9
+ - pi0
10
+ - pi05
11
+ - so101
12
+ - manipulation
13
+ - imitation-learning
14
+ - behavior-cloning
15
+ datasets:
16
+ - CoRL2026-CSI/SO101-teleop_close_pot_lid_100epi
17
+ base_model: lerobot/pi05_base
18
+ language:
19
+ - en
20
+ model-index:
21
+ - name: pi05_close_pot
22
+ results: []
23
+ ---
24
+
25
+ # ฯ€0.5 โ€” SO-101 `close_pot_lid`
26
+
27
+ `lerobot/pi05_base` ๋ฅผ SO-101 ์–‘ํŒ”(top + left wrist) ์นด๋ฉ”๋ผ ์…‹์—…์—์„œ
28
+ **๋ƒ„๋น„ ๋šœ๊ป‘ ๋‹ซ๊ธฐ(`close_pot_lid`)** ๋‹จ์ผ ํƒœ์Šคํฌ์— ๋Œ€ํ•ด 100 ์—ํ”ผ์†Œ๋“œ(57,173 ํ”„๋ ˆ์ž„)
29
+ ์›๊ฒฉ์กฐ์ž‘ ๋ฐ๋ชจ๋กœ ํŒŒ์ธํŠœ๋‹ํ•œ ฯ€0.5 (PaliGemma-2B + Action Expert 300M) ์ •์ฑ…์ž…๋‹ˆ๋‹ค.
30
+
31
+ ํ•™์Šต ์ฝ”๋“œ: [`scripts/train_pi05_close_pot_lid.sh`](https://github.com/HyeonseokE/train_with_lerobot/blob/main/scripts/train_pi05_close_pot_lid.sh)
32
+ ํ”„๋ ˆ์ž„์›Œํฌ: [LeRobot](https://github.com/huggingface/lerobot)
33
+
34
+ ---
35
+
36
+ ## ๋ชจ๋ธ ๊ฐœ์š”
37
+
38
+ | ํ•ญ๋ชฉ | ๊ฐ’ |
39
+ |---|---|
40
+ | Architecture | ฯ€0.5 (PaliGemma-2B VLM + Gemma-300M action expert, flow-matching head) |
41
+ | Base checkpoint | [`lerobot/pi05_base`](https://huggingface.co/lerobot/pi05_base) |
42
+ | Action chunk size | 50 |
43
+ | Inference steps (flow-matching) | 10 |
44
+ | Image resolution | 224 ร— 224 |
45
+ | Cameras | `base_0_rgb`, `left_wrist_0_rgb`, `right_wrist_0_rgb` |
46
+ | State dim (padded) | 32 |
47
+ | Action dim (์‹คํšจ / padded) | **6** / 32 |
48
+ | dtype | bfloat16 |
49
+
50
+ ### ์•ก์…˜ / ์นด๋ฉ”๋ผ ๋งคํ•‘
51
+
52
+ ๋ฐ์ดํ„ฐ์…‹ โ†’ ์ •์ฑ… ์ž…๋ ฅ ํ‚ค rename:
53
+
54
+ ```
55
+ observation.images.top โ†’ observation.images.base_0_rgb
56
+ observation.images.wrist โ†’ observation.images.left_wrist_0_rgb
57
+ ```
58
+
59
+ > `right_wrist_0_rgb` ๋Š” ๋ชจ๋ธ ์ž…๋ ฅ ์Šฌ๋กฏ์ด์ง€๋งŒ SO-101 ๋‹จ์ผํŒ”์—์„œ๋Š” ๋นˆ ์นด๋ฉ”๋ผ๋กœ ์ฒ˜๋ฆฌ๋ฉ๋‹ˆ๋‹ค.
60
+
61
+ ์•ก์…˜ ํ”ผ์ฒ˜(6 DoF, SO-101):
62
+
63
+ ```
64
+ shoulder_pan.pos
65
+ shoulder_lift.pos
66
+ elbow_flex.pos
67
+ wrist_flex.pos
68
+ wrist_roll.pos
69
+ gripper.pos
70
+ ```
71
+
72
+ ์ •๊ทœํ™”: `ACTION = MEAN_STD`, `STATE = MEAN_STD`, `VISUAL = IDENTITY`.
73
+
74
+ ---
75
+
76
+ ## ํ•™์Šต ๋ฐ์ดํ„ฐ
77
+
78
+ - **๋ฐ์ดํ„ฐ์…‹**: [`CoRL2026-CSI/SO101-teleop_close_pot_lid_100epi`](https://huggingface.co/datasets/CoRL2026-CSI/SO101-teleop_close_pot_lid_100epi)
79
+ - **์—ํ”ผ์†Œ๋“œ**: 100
80
+ - **์ด ํ”„๋ ˆ์ž„**: 57,173
81
+ - **๋กœ๋ด‡ / ํƒœ์Šคํฌ**: SO-101, ๋ƒ„๋น„ ๋šœ๊ป‘์„ ์žก์•„ ๋ณธ์ฒด ์œ„์— ๋‹ซ๊ธฐ
82
+ - **์ˆ˜์ง‘ ๋ฐฉ์‹**: human teleoperation
83
+ - **์นด๋ฉ”๋ผ**: top + wrist (๋‘˜ ๋‹ค 224 ร— 224 ์œผ๋กœ ๋ฆฌ์‚ฌ์ด์ฆˆ)
84
+
85
+ ### Image augmentation (ํ•™์Šต ์‹œ)
86
+
87
+ `max_num_transforms=3`, `random_order=True`, ํ›„๋ณด:
88
+
89
+ | ๋ณ€ํ™˜ | ํŒŒ๋ผ๋ฏธํ„ฐ |
90
+ |---|---|
91
+ | ColorJitter brightness | `[0.8, 1.2]` |
92
+ | ColorJitter contrast | `[0.8, 1.2]` |
93
+ | ColorJitter saturation | `[0.5, 1.5]` |
94
+ | ColorJitter hue | `[-0.05, 0.05]` |
95
+ | SharpnessJitter | `[0.5, 1.5]` |
96
+ | RandomAffine | degrees `[-5, 5]`, translate `[0.05, 0.05]` |
97
+
98
+ ---
99
+
100
+ ## ํ•™์Šต ์„ค์ •
101
+
102
+ | ํ•ญ๋ชฉ | ๊ฐ’ |
103
+ |---|---|
104
+ | Hardware | 4 ร— GPU (DDP via ๐Ÿค— Accelerate) |
105
+ | Per-device batch size | 32 |
106
+ | Gradient accumulation | 2 |
107
+ | **Effective global batch** | **256** (32 ร— 4 ร— 2) |
108
+ | Steps | 11,200 |
109
+ | โ‰ˆ Epochs | 50 (`57,173 ร— 50 / 256 โ‰ˆ 11,167`) |
110
+ | Optimizer | AdamW (ฮฒ=(0.9, 0.95), eps=1e-8, wd=0.01) |
111
+ | Peak LR | 2.5e-5 |
112
+ | Decay LR | 2.5e-6 |
113
+ | Scheduler | cosine decay, warmup 1000, decay 30000 |
114
+ | Grad clip | 1.0 |
115
+ | Mixed precision | none (bf16 native) |
116
+ | Gradient checkpointing | on |
117
+ | `compile_model` | off |
118
+ | `freeze_vision_encoder` | off |
119
+ | `train_expert_only` | off |
120
+ | Seed | 1000 |
121
+
122
+ ์ฒดํฌํฌ์ธํŠธ: 11,200 step (ํ•™์Šต ์ข…๋ฃŒ ์‹œ์ ).
123
+
124
+ ---
125
+
126
+ ## ์‚ฌ์šฉ ๋ฐฉ๋ฒ•
127
+
128
+ ### 1. ๋ชจ๋ธ ๋กœ๋“œ
129
+
130
+ ```python
131
+ from lerobot.policies.pi05.modeling_pi05 import PI05Policy
132
+
133
+ policy = PI05Policy.from_pretrained("CoRL2026-CSI/pi05_close_pot")
134
+ policy.eval().to("cuda")
135
+ ```
136
+
137
+ ### 2. ์ถ”๋ก  (์ „์ฒ˜๋ฆฌ/ํ›„์ฒ˜๋ฆฌ ํŒŒ์ดํ”„๋ผ์ธ ํฌํ•จ)
138
+
139
+ LeRobot์˜ ํ‘œ์ค€ inference ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์„ธ์š”:
140
+
141
+ ```bash
142
+ lerobot-eval \
143
+ --policy.path=CoRL2026-CSI/pi05_close_pot \
144
+ --env.type=<your_env> \
145
+ --eval.n_episodes=20
146
+ ```
147
+
148
+ ๋˜๋Š” ์‹ค์‹œ๊ฐ„ ๋กœ๋ด‡ ์ œ์–ด์šฉ ์Šคํฌ๋ฆฝํŠธ๋Š” ์ €์žฅ์†Œ
149
+ [`scripts/infer_smolvla.py`](https://github.com/HyeonseokE/train_with_lerobot/blob/main/scripts/infer_smolvla.py) ์™€ ๋™์ผํ•œ ํŒจํ„ด์„
150
+ ์ฐธ์กฐํ•ด `pi05` ๋กœ ๊ต์ฒดํ•ด ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
151
+
152
+ ### 3. ์นด๋ฉ”๋ผ ํ‚ค ์ฃผ์˜
153
+
154
+ ํ•™์Šต ์‹œ ๋ฐ์ดํ„ฐ์…‹์˜ `observation.images.top` / `.wrist` ๊ฐ€ ์ •์ฑ… ์ž…๋ ฅ
155
+ `base_0_rgb` / `left_wrist_0_rgb` ๋กœ rename ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๋‹ค๋ฅธ ํ™˜๊ฒฝ์—์„œ
156
+ ์‚ฌ์šฉ ์‹œ ๋™์ผํ•œ ํ‚ค๋กœ ๋ณ€ํ™˜ํ•˜๊ฑฐ๋‚˜ `--rename_map` ์ธ์ž๋ฅผ ์‚ฌ์šฉํ•˜์„ธ์š”.
157
+
158
+ ---
159
+
160
+ ## ํ•œ๊ณ„ ๋ฐ ๊ถŒ๊ณ 
161
+
162
+ - **๋‹จ์ผ ํƒœ์Šคํฌ / ๋‹จ์ผ ์‹œ๋“œ**: `close_pot_lid` 100 ์—ํ”ผ์†Œ๋“œ ์™ธ ๋ถ„ํฌ์—์„œ๋Š” ์ผ๋ฐ˜ํ™” ๋ณด์žฅ ์—†์Œ.
163
+ - **๋‹จ์ผํŒ”(SO-101) ์ „์ œ**: `right_wrist_0_rgb` ๋Š” ๋นˆ ์นด๋ฉ”๋ผ๋กœ ํ•™์Šต๋˜์–ด ๋‹ค๋ฅธ ์–‘ํŒ” ์…‹์—…์—์„œ๋Š” ์žฌํ•™์Šต ํ•„์š”.
164
+ - **์นด๋ฉ”๋ผ ์œ„์น˜/์กฐ๋ช… ๋ฏผ๊ฐ๋„**: 100 ์—ํ”ผ์†Œ๋“œ + image aug ๋งŒ์œผ๋กœ ํ•™์Šต โ€” ํฐ ๋„๋ฉ”์ธ ์‹œํ”„ํŠธ์—์„œ๋Š” ์„ฑ๋Šฅ ์ €ํ•˜ ๊ฐ€๋Šฅ.
165
+ - **์ •๋Ÿ‰ ํ‰๊ฐ€ ๋ฏธ์ˆ˜๋ก**: ๋ณธ ์นด๋“œ์—๋Š” ์‹ค๋กœ๋ด‡ / ์‹œ๋ฎฌ success rate ๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ์‚ฌ์šฉ ์ „ ์ž์ฒด ํ‰๊ฐ€ ๊ถŒ์žฅ.
166
+
167
+ ---
168
+
169
+ ## ๋ผ์ด์„ ์Šค
170
+
171
+ Apache 2.0 (๋ฒ ์ด์Šค ๋ชจ๋ธ [`lerobot/pi05_base`](https://huggingface.co/lerobot/pi05_base) ๋ผ์ด์„ ์Šค๋ฅผ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค).
172
+
173
+ ## ์ธ์šฉ
174
+
175
+ LeRobot ํ”„๋กœ์ ํŠธ:
176
+
177
+ ```bibtex
178
+ @misc{cadene2024lerobot,
179
+ author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Wolf, Thomas},
180
+ title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
181
+ howpublished = "\url{https://github.com/huggingface/lerobot}",
182
+ year = {2024}
183
+ }
184
+ ```