aailabkaist commited on
Commit
1a59c60
ยท
verified ยท
1 Parent(s): e9f239a

Upload 4 files

Browse files
1. LeRobot Installation.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # LeRobot Setting Guide
2
+ https://huggingface.co/docs/lerobot/installation \
3
+ https://huggingface.co/docs/lerobot/so101
4
+
5
+ last updated Dec/26/2025
6
+
7
+ # 1. LeRobot Installation
8
+
9
+ ## 1-1. Python ๊ฐ€์ƒ ํ™˜๊ฒฝ ์ƒ์„ฑ
10
+ ```bash
11
+ # Create python 3.10 conda env
12
+ conda create -n lerobot python=3.10 -y
13
+
14
+ # Activate env
15
+ conda activate lerobot
16
+ ```
17
+
18
+ ---
19
+
20
+ ## 1-2. ffmpeg, Rerun ์„ค์น˜
21
+ - ์˜์ƒ ์ฒ˜๋ฆฌ๋ฅผ ์œ„ํ•ด `ffmpeg` ํ•„์š”
22
+ - LeRobot ๋ฐ์ดํ„ฐ visualization์„ ์œ„ํ•ด `Rerun` ํ•„์š”
23
+ ```bash
24
+ conda install ffmpeg=7.1.1 -c conda-forge
25
+ pip install rerun-sdk
26
+ ```
27
+
28
+ ---
29
+
30
+ ## 1-3. LeRobot ์†Œ์Šค ์ฝ”๋“œ ๋‹ค์šด๋กœ๋“œ
31
+ ```bash
32
+ # LeRobot github repo clone
33
+ git clone https://github.com/huggingface/lerobot.git
34
+ cd lerobot
35
+ ```
36
+
37
+ ---
38
+
39
+ ## 1-4. LeRobot ์„ค์น˜
40
+ ```bash
41
+ pip install -e .
42
+ ```
43
+
44
+ ---
45
+
46
+ ## 1-5. ๋กœ๋ด‡ ํ•˜๋“œ์›จ์–ด ๋“œ๋ผ์ด๋ฒ„
47
+ ์‚ฌ์šฉํ•˜๋Š” ๋ชจํ„ฐ์— ๋งž๋Š” ๋“œ๋ผ์ด๋ฒ„ ์„ค์น˜
48
+
49
+ ```bash
50
+ # Feetech motor (SO-100, SO-101)
51
+ pip install -e ".[feetech]"
52
+
53
+ # Dynamixel motor (Koch, etc.)
54
+ pip install -e ".[dynamixel]"
55
+ ```
56
+
57
+ ---
58
+
59
+ ## 1-6. ์„ค์น˜ ํ™•์ธ
60
+
61
+ ```bash
62
+ python -c "import lerobot; print(lerobot.__version__)"
63
+ ```
64
+
65
+ ---
66
+
67
+ ## (Optional) ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ
68
+ ํŠน์ • ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ์„ ์‚ฌ์šฉํ•˜๋ ค๋ฉด ์ถ”๊ฐ€ ํŒจํ‚ค์ง€ ์„ค์น˜
69
+
70
+ ```bash
71
+ # ALOHA ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ
72
+ pip install -e ".[aloha]"
73
+
74
+ # Push-T ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ
75
+ pip install -e ".[pusht]"
76
+
77
+ # ๋ชจ๋“  ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ
78
+ pip install -e ".[aloha,pusht,xarm]"
79
+ ```
80
+
81
+ ---
82
+
2. LeRobot Setting.md ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 2. LeRobot Setting
2
+ last updated Dec/26/2025
3
+
4
+ ## 2-1. USB port ์ฐพ๊ธฐ
5
+
6
+ 1) Follower arm ๋ณด๋“œ์™€ PC๋ฅผ USB๋กœ ์—ฐ๊ฒฐ
7
+ 2) ์•„๋ž˜ ๋ช…๋ น ์‹คํ–‰
8
+
9
+ ```bash
10
+ lerobot-find-port
11
+ ```
12
+
13
+ 3) ์•ˆ๋‚ด์— ๋”ฐ๋ผ **Follower arm ๋ณด๋“œ์—์„œ USB ์ผ€์ด๋ธ”์„ ๋ฝ‘๊ณ  Enter**๋ฅผ ๋ˆ„๋ฅด๋ฉด, ์•„๋ž˜์™€ ๊ฐ™์€ ๊ฒฐ๊ณผ๊ฐ€ ์ถœ๋ ฅ๋ฉ๋‹ˆ๋‹ค.
14
+
15
+ ```text
16
+ The port of this MotorsBus is '/dev/ttyACM0'
17
+ Reconnect the USB cable
18
+ ```
19
+
20
+ โ†’ ์ด ๊ฒฝ์šฐ์— follower arm ๋ณด๋“œ์˜ USB ํฌํŠธ๋Š” **`/dev/ttyACM0`** ์ž…๋‹ˆ๋‹ค.
21
+
22
+ 4) Leader arm ๋ณด๋“œ๋„ ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ USB ํฌํŠธ๋ฅผ ์ฐพ์•„์ค๋‹ˆ๋‹ค.
23
+
24
+ 5) ๊ฐ ํฌํŠธ๊ฐ€ ์–ด๋–ค arm(Follower/Leader)์— ํ•ด๋‹นํ•˜๋Š”์ง€ **๋ฐ˜๋“œ์‹œ ๊ตฌ๋ถ„**ํ•ด ๋‘์„ธ์š”.
25
+
26
+ 6) ์—ฐ๊ฒฐ๋œ USB ํฌํŠธ์— Read/Write ๊ถŒํ•œ ๋ถ€์—ฌ
27
+ ```bash
28
+ sudo chmod 666 /dev/ttyACM0
29
+ sudo chmod 666 /dev/ttyACM1
30
+ ```
31
+
32
+ ## (Optional) Serial number๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ USB port ๊ณ ์ •ํ•˜๋Š” ๋ฒ•
33
+
34
+ ### 1. ttyACM0 serial ํ™•์ธ
35
+ ```bash
36
+ udevadm info -a -n /dev/ttyACM0 | grep '{serial}' -m 1
37
+ ```
38
+
39
+ ```bash
40
+ # Example print
41
+ ATTRS{serial}=="5A7A057786"
42
+ ```
43
+
44
+ ### 2. ttyACM1 serial ํ™•์ธ
45
+ ```bash
46
+ udevadm info -a -n /dev/ttyACM1 | grep '{serial}' -m 1
47
+ ```
48
+
49
+ ```bash
50
+ # Example print
51
+ ATTRS{serial}=="5A68011549"
52
+ ```
53
+
54
+ ### 3. udev rule ์ƒ์„ฑ
55
+
56
+ ```bash
57
+ # udev rule ์ƒ์„ฑ/ํŽธ์ง‘
58
+ sudo nano /etc/udev/rules.d/99-serial.rules
59
+ ```
60
+
61
+ ```bash
62
+ GNU nano 2.9.3 /etc/udev/rules.d/99-serial.rules
63
+
64
+ # ์•„๋ž˜์˜ serial number๋Š” ์˜ˆ์‹œ์ž…๋‹ˆ๋‹ค. ์ž์‹ ์˜ ํ™˜๊ฒฝ์—์„œ ํ™•์ธํ•œ ๊ฐ’์œผ๋กœ ๋ฐ˜๋“œ์‹œ ๊ต์ฒดํ•˜์„ธ์š”.
65
+ SUBSYSTEM=="tty", ATTRS{serial}=="5A7A057786", SYMLINK+="so101_follower"
66
+ SUBSYSTEM=="tty", ATTRS{serial}=="5A68011549", SYMLINK+="so101_leader"
67
+ ```
68
+
69
+ ### 4. rule reload / trigger
70
+ ```bash
71
+ sudo udevadm control --reload-rules
72
+ sudo udevadm trigger
73
+ ```
74
+
75
+ ### 5. ๊ณ ์ • ํฌํŠธ ํ™•์ธ
76
+ ```bash
77
+ ls -l /dev/so101_*
78
+ ```
79
+
80
+ ```bash
81
+ # Example print
82
+ # USB ํฌํŠธ๋ฅผ ๊ฝ‚๋Š” ์ˆœ์„œ๊ฐ€ ๋ฐ”๋€Œ์–ด๋„ ๋‹ค์Œ ๊ฒฝ๋กœ๋กœ ํ•ญ์ƒ ๋™์ผํ•˜๊ฒŒ ์ธ์‹
83
+ lrwxrwxrwx 1 root root 7 11์›” 24 21:46 /dev/so101_follower -> ttyACM0
84
+ lrwxrwxrwx 1 root root 7 11์›” 24 21:46 /dev/so101_leader -> ttyACM1
85
+ ```
86
+
87
+ ---
88
+
89
+ ## 2-2. ๋ชจํ„ฐ ID ์„ค์ •
90
+
91
+ - ๊ฐ ๋ชจํ„ฐ์˜ **๊ณ ์œ  ID** ์„ค์ •์„ ์œ„ํ•ด ์•„๋ž˜ ๋ช…๋ น์–ด ์‹คํ–‰ ํ›„, ํ„ฐ๋ฏธ๋„์— ํ‘œ์‹œ๋˜๋Š” ๊ฐ€์ด๋“œ์— ๋”ฐ๋ผ **๋ชจํ„ฐ๋ฅผ ์ˆœ์„œ๋Œ€๋กœ ์—ฐ๊ฒฐ**ํ•˜๋ฉฐ ์ง„ํ–‰
92
+ - ์•„๋ž˜ ๋งํฌ์˜ ์˜์ƒ์„ ๋ณด๋ฉด์„œ ํ•˜๋Š” ๊ฑธ ์ถ”์ฒœํ•ฉ๋‹ˆ๋‹ค. \
93
+ https://huggingface.co/docs/lerobot/so101#2-set-the-motors-ids-and-baudrates
94
+
95
+ ### Follower
96
+ ```bash
97
+ lerobot-setup-motors \
98
+ --robot.type=so101_follower \
99
+ --robot.port=/dev/so101_follower # Follower ๋ณด๋“œ์˜ USB port
100
+ ```
101
+
102
+ ### Leader
103
+ ```bash
104
+ lerobot-setup-motors \
105
+ --teleop.type=so101_leader \
106
+ --teleop.port=/dev/so101_leader # Leader ๋ณด๋“œ์˜ USB port
107
+ ```
108
+
109
+ ---
110
+
111
+ ## 2-3. Calibration
112
+
113
+ - Calibration์€ ๋กœ๋ด‡ ํŒ”์˜ ๊ฐ ๊ด€์ ˆ(joint)์ด ์ •ํ™•ํ•œ ์œ„์น˜๋ฅผ ์ธ์‹ํ•˜๊ณ  ๋งคํ•‘ํ•  ์ˆ˜ ์žˆ๋„๋ก ํ•˜๋Š” ์ค‘์š”ํ•œ ๊ณผ์ •์ž…๋‹ˆ๋‹ค.
114
+ - ์•„๋ž˜ ๋งํฌ์˜ ์˜์ƒ์„ ๋ณด๋ฉด์„œ ํ•˜๋Š” ๊ฑธ ์ถ”์ฒœํ•ฉ๋‹ˆ๋‹ค. \
115
+ https://huggingface.co/docs/lerobot/so101#calibrate
116
+
117
+ ### Follower
118
+ ```bash
119
+ lerobot-calibrate \
120
+ --robot.type=so101_follower \
121
+ --robot.port=/dev/so101_follower \
122
+ --robot.id=follower
123
+ ```
124
+
125
+ ### Leader
126
+ ```bash
127
+ lerobot-calibrate \
128
+ --teleop.type=so101_leader \
129
+ --teleop.port=/dev/so101_leader \
130
+ --teleop.id=leader
131
+ ```
132
+
133
+ ### Calibration ๊ณผ์ •
134
+ Command ์‹คํ–‰ ํ›„ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋ฉ”์‹œ์ง€๊ฐ€ ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค.
135
+
136
+ ```text
137
+ Move None SO101Leader to the middle of its range of motion and press ENTER....
138
+ Move all joints sequentially through their entire ranges of motion.
139
+ Recording positions. Press ENTER to stop...
140
+ ```
141
+
142
+ ์œ„ ๋ฉ”์‹œ์ง€๋ฅผ ๋”ฐ๋ผ **์ค‘๊ฐ„ ์œ„์น˜(middle position)** ๋ฅผ ์žก์•„์ฃผ๊ณ  **Enter**๋ฅผ ๋ˆŒ๋Ÿฌ์ฃผ์„ธ์š”.
143
+
144
+ #### Joint ์›€์ง์ž„ ์ฒดํฌ๋ฆฌ์ŠคํŠธ
145
+ - ๋กœ๋ด‡ ํŒ”์„ **์ค‘๊ฐ„ ์œ„์น˜(middle position)** ๋กœ ์ด๋™
146
+ - ๋ชจ๋“  ๊ด€์ ˆ(joints)์„ **์ˆœ์ฐจ์ ์œผ๋กœ** ์ „์ฒด ๋ฒ”์œ„์—์„œ ์›€์ง์—ฌ๋ณด๊ธฐ
147
+ - **ENTER** ํ‚ค๋ฅผ ๋ˆŒ๋Ÿฌ ๊ธฐ๋ก ์ค‘์ง€
148
+
149
+ ### Calibration ๊ฒฐ๊ณผ
150
+ Calibration์ด ์™„๋ฃŒ๋˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๊ด€์ ˆ ์œ„์น˜ ์ •๋ณด๊ฐ€ ์ถœ๋ ฅ๋ฉ๋‹ˆ๋‹ค(์˜ˆ์‹œ).
151
+
152
+ ```bash
153
+ -------------------------------------------
154
+ -------------------------------------------
155
+ NAME | MIN | POS | MAX
156
+ shoulder_pan | 1955 | 2007 | 3132
157
+ shoulder_lift | 709 | 724 | 2057
158
+ elbow_flex | 1605 | 3036 | 3110
159
+ wrist_flex | 808 | 2797 | 2803
160
+ wrist_roll | 2037 | 2048 | 2055
161
+ gripper | 2019 | 2046 | 3383
162
+ ```
163
+
164
+ ### ํŒŒ์ผ ์ €์žฅ ํ™•์ธ
165
+ ์„ฑ๊ณต์ ์œผ๋กœ ์™„๋ฃŒ๋˜๋ฉด ๋‹ค์Œ ๋ฉ”์‹œ์ง€๊ฐ€ ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค.
166
+
167
+ ```bash
168
+ Calibration saved to ~/.cache/huggingface/lerobot/calibration/teleoperators/so101_leader/leader.json
169
+ ```
170
+
171
+ ### Calibration ํŒŒ์ผ
172
+
173
+ Calibration์ด ์™„๋ฃŒ๋˜๋ฉด ๋‹ค์Œ ์œ„์น˜์— ํŒŒ์ผ์ด ์ €์žฅ๋ฉ๋‹ˆ๋‹ค.
174
+
175
+ - ๋ฆฌ๋” ์•”:
176
+ `~/.cache/huggingface/lerobot/calibration/teleoperators/so101_leader/leader.json`
177
+ - ํŒ”๋กœ์›Œ ์•”:
178
+ `~/.cache/huggingface/lerobot/calibration/robots/so101_follower/follower.json`
179
+
180
+
181
+ ๊ฐ Calibration ํŒŒ์ผ์—๋Š” ๋‹ค์Œ ์ •๏ฟฝ๏ฟฝ๊ฐ€ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.
182
+
183
+ - ๊ฐ ๊ด€์ ˆ์˜ ์ตœ์†Œ๊ฐ’(MIN)
184
+ - ํ˜„์žฌ ์œ„์น˜(POS)
185
+ - ์ตœ๋Œ€๊ฐ’(MAX)
186
+
187
+ ---
3. LeRobot Manipulation.md ADDED
@@ -0,0 +1,211 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 3. LeRobot Manipulation
2
+ last updated Dec/26/2025
3
+
4
+ ## 3-1. Teleoperation
5
+
6
+ ### ์นด๋ฉ”๋ผ index, ์ŠคํŽ™, fps ํ™•์ธ
7
+ 2๊ฐœ ์ด์ƒ์˜ ์นด๋ฉ”๋ผ๋ฅผ ์—ฐ๊ฒฐํ•ด ๋ฐ์ดํ„ฐ์…‹ ๊ตฌ์ถ•/๋ชจ๋ธ ์ถ”๋ก ์— ํ™œ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
8
+ PC์— ์—ฐ๊ฒฐ๋œ ์นด๋ฉ”๋ผ์˜ index, ํ•ด์ƒ๋„, fps ๋“ฑ ์ •๋ณด๋ฅผ ํ™•์ธํ•˜๋ ค๋ฉด ์•„๋ž˜ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์„ธ์š”.
9
+
10
+ ```bash
11
+ lerobot-find-cameras
12
+ ```
13
+
14
+ ```bash
15
+ # Example print
16
+ --- Detected Cameras ---
17
+ Camera #0:
18
+ Name: OpenCV Camera @ /dev/video0
19
+ Type: OpenCV
20
+ Id: /dev/video0
21
+ Backend api: V4L2
22
+ Default stream profile:
23
+ Format: 0.0
24
+ Fourcc: YUYV
25
+ Width: 640
26
+ Height: 480
27
+ Fps: 30.0
28
+ --------------------
29
+ Camera #1:
30
+ Name: OpenCV Camera @ /dev/video2
31
+ Type: OpenCV
32
+ Id: /dev/video2
33
+ Backend api: V4L2
34
+ Default stream profile:
35
+ Format: 0.0
36
+ Fourcc: YUYV
37
+ Width: 640
38
+ Height: 480
39
+ Fps: 30.0
40
+ --------------------
41
+
42
+ Finalizing image saving...
43
+ Image capture finished. Images saved to outputs/captured_images
44
+ ```
45
+
46
+ > **TIP**: ์นด๋ฉ”๋ผ ์—ฐ๊ฒฐ ์ˆœ์„œ์— ๋”ฐ๋ผ index๊ฐ€ ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์œผ๋‹ˆ, ์‹คํ–‰ ์ „ํ›„๋กœ ํ•ญ์ƒ `lerobot-find-cameras` ๋‹ค์Œ์— `.../lerobot/outputs/captured_images`์—์„œ ์บก์ฒ˜๋œ ์‚ฌ์ง„์„ ํ†ตํ•ด ๊ฐ camera์˜ ํฌํŠธ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.
47
+
48
+ ### Teleoperation command
49
+ ```bash
50
+ lerobot-teleoperate \
51
+ --robot.type=so101_follower \
52
+ --robot.port=/dev/so101_follower \
53
+ --robot.id=follower_new_gripper \
54
+ --robot.cameras="{
55
+ top: {type: opencv, index_or_path: "/dev/video0", width: 640, height: 480, fps: 30},
56
+ wrist: {type: opencv, index_or_path: "/dev/video2", width: 640, height: 480, fps: 30}
57
+ }" \
58
+ --teleop.type=so101_leader \
59
+ --teleop.port=/dev/so101_leader \
60
+ --teleop.id=leader \
61
+ --display_data=true
62
+ ```
63
+
64
+ ### ์˜ต์…˜ ์„ค๋ช…
65
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… | ์˜ˆ์‹œ ๊ฐ’ |
66
+ |---|---|---|
67
+ | `--robot.type` | follower ๋กœ๋ด‡ ํƒ€์ž… | `so101_follower` |
68
+ | `--robot.port` | follower USB ํฌํŠธ | `/dev/so101_follower` |
69
+ | `--robot.id` | follower ๋กœ๋ด‡ ๊ณ ์œ  ID (calibration ๋•Œ ์„ค์ •ํ•œ ID) |
70
+ | `--robot.cameras` | ์นด๋ฉ”๋ผ ์ •๋ณด | `lerobot-find-cameras`๋กœ ํ™•์ธํ•œ ๊ฐ’ |
71
+ | `--teleop.type` | leader ๋กœ๋ด‡ ํƒ€์ž… | `so101_leader` |
72
+ | `--teleop.port` | leader USB ํฌํŠธ | `/dev/so101_leader` |
73
+ | `--teleop.id` | leader ์ด๋ฆ„ | `leader` |
74
+
75
+ ---
76
+
77
+ ## 3-2. Record
78
+
79
+ ### HuggingFace ๊ณ„์ • ์„ค์ •
80
+
81
+ ```bash
82
+ # HuggingFace CLI ํ† ํฐ ๋กœ๊ทธ์ธ
83
+ hf auth login --token ${YOUR_TOKEN_HERE} --add-to-git-credential
84
+
85
+ # ๋กœ๊ทธ์ธ ํ™•์ธ
86
+ HF_USER=$(hf auth whoami | head -n 1)
87
+ export HF_USER=$(python -c "from huggingface_hub import whoami; print(whoami().get(\"name\", \"\"))")
88
+
89
+ # (Optional) ์˜๊ตฌ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
90
+ echo 'export HF_USER=$(python -c "from huggingface_hub import whoami; print(whoami().get(\"name\", \"\"))")' >> ~/.bashrc
91
+ source ~/.bashrc
92
+ echo ${HF_USER}
93
+
94
+ # Task Name ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
95
+ echo export ${TASK_NAME}=TASK_NAME >> ~/.bashrc # Task ๋ฐ”๋€” ๋•Œ๋งˆ๋‹ค ์ƒˆ๋กœ ์„ค์ •
96
+
97
+ #-------------------------------------------------------------------
98
+ # Huggingface ๋กœ๊ทธ์ธ ํ•˜์ง€ ์•Š๊ณ  ๋กœ์ปฌ์—์„œ ์ง„ํ–‰ํ•˜๋Š” ๊ฒฝ์šฐ
99
+ echo export ${HF_USER}=YOUR_NAME >> ~/.bashrc
100
+ echo export ${TASK_NAME}=TASK_NAME >> ~/.bashrc
101
+ source ~/.bashrc
102
+ echo ${HF_USER} && echo ${TASK_NAME}
103
+ ```
104
+
105
+ - HuggingFace [ํ† ํฐ ์„ค์ • ํŽ˜์ด์ง€](https://huggingface.co/settings/tokens)์—์„œ ์ƒˆ ํ† ํฐ ์ƒ์„ฑ
106
+ - ๊ถŒํ•œ: `Write` ์„ ํƒ (๋‚˜์ค‘์— pretrained model์„ ํ•™์Šต์‹œํ‚ค๋Š” ๊ฒฝ์šฐ `Read` ๊ถŒํ•œ๋„ ํ•„์š”ํ•˜๋ฏ€๋กœ `Fine-grained` ์„ ํƒ ์ถ”์ฒœ)
107
+ - ์ƒ์„ฑ๋œ ํ† ํฐ์„ ์œ„ ๋ช…๋ น์–ด์˜ `YOUR_TOKEN_HERE`์— ์ž…๋ ฅ
108
+
109
+
110
+ ### Record command
111
+ ```bash
112
+ lerobot-record \
113
+ --robot.type=so101_follower \
114
+ --robot.port=/dev/ttyACM0 \
115
+ --robot.id=follower_new_gripper \
116
+ --robot.cameras="{
117
+ top: {type: opencv, index_or_path: "/dev/video0", width: 640, height: 480, fps: 30},
118
+ wrist: {type: opencv, index_or_path: "/dev/video2", width: 640, height: 480, fps: 30}
119
+ }" \
120
+ --teleop.type=so101_leader \
121
+ --teleop.port=/dev/ttyACM1 \
122
+ --teleop.id=leader \
123
+ --dataset.single_task="make a cup of coffee." \
124
+ --dataset.repo_id=${HF_USER}/${TASK_NAME} \
125
+ --dataset.num_episodes=10 \
126
+ --dataset.episode_time_s=1200 \
127
+ --dataset.reset_time_s=3 \
128
+ --display_data=true \
129
+ --dataset.push_to_hub=true \
130
+ --resume=true # ์ด๋ฏธ ์ƒ์„ฑ๋œ ๋ฐ์ดํ„ฐ์…‹ repo์— ์ถ”๊ฐ€๋กœ ๋ฐ์ดํ„ฐ ์ˆ˜์ง‘ํ•˜๋Š” ๊ฒฝ์šฐ
131
+ ```
132
+
133
+ ### ์ฃผ์š” Record ์˜ต์…˜
134
+
135
+ | ์˜ต์…˜ | ์„ค๋ช… |
136
+ |------|------|
137
+ | `--robot.type` | follower ๋กœ๋ด‡ ํƒ€์ž… |
138
+ | `--robot.port` | follower USB ํฌํŠธ |
139
+ | `--robot.id` | follower ๋กœ๋ด‡ ๊ณ ์œ  ID (calibration ๋•Œ ์„ค์ •ํ•œ ID) |
140
+ | `--robot.cameras` | ์นด๋ฉ”๋ผ ์ •๋ณด |
141
+ | `--teleop.type` | leader ๋กœ๋ด‡ ํƒ€์ž… |
142
+ | `--robot.port` | leader USB ํฌํŠธ |
143
+ | `--teleop.id` | leader ๋กœ๋ด‡ ๊ณ ์œ  ID (calibration ๋•Œ ์„ค์ •ํ•œ ID) |
144
+ | `--dataset.single_task` | language instruction |
145
+ | `--dataset.repo_id` | ๋ฐ์ดํ„ฐ์…‹ repo ์ด๋ฆ„ |
146
+ | `--dataset.num_episodes` | ์ˆ˜์ง‘ํ•  ์—ํ”ผ์†Œ๋“œ ์ˆ˜ |
147
+ | `--dataset.episode_time_s` | ๊ฐ ์—ํ”ผ์†Œ๋“œ ๋…นํ™” ์‹œ๊ฐ„ |
148
+ | `--dataset.reset_time_s` | ์—ํ”ผ์†Œ๋“œ ๏ฟฝ๏ฟฝ ๋ฆฌ์…‹ ์‹œ๊ฐ„ |
149
+ | `--display_data` | ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์‹œ๊ฐํ™” ์—ฌ๋ถ€ (`Rerun`) |
150
+ | `--dataset.push_to_hub` | Huggingface ์ž๋™ ์—…๋กœ๋“œ ์—ฌ๋ถ€ |
151
+
152
+
153
+ ### Record ์‹œ ํ‚ค๋ณด๋“œ ๋‹จ์ถ•ํ‚ค
154
+ | ํ‚ค | ๊ธฐ๋Šฅ |
155
+ |----|------|
156
+ | โ†’ | ์—ํ”ผ์†Œ๋“œ ์กฐ๊ธฐ ์ข…๋ฃŒ |
157
+ | โ† | ์—ํ”ผ์†Œ๋“œ ์žฌ๊ธฐ๋ก |
158
+ | ESC | Command ์ข…๋ฃŒ ๋ฐ ์—…๋กœ๋“œ |
159
+
160
+
161
+ ### FileExistsError
162
+ ๋ฐ์ดํ„ฐ์…‹ repo๊ฐ€ ์ด๋ฏธ ์กด์žฌํ•˜๋ฉด ๊ธฐ์กด ๋””๋ ‰ํ† ๋ฆฌ๋ฅผ ๋ฎ์–ด์“ฐ์ง€ ์•Š๊ณ  FileExistsError๊ฐ€ ๋ฐœ์ƒํ•˜๋ฏ€๋กœ ๊ธฐ์กด repo๋ฅผ ์‚ญ์ œํ•˜๊ฑฐ๋‚˜ `--resume=true` ์˜ต์…˜์œผ๋กœ ๊ธฐ์กด repo์— ์ด์–ด์„œ ์ˆ˜์ง‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
163
+
164
+ ```bash
165
+ FileExistsError: [Errno 17] File exists: '/home/.../so101_pickplace'
166
+ ```
167
+
168
+ **ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•:**
169
+ ```bash
170
+ rm -rf ~/.cache/huggingface/lerobot/${HF_USER}/${TASK_NAME}
171
+ ```
172
+
173
+ ---
174
+
175
+ ## 3-3. Replay
176
+
177
+ ```bash
178
+ lerobot-replay \
179
+ --robot.type=so101_follower \
180
+ --robot.port=/dev/so101_follower \
181
+ --robot.id=follower \
182
+ --dataset.repo_id=${HF_USER}/${TASK_NAME} \
183
+ --dataset.episode=0
184
+ ```
185
+
186
+ ### ์ฃผ์š” ์˜ต์…˜
187
+ | ์˜ต์…˜ | ์„ค๋ช… |
188
+ |------|------|
189
+ | `--dataset.repo_id` | ์žฌ์ƒํ•  ๋ฐ์ดํ„ฐ์…‹ repo ์ด๋ฆ„ |
190
+ | `--dataset.episode` | ์žฌ์ƒํ•  ์—ํ”ผ์†Œ๋“œ ๋ฒˆํ˜ธ |
191
+ | `--dataset.fps` | ์žฌ์ƒ ์†๋„ |
192
+
193
+ ---
194
+
195
+ ## 3-4. Visualization (๋ฐ์ดํ„ฐ์…‹ ํ™•์ธ)
196
+
197
+ ### ๋ฐฉ๋ฒ• 1: HuggingFace ์›น์—์„œ ํ™•์ธ
198
+ [LeRobot Dataset Visualizer](https://huggingface.co/spaces/lerobot/visualize_dataset) ํŽ˜์ด์ง€์—์„œ ๋ฐ์ดํ„ฐ์…‹ repo๋ฅผ ์ž…๋ ฅ ํ›„ ํ™•์ธ
199
+
200
+ ### ๋ฐฉ๋ฒ• 2: ๋กœ์ปฌ ์‹œ๊ฐํ™”
201
+
202
+ ```bash
203
+ lerobot-dataset-viz \
204
+ --repo-id=${HF_USER}/${TASK_NAME} \
205
+ --episode-index=0 # visualizeํ•  ์—ํ”ผ์†Œ๋“œ ๋ฒˆํ˜ธ
206
+ ```
207
+ - Rerun ๋ทฐ์–ด ์ž๋™ ์‹คํ–‰
208
+ - ์นด๋ฉ”๋ผ ์˜์ƒ, ๋กœ๋ด‡ ์•ก์…˜, ์ƒํƒœ ์ •๋ณด ํ™•์ธ ๊ฐ€๋Šฅ
209
+
210
+ ---
211
+
4. LeRobot VLA.md ADDED
@@ -0,0 +1,246 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 4. LeRobot VLA
2
+ last updated Dec/26/2025
3
+
4
+ ## 4-1. Train a policy
5
+
6
+ ### Example train command (pi0.5)
7
+
8
+ #### (A) Multi-GPU (Accelerate)
9
+ - ๋ฉ€ํ‹ฐ GPU๋ฅผ ์‚ฌ์šฉํ•  ๊ฒฝ์šฐ ์ผ๋ฐ˜์ ์œผ๋กœ `CUDA_VISIBLE_DEVICES`์— ๋…ธ์ถœํ•œ **GPU ๊ฐœ์ˆ˜ = `--num_processes`** ๋กœ ๋งž์ถฅ๋‹ˆ๋‹ค.
10
+
11
+ ```bash
12
+ # Multi-GPU example (4 GPUs)
13
+ CUDA_VISIBLE_DEVICES=0,1,2,3 \
14
+ accelerate launch --num_processes=4 $(which lerobot-train) \
15
+ --wandb.enable=true \
16
+ --wandb.project=lerobot \
17
+ --dataset.repo_id=${HF_USER}/${TASK_NAME} \
18
+ --output_dir=${HF_USER}/pi05_pickplace \
19
+ --policy.type=pi05 \
20
+ --policy.pretrained_path=lerobot/pi05_base \
21
+ --policy.dtype=bfloat16 \
22
+ --policy.device=cuda \
23
+ --steps=3000 \
24
+ --batch_size=64 \
25
+ --log_freq=100 \
26
+ --save_checkpoint=true \
27
+ --save_freq=500 \
28
+ --policy.push_to_hub=true \
29
+ --policy.repo_id=${HF_USER}/pi05_pickplace
30
+ ```
31
+
32
+ #### (B) Single GPU
33
+ - ์‹ฑ๊ธ€ GPU์—์„œ๋Š” `accelerate launch ...` ๋Œ€์‹  `lerobot-train`์„ ์ง์ ‘ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.
34
+
35
+ ```bash
36
+ CUDA_VISIBLE_DEVICES=0 \
37
+ lerobot-train \
38
+ --wandb.enable=true \
39
+ --wandb.project=lerobot \
40
+ --dataset.repo_id=${HF_USER}/${TASK_NAME} \
41
+ --output_dir=${HF_USER}/pi05_pickplace \
42
+ --policy.type=pi05 \
43
+ --policy.pretrained_path=lerobot/pi05_base \
44
+ --policy.dtype=bfloat16 \
45
+ --policy.device=cuda \
46
+ --steps=3000 \
47
+ --batch_size=64 \
48
+ --log_freq=100 \
49
+ --save_checkpoint=true \
50
+ --save_freq=500 \
51
+ --policy.push_to_hub=true \
52
+ --policy.repo_id=${HF_USER}/pi05_pickplace
53
+ ```
54
+
55
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… | ๋น„๊ณ  |
56
+ |---|---|---|
57
+ | `--policy.device` | ์‚ฌ์šฉ device | |
58
+ | `--policy.dtype` | ํ•™์Šต dtype | |
59
+ | `--steps` | ํ•™์Šต ์Šคํ… ์ˆ˜ | |
60
+ | `--batch_size` | ํ”„๋กœ์„ธ์Šค๋‹น ๋ฐฐ์น˜ ์‚ฌ์ด์ฆˆ | |
61
+ | `--optimizer.grad_clip_norm` | gradient clipping max norm | ์ดˆ๋ฐ˜ ๋ฐœ์‚ฐ/NaN์„ ๋ง‰๋Š” ์•ˆ์ „์žฅ์น˜. ๋„ˆ๋ฌด ์ž‘์œผ๋ฉด ํ•™์Šต์ด ๋‹ต๋‹ตํ•ด์งˆ ์ˆ˜ ์žˆ์–ด โ€œ์•ˆ์ •ํ™” โ†’ ํ•„์š” ์‹œ ์™„ํ™”โ€ ์ˆœ์„œ๋กœ ์กฐ์ •ํ•˜๋Š” ๊ฒŒ ๋ฌด๋‚œํ•จ. |
62
+ | `--num_workers` | DataLoader ์›Œ์ปค ์ˆ˜ | |
63
+ | `--dataset.streaming` | ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ์…‹ ๋ชจ๋“œ | ๋ฐ์ดํ„ฐ๋ฅผ ๋ฏธ๋ฆฌ ๋‹ค cachingํ•˜์ง€ ์•Š๊ณ  ์ˆœ์ฐจ ๊ณต๊ธ‰๋ฐ›๋Š” ๋ชจ๋“œ. streaming์€ ์…”ํ”Œ/๋žœ๋ค ์ ‘๊ทผ์ด ์ œํ•œ๋˜๊ธฐ ์‰ฌ์›Œ ํ•™์Šต ๊ณก์„ ์ด ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์Œ. ๋ฐ์ดํ„ฐ๊ฐ€ ๋„ˆ๋ฌด ์ปค์„œ local cache๊ฐ€ ๋ถ€๋‹ด์ธ ๊ฒฝ์šฐ ์œ ๋ฆฌ. |
64
+ | `--policy.drop_n_last_frames` | ์—ํ”ผ์†Œ๋“œ ๋งˆ์ง€๋ง‰ Nํ”„๋ ˆ์ž„ ์ œ๊ฑฐ | |
65
+ | `--wandb.enable` | W&B ๋กœ๊น… ์‚ฌ์šฉ ์—ฌ๋ถ€ | |
66
+ | `--wandb.project` | W&B ํ”„๋กœ์ ํŠธ๋ช… | |
67
+ | `--job_name` | ํ•™์Šต ์ž‘์—… ์ด๋ฆ„ | |
68
+ | `--output_dir` | ๊ฒฐ๊ณผ/์ฒดํฌํฌ์ธํŠธ ์ถœ๋ ฅ ๊ฒฝ๋กœ | |
69
+ | `--save_checkpoint` | ์ฒดํฌํฌ์ธํŠธ ์ €์žฅ ์—ฌ๋ถ€ | |
70
+ | `--save_freq` | ์ฒดํฌํฌ์ธํŠธ ์ €์žฅ ์ฃผ๊ธฐ(step) | |
71
+ | `--seed` | ๋žœ๋ค ์‹œ๋“œ | |
72
+ | `--resume` | ์ฒดํฌํฌ์ธํŠธ์—์„œ ์žฌ์‹œ์ž‘ | |
73
+ | `--checkpoint_path` | ์žฌ์‹œ์ž‘ ์ฒดํฌํฌ์ธํŠธ ๊ฒฝ๋กœ | |
74
+ | `--rename_map` | observation ํ‚ค ์ด๋ฆ„ ๋งคํ•‘ | ๋ฐ์ดํ„ฐ์…‹์˜ ์นด๋ฉ”๋ผ ์ด๋ฆ„๊ณผ policy๊ฐ€ ๊ธฐ๋Œ€ํ•˜๋Š” ์นด๋ฉ”๋ผ ์ด๋ฆ„์ด ๋‹ค๋ฅผ ๋•Œ ์‚ฌ์šฉ. |
75
+ | `--eval_freq` | ํ•™์Šต ์ค‘ ํ‰๊ฐ€ ์ฃผ๊ธฐ(step) | `eval_freq > 0`์ด์–ด๋„ env ์„ค์ •์ด ํ•จ๊ป˜ ์žˆ์–ด์•ผ ํ‰๊ฐ€ ํ™˜๊ฒฝ์ด ๋งŒ๋“ค์–ด์ง. |
76
+ | `--eval.batch_size` | ํ‰๊ฐ€์šฉ env ๊ฐœ์ˆ˜(n_envs) | |
77
+ | `--eval.use_async_envs` | ๋น„๋™๊ธฐ ํ‰๊ฐ€ env ์‚ฌ์šฉ | |
78
+ | `--eval.n_episodes` | ํ‰๊ฐ€ ์—ํ”ผ์†Œ๋“œ ์ˆ˜ | |
79
+ | `--env.task` | (์‹œ๋ฎฌ) ํ‰๊ฐ€ ํƒœ์Šคํฌ | ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ‰๊ฐ€์—์„œ ์–ด๋–ค ํƒœ์Šคํฌ๋ฅผ ๋Œ๋ฆด์ง€ ์ง€์ •. |
80
+ | `--env.max_parallel_tasks` | (์‹œ๋ฎฌ) ํ‰๊ฐ€ ๋ณ‘๋ ฌ ์ œํ•œ | |
81
+
82
+
83
+ ### ํ•™์Šต ์žฌ์‹œ์ž‘(resume) ์˜ˆ์‹œ
84
+ ```bash
85
+ lerobot-train \
86
+ --resume=true \
87
+ --checkpoint_path="<checkpoint_dir>" ... (๋‚˜๋จธ์ง€ ๋™์ผ)
88
+ ```
89
+
90
+ ### ํ•™์Šต ์ค‘ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ‰๊ฐ€ ์ผœ๊ธฐ
91
+ ํ•™์Šต ์ค‘ ํ‰๊ฐ€๋ฅผ ๋Œ๋ฆฌ๋ ค๋ฉด `--eval_freq`๋ฅผ ์–‘์ˆ˜๋กœ ๋‘๊ณ , env ์„ค์ •๊ณผ eval ์„ค์ •์„ ํ•จ๊ป˜ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค.
92
+
93
+ ```bash
94
+ lerobot-train \
95
+ --eval_freq=500 \
96
+ --env.task="<your_env_task>" \
97
+ --env.max_parallel_tasks=8 \
98
+ --eval.batch_size=8 \
99
+ --eval.use_async_envs=true \
100
+ --eval.n_episodes=50 ... (๋‚˜๋จธ์ง€ ๋™์ผ)
101
+ ```
102
+
103
+ ### (Optional) [RA-BC (Reward-Aligned Behavior Cloning)](https://huggingface.co/docs/lerobot/sarm)
104
+
105
+ RA-BC๋Š” โ€œ๊ทธ๋ƒฅ ์‹œ์—ฐ์„ ๊ทธ๋Œ€๋กœ ๋”ฐ๋ผ ๋ฐฐ์šฐ๋Š”(BC)โ€ ๋Œ€์‹ , **์ง„ํ–‰์ด ์ข‹์•„์ง€๋Š”(=์„ฑ๊ณต์— ๊ฐ€๊นŒ์›Œ์ง€๋Š”) ๊ตฌ๊ฐ„์„ ๋” ๊ฐ•ํ•˜๊ฒŒ ํ•™์Šต**ํ•˜๋„๋ก ์ƒ˜ํ”Œ๋ณ„ ๊ฐ€์ค‘์น˜๋ฅผ ์ ์šฉํ•˜๋Š” ๋ฐฉ์‹์ž…๋‹ˆ๋‹ค.
106
+ LeRobot ํ•™์Šต ์ฝ”๋“œ์—์„œ๋Š” RA-BC๋ฅผ ์ผœ๋ฉด ๋ฐฐ์น˜ ์•ˆ์—์„œ ์ƒ˜ํ”Œ๋งˆ๋‹ค weight๋ฅผ ์ ์šฉํ•ด **๊ฐ€์ค‘ ํ‰๊ท  loss**๋กœ ์—…๋ฐ์ดํŠธํ•˜๋Š” ํ๋ฆ„์„ ํƒ‘๋‹ˆ๋‹ค.
107
+
108
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… | ๋ถ€๊ฐ€ ์„ค๋ช…(์–ธ์ œ/์™œ ์“ฐ๋Š”์ง€) |
109
+ |---|---|---|
110
+ | `--use_rabc` | RA-BC ์‚ฌ์šฉ ์—ฌ๋ถ€ | |
111
+ | `--rabc_progress_path` | progress/๊ฐ€์ค‘์น˜ ํŒŒ์ผ ๊ฒฝ๋กœ | ์‚ฌ์ „์— ๊ณ„์‚ฐ๋œ progress/weight ํŒŒ์ผ ํ•„์š”. |
112
+ | `--rabc_head_mode` | progress head ์„ ํƒ | progress ์˜ˆ์ธก์˜ ์–ด๋–ค head๋ฅผ ์‚ฌ์šฉํ• ์ง€ ์„ ํƒ. |
113
+ | `--rabc_kappa` | ๊ฐ€์ค‘์น˜ ๋ฏผ๊ฐ๋„(์„ธ๊ธฐ) | ๊ฐ’์ด ์ปค์งˆ์ˆ˜๋ก ํŠน์ • ๊ตฌ๊ฐ„์— ๊ณผ๋„ํ•˜๊ฒŒ ์ง‘์ค‘ํ•  ์ˆ˜ ์žˆ์Œ. |
114
+ | `--rabc_epsilon` | ์ˆ˜์น˜ ์•ˆ์ •ํ™” ์ƒ์ˆ˜ | ๋ถ„๋ชจ 0 ๋ฐฉ์ง€ |
115
+
116
+ ---
117
+
118
+ ## 4-2. Run inference
119
+
120
+ ### ๋ฐฉ๋ฒ• 1: Inference by `lerobot-record`
121
+ ```bash
122
+ CUDA_VISIBLE_DEVICES=1 \
123
+ lerobot-record \
124
+ --robot.type=so101_follower \
125
+ --robot.port=/dev/ttyACM0 \
126
+ --robot.id=follower_new_gripper \
127
+ --robot.cameras="{
128
+ top: {type: opencv, index_or_path: /dev/video0, width: 640, height: 480, fps: 30},
129
+ wrist: {type: opencv, index_or_path: /dev/video2, width: 640, height: 480, fps: 30}
130
+ }" \
131
+ --dataset.repo_id=${HF_USER}/eval_pi05 \ # The name of dataset begins by eval to reflect that you are running inference (e.g. ${HF_USER}/eval_pi05)
132
+ --dataset.push_to_hub=false \
133
+ --dataset.single_task="pick a cup and place on the pink circle." \
134
+ --policy.path=${HF_USER}/pi05_pickplace \
135
+ --display_data=true
136
+ ```
137
+
138
+ ### ์ฃผ์š” ์˜ต์…˜ ์„ค๋ช…
139
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… |
140
+ |---|---|
141
+ | `--robot.type` | follower ๋กœ๋ด‡ ํƒ€์ž… |
142
+ | `--robot.port` | follower USB ํฌํŠธ |
143
+ | `--robot.id` | follower ๋กœ๋ด‡ ๊ณ ์œ  ID (calibration ๋•Œ ์„ค์ •ํ•œ ID) |
144
+ | `--robot.cameras` | ์นด๋ฉ”๋ผ ์ •๋ณด |
145
+ | `--dataset.repo_id` | ๊ธฐ๋ก/ํ‰๊ฐ€ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•  HF ๋ฐ์ดํ„ฐ์…‹ repo |
146
+ | `--dataset.push_to_hub` | Huggingface ์ž๋™ ์—…๋กœ๋“œ ์—ฌ๋ถ€ |
147
+ | `--dataset.single_task` | language instruction |
148
+ | `--policy.path` | ์‚ฌ์šฉํ•  ์ •์ฑ…(๋ชจ๋ธ) ๊ฒฝ๋กœ |
149
+ | `--display_data` | ํ™”๋ฉด์— ๋ฐ์ดํ„ฐ ํ‘œ์‹œ ์—ฌ๋ถ€ (Rerun) |
150
+
151
+ ---
152
+
153
+ ### ๋ฐฉ๋ฒ• 2: [Asynchronous inference (PolicyServer + RobotClient)](https://huggingface.co/docs/lerobot/async)
154
+
155
+ ๋น„๋™๊ธฐ(Async) ์ถ”๋ก ์€ **action prediction(์„œ๋ฒ„)** ๊ณผ **action execution(๋กœ๋ด‡)** ์„ ๋ถ„๋ฆฌํ•ฉ๋‹ˆ๋‹ค.
156
+ ๋กœ๋ด‡์€ ์ด๋ฏธ ๋ฐ›์€ action chunk๋ฅผ ์‹คํ–‰ํ•˜๋Š” ๋™์•ˆ, ์„œ๋ฒ„๋Š” ๋‹ค์Œ chunk๋ฅผ ๋ฏธ๋ฆฌ ๊ณ„์‚ฐํ•ด โ€œ๋ฉˆ์นซ๊ฑฐ๋ฆผ(wait-for-inference)โ€์„ ์ค„์ž…๋‹ˆ๋‹ค.
157
+
158
+ #### 0) ์„ค์น˜
159
+
160
+ ```bash
161
+ pip install -e ".[async]"
162
+ ```
163
+
164
+ ---
165
+
166
+ #### 1) Policy Server ์‹คํ–‰ (์„œ๋ฒ„/๋ณ„๋„ ํ„ฐ๋ฏธ๋„)
167
+
168
+ `PolicyServer`๋Š” ์ฒ˜์Œ์—๋Š” โ€œ๋นˆ ์ปจํ…Œ์ด๋„ˆโ€๋กœ ๋œจ๊ณ , **์ฒซ ์—ฐ๊ฒฐ(handshake) ๋•Œ ํด๋ผ์ด์–ธํŠธ๊ฐ€ ์–ด๋–ค policy๋ฅผ ์–ด๋–ค device๋กœ ๋Œ๋ฆด์ง€**๋ฅผ ์ „๋‹ฌํ•ฉ๋‹ˆ๋‹ค.
169
+
170
+ ```bash
171
+ # ๊ธฐ๋ณธ ์˜ˆ์‹œ (localhost:8080)
172
+ CUDA_VISIBLE_DEVICES=0,1,2,3 \
173
+ python -m lerobot.async_inference.policy_server \
174
+ --host=127.0.0.1 \
175
+ --port=8080
176
+ ```
177
+
178
+ ---
179
+
180
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… | ๋ถ€๊ฐ€ ์„ค๋ช… |
181
+ |---|---|---|
182
+ | `--host` | ์„œ๋ฒ„ ๋ฐ”์ธ๋”ฉ ์ฃผ์†Œ | ๊ฐ™์€ ๋จธ์‹ ์ด๋ฉด `127.0.0.1`/`localhost`. ๋„คํŠธ์›Œํฌ๋กœ ๋ถ™์ผ ๊ฑฐ๋ฉด `0.0.0.0` ๋˜๋Š” ์„œ๋ฒ„ IP๋ฅผ ์‚ฌ์šฉ. |
183
+ | `--port` | ์„œ๋ฒ„ ํฌํŠธ | ๊ธฐ๋ณธ `8080`. |
184
+ | `--fps` | ์„œ๋ฒ„ ๊ธฐ์ค€ FPS | ๊ธฐ๋ณธ 30. `environment_dt = 1/fps`๋กœ ๋‚ด๋ถ€ timestep์„ ๊ณ„์‚ฐ. |
185
+ | `--inference_latency` | ๋ชฉํ‘œ ์ถ”๋ก  ์ง€์—ฐ(์ดˆ) | ๊ธฐ๋ณธ `1/fps`. ์„œ๋ฒ„๋Š” action chunk๋ฅผ ๋งŒ๋“  ๋’ค, ์ด ๊ฐ’์— ๋งž์ถฐ pacing(sleep)ํ•ฉ๋‹ˆ๋‹ค. |
186
+ | `--obs_queue_timeout` | observation queue timeout(์ดˆ) | ๊ธฐ๋ณธ 2์ดˆ. timeout ๋‚ด์— obs๊ฐ€ ์—†์œผ๋ฉด ์„œ๋ฒ„๋Š” ๋นˆ ์‘๋‹ต์„ ๋ฐ˜ํ™˜. |
187
+
188
+ > ์ฐธ๊ณ : ์„œ๋ฒ„๋Š” observation queue๋ฅผ `maxsize=1`๋กœ ๋‘๊ณ  โ€œ๊ฐ€์žฅ ์ตœ์‹  obsโ€ ์œ„์ฃผ๋กœ ์ฒ˜๋ฆฌํ•ฉ๋‹ˆ๋‹ค.
189
+
190
+ ---
191
+
192
+ #### 2) Robot Client ์‹คํ–‰ (๋กœ๋ด‡/๋ณ„๋„ ํ„ฐ๋ฏธ๋„)
193
+
194
+ ํด๋ผ์ด์–ธํŠธ๋Š” ๋กœ๋ด‡์—์„œ observation์„ ์ฝ์–ด ์„œ๋ฒ„๋กœ ๋ณด๋‚ด๊ณ , ์„œ๋ฒ„๊ฐ€ ๋งŒ๋“  action chunk๋ฅผ ๋ฐ›์•„ **๋กœ์ปฌ ํ์— ์Œ“์•„ ์‹คํ–‰**ํ•ฉ๋‹ˆ๋‹ค.
195
+
196
+ ```bash
197
+ python -m lerobot.async_inference.robot_client \
198
+ --server_address=127.0.0.1:8080 \
199
+ --robot.type=so101_follower \
200
+ --robot.port=/dev/so101_follower \
201
+ --robot.id=follower \
202
+ --robot.cameras='{
203
+ top: {type: opencv, index_or_path: "/dev/video0", width: 640, height: 480, fps: 30},
204
+ wrist: {type: opencv, index_or_path: "/dev/video2", width: 640, height: 480, fps: 30}
205
+ }' \
206
+ --task="pick a cup and place on the pink circle." \
207
+ --policy_type=pi05 \
208
+ --pretrained_name_or_path=${HF_USER}/pi05_pickplace \
209
+ --policy_device=cuda \
210
+ --actions_per_chunk=50 \
211
+ --chunk_size_threshold=0.7 \
212
+ --aggregate_fn_name=weighted_average \
213
+ --debug_visualize_queue_size=True
214
+ ```
215
+
216
+ | ์˜ต์…˜๋ช… | ์„ค๋ช… | ๋ถ€๊ฐ€ ์„ค๋ช… |
217
+ |---|---|---|
218
+ | `--server_address` | ์„œ๋ฒ„ ์ฃผ์†Œ | `host:port` ํ˜•ํƒœ. |
219
+ | `--robot.type` | ๋กœ๋ด‡ ํƒ€์ž… | |
220
+ | `--robot.port` | ๋กœ๋ด‡ ํฌํŠธ | |
221
+ | `--robot.id` | ๋กœ๋ด‡ ID | |
222
+ | `--robot.cameras` | ์นด๋ฉ”๋ผ ์„ค์ • | **ํ‚ค ์ด๋ฆ„์ด policy๊ฐ€ ๊ธฐ๋Œ€ํ•˜๋Š” ์ด๋ฏธ์ง€ ํ‚ค**์™€ ๋งž์•„์•ผ ํ•จ. (ํ‚ค ๋ถˆ์ผ์น˜ ์‹œ ์ด๋ฏธ์ง€๊ฐ€ โ€œ์—†๋Š” ๊ฒƒ์ฒ˜๋Ÿผโ€ ๋“ค์–ด๊ฐ€๊ฑฐ๋‚˜ ์ „์ฒ˜๋ฆฌ์—์„œ ๋ฌธ์ œ ๋ฐœ์ƒ) |
223
+ | `--task` | language instruction | |
224
+ | `--policy_type` | ์ •์ฑ… ํƒ€์ž… | |
225
+ | `--pretrained_name_or_path` | ๋ชจ๋ธ ๊ฒฝ๋กœ | |
226
+ | `--policy_device` | (์„œ๋ฒ„) ์ •์ฑ… ์‹คํ–‰ device | ์ด ๊ฐ’์€ โ€œํด๋ผ์ด์–ธํŠธ ์˜ต์…˜โ€์ด์ง€๋งŒ, ์‹ค์ œ๋กœ๋Š” **์„œ๋ฒ„ ์ชฝ์—์„œ** ๋ชจ๋ธ์„ ์˜ฌ๋ฆด device๋ฅผ ์ง€์ •ํ•จ. |
227
+ | `--actions_per_chunk` | ํ•œ ๋ฒˆ์— ๋ฐ›์„ action ์ˆ˜ | ๊ถŒ์žฅ ๋ฒ”์œ„๋Š” ๋ณดํ†ต 10~50. ๊ฐ’์ด ํฌ๋ฉด ๋Š๊น€์ด ์ค„ ์ˆ˜ ์žˆ์ง€๋งŒ, ๋” ๋จผ horizon ์˜ˆ์ธก์œผ๋กœ ์˜ค์ฐจ ๋ˆ„์  ๊ฐ€๋Šฅ์„ฑ์ด ์ปค์ง. |
228
+ | `--chunk_size_threshold` | obs ์ „์†ก ํŠธ๋ฆฌ๊ฑฐ | ํ๊ฐ€ โ€œ์–ผ๋งˆ๋‚˜ ๋น„์—ˆ์„ ๋•Œโ€ ์ƒˆ obs๋ฅผ ๋ณด๋‚ด ๋‹ค์Œ chunk๋ฅผ ๋ฏธ๋ฆฌ ๊ณ„์‚ฐํ• ์ง€ ๊ฒฐ์ •. ๊ฐ’์ด ํด์ˆ˜๋ก ์ ์‘์„ฑโ†‘(๋” ์ž์ฃผ ์—…๋ฐ์ดํŠธ) / ์„œ๋ฒ„ ๋ถ€๋‹ดโ†‘. |
229
+ | `--aggregate_fn_name` | action ๊ฒน์นจ ๊ตฌ๊ฐ„ ๋ณ‘ํ•ฉ ํ•จ์ˆ˜ | action chunk๊ฐ€ ๊ฒน์น˜๋ฉด ๊ฐ™์€ timestep์˜ action์„ ์„ž์Œ. (์˜ˆ: `weighted_average`, `latest_only`, `average`, `conservative`) |
230
+ | `--debug_visualize_queue_size` | queue ํฌ๊ธฐ ์‹œ๊ฐํ™” | ์ข…๋ฃŒ ์‹œ queue ํฌ๊ธฐ ๋ณ€ํ™”๋ฅผ plotting |
231
+ | `--fps` | ํด๋ผ์ด์–ธํŠธ ์ œ์–ด ๋ฃจํ”„ FPS | ๊ธฐ๋ณธ 30. `environment_dt=1/fps`๋กœ sleep์„ ์กฐ์ ˆ. |
232
+
233
+ ---
234
+
235
+ #### 3) ํŠœ๋‹ ํŒ (actions_per_chunk / chunk_size_threshold)
236
+
237
+ - `actions_per_chunk` โ†‘: actions ๊ณ ๊ฐˆ ์œ„ํ—˜ โ†“ โ†’ ๋Š๊น€ โ†“ / ํ•˜์ง€๋งŒ ๊ธด horizon ์˜ˆ์ธก โ†’ ์˜ค์ฐจ ๋ˆ„์  ๊ฐ€๋Šฅ์„ฑ โ†‘
238
+ - `chunk_size_threshold` โ†‘: ๋” ์ž์ฃผ ์ƒˆ chunk ์š”์ฒญ โ†’ ์ ์‘์„ฑ โ†‘ / ํ•˜์ง€๋งŒ ์„œ๋ฒ„ ์š”์ฒญ๋Ÿ‰ โ†‘ โ†’ ์ถ”๋ก  ํŒŒ์ดํ”„๋ผ์ธ ๋ถ€๋‹ด โ†‘
239
+
240
+ ---
241
+
242
+ #### 4) ์ž์ฃผ ๋‚˜๋Š” ๋ฌธ์ œ
243
+ - **์„œ๋ฒ„๋Š” ๋œจ๋Š”๋ฐ ๋กœ๋ด‡์ด ์•ˆ ์›€์ง์ž„**: `server_address`๊ฐ€ ์„œ๋ฒ„์˜ `host:port`์™€ ์ผ์น˜ํ•˜๋Š”์ง€, ๋ฐฉํ™”๋ฒฝ/ํฌํŠธ๊ฐ€ ๋ง‰ํ˜€์žˆ์ง€ ์•Š์€์ง€ ํ™•์ธ
244
+ - **์นด๋ฉ”๋ผ ํ‚ค ๋งค์นญ ์˜ค๋ฅ˜**: `--robot.cameras`์˜ ํ‚ค ์ด๋ฆ„์ด policy๊ฐ€ ๊ธฐ๋Œ€ํ•˜๋Š” ์ด๋ฏธ์ง€ ํ‚ค์™€ ์ผ์น˜ํ•ด์•ผ ํ•จ
245
+ - **Stop-and-go**: `actions_per_chunk`๋ฅผ ๋Š˜๋ฆฌ๊ฑฐ๋‚˜ `chunk_size_threshold`๋ฅผ ํ‚ค์šฐ๋˜, ์„œ๋ฒ„ ๋ฆฌ์†Œ์Šค/๋„คํŠธ์›Œํฌ ์ง€์—ฐ์„ ํ•จ๊ป˜ ๊ณ ๋ ค
246
+