Robotics
LeRobot
Safetensors
diffusion
Eshwar-2123 commited on
Commit
9f6bc81
·
verified ·
1 Parent(s): 45d8c18

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +144 -26
README.md CHANGED
@@ -1,34 +1,152 @@
1
- # Diffusion Policy for Pick-and-Place (SO-100)
 
 
 
 
 
 
 
 
 
 
2
 
3
- ## Overview
4
- This model is a Diffusion Policy trained using Hugging Face LeRobot for a single-task pick-and-place manipulation problem.
5
 
6
- ## Robot
7
- - SO-100 follower arm
8
- - Front camera
9
- - Wrist camera
10
 
11
- ## Task
12
- Pick the blue cube from one of four predefined locations and place it into a fixed target box.
13
 
14
- ## Dataset
15
- - 40 teleoperated demonstrations
16
- - Dual-camera observations
17
- - Dataset:
18
- Eshwar-2123/pick_place_blue_cube_box_clean_v1_20260706_125359
19
 
20
- ## Training
21
- - Policy: Diffusion Policy
22
- - Steps: 30,000
23
- - Batch size: 16
24
- - GPU: NVIDIA RTX 5080
25
 
26
- ## Results
27
- - Lowest validation loss: 0.037 (10k steps)
28
- - Final training loss: 0.0036
29
- - Final validation loss: 0.1303
30
 
31
- ## Usage
32
 
33
- ```python
34
- from lerobot.policies import DiffusionPolicy
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets: Eshwar-2123/pick_place_blue_cube_box_clean_v1_20260706_125359
3
+ library_name: lerobot
4
+ license: apache-2.0
5
+ model_name: diffusion
6
+ pipeline_tag: robotics
7
+ tags:
8
+ - lerobot
9
+ - robotics
10
+ - diffusion
11
+ ---
12
 
13
+ # Model Card for Diffusion Policy
 
14
 
15
+ [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://huggingface.co/papers/2303.04137) is an imitation learning policy that models robot actions as a denoising diffusion process. It is particularly effective for learning multi-modal manipulation behaviors from demonstrations while producing smooth action trajectories.
 
 
 
16
 
17
+ <!--
18
+ Add a demo GIF after benchmarking.
19
 
20
+ <p align="center">
21
+ <img src="https://huggingface.co/Eshwar-2123/diffusion_pick_place_clean/resolve/main/demo.gif" width="60%"/>
22
+ </p>
23
+ -->
 
24
 
25
+ This policy has been trained and pushed to the Hub using [LeRobot](https://github.com/huggingface/lerobot).
 
 
 
 
26
 
27
+ Learn how to train and run it in the [LeRobot Diffusion Policy guide](https://huggingface.co/docs/lerobot/main/en/diffusion), or browse the [full documentation](https://huggingface.co/docs/lerobot/index).
 
 
 
28
 
29
+ ---
30
 
31
+ # Model Details
32
+
33
+ - **License:** apache-2.0
34
+ - **Robot type:** `so_follower`
35
+ - **Cameras:** `front`, `wrist`
36
+
37
+ ## Inputs & Outputs
38
+
39
+ The policy consumes these observation features and produces these action features.
40
+
41
+ ### Inputs
42
+
43
+ | Feature | Type | Shape |
44
+ | --- | --- | --- |
45
+ | `observation.state` | STATE | `(6,)` |
46
+ | `observation.images.front` | VISUAL | `(3, 480, 640)` |
47
+ | `observation.images.wrist` | VISUAL | `(3, 480, 640)` |
48
+
49
+ ### Outputs
50
+
51
+ | Feature | Type | Shape |
52
+ | --- | --- | --- |
53
+ | `action` | ACTION | `(6,)` |
54
+
55
+ ---
56
+
57
+ # Training Dataset
58
+
59
+ - **Repository:** [Eshwar-2123/pick_place_blue_cube_box_clean_v1_20260706_125359](https://huggingface.co/datasets/Eshwar-2123/pick_place_blue_cube_box_clean_v1_20260706_125359)
60
+ - **Episodes:** 47
61
+ - **Frames:** 20321
62
+ - **Frame rate:** 30 FPS
63
+ - **Task(s):** ""
64
+
65
+ <a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=Eshwar-2123/pick_place_blue_cube_box_clean_v1_20260706_125359">
66
+ <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
67
+ <img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
68
+ </a>
69
+
70
+ ---
71
+
72
+ # Training Configuration
73
+
74
+ | Setting | Value |
75
+ | --- | --- |
76
+ | Training steps | 30000 |
77
+ | Batch size | 16 |
78
+ | Optimizer | adamw |
79
+ | Learning rate | 1e-04 |
80
+ | Seed | 1000 |
81
+ | LeRobot version | 0.5.2 |
82
+
83
+ ---
84
+
85
+ # How to Get Started with the Model
86
+
87
+ New to LeRobot? These guides cover the full workflow:
88
+
89
+ - **[Install LeRobot](https://huggingface.co/docs/lerobot/main/en/installation)** — set up the `lerobot` package.
90
+ - **[Hardware setup](https://huggingface.co/docs/lerobot/main/en/hardware_guide)** — assemble, wire, and calibrate your robot and cameras.
91
+ - **[Record data & train a policy](https://huggingface.co/docs/lerobot/en/il_robots)** — the end-to-end imitation-learning walkthrough.
92
+ - **[CLI cheat-sheet](https://huggingface.co/docs/lerobot/main/en/cheat-sheet)** — quick reference for the `lerobot-*` commands.
93
+
94
+ ## Run the policy on your robot
95
+
96
+ ```bash
97
+ lerobot-rollout \
98
+ --strategy.type=base \
99
+ --robot.type=so_follower \
100
+ --robot.port=<your_robot_port> \
101
+ --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
102
+ --policy.path=Eshwar-2123/diffusion_pick_place_clean \
103
+ --task="" \
104
+ --duration=60
105
+ ```
106
+
107
+ Replace the remaining `<...>` placeholders with your own values.
108
+
109
+ ---
110
+
111
+ ## Train your own policy
112
+
113
+ ```bash
114
+ lerobot-train \
115
+ --dataset.repo_id=${HF_USER}/<dataset> \
116
+ --policy.type=diffusion \
117
+ --output_dir=outputs/train/<policy_repo_id> \
118
+ --job_name=lerobot_training \
119
+ --policy.device=cuda \
120
+ --policy.repo_id=${HF_USER}/<policy_repo_id> \
121
+ --wandb.enable=true
122
+ ```
123
+
124
+ _Writes checkpoints to `outputs/train/<policy_repo_id>/checkpoints/`._
125
+
126
+ ---
127
+
128
+ # Evaluation
129
+
130
+ _No evaluation results have been provided for this policy yet._
131
+
132
+ ---
133
+
134
+ # Citation
135
+
136
+ If you use this policy, please cite **Diffusion Policy**, along with **LeRobot**.
137
+
138
+ ```bibtex
139
+ @article{chi2023diffusionpolicy,
140
+ title={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion},
141
+ author={Chi, Cheng and Feng, Siyuan and Du, Yilun and Xu, Zheng and Cousineau, Eric and Burchfiel, Benjamin and Song, Shuran},
142
+ journal={RSS},
143
+ year={2023}
144
+ }
145
+
146
+ @misc{cadene2024lerobot,
147
+ author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
148
+ title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
149
+ howpublished = "\url{https://github.com/huggingface/lerobot}",
150
+ year = {2024}
151
+ }
152
+ ```