Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
|
|
| 1 |
---
|
| 2 |
-
datasets:
|
|
|
|
| 3 |
library_name: lerobot
|
| 4 |
license: apache-2.0
|
| 5 |
model_name: diffusion
|
|
@@ -8,55 +10,120 @@ tags:
|
|
| 8 |
- lerobot
|
| 9 |
- robotics
|
| 10 |
- diffusion
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
#
|
| 14 |
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
| 16 |
|
|
|
|
| 17 |
|
| 18 |
-
|
| 19 |
|
|
|
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
---
|
| 25 |
|
| 26 |
-
## How to
|
| 27 |
|
| 28 |
-
|
| 29 |
-
Below is the short version on how to train and run inference/eval:
|
| 30 |
|
| 31 |
-
###
|
| 32 |
|
| 33 |
```bash
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
--output_dir=outputs/train/<desired_policy_repo_id> \
|
| 38 |
-
--job_name=lerobot_training \
|
| 39 |
-
--policy.device=cuda \
|
| 40 |
-
--policy.repo_id=${HF_USER}/<desired_policy_repo_id>
|
| 41 |
-
--wandb.enable=true
|
| 42 |
-
```
|
| 43 |
|
| 44 |
-
|
| 45 |
|
| 46 |
-
|
| 47 |
|
| 48 |
```bash
|
| 49 |
-
|
| 50 |
-
--
|
| 51 |
-
--
|
| 52 |
-
--
|
| 53 |
-
--
|
|
|
|
|
|
|
| 54 |
```
|
| 55 |
|
| 56 |
-
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
-
|
|
|
|
| 1 |
+
````markdown
|
| 2 |
---
|
| 3 |
+
datasets:
|
| 4 |
+
- lerobot/aloha_sim_insertion_human
|
| 5 |
library_name: lerobot
|
| 6 |
license: apache-2.0
|
| 7 |
model_name: diffusion
|
|
|
|
| 10 |
- lerobot
|
| 11 |
- robotics
|
| 12 |
- diffusion
|
| 13 |
+
- aloha
|
| 14 |
+
- imitation-learning
|
| 15 |
+
- benchmark
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# Diffusion Policy for Aloha Insertion (200k Steps)
|
| 19 |
|
| 20 |
+
**Task**: Aloha Insertion (Simulated, 3D Manipulation)
|
| 21 |
+
**Algorithm**: [Diffusion Policy](https://huggingface.co/papers/2303.04137) (DDPM)
|
| 22 |
+
**Training Steps**: 200,000
|
| 23 |
+
**Author**: Graduate Student, UESTC (University of Electronic Science and Technology of China)
|
| 24 |
|
| 25 |
+
This model represents a benchmark experiment for **Diffusion Policy** on the challenging **Aloha Insertion** task (Simulated). It was trained using the [LeRobot](https://github.com/huggingface/lerobot) framework to evaluate the algorithm's performance on complex, high-dimensional 3D manipulation tasks compared to baseline methods (ACT).
|
| 26 |
|
| 27 |
+
### 🔬 Benchmark Results
|
| 28 |
|
| 29 |
+
This experiment highlights the significant difficulty of the Aloha Insertion task for generative policies under limited compute constraints (Batch Size=8).
|
| 30 |
|
| 31 |
+
**Evaluation Metrics (50 Episodes):**
|
| 32 |
+
|
| 33 |
+
| Metric | Value | Note |
|
| 34 |
+
| :--- | :--- | :--- |
|
| 35 |
+
| **Success Rate** | **0.0%** | Task difficulty is extremely high; model struggled to complete final insertion. |
|
| 36 |
+
| **Avg Max Reward** | **0.10** | Indicates partial success in grasping/moving, but failed alignment. |
|
| 37 |
+
| **Avg Sum Reward** | **8.20** | Shows the model learned valid trajectories but lacked precision at the goal state. |
|
| 38 |
+
|
| 39 |
+
**Analysis:**
|
| 40 |
+
Unlike the 2D Push-T task where Diffusion Policy excelled, the high-dimensional visual input (3 cameras) and precise 3D spatial reasoning required for Aloha Insertion proved challenging. While the ACT baseline achieved a 2% success rate (1/50), Diffusion Policy (at 200k steps) demonstrated trajectory learning but failed to finalize the insertion, suggesting a need for longer training or larger batch sizes for this specific domain.
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
|
| 44 |
+
## Model Details
|
| 45 |
+
|
| 46 |
+
- **Architecture**: ResNet18 (Vision Backbone) + U-Net (Diffusion Head)
|
| 47 |
+
- **Input**: 3 Camera Views (Top, Left, Right)
|
| 48 |
+
- **Prediction Horizon**: 16 steps
|
| 49 |
+
- **Observation History**: 2 steps
|
| 50 |
+
- **Action Steps**: 8 steps
|
| 51 |
+
- **Image Resolution**: 480x640 (Cropped to 420x560)
|
| 52 |
+
- **Total Parameters**: ~263 Million
|
| 53 |
|
| 54 |
---
|
| 55 |
|
| 56 |
+
## How to Use This Model
|
| 57 |
|
| 58 |
+
You can evaluate this model or visualize its performance using `lerobot`.
|
|
|
|
| 59 |
|
| 60 |
+
### 1. Installation
|
| 61 |
|
| 62 |
```bash
|
| 63 |
+
# Install LeRobot
|
| 64 |
+
pip install lerobot
|
| 65 |
+
````
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
### 2\. Evaluate / Visualize
|
| 68 |
|
| 69 |
+
Run the following command in your terminal to evaluate the model for 50 episodes and save the visualization videos:
|
| 70 |
|
| 71 |
```bash
|
| 72 |
+
python -m lerobot.scripts.lerobot_eval \
|
| 73 |
+
--policy.type diffusion \
|
| 74 |
+
--policy.pretrained_path Lemon-03/DP_Aloha_Insertion_test \
|
| 75 |
+
--eval.n_episodes 50 \
|
| 76 |
+
--eval.batch_size 8 \
|
| 77 |
+
--env.type aloha \
|
| 78 |
+
--env.task AlohaInsertion-v0
|
| 79 |
```
|
| 80 |
|
| 81 |
+
-----
|
| 82 |
|
| 83 |
+
## Training Configuration (Reference)
|
| 84 |
|
| 85 |
+
For reproducibility, here are the key parameters used during the training session:
|
| 86 |
+
|
| 87 |
+
- **Batch Size**: 8 (Limited by 8GB VRAM)
|
| 88 |
+
- **Optimizer**: Adam (lr=1e-4, betas=[0.95, 0.999])
|
| 89 |
+
- **Scheduler**: Cosine with warmup (500 steps)
|
| 90 |
+
- **Vision**: ResNet18 with GroupNorm
|
| 91 |
+
- **Precision**: Mixed Precision (AMP) enabled
|
| 92 |
+
|
| 93 |
+
<!-- end list -->
|
| 94 |
+
|
| 95 |
+
```bash
|
| 96 |
+
# Original Training Command
|
| 97 |
+
python -m lerobot.scripts.lerobot_train \
|
| 98 |
+
--config_path diffusion_aloha.yaml \
|
| 99 |
+
--env.type aloha \
|
| 100 |
+
--env.task AlohaInsertion-v0 \
|
| 101 |
+
--dataset.repo_id lerobot/aloha_sim_insertion_human \
|
| 102 |
+
--wandb.enable true \
|
| 103 |
+
--job_name DP_Aloha_Insertion \
|
| 104 |
+
--policy.repo_id Lemon-03/DP_Aloha_Insertion_test
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
### Config File (`diffusion_aloha.yaml`) Snippet:
|
| 108 |
+
|
| 109 |
+
```yaml
|
| 110 |
+
# Training
|
| 111 |
+
steps: 200000
|
| 112 |
+
eval_freq: 20000
|
| 113 |
+
save_freq: 20000
|
| 114 |
+
batch_size: 8
|
| 115 |
+
|
| 116 |
+
# Policy
|
| 117 |
+
policy:
|
| 118 |
+
type: diffusion
|
| 119 |
+
vision_backbone: resnet18
|
| 120 |
+
crop_shape: [420, 560]
|
| 121 |
+
n_action_steps: 8
|
| 122 |
+
n_obs_steps: 2
|
| 123 |
+
horizon: 16
|
| 124 |
+
num_inference_steps: 100
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
```
|
| 128 |
|
| 129 |
+
---
|