xtli commited on
Commit
9cdd667
·
verified ·
1 Parent(s): f6e1c23

Upload 4 files

Browse files
Files changed (4) hide show
  1. LEHOME_REPO_README.md +184 -0
  2. README.md +61 -3
  3. UPLOAD_TO_HF.md +14 -0
  4. train_act_1.yaml +39 -0
LEHOME_REPO_README.md ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <p align="center">
2
+ <h1 align="center">
3
+ LeHome Challenge 2026
4
+ </h1>
5
+ <h2 align="center">
6
+ Challenge on Garment Manipulation Skill Learning in Household Scenarios
7
+ </h2>
8
+
9
+ <h3 align="center">
10
+ <a href="https://lehome-challenge.com/">Competition Website</a>
11
+ </h3>
12
+ </p>
13
+
14
+ <div align="center">
15
+
16
+ [![Python](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/)
17
+ [![Isaac Lab](https://img.shields.io/badge/Isaac%20Lab-2.3.1-green.svg)](https://isaac-sim.github.io/IsaacLab/main/index.html)
18
+ [![LeRobot](https://img.shields.io/badge/LeRobot-0.4.2-yellow.svg)](https://github.com/huggingface/lerobot)
19
+ [![License](https://img.shields.io/badge/license-Apache%202.0-red.svg)](LICENSE)
20
+ [![ICRA](https://img.shields.io/badge/ICRA-2026-orange.svg)](https://2026.ieee-icra.org/program/competitions/)
21
+
22
+ </div>
23
+
24
+ ## 📑 Table of Contents
25
+
26
+ - [📑 Table of Contents](#-table-of-contents)
27
+ - [🚀 Quick Start](#-quick-start)
28
+ - [1. Installation](#1-installation)
29
+ - [Use UV](#use-uv)
30
+ - [Use Docker](#use-docker)
31
+ - [2. Assets \& Data Preparation](#2-assets--data-preparation)
32
+ - [Download Simulation Assets](#download-simulation-assets)
33
+ - [Download Example Dataset](#download-example-dataset)
34
+ - [Collect Your Own Data](#collect-your-own-data)
35
+ - [3. Train](#3-train)
36
+ - [Quick Start](#quick-start)
37
+ - [4. Eval](#4-eval)
38
+ - [Common Options](#common-options)
39
+ - [Garment Test Configuration](#garment-test-configuration)
40
+ - [📮 Submission](#-submission)
41
+ - [🧩 Acknowledgments](#-acknowledgments)
42
+
43
+ ## 🚀 Quick Start
44
+
45
+ > ⚠️ **IMPORTANT**:
46
+ > For Ubuntu version and GPU-related settings, please refer to the [IsaacSim 5.1.0 Documentation](https://docs.isaacsim.omniverse.nvidia.com/5.1.0/installation/requirements.html). And the simulation currently only supports CPU devices.
47
+
48
+ ### 1. Installation
49
+ We offer two installation methods: UV and Docker for submission and local evaluation.
50
+
51
+ #### Use UV
52
+
53
+ The simulation environment is based on the IssacLab and LeRobot repositories; please refer to [UV installation guide](docs/installation.md).
54
+
55
+ #### Use Docker
56
+
57
+ ***Docker is not ready yet; please use uv for Installation for now.***
58
+
59
+ ### 2. Assets & Data Preparation
60
+
61
+ #### Download Simulation Assets
62
+
63
+ Download the required simulation assets (scenes, objects, robots) from HuggingFace:
64
+
65
+ ```bash
66
+ # This creates the Assets/ directory with all required simulation resources
67
+ hf download lehome/asset_challenge --repo-type dataset --local-dir Assets
68
+ ```
69
+
70
+ #### Download Example Dataset
71
+
72
+ We provide demonstrations for four types of garments. Download from HuggingFace:
73
+
74
+ ```bash
75
+ hf download lehome/dataset_challenge_merged --repo-type dataset --local-dir Datasets/example
76
+ ```
77
+
78
+ If you need depth information or individual data for each garment. Download from HuggingFace:
79
+
80
+ ```bash
81
+ hf download lehome/dataset_challenge --repo-type dataset --local-dir Datasets/example
82
+ ```
83
+
84
+ #### Collect Your Own Data
85
+ For detailed instructions on teleoperation data collection and dataset processing, please refer to our [Dataset Collection and Processing Guide](docs/datasets.md) ( using SO101 Leader is strongly recommended).
86
+
87
+ ### 3. Train
88
+
89
+ We provide several training examples; the models and training framework are from LeRobot.
90
+
91
+ #### Quick Start
92
+
93
+ Train using one of the pre-configured training files:
94
+
95
+ ```bash
96
+ lerobot-train --config_path=configs/train_<policy>.yaml
97
+ ```
98
+
99
+ **Available config files:**
100
+ - `configs/train_act.yaml` - ACT
101
+ - `configs/train_dp.yaml` - DP
102
+ - `configs/train_smolvla.yaml` - SmolVLA
103
+
104
+ **Key configuration options:**
105
+ - **Dataset path**: Update `dataset.root` to point to your dataset
106
+ - **Input/Output features**: Specify which observations and actions to use
107
+ - **Training parameters**: Adjust `batch_size`, `steps`, `save_freq`, etc.
108
+ - **Output directory**: Modify `output_dir` to save models elsewhere
109
+
110
+ > 📖 **For detailed training instructions, feature selection guide, and configuration options, see our [Training Guide](docs/training.md).**
111
+
112
+ ### 4. Eval
113
+
114
+ Evaluate your trained policy on the challenge garments. The framework supports LeRobot policies and custom implementations.
115
+
116
+ **Examples:**
117
+
118
+ ```bash
119
+ # Evaluate using LeRobot policy
120
+ # Note: --policy_path and --dataset_root are required parameters for LeRobot policies, ready to run once the dataset and model checkpoints are prepared.
121
+ python -m scripts.eval \
122
+ --policy_type lerobot \
123
+ --policy_path outputs/train/act_top_long/checkpoints/last/pretrained_model \
124
+ --garment_type "top_long" \
125
+ --dataset_root Datasets/example/top_long_merged \
126
+ --num_episodes 2 \
127
+ --enable_cameras \
128
+ --device cpu
129
+
130
+ # Evaluate custom policy
131
+ # Note: Participants can define their own model loading logic within the policy class. Provides flexibility for participants to implement specialized loading and inference logic.
132
+ python -m scripts.eval \
133
+ --policy_type custom \
134
+ --garment_type "top_long" \
135
+ --num_episodes 5 \
136
+ --enable_cameras \
137
+ --device cpu
138
+ ```
139
+
140
+ #### Common Options
141
+
142
+ | Parameter | Description | Default | Required For |
143
+ |-----------|-------------|---------|--------------|
144
+ | `--policy_type` | Policy type: `lerobot`, `custom` | `lerobot` | All |
145
+ | `--policy_path` | Path to model checkpoint | - | All (passed as `model_path` for custom) |
146
+ | `--dataset_root` | Dataset path (for metadata) | - | **LeRobot only** |
147
+ | `--garment_type` | Type of garments: `top_long`, `top_short`, `pant_long`, `pant_short`, `custom` | `top_long` | All |
148
+ | `--num_episodes` | Episodes per garment | `5` | All |
149
+ | `--max_steps` | Max steps per episode | `600` | All |
150
+ | `--save_video` | Save evaluation videos | | All |
151
+ | `--video_dir` | Directory to save evaluation videos | `outputs/eval_videos` | `--save_video` |
152
+ | `--enable_cameras` | Enable camera rendering | | All |
153
+ | `--device` | Device for inference: only `cpu` |'cpu'| All |
154
+ | `--headless` | Used for evaluation without GUI | disabled | All |
155
+
156
+ **Parameter Descriptions:**
157
+
158
+ * **Required for LeRobot Policy**: `--policy_path` (model path) and `--dataset_root` (dataset path, used for loading metadata).
159
+ * **Custom Policy**: `--policy_path` is passed to the policy constructor as `model_path`. Participants can define their own model loading logic (refer to `scripts/eval_policy/example_participant_policy.py`).
160
+
161
+
162
+ #### Garment Test Configuration
163
+ Evaluation is performed on the `Release` set of garments. Under the directory `Assets/objects/Challenge_Garment/Release`, each garment category folder contains a corresponding text file listing the garment names (e.g., `Top_Long/Top_Long.txt`).
164
+
165
+ * **Evaluate a Category**: Set `--garment_type` to `top_long`, `top_short`, `pant_long`, or `pant_short` to evaluate all garments within that category.
166
+ * **Evaluate Specific Garments**: Edit `Assets/objects/Challenge_Garment/Release/Release_test_list.txt` to include only the garments you want to test, then run with `--garment_type custom`.
167
+
168
+ > 📖 **For detailed policy evaluation guide**, see [eval_guide](docs/policy_eval.md).
169
+
170
+
171
+ ## 📮 Submission
172
+
173
+ Once you are satisfied with your model's performance, follow these steps to submit your results to the competition leaderboard:
174
+
175
+ >Submission instructions will be available on the [competition website](https://lehome-challenge.com/).
176
+
177
+ ## 🧩 Acknowledgments
178
+
179
+ This project stands on the shoulders of giants. We utilize and build upon the following excellent open-source projects:
180
+
181
+ - **[Isaac Sim](https://docs.isaacsim.omniverse.nvidia.com/5.1.0/index.html)** - For photorealistic physics simulation
182
+ - **[Isaac Lab](https://isaac-sim.github.io/IsaacLab/main/index.html)** - For modular robot learning environments
183
+ - **[LeRobot](https://github.com/huggingface/lerobot)** - For state-of-the-art Imitation Learning algorithms
184
+ - **[Marble](https://marble.worldlabs.ai/)** - For diverse simulation scene generation
README.md CHANGED
@@ -1,3 +1,61 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - lerobot
5
+ - lehome
6
+ - act
7
+ - robotics
8
+ ---
9
+
10
+ # ACT Policy For LeHome Challenge
11
+
12
+ This repository contains a LeRobot-format ACT checkpoint exported from:
13
+
14
+ - Experiment: `outputs/train/act_top_longee_depth_bs32`
15
+ - Checkpoint: `outputs/train/act_top_longee_depth_bs32/checkpoints/0000125000`
16
+ - Team: `HandX`
17
+ - Registration ID: `r26`
18
+ - Contact: `xtli312@163.com`
19
+
20
+ ## Expected Layout
21
+
22
+ - `pretrained_model/`
23
+ - `train_act_1.yaml`
24
+ - `LEHOME_REPO_README.md`
25
+
26
+ ## Evaluation
27
+
28
+ Use this checkpoint with the LeHome evaluation script:
29
+
30
+ ```bash
31
+ python -m scripts.eval \
32
+ --policy_type lerobot \
33
+ --policy_path "<downloaded_bundle>/pretrained_model" \
34
+ --dataset_root "Datasets/example/top_long_merged" \
35
+ --garment_type "custom" \
36
+ --num_episodes 5 \
37
+ --task_description "fold the garment on the table" \
38
+ --enable_cameras \
39
+ --device cpu
40
+ ```
41
+
42
+ ## Required Dependencies
43
+
44
+ - Python 3.11
45
+ - Isaac Sim 5.1.0
46
+ - Isaac Lab 2.3.1
47
+ - `lerobot`
48
+ - LeHome repository source code
49
+
50
+ ## Notes
51
+
52
+ - The challenge organizers also need the official assets and dataset metadata.
53
+ - If you upload this bundle to Hugging Face, point the Google Form `Policy Submission` field to that repository URL.
54
+ - Local self-reported evaluation on `Release_test_list.txt` with 5 episodes per garment:
55
+ - Total Episodes: `15`
56
+ - Average Return: `183.60 +- 67.87`
57
+ - Success Rate: `20.00%`
58
+ - `Top_Long_Seen_6`: `20.00%` success, `224.89` avg return
59
+ - `Top_Long_Seen_7`: `0.00%` success, `205.17` avg return
60
+ - `Top_Long_Unseen_1`: `40.00%` success, `120.74` avg return
61
+ - Generated on 2026-04-02T12:02:18.
UPLOAD_TO_HF.md ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Upload Instructions
2
+
3
+ From the repository root:
4
+
5
+ ```bash
6
+ huggingface-cli repo create <your-hf-repo-name> --type model
7
+ huggingface-cli upload <your-username>/<your-hf-repo-name> "/data3/lxt-24/lehome-challenge/submission/hf/act_top_longee_depth_bs32_0000125000_hf_bundle" .
8
+ ```
9
+
10
+ After upload, use:
11
+
12
+ `https://huggingface.co/<your-username>/<your-hf-repo-name>`
13
+
14
+ as the `Policy Submission` link in the leaderboard form.
train_act_1.yaml ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ dataset:
2
+ repo_id: repo_act
3
+ root: Datasets/example/top_long_merged
4
+
5
+ policy:
6
+ type: act
7
+ device: cuda
8
+ push_to_hub: false
9
+
10
+ input_features:
11
+ observation.ee_pose:
12
+ type: STATE
13
+ shape: [16]
14
+ observation.images.top_rgb:
15
+ type: VISUAL
16
+ shape: [3, 480, 640]
17
+ observation.images.left_rgb:
18
+ type: VISUAL
19
+ shape: [3, 480, 640]
20
+ observation.images.right_rgb:
21
+ type: VISUAL
22
+ shape: [3, 480, 640]
23
+ observation.top_depth:
24
+ type: STATE
25
+ shape: [1, 480, 640]
26
+
27
+ output_features:
28
+ action:
29
+ type: ACTION
30
+ shape: [12]
31
+
32
+ output_dir: outputs/train/act_top_longee_depth_bs32
33
+ batch_size: 32
34
+ steps: 3000000000
35
+ save_freq: 5000
36
+ log_freq: 1000
37
+
38
+ wandb:
39
+ enable: false