README.md

Manual Evaluation Guide

This guide provides step-by-step instructions for evaluating our trained policies.

Actually, our required enviroment is consistent with the official.

Prerequisites

• Python 3.11
• uv package manager
• GPU driver and CUDA supporting IsaacSim5.1.0.

Installation Steps

1. Clone the Repository

git clone https://github.com/lehome-official/lehome-challenge.git
cd lehome-challenge

2. Install Dependencies with uv

uv sync

This will create a virtual environment and install all required dependencies.

3. Clone and Configure IsaacLab

cd third_party
git clone https://github.com/lehome-official/IsaacLab.git
cd ..

4. Install IsaacLab

Activate the virtual environment and install IsaacLab:

source .venv/bin/activate
./third_party/IsaacLab/isaaclab.sh -i none

5. Install LeHome Package

Finally, install the LeHome package in development mode:

uv pip install -e ./source/lehome

---

Download Assets

1.Download Simulation Assets

Download the required simulation assets (scenes, objects, robots) from HuggingFace:

# This creates the Assets/ directory with all required simulation resources
hf download lehome/asset_challenge --repo-type dataset --local-dir Assets

2.Download Example Dataset

Four types of garments are provided. Download from HuggingFace:

hf download lehome/dataset_challenge_merged --repo-type dataset --local-dir Datasets/example

Evaluation

python -m scripts.eval \
    --policy_type lerobot \
    --policy_path outputs/train/act_four_types_0331/checkpoints/060000/pretrained_model \
    --garment_type "custom" \
    --dataset_root Datasets/example/four_types_merged \
    --num_episodes 5 \
    --enable_cameras \
    --device cpu    

The policy_path, garment_type and dataset_root may have to be changed to the actual ones used.