robometer/dataset_upload/dataset_guides/LIBERO.md

LIBERO Dataset Guide

LIBERO is a benchmark for lifelong robot learning with built-in support in the Robometer training pipeline.

Overview

• 📁 Local File Support: Processes HDF5 files from local storage
• 🎮 Simulation Data: High-quality manipulation tasks
• 🏠 Multiple Environments: Living room, kitchen, office, and study scenarios
• 📊 Structured Tasks: Clear task descriptions and optimal trajectories

Prerequisites

Download LIBERO Dataset

# Clone or download LIBERO dataset
git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git
# Follow LIBERO installation instructions for dataset download

# This should work too
git submodule update --init --recursive

cd deps/libero/LIBERO
uv run python benchmark_scripts/download_libero_datasets.py --datasets DATASET

where DATASET is chosen from [libero_spatial, libero_object, libero_100, libero_goal].

Quick Start

0. Set Hugging Face repo ID

Before we start, you must have an HF account which will be pushed to. You will set this by setting

export HF_USERNAME=<insert HF username here>

Then, for each dataset, run with the all the datasets you would like to process

Option 1: Use Default Configuration

uv run python dataset_upload/generate_hf_dataset.py \
    --config_path=dataset_upload/configs/data_gen_configs/libero.yaml\
    --dataset.dataset_path=deps/libero/LIBERO/libero/datasets/libero_90 \
    --dataset.dataset_name=libero_90

If all your LIBERO data exists in the path above, you can use the following utility script

uv run bash dataset_upload/data_scripts/libero/gen_all_libero.sh

Option 2: Custom Configuration

uv run python dataset_upload/generate_hf_dataset.py \
    --config_path=dataset_upload/configs/data_gen_configs/libero.yaml \
    --dataset.dataset_path=/path/to/your/libero/dataset \
    --dataset.dataset_name=libero_custom \
    --output.output_dir=libero_robometer_dataset \
    --output.max_trajectories=1000 \
    --output.use_video=true \
    --output.fps=10

Configuration Options

Create a custom config file configs/data_gen_configs/libero.yaml:

dataset:
  dataset_path: LIBERO/libero/datasets/libero_90
  dataset_name: libero_90

output:
  output_dir: libero_dataset
  max_trajectories: -1  # Process all trajectories
  max_frames: 32
  use_video: true
  fps: 10

hub:
  push_to_hub: false
  hub_repo_id: your-username/libero_rfm

Data Structure Processed

LIBERO Dataset:
├── *.hdf5 files             ← PROCESSED
│   ├── /data/
│   │   └── trajectory_*/
│   │       ├── obs/
│   │       │   └── agentview_rgb    ← EXTRACTED as frames
│   │       └── actions              ← EXTRACTED as actions
└── Generated Output:
    ├── frames: List[np.ndarray]     ← RGB video frames
    ├── actions: np.ndarray          ← Robot actions
    ├── task: str                    ← Parsed from filename
    └── optimal: "optimal"           ← All LIBERO data assumed optimal

Supported LIBERO Variants

• LIBERO-90: 90 tasks across 4 environments
• LIBERO-10: 10 benchmark tasks
• Custom datasets: Any LIBERO-format HDF5 files

Sample Output

Loading LIBERO dataset from: LIBERO/libero/datasets/libero_90
Found 90 HDF5 files
Processing LIBERO dataset, 90 files: 100%|██████████| 90/90

Sample trajectory:
- Task: "stack the right bowl on the left bowl and place them in the tray"
- Frames: (128, 128, 128, 3) RGB images
- Actions: (128, 7) joint positions
- Environment: LIVING_ROOM_SCENE4

File Name Parsing

LIBERO dataset automatically parses task information from HDF5 filenames:

LIVING_ROOM_SCENE4_stack_the_right_bowl_on_the_left_bowl_and_place_them_in_the_tray.hdf5
│              │    │
│              │    └── Task description
│              └── Scene identifier  
└── Environment type

Performance Notes

• Processing Speed: ~2-5 files/second
• Memory Usage: Moderate (loads one HDF5 file at a time)
• Storage: Variable (depends on trajectory length)
• Video Encoding: Converts RGB arrays to MP4 format

Troubleshooting

HDF5 File Issues

# Check HDF5 file structure
import h5py
with h5py.File('path/to/file.hdf5', 'r') as f:
    print(list(f.keys()))  # Should show 'data'
    print(list(f['data'].keys()))  # Should show trajectory keys

Missing Observations

Ensure your LIBERO dataset has the expected structure:

/data/demo_0/obs/agentview_rgb  # RGB frames
/data/demo_0/actions            # Action sequences

Memory Issues

For large LIBERO datasets:

# Process in chunks
uv run python data/generate_hf_dataset.py \
    --config_path=configs/data_gen_configs/libero.yaml \
    --output.max_trajectories=100  # Limit trajectories

Integration with Robometer Training

# Train on processed LIBERO dataset
uv run accelerate launch --config_file configs/fsdp.yaml train.py \
    --config_path=configs/config.yaml \
    --dataset.dataset_path=libero_dataset/libero_90