robometer/dataset_upload/dataset_guides/Galaxea.md

Galaxea Open-World Dataset Guide

This guide explains how to integrate and use the Galaxea Open-World RLDS dataset with the Robometer training pipeline.

Source: https://huggingface.co/datasets/OpenGalaxea/Galaxea-Open-World-Dataset

Can download it with hf download OpenGalaxea/Galaxea-Open-World-Dataset --repo-type dataset --include "*rlds*" --local-dir ./datasets/galaxea

Also, need to install some extra dependencies:

uv pip install tensorflow-datasets
uv pip install tensorflow
uv pip install tf-keras

Overview

• 500+ hours of real-world mobile manipulation data in RLDS and LeRobot formats.
• Fine-grained subtask language annotations at step level via language_instruction.
• Multiple RLDS builders (e.g., part1_r1_lite, sample_r1_lite) under a common rlds/ root.

Directory Structure

<dataset_path>/
  rlds/
    sample_r1_lite/
      1.0.0/
        dataset_info.json
        features.json
        merge_dataset_large_r1_lite-train.tfrecord-00000-of-01024
        ...
    part1_r1_lite/
      1.0.0/
      ...

Language Instruction Schema

As documented, language_instruction encodes three parts separated by @:

• high_level @ low_level_chinese @ low_level_english

We extract the low-level English (the third part) and use it as the task string for embeddings.

Configuration (configs/data_gen_configs/galaxea.yaml)

# configs/data_gen_configs/galaxea.yaml

dataset:
  dataset_path: ./datasets/galaxea
  dataset_name: galaxea_part1_r1_lite # choose from part1_r1_lite, part2_r1_lite, part3_r1_lite, part4_r1_lite, part5_r1_lite, ...

output:
  output_dir: ./robometer_dataset/galaxea_rfm
  max_trajectories: -1
  max_frames: 64
  use_video: true
  fps: 10
  shortest_edge_size: 240
  center_crop: false
  num_workers: 4

hub:
  push_to_hub: true
  hub_repo_id: galaxea_rfm

Usage

bash dataset_upload/data_scripts/galaxea/gen_all_galaxea.sh

This will:

• Iterate the listed RLDS builders under rlds/
• For each episode, parse language_instruction and extract the low-level English instruction
• Select camera views (image_camera_head, image_camera_wrist_left, image_camera_wrist_right)
• Convert frames to web-optimized videos and create a HuggingFace dataset

Data Fields

Each trajectory includes:

• id: Unique identifier
• task: Low-level English instruction (parsed from language_instruction)
• frames: Relative path to the generated clip video
• is_robot: True
• quality_label: "successful"
• partial_success: N/A (fixed by pipeline)
• data_source: galaxea

Troubleshooting

• Builder not found: Ensure the RLDS version directories exist under rlds/<name>/.
• Missing instruction: If no language_instruction is present or malformed, the episode is skipped.
• Performance: Adjust num_workers and batch size inside the loader if needed.