Humanoid Everyday Dataset Guide

This guide explains how to integrate and use the Humanoid Everyday dataset with the Robometer training pipeline.

Source: https://github.com/ausbxuse/Humanoid-Everyday

Overview

The Humanoid Everyday Dataset is a diverse collection of humanoid robot (Unitree G1 and H1) demonstrations recorded at 30 Hz across everyday tasks. This dataset supports research in robot learning, imitation, and perception.

Total Download Size: ~500 GB (across 250 tasks), over 100,000 time-step recorded
Tasks: 260 diverse scenarios (loco-manipulation, basic manipulation, tool use, deformables, articulated objects, human–robot interaction)
Episodes per task: 40
Recording Frequency: 30 Hz

Modalities captured

Low-dimensional:
- Joint states (arm, leg, hand)
- IMU (orientation, accelerometer, gyroscope, RPY)
- Odometry/Kinematics (position, velocity, orientation)
- Hand pressure sensors (G1 only)
- Teleoperator hands/head actions from Apple Vision Pro
- Inverse kinematics data
High-dimensional:
- Egocentric RGB images (480x640x3, PNG)
- Depth maps (480x640, uint16)
- LiDAR point clouds (~6k points per step, PCD)

Prerequisites

Install humanoid_everyday dataloader

git clone https://github.com/ausbxuse/Humanoid-Everyday
cd Humanoid-Everyday
uv pip install -e .

Download dataset

Please visit the task spreadsheet to download your task of interest, or use the provided download script:

bash dataset_upload/data_scripts/humanoid_everyday/download_humanoid_everyday.sh

Directory Structure

<dataset_path>/
  task1.zip
  task2.zip
  ...
  taskN.zip

Each zip file contains a complete task dataset with multiple episodes.

Data Schema

Each time step is represented by a Python dictionary with the following fields:

{
    # Scalar identifiers
    "time": np.float64,                # UNIX timestamp (s)
    "robot_type": np.str_,             # Robot model identifier (G1 only)

    # Robot states
    "states": {
        "arm_state": np.ndarray((14,), dtype=np.float64),   # 14 joint angles
        "leg_state": np.ndarray((15 or 13,), dtype=np.float64),  # 15 joint angles for G1, 13 for H1_2
        "hand_state": np.ndarray((14 or 12,), dtype=np.float64), # 14 joint angles for Unitree Dex3 Hand, 12 for Inspire Dextrous Hand
        "hand_pressure_state": [...],  # List of per-sensor readings (9 sensors per hand)
        "imu": {
            "quaternion": np.ndarray((4,), dtype=np.float64),    # [w, x, y, z]
            "accelerometer": np.ndarray((3,), dtype=np.float64), # [ax, ay, az]
            "gyroscope": np.ndarray((3,), dtype=np.float64),     # [gx, gy, gz]
            "rpy": np.ndarray((3,), dtype=np.float64)            # [roll, pitch, yaw]
        },
        "odometry": {
            "position": np.ndarray((3,), dtype=np.float64),  # [x, y, z]
            "velocity": np.ndarray((3,), dtype=np.float64),  # [vx, vy, vz]
            "rpy": np.ndarray((3,), dtype=np.float64),      # [roll, pitch, yaw]
            "quat": np.ndarray((4,), dtype=np.float64)      # [w, x, y, z]
        }
    },

    # Control commands and solutions
    "actions": {
        "right_angles": np.ndarray((7,), dtype=np.float64),  # commanded joint angles
        "left_angles": np.ndarray((7,), dtype=np.float64),   # commanded joint angles
        "armtime": np.float64,                               # timestamp
        "iktime": np.float64,                                # timestamp
        "sol_q": np.ndarray((14,), dtype=np.float64),       # solution joint angles
        "tau_ff": np.ndarray((14,), dtype=np.float64),      # feedforward torques
        "head_rmat": np.ndarray((3, 3), dtype=np.float64),  # rotation matrix
        "left_pose": np.ndarray((4, 4), dtype=np.float64),  # homogeneous transform
        "right_pose": np.ndarray((4, 4), dtype=np.float64)  # homogeneous transform
    },

    # High-dimensional observations
    "image": np.ndarray((480, 640, 3), dtype=np.uint8),     # RGB image
    "depth": np.ndarray((480, 640), dtype=np.uint16),       # Depth map
    "lidar": np.ndarray((~6000, 3), dtype=np.float64)       # around 6000 points for lidar point cloud
}

Configuration (configs/data_gen_configs/humanoid_everyday.yaml)

# configs/data_gen_configs/humanoid_everyday.yaml

dataset:
  dataset_path: "./datasets/humanoid_everyday"  # Path containing zip files
  dataset_name: humanoid_everyday_rfm

output:
  output_dir: ./robometer_dataset/humanoid_everyday_rfm
  max_trajectories: -1
  max_frames: 64
  use_video: true
  fps: 10
  shortest_edge_size: 240
  center_crop: false
  num_workers: 4

hub:
  push_to_hub: true
  hub_repo_id: humanoid_everyday_rfm

Usage

uv run python -m dataset_upload.generate_hf_dataset --config dataset_upload/configs/data_gen_configs/humanoid_everyday.yaml

This will:

Find all zip files in the specified dataset path
For each zip file, extract the task name and load episodes using the humanoid_everyday dataloader
Extract RGB images from each episode
Convert frames to web-optimized videos and create a HuggingFace dataset
Use the zip filename (without extension) as the task description

Data Fields

Each trajectory includes:

id: Unique identifier
task: Task name extracted from zip filename
frames: Relative path to the generated clip video
is_robot: True
quality_label: "successful"
partial_success: N/A (fixed by pipeline)
data_source: humanoid_everyday

Example Usage with Dataloader

from humanoid_everyday import Dataloader

# Load your downloaded task's dataset zip file (e.g., the "push_a_button" task)
ds = Dataloader("~/Downloads/push_a_button.zip")
print("Episode length of dataset:", len(ds))

# Displaying high dimensional data at first episode, second timestep.
ds.display_image(0, 1)
ds.display_depth_point_cloud(0, 1)
ds.display_lidar_point_cloud(0, 1)

for i, episode in enumerate(ds):
    if i == 1:  # episode 1
        print("RGB image shape:", episode[0]["image"].shape)  # (480, 640, 3)
        print("Depth map shape:", episode[0]["depth"].shape)  # (480, 640)
        print("LiDAR points shape:", episode[0]["lidar"].shape)  # (~6000, 3)

        batch = episode[0:4]  # batch loading episodes
        print(batch[1]["image"].shape)
        print(batch[0]["image"].shape)

Troubleshooting

Missing humanoid_everyday package: Install it with pip install humanoid_everyday or clone and install from the GitHub repository
No zip files found: Ensure the dataset_path contains zip files with humanoid everyday datasets
Import errors: Make sure the humanoid_everyday package is properly installed and accessible
Memory issues: Adjust max_frames and num_workers parameters to reduce memory usage
Long episodes: Episodes longer than 1000 frames are automatically skipped to prevent memory issues

License

This dataset is released under the MIT License.