robometer/dataset_upload/dataset_guides/FailSafe.md

FailSafe Dataset Guide

This guide explains how to integrate and use the FailSafe dataset (correctdata_v2) with the Robometer pipeline.

Source: https://huggingface.co/datasets/onepiece1999/correctdata_v2/tree/main

Overview

• The dataset contains three tasks with failures and successes:
  • FailPickCube-v1: "Pick up the red cube"
  • FailPushCube-v1: "push and move a cube to a goal region in front of it"
  • FailStackCube-v1: "Pick up a red cube and stack it on top of a green cube and let go of the cube without it falling."
• Each task has ~100 seed folders. Each seed contains a Ground_Truth/ (success) and multiple failure attempt folders.
• Each attempt has camera views front/, side/, wrist/ with ordered .png frames.
• Additionally, JSON files (e.g., vla_data_FailPickCube-v1.json, vla_data_GT_PickCube-v1.json) provide sub-task annotated mini-trajectories with success/failure labels.

Directory Structure

<dataset_path>/
  FailPickCube-v1/
    0/
      Ground_Truth/
        front/0.png ...
        side/*.png
        wrist/*.png
      1_trans_x_2/
        front/*.png
        side/*.png
        wrist/*.png
    1/
      ...
  FailPushCube-v1/
    ...
  FailStackCube-v1/
    ...
  json_files/
    vla_data_FailPickCube-v1.json
    vla_data_GT_PickCube-v1.json
    ...

Configuration (configs/data_gen_configs/failsafe.yaml)

# configs/data_gen_configs/failsafe.yaml

dataset:
  dataset_path: ./datasets/failsafe
  dataset_name: failsafe

output:
  output_dir: ./robometer_dataset/failsafe_rfm
  max_trajectories: -1
  max_frames: 64
  use_video: true
  fps: 10
  shortest_edge_size: 240
  center_crop: false

hub:
  push_to_hub: true
  hub_repo_id: failsafe_rfm

Loader

• File: dataset_upload/dataset_loaders/failsafe_loader.py
• Function: load_failsafe_dataset(dataset_path)
• The loader:
  • Builds full episodes from Ground_Truth/ (success) and failure attempt folders for the chosen camera view.
  • Parses vla_data_*.json entries to create sub-task mini-trajectories using the specified image paths and labels; includes the sub_task text in the trajectory task field.
  • Uses the per-task instructions above when JSONs do not specify an instruction.

Usage

uv run python -m dataset_upload.generate_hf_dataset --config_path=dataset_upload/configs/data_gen_configs/failsafe.yaml

This will:

• Load full episodes and sub-task mini-trajectories for the selected view
• Generate web-optimized videos
• Create a HuggingFace dataset ready to push or save

Notes

• If JSON files are at the dataset root instead of json_files/, the loader will look there as a fallback.
• You can switch view in the YAML to side or wrist to regenerate the dataset for a different camera.