| # RACER-augmented RLBench Dataset Guide |
|
|
| This guide explains how to load and convert the RACER-augmented RLBench dataset with the Robometer pipeline. |
|
|
| Sources: |
| - Dataset card: `https://huggingface.co/datasets/sled-umich/RACER-augmented_rlbench` |
| - Example JSON: `https://huggingface.co/datasets/sled-umich/RACER-augmented_rlbench/blob/main/samples/close_jar/0/language_description.json` |
|
|
| ## Overview |
|
|
| - Train/validation split under `train/` and `val/` directories (sometimes `train/samples/`) |
| - Each task (e.g., `close_jar`) contains multiple numbered episodes with: |
| - `language_description.json` (contains `task_goal` and per-frame `subgoal` entries) |
| - Camera folders: `front_rgb/`, `left_shoulder_rgb/`, `right_shoulder_rgb` (or `right_shoudler_rgb`), `wrist_rgb/` |
|
|
| We use `task_goal` as the language instruction for all trajectories. |
| - Success trajectories: full expert episode for each camera view |
| - Failure trajectories: for any expert frame with heuristic failures in `augmentation`, construct a failure episode consisting of expert frames up to that frame (inclusive), for each camera view |
|
|
| ## Configuration |
|
|
| ```yaml |
| # configs/data_gen_configs/racer_train.yaml |
| |
| dataset: |
| dataset_path: ./datasets/racer |
| dataset_name: racer_train |
| |
| output: |
| output_dir: ./robometer_dataset/racer_train_rfm |
| max_trajectories: -1 |
| max_frames: 64 |
| use_video: true |
| fps: 10 |
| shortest_edge_size: 240 |
| center_crop: false |
| |
| hub: |
| push_to_hub: true |
| hub_repo_id: racer_train_rfm |
| ``` |
|
|
| For validation, use `racer_val.yaml` analogously. |
|
|
| ## Loader |
|
|
| - File: `dataset_upload/dataset_loaders/racer_loader.py` |
| - Function: `load_racer_dataset(dataset_path, dataset_name)` |
| - Notes: |
| - Handles both `train/` and `validation/` (and the `samples/` subfolder if present) |
| - Uses `task_goal` from `language_description.json` |
| - Builds successes and heuristic failure truncations per camera view |
|
|
| ## Usage |
|
|
| ```bash |
| uv run python -m dataset_upload.generate_hf_dataset --config_path=dataset_upload/configs/data_gen_configs/racer_train.yaml |
| ``` |
|
|
| This will: |
| - Load expert and heuristic failure episodes |
| - Generate web-optimized videos per camera view |
| - Create a HuggingFace dataset for the train split (use the val YAML for validation) |
|
|
