| # MoTiF Dataset Guide |
|
|
| This guide explains how to integrate and use the MoTiF-1K dataset with the Robometer data pipeline using a FrameLoader (no HuggingFace conversion required). |
|
|
| Source: `https://github.com/Minyoung1005/motif#data-structure` |
|
|
| ## Overview |
|
|
| - 1K trajectories across 13 task categories; both human and robot (Stretch) motions |
| - Visual motion representations provided; we support raw video or frame directories |
| - We use a simple FrameLoader to load frames on-demand for each trajectory |
|
|
| ## Directory Structure |
|
|
| As per the MoTiF README, after unzipping `MotIF.zip` under `./data`, `./data/MotIF` contains at least: |
|
|
| ``` |
| ./data/MotIF/ |
| annotations/ |
| human_motion/ |
| stretch_motion/ |
| ``` |
|
|
| Our loader first looks for annotations under `annotations/` to pair sources with language text; if absent, it will scan `human_motion/` and `stretch_motion/` for videos or frame directories. |
|
|
| ## Loader |
|
|
| - File: `dataset_upload/dataset_loaders/motif_loader.py` |
| - Exposes `load_motif_dataset(dataset_path: str) -> dict[str, list[dict]]` |
| - Each trajectory dictionary contains: |
| - `id`: unique id |
| - `task`: from annotations if available, otherwise "MoTiF" |
| - `frames`: `MotifFrameLoader` that lazily reads frames (video file or directory of images) |
| - `is_robot`: inferred from path (`stretch`/`robot` -> True, `human` -> False) |
| - `quality_label`: "successful" |
| - `partial_success`: 1 |
| - `data_source`: "motif" |
|
|
| ## Configuration (configs/data_gen_configs/motif.yaml) |
|
|
| ```yaml |
| # configs/data_gen_configs/motif.yaml |
| |
| dataset: |
| dataset_path: ./datasets/MotIF |
| dataset_name: motif |
| |
| output: |
| output_dir: ./robometer_dataset/motif_rfm |
| max_trajectories: -1 |
| max_frames: 64 |
| use_video: true |
| fps: 10 |
| shortest_edge_size: 240 |
| center_crop: false |
| |
| hub: |
| push_to_hub: false |
| hub_repo_id: motif_rfm |
| ``` |
|
|
| ## Usage Example |
|
|
| ```bash |
| uv run python -m dataset_upload.generate_hf_dataset --config dataset_upload/configs/data_gen_configs/motif.yaml |
| ``` |
|
|
| This will: |
| - Find all zip files in the specified dataset path |
| - For each zip file, extract the task name and load episodes using the humanoid_everyday dataloader |
| - Extract RGB images from each episode |
| - Convert frames to web-optimized videos and create a HuggingFace dataset |
| - Use the zip filename (without extension) as the task description |
| |
| ## Notes |
| |
| - Annotations: The loader tries to parse any JSON/JSONL files under `annotations/` to find `(source_path, text)` pairs. Supported keys include `video_path|path|image_dir|frames_dir` and `narration|instruction|task|description|caption`. |
| - Frame directories: If a directory contains images (e.g., `.jpg`, `.png`), it is treated as a sequence of frames. |
| - Video support: Common video formats are supported via OpenCV (e.g., `.mp4`, `.mov`). |
| - If you need to use specific MoTiF visual motion representations (e.g., storyboard, optical flow), point `source_path` to those assets and the FrameLoader will load images in order. |
|
|
