MoTiF Dataset Guide

This guide explains how to integrate and use the MoTiF-1K dataset with the Robometer data pipeline using a FrameLoader (no HuggingFace conversion required).

Source: https://github.com/Minyoung1005/motif#data-structure

Overview

1K trajectories across 13 task categories; both human and robot (Stretch) motions
Visual motion representations provided; we support raw video or frame directories
We use a simple FrameLoader to load frames on-demand for each trajectory

Directory Structure

As per the MoTiF README, after unzipping MotIF.zip under ./data, ./data/MotIF contains at least:

./data/MotIF/
  annotations/
  human_motion/
  stretch_motion/

Our loader first looks for annotations under annotations/ to pair sources with language text; if absent, it will scan human_motion/ and stretch_motion/ for videos or frame directories.

Loader

File: dataset_upload/dataset_loaders/motif_loader.py
Exposes load_motif_dataset(dataset_path: str) -> dict[str, list[dict]]
Each trajectory dictionary contains:
- id: unique id
- task: from annotations if available, otherwise "MoTiF"
- frames: MotifFrameLoader that lazily reads frames (video file or directory of images)
- is_robot: inferred from path (stretch/robot -> True, human -> False)
- quality_label: "successful"
- partial_success: 1
- data_source: "motif"

Configuration (configs/data_gen_configs/motif.yaml)

# configs/data_gen_configs/motif.yaml

dataset:
  dataset_path: ./datasets/MotIF
  dataset_name: motif

output:
  output_dir: ./robometer_dataset/motif_rfm
  max_trajectories: -1
  max_frames: 64
  use_video: true
  fps: 10
  shortest_edge_size: 240
  center_crop: false

hub:
  push_to_hub: false
  hub_repo_id: motif_rfm

Usage Example

uv run python -m dataset_upload.generate_hf_dataset --config dataset_upload/configs/data_gen_configs/motif.yaml

This will:

Find all zip files in the specified dataset path
For each zip file, extract the task name and load episodes using the humanoid_everyday dataloader
Extract RGB images from each episode
Convert frames to web-optimized videos and create a HuggingFace dataset
Use the zip filename (without extension) as the task description

Notes

Annotations: The loader tries to parse any JSON/JSONL files under annotations/ to find (source_path, text) pairs. Supported keys include video_path|path|image_dir|frames_dir and narration|instruction|task|description|caption.
Frame directories: If a directory contains images (e.g., .jpg, .png), it is treated as a sequence of frames.
Video support: Common video formats are supported via OpenCV (e.g., .mp4, .mov).
If you need to use specific MoTiF visual motion representations (e.g., storyboard, optical flow), point source_path to those assets and the FrameLoader will load images in order.