README.md

MolmoAct2-BimanualYAM — Rerun RRD conversion

Rerun recordings of the MolmoAct2-BimanualYAM dataset: 32,246 teleoperated episodes (about 704 hours) of bimanual tabletop manipulation on two I2RT YAM arms, with three cameras. Converted from the full release.

Each episode becomes several .rrd files, one per layer, sharing a single recording_id. Open them in the Rerun Viewer, or register them as a catalog dataset. Layers live in their own directories, so one episode is spread across the bucket like this:

base/episode_<NNNNN>.rrd       # the episode as recorded ('base' layer), with its metadata as properties
urdf/…                         # the two posed arms: forward kinematics from the joint state
cameras/…                      # the three cameras placed in the scene
stats/…                        # the source's statistics of the episode, and how far each arm moves
assets/urdf-model-left.rrd     # the left arm's meshes, one for the whole dataset
assets/urdf-model-right.rrd    # the right arm's meshes, one for the whole dataset
blueprints/default.rbl         # default viewer layout, one for the whole dataset

<NNNNN> is the source episode_index, zero-padded to five digits (00000…32245).

Provenance and attribution

This dataset is a converted derivative of allenai/MolmoAct2-BimanualYAM-Dataset, released by the Allen Institute for AI. The original authors released it under the Apache License 2.0, at source revision e9f21ae1, dated 2026-06-11.

All credit for the underlying data collection belongs to the original authors. The original authors do not endorse this conversion.

Changes from the source

• The source packs many consecutive episodes into each parquet and mp4 file. Every episode becomes its own Rerun recording.
• The base layer keeps the state, action, and task of every frame, and copies each camera's AV1 samples without re-encoding, except that a video window starting on a delta frame has its frames up to the next keyframe re-encoded.
• Three layers are added beyond the base: urdf poses the two arms, cameras places the three cameras, and stats holds the per-episode statistics from meta/episodes (not the dataset-wide meta/stats.json) and how far each arm moves. The source has no calibration, so the arm spacing, the top camera pose, and the camera intrinsics are estimates.

Size and scale

The full RRD dataset is 2.42 TB in 128,987 files, converted from 2.35 TB of source. A single episode's base layer ranges roughly from 0.5 to 200 MB, with a median of about 50 MB. The base layer is about the size of the source. The urdf layer adds about 2% on top, the cameras layer about 27 KB per episode, and the stats layer about 40 KB. The two arm models are 3.3 MB each.

How to use

Fetch one episode from the bucket and view it locally:

pip install "huggingface_hub[cli]" rerun-sdk

# Every layer of one episode, plus the two arm models and the default blueprint
hf buckets sync hf://buckets/rerun/molmoact2 . \
  --include "*/episode_<NNNNN>.rrd" \
  --include "assets/*.rrd" \
  --include "blueprints/default.rbl"

rerun */episode_<NNNNN>.rrd blueprints/default.rbl

Keep the * so every layer loads.

Note: The viewer opens local files through its Viewer catalog by default, and it can then ignore the blueprint file. Turn off Settings → Viewer catalog → Load files via Viewer catalog before running the command above.

Opened as above, the 3D view shows the three cameras but not the arms.

The recommended way of viewing the dataset with a 3D view is to register the episodes on a catalog with the two assets/urdf-model-*.rrd files as the dataset's assets. The Local Catalog section in the code repo provides instructions.

These recordings were written with Rerun 0.38.1. A newer viewer reads them as they are in most cases; rerun rrd migrate updates them if it cannot.

To copy the whole bucket (about 2.4 TB), mirror it bucket-to-bucket rather than to a local disk. The bucket is reachable over an S3-compatible API, so rclone can stream it straight into your own S3-compatible storage.

Scope

Conversion code is published at rerun-io/rrd-datasets. The code and these recordings demonstrate the conversion; they are not a versioned dataset release. Files may be regenerated or replaced, and converting the same episode twice does not give identical bytes.

Interesting observations of the source data, including known issues, are listed in observations.md.

License

Apache License 2.0, the same license as the source dataset — the full text ships as the LICENSE file at the root of this bucket. If you redistribute or build on this data, keep the attribution above and comply with the license terms.

The two arm models, assets/urdf-model-left.rrd and assets/urdf-model-right.rrd, are the only files not derived from the source dataset. They carry geometry from the YAM arm model in i2rt-robotics/i2rt, under the MIT License, Copyright (c) I2RT Robotics. Its full text ships as LICENSE-YAM at the root of this bucket, and applies to those two files.