PointWorld-BEHAVIOR WebDataset shards

WebDataset (WDS) train/test shards for training and evaluating PointWorld, converted from the nvidia/PointWorld-BEHAVIOR release. This is not an official NVIDIA release.

License

NVIDIA License, non-commercial: research and evaluation use only. See LICENSE.pdf.

This data is a derivative work of nvidia/PointWorld-BEHAVIOR, which is distributed under the NVIDIA License, and is redistributed under that same license. The NVIDIA License permits use of this data and of anything derived from it only for non-commercial purposes, meaning research or evaluation. The underlying simulation episodes come from BEHAVIOR-1K (behavior-1k/2025-challenge-rawdata); see its terms as well.

Contents

behavior/wds/
├── train/train-rank{R}-NNNNNN.tar
├── test/test-rank{R}-NNNNNN.tar
├── metadata_rank{R}.json
└── {train,test}_source_paths_rank{R}.txt
_jobs/
├── wds_index.json                         # samples per tar
├── integrity_check.json                   # merged integrity results (valid clips)
├── unavailable_test_clips.json            # paper test clips not in the source release
└── ...                                    # build bookkeeping

The layout matches what the PointWorld main branch expects: point LOCAL_DATASET_DIR at a directory containing behavior/wds. You can mount the bucket there or sync it locally (about 1.8 TB).

How it was made

Built with the data branch pipeline (data_integrity_check.py + convert_wds.py), run per source task on Hugging Face Jobs. Very minor differences from the paper:

• Split: the paper split manifests/behavior_paper_split_manifest.json, except for 483 manifest test clips from 19 episodes. These have legacy ids (e.g. boxing_books_up_for_storage_1753068141443199_boxing_books_up_for_storage-2735:2746) with no counterpart in the source release, so the test split has 173,104 of the manifest's 173,587 clips. The 483 ids are listed in _jobs/unavailable_test_clips.json.
• Clip totals: the source release yields 1,730,593 valid clips against the paper's 1,735,872. The gap of 5,279 matches those 19 episodes (train + test) plus about 450 clips, so the episodes appear to be absent from the release rather than present in train.
• Content: WDS samples contain the fields listed by get_wds_data_keys("behavior") in shared/data_contract.py, plus clip_attributes. Clips excluded by the BEHAVIOR integrity filtering in data_integrity_check.py are not included.

Citation

If you use this data, cite the PointWorld paper and the original BEHAVIOR dataset (https://behavior.stanford.edu/).

@article{huang2026pointworld,
  title={PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation},
  author={Huang, Wenlong and Chao, Yu-Wei and Mousavian, Arsalan and Liu, Ming-Yu and Fox, Dieter and Mo, Kaichun and Li, Fei-Fei},
  journal={arXiv preprint arXiv:2601.03782},
  year={2026}
}