Hugging Face's logo

Spaces:
Rice-RobotPI-Lab
/
EgoInfinity
Running

App Files Community
EgoInfinity
File size: 2,789 Bytes
66d097c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
title: EgoInfinity Browser
emoji: 🪞
colorFrom: indigo
colorTo: pink
sdk: static
pinned: false
license: mit
short_description: Static viewer for the EgoInfinity dataset
---

# EgoInfinity Browser

A static, browser-only viewer for curated EgoInfinity clips. No backend, no
Python, no live viser server. Every clip is served as pre-baked assets and
rendered with three.js / the viser-build-client.

Source code: <https://github.com/Rice-RobotPI-Lab/EgoInfinity>

Dataset: <https://huggingface.co/datasets/Rice-RobotPI-Lab/egoinfinity>

## What you see

For each clip:

- **YouTube iframe**: the original source video, embedded from YouTube.
- **Depth video** (MoGe-2) and **optical flow** (MEMFOF), rendered as colormapped MP4s.
- **SAM-tracked object mask** highlighting the active object across the clip.
- **3D scene**: viser recording showing the per-frame point cloud, hand mesh,
  object meshes (Gaussian Splat baked), and skeleton.
- **Hand skeleton overlay** projected onto the depth video.
- **Action signals**: per-frame timeseries (hand velocity, contact, etc.).

## Get the data

The full processed dataset is hosted on Hugging Face Datasets:

```bash
# Snapshot download (entire dataset, ~370 MB)
pip install huggingface_hub
hf download Rice-RobotPI-Lab/egoinfinity --repo-type=dataset --local-dir ./egoinfinity-data

# Or browse interactively at:
# https://huggingface.co/datasets/Rice-RobotPI-Lab/egoinfinity
```

While this Space is private, you must be logged into a Hugging Face account
with read access to both repositories for assets to load in the viewer.

## License & attribution

- **Code** in this Space (HTML, JS, viser-client) is released under the MIT
  License. See [LICENSE](LICENSE).
- **Data assets** (depth/flow/mask videos, viser recordings, hand parameters,
  scene metadata) are derivative works of [Action100M (Meta FAIR)] and are
  governed by the FAIR Noncommercial Research License v1. See
  [LICENSE-Action100M](LICENSE-Action100M).
- **Original videos** remain on YouTube and are embedded via the YouTube
  IFrame API. Only the SAM-tracked region of each frame appears in the
  derivative `mask.mp4` (the rest is painted black).
- **Hand visualization** uses pose parameters derived from the MANO hand model
  (Romero et al., 2017). MANO is not redistributed; only baked vertex positions
  appear in the viewer.
- **WiLoR** weights (CC-BY-NC-ND) are not redistributed.

[Action100M (Meta FAIR)]: https://github.com/facebookresearch/Action100M

## Citation

If you find this useful in your research, please cite:

```bibtex
@misc{egoinfinity2026,
  title  = {EgoInfinity: A Web-Scale Data Engine for Video-to-Action Robot Learning through Egocentric Views},
  author = {Rice RobotPI Lab},
  year   = {2026},
  note   = {Preview release}
}
```