Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>ego-multimodal — Rerun Viewer</title> | |
| <style> | |
| * { margin: 0; padding: 0; box-sizing: border-box; } | |
| body { | |
| font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; | |
| background: #0d1117; | |
| color: #c9d1d9; | |
| min-height: 100vh; | |
| } | |
| .header { | |
| background: #161b22; | |
| border-bottom: 1px solid #30363d; | |
| padding: 20px 32px; | |
| } | |
| .header h1 { font-size: 1.5rem; color: #f0f6fc; margin-bottom: 4px; } | |
| .header p { font-size: 0.9rem; color: #8b949e; } | |
| .container { max-width: 900px; margin: 0 auto; padding: 32px; } | |
| .card { | |
| background: #161b22; | |
| border: 1px solid #30363d; | |
| border-radius: 8px; | |
| padding: 24px; | |
| margin-bottom: 20px; | |
| } | |
| .card h2 { font-size: 1.15rem; color: #f0f6fc; margin-bottom: 8px; } | |
| .card p { font-size: 0.9rem; color: #8b949e; margin-bottom: 16px; } | |
| .size { color: #58a6ff; font-weight: 500; } | |
| .btn { | |
| display: inline-block; | |
| padding: 8px 18px; | |
| background: #238636; | |
| color: #fff; | |
| text-decoration: none; | |
| border-radius: 6px; | |
| font-size: 0.85rem; | |
| font-weight: 500; | |
| margin-right: 8px; | |
| margin-bottom: 8px; | |
| transition: background 0.15s; | |
| } | |
| .btn:hover { background: #2ea043; } | |
| .btn-secondary { background: #30363d; } | |
| .btn-secondary:hover { background: #484f58; } | |
| .code-block { | |
| background: #0d1117; | |
| border: 1px solid #30363d; | |
| border-radius: 6px; | |
| padding: 16px; | |
| font-family: 'SF Mono', 'Fira Code', monospace; | |
| font-size: 0.8rem; | |
| overflow-x: auto; | |
| margin-top: 12px; | |
| color: #e6edf3; | |
| line-height: 1.5; | |
| } | |
| .tag { | |
| display: inline-block; | |
| padding: 2px 8px; | |
| background: #1f6feb22; | |
| color: #58a6ff; | |
| border-radius: 12px; | |
| font-size: 0.75rem; | |
| margin-right: 6px; | |
| } | |
| .note { | |
| background: #0d1117; | |
| border-left: 3px solid #58a6ff; | |
| padding: 12px 16px; | |
| border-radius: 0 6px 6px 0; | |
| margin-top: 12px; | |
| font-size: 0.85rem; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="header"> | |
| <h1>ego-multimodal — MoWare IMU Motion Capture + GMR</h1> | |
| <p>Interactive Rerun visualizations for mocap, video, segmentation, and General Motion Retargeting</p> | |
| </div> | |
| <div class="container"> | |
| <div class="card"> | |
| <h2>Mocap</h2> | |
| <p>Video with activity annotations, hand-proximal segmentation masks, and full-body IMU mocap avatar.</p> | |
| <span class="tag">video</span> | |
| <span class="tag">segmentation</span> | |
| <span class="tag">annotations</span> | |
| <span class="tag">mocap avatar</span> | |
| <br><br> | |
| <span class="size">2.9 GB</span> | |
| <br><br> | |
| <a class="btn" href="https://app.rerun.io/version/0.30.0/?url=https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/video.rrd" target="_blank"> | |
| Open in Rerun Viewer | |
| </a> | |
| <a class="btn btn-secondary" href="https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/video.rrd" target="_blank"> | |
| Download .rrd | |
| </a> | |
| </div> | |
| <div class="card"> | |
| <h2>Finger Dexterity</h2> | |
| <p>3D pinhole camera projection with head tracking, full mocap skeleton, and close-up views of both hands.</p> | |
| <span class="tag">pinhole camera</span> | |
| <span class="tag">head tracking</span> | |
| <span class="tag">hand close-ups</span> | |
| <br><br> | |
| <span class="size">1.8 GB</span> | |
| <br><br> | |
| <a class="btn" href="https://app.rerun.io/version/0.30.0/?url=https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/pinhole.rrd" target="_blank"> | |
| Open in Rerun Viewer | |
| </a> | |
| <a class="btn btn-secondary" href="https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/pinhole.rrd" target="_blank"> | |
| Download .rrd | |
| </a> | |
| </div> | |
| <div class="card"> | |
| <h2>General Motion Retargeting</h2> | |
| <p>Human motion retargeted to 3 humanoid robots via GMR transfer functions: Unitree G1, Fourier N1, and Booster T1.</p> | |
| <span class="tag">Unitree G1</span> | |
| <span class="tag">Fourier N1</span> | |
| <span class="tag">Booster T1</span> | |
| <span class="tag">GMR</span> | |
| <br><br> | |
| <span class="size">10 GB</span> | |
| <br><br> | |
| <a class="btn" href="https://app.rerun.io/version/0.30.0/?url=https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/gmr_robots.rrd" target="_blank"> | |
| Open in Rerun Viewer | |
| </a> | |
| <a class="btn btn-secondary" href="https://huggingface.co/datasets/phi-9/ego-multimodal/resolve/main/visualizations/gmr_robots.rrd" target="_blank"> | |
| Download .rrd | |
| </a> | |
| <div class="note"> | |
| This file is 10 GB. The Rerun web viewer may struggle with files this large. For best results, download the .rrd and open locally with <code>rerun gmr_robots.rrd</code>. | |
| </div> | |
| </div> | |
| <div class="card"> | |
| <h2>Programmatic Access</h2> | |
| <p>Load the structured data (quaternions, trajectories, annotations) directly via the <code>datasets</code> library:</p> | |
| <div class="code-block"> | |
| <span style="color:#ff7b72">from</span> datasets <span style="color:#ff7b72">import</span> load_dataset | |
| <span style="color:#8b949e"># Load motion capture quaternion data (78K rows)</span> | |
| ds = load_dataset(<span style="color:#a5d6ff">"phi-9/ego-multimodal"</span>, <span style="color:#a5d6ff">"mocap"</span>) | |
| <span style="color:#8b949e"># Load activity annotations</span> | |
| ann = load_dataset(<span style="color:#a5d6ff">"phi-9/ego-multimodal"</span>, <span style="color:#a5d6ff">"annotations"</span>) | |
| <span style="color:#8b949e"># Load 3D trajectories</span> | |
| head = load_dataset(<span style="color:#a5d6ff">"phi-9/ego-multimodal"</span>, <span style="color:#a5d6ff">"trajectory_head_only"</span>) | |
| <span style="color:#8b949e"># Download a Rerun visualization</span> | |
| <span style="color:#ff7b72">from</span> huggingface_hub <span style="color:#ff7b72">import</span> hf_hub_download | |
| path = hf_hub_download(<span style="color:#a5d6ff">"phi-9/ego-multimodal"</span>, <span style="color:#a5d6ff">"visualizations/video.rrd"</span>, repo_type=<span style="color:#a5d6ff">"dataset"</span>) | |
| </div> | |
| </div> | |
| <div class="card"> | |
| <h2>Dataset Contents</h2> | |
| <p>Full file listing at <a href="https://huggingface.co/datasets/phi-9/ego-multimodal" style="color:#58a6ff">phi-9/ego-multimodal</a></p> | |
| <table style="width:100%; font-size:0.85rem; margin-top:12px; border-collapse:collapse;"> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">data/mocap_frames.parquet</td> | |
| <td style="text-align:right; color:#58a6ff">883 KB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">data/calibration_frame.parquet</td> | |
| <td style="text-align:right; color:#58a6ff">3.5 KB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">data/trajectory_*.parquet (×4)</td> | |
| <td style="text-align:right; color:#58a6ff">~9 KB each</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">data/annotations.parquet</td> | |
| <td style="text-align:right; color:#58a6ff">3.2 KB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">visualizations/video.rrd</td> | |
| <td style="text-align:right; color:#58a6ff">2.9 GB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">visualizations/pinhole.rrd</td> | |
| <td style="text-align:right; color:#58a6ff">1.8 GB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">visualizations/gmr_robots.rrd</td> | |
| <td style="text-align:right; color:#58a6ff">10 GB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">segmentation/l2.hand_prox_seg.json</td> | |
| <td style="text-align:right; color:#58a6ff">1.1 GB</td> | |
| </tr> | |
| <tr style="border-bottom:1px solid #30363d;"> | |
| <td style="padding:6px 0;">raw/l2.mp4</td> | |
| <td style="text-align:right; color:#58a6ff">76 MB</td> | |
| </tr> | |
| <tr> | |
| <td style="padding:6px 0;">raw/l2_mod_combined.decoded.json</td> | |
| <td style="text-align:right; color:#58a6ff">27 MB</td> | |
| </tr> | |
| </table> | |
| </div> | |
| </div> | |
| </body> | |
| </html> | |