Spaces:
Sleeping
Take the dataset repo as user input and stream windows on demand
Browse filesThe repo is now typed into the app rather than baked in via DATASET_REPO,
which becomes only the prefilled default. Accepts owner/name, a revision
suffix, or a pasted dataset URL.
Generalises the layout assumptions so an arbitrary repo renders: the manifest
and metrics files are optional, the window list falls back to listing
windows/ one level deep, per-window metadata is backfilled from meta.json,
the camera count is inferred from the frame aspect ratio, and window length
and critical-frame markers are no longer fixed at 33/16.
Storage is now per session and bounded, so large datasets are never pulled
whole: loading a repo fetches only the index, window files arrive when the
window is opened, at most MAX_WINDOWS_ON_DISK stay resident, and the
session's directory is deleted when the tab closes. A janitor reaps idle
sessions and directories orphaned by an earlier process.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@@ -13,51 +13,73 @@ short_description: Frame-locked viewer for rollouts vs. ground truth
|
|
| 13 |
|
| 14 |
# Rollout vs. ground truth
|
| 15 |
|
| 16 |
-
Side-by-side viewer for forward-dynamics world-model rollouts.
|
| 17 |
-
|
|
|
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|
| 21 |
|
| 22 |
## What it shows
|
| 23 |
|
| 24 |
- **Playback** — GT and generated rendered into one video, so the two panels stay frame-locked
|
| 25 |
instead of drifting apart the way two independent players do. Optional `|GT − generated|`
|
| 26 |
-
difference panel. Playback fps is adjustable (
|
| 27 |
-
- **Frame stepper** — a single frame from both videos side by side, defaulting to
|
| 28 |
-
|
| 29 |
-
- **Driving actions** — the
|
| 30 |
-
left-arm
|
| 31 |
-
- **All windows** — sortable table of every window
|
| 32 |
|
| 33 |
-
Filter by task, gripper edge
|
| 34 |
-
|
| 35 |
|
| 36 |
## Expected dataset layout
|
| 37 |
|
| 38 |
```
|
| 39 |
-
windows_manifest.json # list of window dicts
|
| 40 |
-
#
|
| 41 |
-
metrics_summary.json #
|
| 42 |
-
# crit_mse, crit_psnr}
|
| 43 |
-
windows/<episode_id>/gt.mp4 #
|
| 44 |
-
windows/<episode_id>/generated.mp4 #
|
| 45 |
-
windows/<episode_id>/actions.json # [
|
| 46 |
-
windows/<episode_id>/meta.json
|
| 47 |
```
|
| 48 |
|
| 49 |
-
|
| 50 |
-
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
| `HF_TOKEN` | **Required if the dataset is private.** Add it under *Settings → Variables and secrets → Secrets*. |
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
## Running locally
|
| 63 |
|
|
|
|
| 13 |
|
| 14 |
# Rollout vs. ground truth
|
| 15 |
|
| 16 |
+
Side-by-side viewer for forward-dynamics world-model rollouts. Type a **dataset repo id** into
|
| 17 |
+
the box at the top and it renders that repo — the Space holds no data of its own and is not tied
|
| 18 |
+
to any one dataset.
|
| 19 |
|
| 20 |
+
Accepted forms: `owner/name`, `owner/name@revision`, or a pasted `https://huggingface.co/datasets/…`
|
| 21 |
+
URL. `DATASET_REPO` sets what is prefilled at startup.
|
| 22 |
|
| 23 |
## What it shows
|
| 24 |
|
| 25 |
- **Playback** — GT and generated rendered into one video, so the two panels stay frame-locked
|
| 26 |
instead of drifting apart the way two independent players do. Optional `|GT − generated|`
|
| 27 |
+
difference panel. Playback fps is adjustable (a 33-frame window is only ~1 s at 30 fps).
|
| 28 |
+
- **Frame stepper** — a single frame from both videos side by side, defaulting to the critical
|
| 29 |
+
frame when the dataset marks one.
|
| 30 |
+
- **Driving actions** — the action chunk fed to the model. A 14-D chunk is split into grippers /
|
| 31 |
+
left-arm / right-arm; any other width is plotted as raw channels.
|
| 32 |
+
- **All windows** — sortable table of every window; click a row to load it.
|
| 33 |
|
| 34 |
+
Filter by task, gripper edge and arm; sort by critical-frame PSNR to jump to the hardest or
|
| 35 |
+
easiest cases.
|
| 36 |
|
| 37 |
## Expected dataset layout
|
| 38 |
|
| 39 |
```
|
| 40 |
+
windows_manifest.json # optional: list of window dicts (episode_id, task,
|
| 41 |
+
# edge_type, arm, start_frame, critical_frame, ...)
|
| 42 |
+
metrics_summary.json # optional: [{id, task, edge, arm, full_mse, full_psnr,
|
| 43 |
+
# crit_mse, crit_psnr}, ...]
|
| 44 |
+
windows/<episode_id>/gt.mp4 # required
|
| 45 |
+
windows/<episode_id>/generated.mp4 # required, same geometry and length as gt.mp4
|
| 46 |
+
windows/<episode_id>/actions.json # optional: [T, D] driving actions
|
| 47 |
+
windows/<episode_id>/meta.json # optional: per-window metadata
|
| 48 |
```
|
| 49 |
|
| 50 |
+
Only the two mp4s are required. With no `windows_manifest.json` the window list is discovered by
|
| 51 |
+
listing `windows/` one level deep, and per-window metadata is read from each `meta.json` as you
|
| 52 |
+
open it. Videos may stack **N cameras vertically** — N is inferred from the frame aspect ratio,
|
| 53 |
+
and the camera selector adapts. Black letterbox bars are trimmed automatically, identically for
|
| 54 |
+
GT and generated.
|
| 55 |
|
| 56 |
+
## Storage model
|
| 57 |
+
|
| 58 |
+
Nothing is downloaded at build time, and no repo is ever pulled whole — important for datasets
|
| 59 |
+
too large to fit on the Space.
|
| 60 |
+
|
| 61 |
+
- Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory
|
| 62 |
+
listing when there is no manifest.
|
| 63 |
+
- A window's mp4s, `actions.json` and `meta.json` are fetched **the first time that window is
|
| 64 |
+
opened**, never in bulk.
|
| 65 |
+
- Each browser session gets its own scratch directory. At most `MAX_WINDOWS_ON_DISK` windows stay
|
| 66 |
+
resident; beyond that the least recently used are deleted. Decoded frames are capped at 2
|
| 67 |
+
windows in RAM and composed strips at 1.
|
| 68 |
+
- The whole directory is deleted when the session ends. A janitor also reaps sessions idle past
|
| 69 |
+
`SESSION_TTL` and directories orphaned by an earlier process, so nothing accumulates if a tab
|
| 70 |
+
dies without a clean shutdown.
|
| 71 |
|
| 72 |
+
The sidebar shows live occupancy (`cached this session: n/N windows`).
|
| 73 |
+
|
| 74 |
+
## Configuration
|
|
|
|
| 75 |
|
| 76 |
+
| Variable | Default | Purpose |
|
| 77 |
+
|---|---|---|
|
| 78 |
+
| `DATASET_REPO` | `OneWorld-AI/abc-critical-window-rollout` | Repo prefilled in the input box. |
|
| 79 |
+
| `HF_TOKEN` | — | **Required for private datasets.** Set under *Settings → Variables and secrets*. |
|
| 80 |
+
| `MAX_WINDOWS_ON_DISK` | `12` | Windows kept per session before LRU eviction. |
|
| 81 |
+
| `MAX_DISCOVER` | `5000` | Cap on windows listed when a repo has no manifest. |
|
| 82 |
+
| `SESSION_TTL` | `3600` | Seconds of idleness before a session's storage is reclaimed. |
|
| 83 |
|
| 84 |
## Running locally
|
| 85 |
|
|
@@ -1,47 +1,68 @@
|
|
| 1 |
"""
|
| 2 |
-
Side-by-side viewer for
|
| 3 |
|
| 4 |
-
|
| 5 |
|
| 6 |
-
windows_manifest.json
|
| 7 |
-
metrics_summary.json
|
| 8 |
-
windows/<episode_id>/gt.mp4 #
|
| 9 |
-
windows/<episode_id>/generated.mp4 #
|
| 10 |
-
windows/<episode_id>/actions.json # [
|
| 11 |
-
windows/<episode_id>/meta.json
|
| 12 |
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
"""
|
| 16 |
|
| 17 |
from __future__ import annotations
|
| 18 |
|
| 19 |
-
import functools
|
| 20 |
import json
|
| 21 |
import os
|
|
|
|
|
|
|
| 22 |
import tempfile
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
from pathlib import Path
|
| 24 |
|
| 25 |
import cv2
|
| 26 |
import gradio as gr
|
| 27 |
import imageio_ffmpeg
|
| 28 |
import numpy as np
|
| 29 |
-
from huggingface_hub import hf_hub_download
|
| 30 |
from matplotlib.figure import Figure
|
| 31 |
|
| 32 |
-
|
| 33 |
HF_TOKEN = os.environ.get("HF_TOKEN") or os.environ.get("HUGGING_FACE_HUB_TOKEN") or None
|
| 34 |
|
| 35 |
-
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
FONT = cv2.FONT_HERSHEY_SIMPLEX
|
| 43 |
HDR, FTR, SEP = 26, 26, 4
|
| 44 |
-
SCALE = 2 #
|
| 45 |
|
| 46 |
GT_COLOR = (130, 225, 140)
|
| 47 |
GEN_COLOR = (120, 180, 255)
|
|
@@ -56,62 +77,281 @@ SORTS = [
|
|
| 56 |
]
|
| 57 |
|
| 58 |
|
| 59 |
-
# ---------------------------------------------------------------------------
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
def
|
| 63 |
-
"""
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
repo_type="dataset",
|
| 68 |
token=HF_TOKEN,
|
| 69 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
-
|
| 73 |
-
def
|
| 74 |
-
""
|
| 75 |
-
manifest = json.load(open(fetch("windows_manifest.json")))
|
| 76 |
-
try:
|
| 77 |
-
metrics = json.load(open(fetch("metrics_summary.json")))
|
| 78 |
-
except Exception:
|
| 79 |
-
metrics = []
|
| 80 |
-
by_id = {m["id"]: m for m in metrics}
|
| 81 |
-
|
| 82 |
-
rows = []
|
| 83 |
-
for i, w in enumerate(manifest):
|
| 84 |
-
ep = w["episode_id"]
|
| 85 |
-
m = by_id.get(ep, {})
|
| 86 |
-
rows.append(
|
| 87 |
-
{
|
| 88 |
-
"order": i,
|
| 89 |
-
"episode_id": ep,
|
| 90 |
-
"task": w.get("task", "?"),
|
| 91 |
-
"edge": w.get("edge_type", "?"),
|
| 92 |
-
"arm": w.get("arm", "?"),
|
| 93 |
-
"amplitude": w.get("amplitude"),
|
| 94 |
-
"start_frame": w.get("start_frame"),
|
| 95 |
-
"window": w.get("window"),
|
| 96 |
-
"critical_frame": w.get("critical_frame"),
|
| 97 |
-
"n_frames": w.get("n_frames"),
|
| 98 |
-
"full_psnr": m.get("full_psnr"),
|
| 99 |
-
"crit_psnr": m.get("crit_psnr"),
|
| 100 |
-
"full_mse": m.get("full_mse"),
|
| 101 |
-
"crit_mse": m.get("crit_mse"),
|
| 102 |
-
}
|
| 103 |
-
)
|
| 104 |
-
return rows
|
| 105 |
|
| 106 |
|
| 107 |
def label_for(row: dict) -> str:
|
| 108 |
psnr = row["crit_psnr"]
|
| 109 |
-
head = f"{psnr:5.1f} dB" if psnr
|
| 110 |
-
|
|
|
|
|
|
|
| 111 |
|
| 112 |
|
| 113 |
-
def filter_rows(query: str, edge: str, arm: str, sort: str) -> list[dict]:
|
| 114 |
-
rows =
|
| 115 |
q = (query or "").strip().lower()
|
| 116 |
if q:
|
| 117 |
rows = [r for r in rows if q in r["task"].lower() or q in r["episode_id"].lower()]
|
|
@@ -120,24 +360,35 @@ def filter_rows(query: str, edge: str, arm: str, sort: str) -> list[dict]:
|
|
| 120 |
if arm != "any":
|
| 121 |
rows = [r for r in rows if r["arm"] == arm]
|
| 122 |
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
| 124 |
if sort == SORTS[0]:
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
rows = sorted(rows, key=lambda r: r["order"])
|
| 132 |
-
return rows
|
| 133 |
|
| 134 |
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
for r in load_index():
|
| 138 |
if r["episode_id"] == episode_id:
|
| 139 |
-
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
|
| 142 |
|
| 143 |
# --------------------------------------------------------------------------- video
|
|
@@ -158,18 +409,22 @@ def read_video(path: str) -> np.ndarray:
|
|
| 158 |
return np.stack(frames)
|
| 159 |
|
| 160 |
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
return gt[:t], gen[:t]
|
| 167 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
|
|
|
| 173 |
return [frames[:, i * h : (i + 1) * h] for i in rows]
|
| 174 |
|
| 175 |
|
|
@@ -185,7 +440,7 @@ def letterbox_rows(*bands: np.ndarray, thresh: int = 16) -> tuple[int, int]:
|
|
| 185 |
return int(idx[0]), int(idx[-1]) + 1
|
| 186 |
|
| 187 |
|
| 188 |
-
def crop_view(gt: np.ndarray, gen: np.ndarray, view:
|
| 189 |
"""Select cameras and trim the letterbox, identically for both videos."""
|
| 190 |
gt_out, gen_out = [], []
|
| 191 |
for g, p in zip(cam_slices(gt, view), cam_slices(gen, view)):
|
|
@@ -219,11 +474,9 @@ def diff_frames(gt: np.ndarray, gen: np.ndarray, gain: float = 3.0) -> np.ndarra
|
|
| 219 |
return np.stack([cv2.applyColorMap(f, cv2.COLORMAP_INFERNO)[:, :, ::-1] for f in d])
|
| 220 |
|
| 221 |
|
| 222 |
-
|
| 223 |
-
def compose(episode_id: str, view: str, show_diff: bool, crit_idx: int) -> np.ndarray:
|
| 224 |
"""Build the labelled side-by-side strip: [T, H, W, 3] RGB."""
|
| 225 |
-
|
| 226 |
-
gt_c, gen_c = crop_view(gt, gen, view)
|
| 227 |
gt_v, gen_v = upscale(gt_c, SCALE), upscale(gen_c, SCALE)
|
| 228 |
|
| 229 |
panels = [panel(gt_v, "GROUND TRUTH", GT_COLOR), panel(gen_v, "GENERATED", GEN_COLOR)]
|
|
@@ -240,14 +493,15 @@ def compose(episode_id: str, view: str, show_diff: bool, crit_idx: int) -> np.nd
|
|
| 240 |
t, h, w, _ = comp.shape
|
| 241 |
out = np.zeros((t, h + FTR, w, 3), np.uint8)
|
| 242 |
out[:, :h] = comp
|
|
|
|
| 243 |
tl0, tl1, y = int(w * 0.45), w - 14, FTR // 2
|
| 244 |
for i in range(t):
|
| 245 |
bar = np.full((FTR, w, 3), 22, np.uint8)
|
| 246 |
-
is_crit = i == crit_idx
|
| 247 |
text = f"frame {i:02d}/{t - 1}" + (" CRITICAL FRAME" if is_crit else "")
|
| 248 |
cv2.putText(bar, text, (8, FTR - 8), FONT, 0.5, CRIT_COLOR if is_crit else (200, 200, 200), 1, cv2.LINE_AA)
|
| 249 |
cv2.line(bar, (tl0, y), (tl1, y), (70, 70, 70), 2, cv2.LINE_AA)
|
| 250 |
-
if
|
| 251 |
cx = int(tl0 + (tl1 - tl0) * crit_idx / max(t - 1, 1))
|
| 252 |
cv2.line(bar, (cx, y - 7), (cx, y + 7), CRIT_COLOR, 2, cv2.LINE_AA)
|
| 253 |
px = int(tl0 + (tl1 - tl0) * i / max(t - 1, 1))
|
|
@@ -264,7 +518,7 @@ def compose(episode_id: str, view: str, show_diff: bool, crit_idx: int) -> np.nd
|
|
| 264 |
|
| 265 |
|
| 266 |
def write_mp4(frames: np.ndarray, path: Path, fps: float) -> str:
|
| 267 |
-
|
| 268 |
writer = imageio_ffmpeg.write_frames(
|
| 269 |
str(path),
|
| 270 |
size=(w, h),
|
|
@@ -282,115 +536,178 @@ def write_mp4(frames: np.ndarray, path: Path, fps: float) -> str:
|
|
| 282 |
return str(path)
|
| 283 |
|
| 284 |
|
| 285 |
-
def render_video(episode_id: str, view:
|
| 286 |
-
|
| 287 |
-
path = CACHE_DIR / f"{key}.mp4"
|
| 288 |
if not path.exists():
|
| 289 |
-
write_mp4(
|
| 290 |
return str(path)
|
| 291 |
|
| 292 |
|
| 293 |
# --------------------------------------------------------------------------- actions plot
|
| 294 |
|
| 295 |
|
| 296 |
-
def plot_actions(episode_id: str, crit_idx: int, cursor: int | None
|
| 297 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 298 |
t = np.arange(len(actions))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 299 |
|
| 300 |
-
fig = Figure(figsize=(7.2,
|
| 301 |
-
axes = fig.subplots(
|
| 302 |
-
specs = [
|
| 303 |
-
("Grippers (1.0 = open, 0.0 = closed)", [(6, "L gripper"), (13, "R gripper")]),
|
| 304 |
-
("Left arm joints", [(i, f"L{i}") for i in range(6)]),
|
| 305 |
-
("Right arm joints", [(i, f"R{i - 7}") for i in range(7, 13)]),
|
| 306 |
-
]
|
| 307 |
for ax, (title, channels) in zip(axes, specs):
|
| 308 |
for idx, name in channels:
|
| 309 |
ax.plot(t, actions[:, idx], lw=1.4, label=name)
|
| 310 |
ax.set_title(title, fontsize=9, loc="left")
|
| 311 |
-
|
|
|
|
| 312 |
if cursor is not None:
|
| 313 |
ax.axvline(cursor, color="0.35", ls=":", lw=1.0, zorder=0)
|
| 314 |
ax.grid(alpha=0.25, lw=0.5)
|
| 315 |
ax.tick_params(labelsize=8)
|
| 316 |
-
|
| 317 |
-
|
|
|
|
|
|
|
| 318 |
return fig
|
| 319 |
|
| 320 |
|
| 321 |
# --------------------------------------------------------------------------- ui glue
|
| 322 |
|
| 323 |
|
| 324 |
-
def metrics_md(row: dict) -> str:
|
| 325 |
def fmt(v, unit=""):
|
| 326 |
return f"{v:.2f}{unit}" if isinstance(v, (int, float)) else "—"
|
| 327 |
|
| 328 |
-
|
| 329 |
-
|
| 330 |
-
|
| 331 |
-
|
| 332 |
-
|
| 333 |
-
|
| 334 |
-
|
| 335 |
-
|
| 336 |
-
|
| 337 |
-
|
| 338 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 339 |
)
|
|
|
|
| 340 |
|
| 341 |
|
| 342 |
def table_rows(rows: list[dict]) -> list[list]:
|
|
|
|
|
|
|
|
|
|
| 343 |
return [
|
| 344 |
-
[
|
| 345 |
-
r["episode_id"][8:16],
|
| 346 |
-
r["task"],
|
| 347 |
-
r["edge"],
|
| 348 |
-
r["arm"],
|
| 349 |
-
round(r["crit_psnr"], 2) if r["crit_psnr"] is not None else None,
|
| 350 |
-
round(r["full_psnr"], 2) if r["full_psnr"] is not None else None,
|
| 351 |
-
]
|
| 352 |
for r in rows
|
| 353 |
]
|
| 354 |
|
| 355 |
|
| 356 |
-
def
|
| 357 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 358 |
choices = [(label_for(r), r["episode_id"]) for r in rows]
|
| 359 |
ids = [r["episode_id"] for r in rows]
|
| 360 |
value = current if current in ids else (ids[0] if ids else None)
|
| 361 |
return (
|
| 362 |
-
gr.update(choices=choices, value=value, label=f"Window ({len(ids)} of
|
| 363 |
table_rows(rows),
|
| 364 |
ids,
|
| 365 |
)
|
| 366 |
|
| 367 |
|
| 368 |
-
def
|
| 369 |
-
|
| 370 |
-
|
| 371 |
-
|
| 372 |
-
|
| 373 |
-
|
| 374 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 375 |
|
| 376 |
-
|
| 377 |
-
|
| 378 |
-
|
| 379 |
-
|
| 380 |
-
|
| 381 |
-
|
| 382 |
-
|
| 383 |
-
i = int(min(frame_idx, len(frames) - 1))
|
| 384 |
-
return video, frames[i], metrics_md(row), plot_actions(episode_id, crit, i)
|
| 385 |
|
| 386 |
|
| 387 |
-
def on_frame(episode_id, view, show_diff, frame_idx):
|
| 388 |
-
|
|
|
|
| 389 |
return None, None
|
| 390 |
-
crit = critical_index(row_by_id(episode_id))
|
| 391 |
-
frames =
|
| 392 |
i = int(min(frame_idx, len(frames) - 1))
|
| 393 |
-
return frames[i], plot_actions(episode_id, crit, i)
|
| 394 |
|
| 395 |
|
| 396 |
def step(ids, current, delta):
|
|
@@ -410,15 +727,23 @@ CSS = """
|
|
| 410 |
.window-still img { max-height: 78vh; object-fit: contain; background: #111; }
|
| 411 |
"""
|
| 412 |
|
| 413 |
-
with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
| 414 |
-
gr.Markdown(
|
| 415 |
-
|
| 416 |
-
|
| 417 |
-
"each centred on a gripper critical frame (frame 16). "
|
| 418 |
-
"Cameras are top / left-wrist / right-wrist."
|
| 419 |
-
)
|
| 420 |
ids_state = gr.State([])
|
| 421 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 422 |
with gr.Row():
|
| 423 |
with gr.Column(scale=1, min_width=320):
|
| 424 |
query = gr.Textbox(label="Search task or episode", placeholder="e.g. napkin, pillowcase…")
|
|
@@ -431,9 +756,9 @@ with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
|
| 431 |
prev_btn = gr.Button("← Prev")
|
| 432 |
next_btn = gr.Button("Next →")
|
| 433 |
gr.Markdown("---")
|
| 434 |
-
view = gr.Radio(
|
| 435 |
show_diff = gr.Checkbox(False, label="Add |GT − generated| difference panel")
|
| 436 |
-
fps = gr.Slider(2, 30, value=8, step=1, label="Playback fps
|
| 437 |
info = gr.Markdown()
|
| 438 |
|
| 439 |
with gr.Column(scale=2):
|
|
@@ -447,10 +772,10 @@ with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
|
| 447 |
elem_classes="window-video",
|
| 448 |
)
|
| 449 |
with gr.Tab("Frame stepper"):
|
| 450 |
-
frame_idx = gr.Slider(0, 32, value=16, step=1, label="Frame in window
|
| 451 |
still = gr.Image(label="Frame comparison", elem_classes="window-still")
|
| 452 |
with gr.Tab("Driving actions"):
|
| 453 |
-
actions_plot = gr.Plot(label="
|
| 454 |
with gr.Tab("All windows"):
|
| 455 |
table = gr.Dataframe(
|
| 456 |
headers=["episode", "task", "edge", "arm", "crit PSNR", "full PSNR"],
|
|
@@ -460,12 +785,23 @@ with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
|
| 460 |
max_height=650,
|
| 461 |
)
|
| 462 |
|
| 463 |
-
filter_inputs = [query, edge, arm, sort]
|
| 464 |
filter_outputs = [picker, table, ids_state]
|
| 465 |
-
view_inputs = [picker, view, show_diff, fps, frame_idx]
|
| 466 |
-
view_outputs = [video, still, info, actions_plot]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 467 |
|
| 468 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 469 |
control.change(on_filter, filter_inputs + [picker], filter_outputs).then(
|
| 470 |
on_select, view_inputs, view_outputs
|
| 471 |
)
|
|
@@ -473,7 +809,7 @@ with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
|
| 473 |
picker.change(on_select, view_inputs, view_outputs)
|
| 474 |
for control in (view, show_diff, fps):
|
| 475 |
control.change(on_select, view_inputs, view_outputs)
|
| 476 |
-
frame_idx.change(on_frame, [picker, view, show_diff, frame_idx], [still, actions_plot])
|
| 477 |
|
| 478 |
prev_btn.click(lambda i, c: step(i, c, -1), [ids_state, picker], picker)
|
| 479 |
next_btn.click(lambda i, c: step(i, c, +1), [ids_state, picker], picker)
|
|
@@ -483,9 +819,5 @@ with gr.Blocks(title="Rollout vs. ground truth", fill_width=True) as demo:
|
|
| 483 |
|
| 484 |
table.select(on_table_select, ids_state, picker)
|
| 485 |
|
| 486 |
-
demo.load(on_filter, filter_inputs + [picker], filter_outputs).then(
|
| 487 |
-
on_select, view_inputs, view_outputs
|
| 488 |
-
)
|
| 489 |
-
|
| 490 |
if __name__ == "__main__":
|
| 491 |
demo.queue(max_size=16).launch(css=CSS)
|
|
|
|
| 1 |
"""
|
| 2 |
+
Side-by-side viewer for forward-dynamics rollouts vs. ground truth.
|
| 3 |
|
| 4 |
+
Point it at any HF dataset repo laid out as:
|
| 5 |
|
| 6 |
+
windows_manifest.json # optional: list of window dicts
|
| 7 |
+
metrics_summary.json # optional: per-window MSE / PSNR
|
| 8 |
+
windows/<episode_id>/gt.mp4 # ground-truth window
|
| 9 |
+
windows/<episode_id>/generated.mp4 # model rollout for the same slice
|
| 10 |
+
windows/<episode_id>/actions.json # optional: [T, D] driving actions
|
| 11 |
+
windows/<episode_id>/meta.json # optional: per-window metadata
|
| 12 |
|
| 13 |
+
Only `windows/<id>/{gt,generated}.mp4` is required. Without a manifest the
|
| 14 |
+
window list is discovered by listing `windows/` one level deep. Videos may
|
| 15 |
+
stack N cameras vertically; N is inferred from the frame aspect ratio.
|
| 16 |
+
|
| 17 |
+
Nothing is downloaded at build time and no repo is ever pulled whole. Each
|
| 18 |
+
browser session gets its own scratch directory; window files arrive only when
|
| 19 |
+
that window is opened, at most MAX_WINDOWS_ON_DISK are kept, and the whole
|
| 20 |
+
directory is deleted when the session ends.
|
| 21 |
+
|
| 22 |
+
DATASET_REPO sets the repo prefilled at startup. Private repos need an
|
| 23 |
+
HF_TOKEN secret with read access.
|
| 24 |
"""
|
| 25 |
|
| 26 |
from __future__ import annotations
|
| 27 |
|
|
|
|
| 28 |
import json
|
| 29 |
import os
|
| 30 |
+
import re
|
| 31 |
+
import shutil
|
| 32 |
import tempfile
|
| 33 |
+
import threading
|
| 34 |
+
import time
|
| 35 |
+
import uuid
|
| 36 |
+
from collections import OrderedDict
|
| 37 |
from pathlib import Path
|
| 38 |
|
| 39 |
import cv2
|
| 40 |
import gradio as gr
|
| 41 |
import imageio_ffmpeg
|
| 42 |
import numpy as np
|
| 43 |
+
from huggingface_hub import HfApi, RepoFolder, hf_hub_download
|
| 44 |
from matplotlib.figure import Figure
|
| 45 |
|
| 46 |
+
DEFAULT_REPO = os.environ.get("DATASET_REPO", "OneWorld-AI/abc-critical-window-rollout")
|
| 47 |
HF_TOKEN = os.environ.get("HF_TOKEN") or os.environ.get("HUGGING_FACE_HUB_TOKEN") or None
|
| 48 |
|
| 49 |
+
BASE_DIR = Path(tempfile.gettempdir()) / "wmviz"
|
| 50 |
+
BASE_DIR.mkdir(parents=True, exist_ok=True)
|
| 51 |
|
| 52 |
+
MAX_WINDOWS_ON_DISK = int(os.environ.get("MAX_WINDOWS_ON_DISK", 12))
|
| 53 |
+
MAX_PAIRS_IN_RAM = 2
|
| 54 |
+
MAX_COMPOSED_IN_RAM = 1
|
| 55 |
+
MAX_DISCOVER = int(os.environ.get("MAX_DISCOVER", 5000))
|
| 56 |
+
SESSION_TTL = int(os.environ.get("SESSION_TTL", 3600))
|
| 57 |
+
|
| 58 |
+
# Conventional names when a video stacks exactly three cameras.
|
| 59 |
+
THREE_CAM_NAMES = ["Top", "Left wrist", "Right wrist"]
|
| 60 |
+
ALL_CAMS = -1
|
| 61 |
+
NO_CRIT = -1
|
| 62 |
|
| 63 |
FONT = cv2.FONT_HERSHEY_SIMPLEX
|
| 64 |
HDR, FTR, SEP = 26, 26, 4
|
| 65 |
+
SCALE = 2 # source frames are small; upscale so overlays stay legible
|
| 66 |
|
| 67 |
GT_COLOR = (130, 225, 140)
|
| 68 |
GEN_COLOR = (120, 180, 255)
|
|
|
|
| 77 |
]
|
| 78 |
|
| 79 |
|
| 80 |
+
# --------------------------------------------------------------------------- session store
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def parse_repo(text: str) -> tuple[str, str | None]:
|
| 84 |
+
"""Split "owner/name@revision" into (repo_id, revision). Tolerates a pasted URL."""
|
| 85 |
+
text = (text or "").strip().rstrip("/")
|
| 86 |
+
m = re.match(r"^https?://huggingface\.co/(?:datasets/)?(.+?)(?:/tree/([^/]+))?$", text)
|
| 87 |
+
if m:
|
| 88 |
+
return m.group(1), m.group(2)
|
| 89 |
+
if "@" in text:
|
| 90 |
+
repo, _, rev = text.partition("@")
|
| 91 |
+
return repo.strip(), rev.strip() or None
|
| 92 |
+
return text, None
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
class Session:
|
| 96 |
+
"""One browser session's scratch space: bounded on disk and in RAM."""
|
| 97 |
+
|
| 98 |
+
def __init__(self, sid: str):
|
| 99 |
+
self.sid = sid
|
| 100 |
+
self.root = BASE_DIR / sid
|
| 101 |
+
self.data = self.root / "data"
|
| 102 |
+
self.render = self.root / "render"
|
| 103 |
+
for d in (self.data, self.render):
|
| 104 |
+
d.mkdir(parents=True, exist_ok=True)
|
| 105 |
+
self.repo = ""
|
| 106 |
+
self.revision: str | None = None
|
| 107 |
+
self.rows: list[dict] = []
|
| 108 |
+
self.note = ""
|
| 109 |
+
self._on_disk: OrderedDict[str, bool] = OrderedDict()
|
| 110 |
+
self._pairs: OrderedDict[str, tuple[np.ndarray, np.ndarray]] = OrderedDict()
|
| 111 |
+
self._composed: OrderedDict[tuple, np.ndarray] = OrderedDict()
|
| 112 |
+
self._meta: dict[str, dict] = {}
|
| 113 |
+
self.touched = time.time()
|
| 114 |
+
|
| 115 |
+
# -- fetching ----------------------------------------------------------
|
| 116 |
+
|
| 117 |
+
def fetch(self, rel: str, optional: bool = False) -> str | None:
|
| 118 |
+
"""Download one repo file into this session's directory."""
|
| 119 |
+
self.touched = time.time()
|
| 120 |
+
try:
|
| 121 |
+
return hf_hub_download(
|
| 122 |
+
repo_id=self.repo,
|
| 123 |
+
filename=rel,
|
| 124 |
+
repo_type="dataset",
|
| 125 |
+
revision=self.revision,
|
| 126 |
+
token=HF_TOKEN,
|
| 127 |
+
local_dir=str(self.data),
|
| 128 |
+
)
|
| 129 |
+
except Exception:
|
| 130 |
+
if optional:
|
| 131 |
+
return None
|
| 132 |
+
raise
|
| 133 |
+
|
| 134 |
+
def window_file(self, episode_id: str, name: str, optional: bool = False) -> str | None:
|
| 135 |
+
path = self.fetch(f"windows/{episode_id}/{name}", optional=optional)
|
| 136 |
+
if path:
|
| 137 |
+
self._keep(episode_id)
|
| 138 |
+
return path
|
| 139 |
+
|
| 140 |
+
def _keep(self, episode_id: str) -> None:
|
| 141 |
+
"""Register a window as resident and evict the least recently used ones."""
|
| 142 |
+
self._on_disk[episode_id] = True
|
| 143 |
+
self._on_disk.move_to_end(episode_id)
|
| 144 |
+
while len(self._on_disk) > MAX_WINDOWS_ON_DISK:
|
| 145 |
+
old, _ = self._on_disk.popitem(last=False)
|
| 146 |
+
shutil.rmtree(self.data / "windows" / old, ignore_errors=True)
|
| 147 |
+
self._pairs.pop(old, None)
|
| 148 |
+
for key in [k for k in self._composed if k[0] == old]:
|
| 149 |
+
self._composed.pop(key, None)
|
| 150 |
+
for f in self.render.glob(f"{old}_*.mp4"):
|
| 151 |
+
f.unlink(missing_ok=True)
|
| 152 |
+
|
| 153 |
+
def disk_windows(self) -> int:
|
| 154 |
+
return len(self._on_disk)
|
| 155 |
+
|
| 156 |
+
# -- bounded RAM caches ------------------------------------------------
|
| 157 |
+
|
| 158 |
+
def pair(self, episode_id: str) -> tuple[np.ndarray, np.ndarray]:
|
| 159 |
+
hit = self._pairs.get(episode_id)
|
| 160 |
+
if hit is not None:
|
| 161 |
+
self._pairs.move_to_end(episode_id)
|
| 162 |
+
return hit
|
| 163 |
+
gt = read_video(self.window_file(episode_id, "gt.mp4"))
|
| 164 |
+
gen = read_video(self.window_file(episode_id, "generated.mp4"))
|
| 165 |
+
t = min(len(gt), len(gen))
|
| 166 |
+
value = (gt[:t], gen[:t])
|
| 167 |
+
self._pairs[episode_id] = value
|
| 168 |
+
while len(self._pairs) > MAX_PAIRS_IN_RAM:
|
| 169 |
+
self._pairs.popitem(last=False)
|
| 170 |
+
return value
|
| 171 |
+
|
| 172 |
+
def composed(self, episode_id: str, view: int, show_diff: bool, crit: int) -> np.ndarray:
|
| 173 |
+
key = (episode_id, view, show_diff, crit)
|
| 174 |
+
hit = self._composed.get(key)
|
| 175 |
+
if hit is not None:
|
| 176 |
+
self._composed.move_to_end(key)
|
| 177 |
+
return hit
|
| 178 |
+
value = compose(self.pair(episode_id), view, show_diff, crit)
|
| 179 |
+
self._composed[key] = value
|
| 180 |
+
while len(self._composed) > MAX_COMPOSED_IN_RAM:
|
| 181 |
+
self._composed.popitem(last=False)
|
| 182 |
+
return value
|
| 183 |
+
|
| 184 |
+
def meta(self, episode_id: str) -> dict:
|
| 185 |
+
if episode_id not in self._meta:
|
| 186 |
+
path = self.window_file(episode_id, "meta.json", optional=True)
|
| 187 |
+
try:
|
| 188 |
+
self._meta[episode_id] = json.load(open(path)) if path else {}
|
| 189 |
+
except Exception:
|
| 190 |
+
self._meta[episode_id] = {}
|
| 191 |
+
return self._meta[episode_id]
|
| 192 |
+
|
| 193 |
+
# -- lifecycle ---------------------------------------------------------
|
| 194 |
+
|
| 195 |
+
def set_repo(self, repo: str, revision: str | None) -> None:
|
| 196 |
+
if (repo, revision) != (self.repo, self.revision):
|
| 197 |
+
self.reset()
|
| 198 |
+
self.repo, self.revision = repo, revision
|
| 199 |
+
|
| 200 |
+
def reset(self) -> None:
|
| 201 |
+
self._on_disk.clear()
|
| 202 |
+
self._pairs.clear()
|
| 203 |
+
self._composed.clear()
|
| 204 |
+
self._meta.clear()
|
| 205 |
+
self.rows = []
|
| 206 |
+
for d in (self.data, self.render):
|
| 207 |
+
shutil.rmtree(d, ignore_errors=True)
|
| 208 |
+
d.mkdir(parents=True, exist_ok=True)
|
| 209 |
+
|
| 210 |
+
def close(self) -> None:
|
| 211 |
+
self._pairs.clear()
|
| 212 |
+
self._composed.clear()
|
| 213 |
+
shutil.rmtree(self.root, ignore_errors=True)
|
| 214 |
+
|
| 215 |
+
|
| 216 |
+
SESSIONS: dict[str, Session] = {}
|
| 217 |
+
_LOCK = threading.Lock()
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
def sweep() -> None:
|
| 221 |
+
"""Drop sessions whose browser tab went away without a clean callback."""
|
| 222 |
+
now = time.time()
|
| 223 |
+
with _LOCK:
|
| 224 |
+
stale = [s for s in SESSIONS.values() if now - s.touched > SESSION_TTL]
|
| 225 |
+
for s in stale:
|
| 226 |
+
SESSIONS.pop(s.sid, None)
|
| 227 |
+
s.close()
|
| 228 |
+
live = set(SESSIONS)
|
| 229 |
+
for d in BASE_DIR.iterdir(): # dirs left behind by a previous process
|
| 230 |
+
try:
|
| 231 |
+
if d.is_dir() and d.name not in live and now - d.stat().st_mtime > SESSION_TTL:
|
| 232 |
+
shutil.rmtree(d, ignore_errors=True)
|
| 233 |
+
except OSError:
|
| 234 |
+
pass
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
def get_session(sid: str | None) -> Session:
|
| 238 |
+
with _LOCK:
|
| 239 |
+
s = SESSIONS.get(sid or "")
|
| 240 |
+
if s is None:
|
| 241 |
+
sid = sid or uuid.uuid4().hex
|
| 242 |
+
s = SESSIONS[sid] = Session(sid)
|
| 243 |
+
s.touched = time.time()
|
| 244 |
+
return s
|
| 245 |
+
|
| 246 |
+
|
| 247 |
+
def release(sid: str | None) -> None:
|
| 248 |
+
"""gr.State delete_callback — fires when the session ends or its TTL expires."""
|
| 249 |
+
with _LOCK:
|
| 250 |
+
s = SESSIONS.pop(sid or "", None)
|
| 251 |
+
if s is not None:
|
| 252 |
+
s.close()
|
| 253 |
+
|
| 254 |
+
|
| 255 |
+
# --------------------------------------------------------------------------- index
|
| 256 |
+
|
| 257 |
+
|
| 258 |
+
def _row(order: int, episode_id: str, w: dict, m: dict) -> dict:
|
| 259 |
+
return {
|
| 260 |
+
"order": order,
|
| 261 |
+
"episode_id": episode_id,
|
| 262 |
+
"task": w.get("task") or "?",
|
| 263 |
+
"edge": w.get("edge_type") or "?",
|
| 264 |
+
"arm": w.get("arm") or "?",
|
| 265 |
+
"amplitude": w.get("amplitude"),
|
| 266 |
+
"start_frame": w.get("start_frame"),
|
| 267 |
+
"window": w.get("window"),
|
| 268 |
+
"critical_frame": w.get("critical_frame"),
|
| 269 |
+
"n_frames": w.get("n_frames"),
|
| 270 |
+
"full_psnr": m.get("full_psnr"),
|
| 271 |
+
"crit_psnr": m.get("crit_psnr"),
|
| 272 |
+
"full_mse": m.get("full_mse"),
|
| 273 |
+
"crit_mse": m.get("crit_mse"),
|
| 274 |
+
}
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
def discover_episodes(session: Session) -> tuple[list[str], bool]:
|
| 278 |
+
"""List windows/ one level deep, capped. Returns (ids, was_truncated)."""
|
| 279 |
+
tree = HfApi().list_repo_tree(
|
| 280 |
+
session.repo,
|
| 281 |
+
path_in_repo="windows",
|
| 282 |
+
recursive=False,
|
| 283 |
+
revision=session.revision,
|
| 284 |
repo_type="dataset",
|
| 285 |
token=HF_TOKEN,
|
| 286 |
)
|
| 287 |
+
eps, truncated = [], False
|
| 288 |
+
for entry in tree:
|
| 289 |
+
if not isinstance(entry, RepoFolder):
|
| 290 |
+
continue
|
| 291 |
+
eps.append(entry.path.split("/")[-1])
|
| 292 |
+
if len(eps) >= MAX_DISCOVER:
|
| 293 |
+
truncated = True
|
| 294 |
+
break
|
| 295 |
+
return sorted(eps), truncated
|
| 296 |
+
|
| 297 |
+
|
| 298 |
+
def build_index(session: Session) -> str:
|
| 299 |
+
"""Populate session.rows from the manifest, or by listing the repo. Returns a note."""
|
| 300 |
+
metrics = []
|
| 301 |
+
path = session.fetch("metrics_summary.json", optional=True)
|
| 302 |
+
if path:
|
| 303 |
+
try:
|
| 304 |
+
metrics = json.load(open(path))
|
| 305 |
+
except Exception:
|
| 306 |
+
metrics = []
|
| 307 |
+
by_id = {m.get("id"): m for m in metrics if isinstance(m, dict)}
|
| 308 |
+
|
| 309 |
+
manifest = None
|
| 310 |
+
path = session.fetch("windows_manifest.json", optional=True)
|
| 311 |
+
if path:
|
| 312 |
+
try:
|
| 313 |
+
loaded = json.load(open(path))
|
| 314 |
+
if isinstance(loaded, list) and loaded:
|
| 315 |
+
manifest = loaded
|
| 316 |
+
except Exception:
|
| 317 |
+
manifest = None
|
| 318 |
+
|
| 319 |
+
if manifest is not None:
|
| 320 |
+
session.rows = [
|
| 321 |
+
_row(i, w.get("episode_id", f"window_{i}"), w, by_id.get(w.get("episode_id"), {}))
|
| 322 |
+
for i, w in enumerate(manifest)
|
| 323 |
+
]
|
| 324 |
+
note = "`windows_manifest.json`"
|
| 325 |
+
else:
|
| 326 |
+
eps, truncated = discover_episodes(session)
|
| 327 |
+
if not eps:
|
| 328 |
+
raise FileNotFoundError(
|
| 329 |
+
"no windows/<episode_id>/ directories found — is this the right repo?"
|
| 330 |
+
)
|
| 331 |
+
session.rows = [_row(i, ep, {}, by_id.get(ep, {})) for i, ep in enumerate(eps)]
|
| 332 |
+
note = "directory listing (no `windows_manifest.json`)"
|
| 333 |
+
if truncated:
|
| 334 |
+
note += f", capped at {MAX_DISCOVER}"
|
| 335 |
|
| 336 |
+
if metrics:
|
| 337 |
+
note += " + `metrics_summary.json`"
|
| 338 |
+
return note
|
| 339 |
|
| 340 |
+
|
| 341 |
+
def short_id(episode_id: str) -> str:
|
| 342 |
+
return re.sub(r"^(episode|window|ep)[-_]", "", episode_id)[:8]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 343 |
|
| 344 |
|
| 345 |
def label_for(row: dict) -> str:
|
| 346 |
psnr = row["crit_psnr"]
|
| 347 |
+
head = f"{psnr:5.1f} dB" if isinstance(psnr, (int, float)) else " -- dB"
|
| 348 |
+
tags = " ".join(t for t in (row["edge"], row["arm"]) if t and t != "?")
|
| 349 |
+
parts = [head] + ([tags] if tags else []) + [row["task"], short_id(row["episode_id"])]
|
| 350 |
+
return " · ".join(parts)
|
| 351 |
|
| 352 |
|
| 353 |
+
def filter_rows(session: Session, query: str, edge: str, arm: str, sort: str) -> list[dict]:
|
| 354 |
+
rows = session.rows
|
| 355 |
q = (query or "").strip().lower()
|
| 356 |
if q:
|
| 357 |
rows = [r for r in rows if q in r["task"].lower() or q in r["episode_id"].lower()]
|
|
|
|
| 360 |
if arm != "any":
|
| 361 |
rows = [r for r in rows if r["arm"] == arm]
|
| 362 |
|
| 363 |
+
def psnr(r, missing):
|
| 364 |
+
v = r["crit_psnr"]
|
| 365 |
+
return v if isinstance(v, (int, float)) else missing
|
| 366 |
+
|
| 367 |
if sort == SORTS[0]:
|
| 368 |
+
return sorted(rows, key=lambda r: psnr(r, float("inf")))
|
| 369 |
+
if sort == SORTS[1]:
|
| 370 |
+
return sorted(rows, key=lambda r: -psnr(r, float("-inf")))
|
| 371 |
+
if sort == SORTS[2]:
|
| 372 |
+
return sorted(rows, key=lambda r: (r["task"], r["episode_id"]))
|
| 373 |
+
return sorted(rows, key=lambda r: r["order"])
|
|
|
|
|
|
|
| 374 |
|
| 375 |
|
| 376 |
+
def row_by_id(session: Session, episode_id: str) -> dict:
|
| 377 |
+
for r in session.rows:
|
|
|
|
| 378 |
if r["episode_id"] == episode_id:
|
| 379 |
+
row = dict(r)
|
| 380 |
+
break
|
| 381 |
+
else:
|
| 382 |
+
row = _row(0, episode_id, {}, {})
|
| 383 |
+
# backfill whatever the manifest did not carry from the window's own meta.json
|
| 384 |
+
if row["task"] == "?" or row["critical_frame"] is None:
|
| 385 |
+
meta = session.meta(episode_id)
|
| 386 |
+
if meta:
|
| 387 |
+
merged = _row(row["order"], episode_id, meta, {})
|
| 388 |
+
for k, v in merged.items():
|
| 389 |
+
if row.get(k) in (None, "?"):
|
| 390 |
+
row[k] = v
|
| 391 |
+
return row
|
| 392 |
|
| 393 |
|
| 394 |
# --------------------------------------------------------------------------- video
|
|
|
|
| 409 |
return np.stack(frames)
|
| 410 |
|
| 411 |
|
| 412 |
+
def n_cameras(frames: np.ndarray) -> int:
|
| 413 |
+
"""How many roughly-square cameras are stacked vertically in the frame."""
|
| 414 |
+
h, w = frames.shape[1], frames.shape[2]
|
| 415 |
+
return max(1, min(6, int(round(h / w))))
|
| 416 |
+
|
|
|
|
| 417 |
|
| 418 |
+
def camera_choices(n: int) -> list[tuple[str, int]]:
|
| 419 |
+
names = THREE_CAM_NAMES if n == 3 else [f"Camera {i + 1}" for i in range(n)]
|
| 420 |
+
choices = [(name, i) for i, name in enumerate(names)]
|
| 421 |
+
return ([("All cameras", ALL_CAMS)] + choices) if n > 1 else choices
|
| 422 |
|
| 423 |
+
|
| 424 |
+
def cam_slices(frames: np.ndarray, view: int) -> list[np.ndarray]:
|
| 425 |
+
n = n_cameras(frames)
|
| 426 |
+
h = frames.shape[1] // n
|
| 427 |
+
rows = range(n) if view == ALL_CAMS or not 0 <= view < n else [view]
|
| 428 |
return [frames[:, i * h : (i + 1) * h] for i in rows]
|
| 429 |
|
| 430 |
|
|
|
|
| 440 |
return int(idx[0]), int(idx[-1]) + 1
|
| 441 |
|
| 442 |
|
| 443 |
+
def crop_view(gt: np.ndarray, gen: np.ndarray, view: int) -> tuple[np.ndarray, np.ndarray]:
|
| 444 |
"""Select cameras and trim the letterbox, identically for both videos."""
|
| 445 |
gt_out, gen_out = [], []
|
| 446 |
for g, p in zip(cam_slices(gt, view), cam_slices(gen, view)):
|
|
|
|
| 474 |
return np.stack([cv2.applyColorMap(f, cv2.COLORMAP_INFERNO)[:, :, ::-1] for f in d])
|
| 475 |
|
| 476 |
|
| 477 |
+
def compose(pair: tuple[np.ndarray, np.ndarray], view: int, show_diff: bool, crit_idx: int) -> np.ndarray:
|
|
|
|
| 478 |
"""Build the labelled side-by-side strip: [T, H, W, 3] RGB."""
|
| 479 |
+
gt_c, gen_c = crop_view(pair[0], pair[1], view)
|
|
|
|
| 480 |
gt_v, gen_v = upscale(gt_c, SCALE), upscale(gen_c, SCALE)
|
| 481 |
|
| 482 |
panels = [panel(gt_v, "GROUND TRUTH", GT_COLOR), panel(gen_v, "GENERATED", GEN_COLOR)]
|
|
|
|
| 493 |
t, h, w, _ = comp.shape
|
| 494 |
out = np.zeros((t, h + FTR, w, 3), np.uint8)
|
| 495 |
out[:, :h] = comp
|
| 496 |
+
has_crit = 0 <= crit_idx < t
|
| 497 |
tl0, tl1, y = int(w * 0.45), w - 14, FTR // 2
|
| 498 |
for i in range(t):
|
| 499 |
bar = np.full((FTR, w, 3), 22, np.uint8)
|
| 500 |
+
is_crit = has_crit and i == crit_idx
|
| 501 |
text = f"frame {i:02d}/{t - 1}" + (" CRITICAL FRAME" if is_crit else "")
|
| 502 |
cv2.putText(bar, text, (8, FTR - 8), FONT, 0.5, CRIT_COLOR if is_crit else (200, 200, 200), 1, cv2.LINE_AA)
|
| 503 |
cv2.line(bar, (tl0, y), (tl1, y), (70, 70, 70), 2, cv2.LINE_AA)
|
| 504 |
+
if has_crit:
|
| 505 |
cx = int(tl0 + (tl1 - tl0) * crit_idx / max(t - 1, 1))
|
| 506 |
cv2.line(bar, (cx, y - 7), (cx, y + 7), CRIT_COLOR, 2, cv2.LINE_AA)
|
| 507 |
px = int(tl0 + (tl1 - tl0) * i / max(t - 1, 1))
|
|
|
|
| 518 |
|
| 519 |
|
| 520 |
def write_mp4(frames: np.ndarray, path: Path, fps: float) -> str:
|
| 521 |
+
_, h, w, _ = frames.shape
|
| 522 |
writer = imageio_ffmpeg.write_frames(
|
| 523 |
str(path),
|
| 524 |
size=(w, h),
|
|
|
|
| 536 |
return str(path)
|
| 537 |
|
| 538 |
|
| 539 |
+
def render_video(session: Session, episode_id: str, view: int, show_diff: bool, fps: float, crit: int) -> str:
|
| 540 |
+
path = session.render / f"{episode_id}_{view}_{int(show_diff)}_{fps:g}.mp4"
|
|
|
|
| 541 |
if not path.exists():
|
| 542 |
+
write_mp4(session.composed(episode_id, view, show_diff, crit), path, fps)
|
| 543 |
return str(path)
|
| 544 |
|
| 545 |
|
| 546 |
# --------------------------------------------------------------------------- actions plot
|
| 547 |
|
| 548 |
|
| 549 |
+
def plot_actions(session: Session, episode_id: str, crit_idx: int, cursor: int | None) -> Figure | None:
|
| 550 |
+
path = session.window_file(episode_id, "actions.json", optional=True)
|
| 551 |
+
if not path:
|
| 552 |
+
return None
|
| 553 |
+
try:
|
| 554 |
+
actions = np.atleast_2d(np.asarray(json.load(open(path)), dtype=float))
|
| 555 |
+
except Exception:
|
| 556 |
+
return None
|
| 557 |
+
|
| 558 |
t = np.arange(len(actions))
|
| 559 |
+
dim = actions.shape[1]
|
| 560 |
+
if dim == 14: # bimanual: [L arm 6, L grip, R arm 6, R grip]
|
| 561 |
+
specs = [
|
| 562 |
+
("Grippers (1.0 = open, 0.0 = closed)", [(6, "L gripper"), (13, "R gripper")]),
|
| 563 |
+
("Left arm joints", [(i, f"L{i}") for i in range(6)]),
|
| 564 |
+
("Right arm joints", [(i, f"R{i - 7}") for i in range(7, 13)]),
|
| 565 |
+
]
|
| 566 |
+
else:
|
| 567 |
+
specs = [(f"Action channels [{len(actions)} x {dim}]", [(i, f"a{i}") for i in range(dim)])]
|
| 568 |
|
| 569 |
+
fig = Figure(figsize=(7.2, 1.8 * len(specs) + 0.6), dpi=110, layout="constrained")
|
| 570 |
+
axes = np.atleast_1d(fig.subplots(len(specs), 1, sharex=True))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 571 |
for ax, (title, channels) in zip(axes, specs):
|
| 572 |
for idx, name in channels:
|
| 573 |
ax.plot(t, actions[:, idx], lw=1.4, label=name)
|
| 574 |
ax.set_title(title, fontsize=9, loc="left")
|
| 575 |
+
if crit_idx >= 0:
|
| 576 |
+
ax.axvline(crit_idx, color="crimson", ls="--", lw=1.2, zorder=0)
|
| 577 |
if cursor is not None:
|
| 578 |
ax.axvline(cursor, color="0.35", ls=":", lw=1.0, zorder=0)
|
| 579 |
ax.grid(alpha=0.25, lw=0.5)
|
| 580 |
ax.tick_params(labelsize=8)
|
| 581 |
+
if len(channels) <= 8:
|
| 582 |
+
ax.legend(fontsize=7, ncol=6, loc="upper right", framealpha=0.7)
|
| 583 |
+
tail = " (dashed red = critical frame)" if crit_idx >= 0 else ""
|
| 584 |
+
axes[-1].set_xlabel(f"frame in window{tail}", fontsize=8)
|
| 585 |
return fig
|
| 586 |
|
| 587 |
|
| 588 |
# --------------------------------------------------------------------------- ui glue
|
| 589 |
|
| 590 |
|
| 591 |
+
def metrics_md(session: Session, row: dict) -> str:
|
| 592 |
def fmt(v, unit=""):
|
| 593 |
return f"{v:.2f}{unit}" if isinstance(v, (int, float)) else "—"
|
| 594 |
|
| 595 |
+
out = [f"### {row['task'].replace('_', ' ')}", f"`{row['episode_id']}`", ""]
|
| 596 |
+
if any(isinstance(row[k], (int, float)) for k in ("full_psnr", "crit_psnr", "full_mse", "crit_mse")):
|
| 597 |
+
out += [
|
| 598 |
+
"| | full window | critical frame |",
|
| 599 |
+
"|---|---|---|",
|
| 600 |
+
f"| **PSNR** | {fmt(row['full_psnr'], ' dB')} | {fmt(row['crit_psnr'], ' dB')} |",
|
| 601 |
+
f"| **MSE** | {fmt(row['full_mse'])} | {fmt(row['crit_mse'])} |",
|
| 602 |
+
"",
|
| 603 |
+
]
|
| 604 |
+
bits = []
|
| 605 |
+
if row["edge"] != "?":
|
| 606 |
+
bits.append(f"**{row['edge']}**")
|
| 607 |
+
if row["arm"] != "?":
|
| 608 |
+
bits.append(f"arm **{row['arm']}**")
|
| 609 |
+
if isinstance(row["amplitude"], (int, float)):
|
| 610 |
+
bits.append(f"amplitude {fmt(row['amplitude'])}")
|
| 611 |
+
win = row.get("window")
|
| 612 |
+
if isinstance(win, (list, tuple)) and len(win) == 2:
|
| 613 |
+
bits.append(f"window frames {win[0]}–{win[1]}")
|
| 614 |
+
if row["n_frames"]:
|
| 615 |
+
bits.append(f"of {row['n_frames']}")
|
| 616 |
+
if row["critical_frame"] is not None:
|
| 617 |
+
bits.append(f"critical frame {row['critical_frame']}")
|
| 618 |
+
if bits:
|
| 619 |
+
out.append(" · ".join(bits))
|
| 620 |
+
out.append(
|
| 621 |
+
f"\n<sub>cached this session: {session.disk_windows()}/{MAX_WINDOWS_ON_DISK} windows "
|
| 622 |
+
"— evicted least-recently-used, all deleted when you close the tab</sub>"
|
| 623 |
)
|
| 624 |
+
return "\n".join(out)
|
| 625 |
|
| 626 |
|
| 627 |
def table_rows(rows: list[dict]) -> list[list]:
|
| 628 |
+
def num(v):
|
| 629 |
+
return round(v, 2) if isinstance(v, (int, float)) else None
|
| 630 |
+
|
| 631 |
return [
|
| 632 |
+
[short_id(r["episode_id"]), r["task"], r["edge"], r["arm"], num(r["crit_psnr"]), num(r["full_psnr"])]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 633 |
for r in rows
|
| 634 |
]
|
| 635 |
|
| 636 |
|
| 637 |
+
def critical_index(row: dict) -> int:
|
| 638 |
+
"""Offset of the critical frame inside the window, or NO_CRIT."""
|
| 639 |
+
start, crit = row.get("start_frame"), row.get("critical_frame")
|
| 640 |
+
if start is None or crit is None:
|
| 641 |
+
return NO_CRIT
|
| 642 |
+
return int(crit) - int(start)
|
| 643 |
+
|
| 644 |
+
|
| 645 |
+
def on_load(repo_text, sid):
|
| 646 |
+
"""Resolve a repo and load only its index — no window data is pulled here."""
|
| 647 |
+
sweep()
|
| 648 |
+
session = get_session(sid)
|
| 649 |
+
repo, revision = parse_repo(repo_text)
|
| 650 |
+
if not repo:
|
| 651 |
+
return session.sid, "Enter a dataset repo id, e.g. `owner/name`."
|
| 652 |
+
session.set_repo(repo, revision)
|
| 653 |
+
try:
|
| 654 |
+
note = build_index(session)
|
| 655 |
+
except Exception as exc:
|
| 656 |
+
session.rows = []
|
| 657 |
+
gr.Warning(f"Could not load {repo}: {exc}")
|
| 658 |
+
return session.sid, f"❌ **{repo}** — {exc}"
|
| 659 |
+
at = f"@{revision}" if revision else ""
|
| 660 |
+
return session.sid, (
|
| 661 |
+
f"✅ **{repo}{at}** — {len(session.rows)} windows from {note}. "
|
| 662 |
+
"Window files download only when you open them."
|
| 663 |
+
)
|
| 664 |
+
|
| 665 |
+
|
| 666 |
+
def on_filter(sid, query, edge, arm, sort, current):
|
| 667 |
+
session = get_session(sid)
|
| 668 |
+
if not session.rows:
|
| 669 |
+
return gr.update(choices=[], value=None), [], []
|
| 670 |
+
rows = filter_rows(session, query, edge, arm, sort)
|
| 671 |
choices = [(label_for(r), r["episode_id"]) for r in rows]
|
| 672 |
ids = [r["episode_id"] for r in rows]
|
| 673 |
value = current if current in ids else (ids[0] if ids else None)
|
| 674 |
return (
|
| 675 |
+
gr.update(choices=choices, value=value, label=f"Window ({len(ids)} of {len(session.rows)})"),
|
| 676 |
table_rows(rows),
|
| 677 |
ids,
|
| 678 |
)
|
| 679 |
|
| 680 |
|
| 681 |
+
def on_select(sid, episode_id, view, show_diff, fps, frame_idx):
|
| 682 |
+
session = get_session(sid)
|
| 683 |
+
if not session.rows or not episode_id:
|
| 684 |
+
return None, None, "*No window selected.*", None, gr.update(), gr.update()
|
| 685 |
+
row = row_by_id(session, episode_id)
|
| 686 |
+
crit = critical_index(row)
|
| 687 |
|
| 688 |
+
cams = n_cameras(session.pair(episode_id)[0])
|
| 689 |
+
choices = camera_choices(cams)
|
| 690 |
+
valid = [v for _, v in choices]
|
| 691 |
+
view = view if view in valid else choices[0][1]
|
| 692 |
+
cam_update = gr.update(choices=choices, value=view)
|
| 693 |
|
| 694 |
+
video = render_video(session, episode_id, view, show_diff, fps, crit)
|
| 695 |
+
frames = session.composed(episode_id, view, show_diff, crit)
|
| 696 |
+
last = len(frames) - 1
|
| 697 |
+
i = int(min(frame_idx, last))
|
| 698 |
+
plot = plot_actions(session, episode_id, crit, i)
|
| 699 |
+
slider = gr.update(maximum=last, value=i, label=f"Frame in window (0–{last})")
|
| 700 |
+
return video, frames[i], metrics_md(session, row), plot, slider, cam_update
|
|
|
|
|
|
|
| 701 |
|
| 702 |
|
| 703 |
+
def on_frame(sid, episode_id, view, show_diff, frame_idx):
|
| 704 |
+
session = get_session(sid)
|
| 705 |
+
if not session.rows or not episode_id:
|
| 706 |
return None, None
|
| 707 |
+
crit = critical_index(row_by_id(session, episode_id))
|
| 708 |
+
frames = session.composed(episode_id, view, show_diff, crit)
|
| 709 |
i = int(min(frame_idx, len(frames) - 1))
|
| 710 |
+
return frames[i], plot_actions(session, episode_id, crit, i)
|
| 711 |
|
| 712 |
|
| 713 |
def step(ids, current, delta):
|
|
|
|
| 727 |
.window-still img { max-height: 78vh; object-fit: contain; background: #111; }
|
| 728 |
"""
|
| 729 |
|
| 730 |
+
with gr.Blocks(title="Rollout vs. ground truth", fill_width=True, delete_cache=(600, 600)) as demo:
|
| 731 |
+
gr.Markdown("# Forward-dynamics rollout vs. ground truth")
|
| 732 |
+
|
| 733 |
+
sid_state = gr.State("", time_to_live=SESSION_TTL, delete_callback=release)
|
|
|
|
|
|
|
|
|
|
| 734 |
ids_state = gr.State([])
|
| 735 |
|
| 736 |
+
with gr.Row():
|
| 737 |
+
repo_box = gr.Textbox(
|
| 738 |
+
DEFAULT_REPO,
|
| 739 |
+
label="Dataset repo",
|
| 740 |
+
info="owner/name, optionally owner/name@revision. A repo URL also works.",
|
| 741 |
+
scale=5,
|
| 742 |
+
submit_btn=True,
|
| 743 |
+
)
|
| 744 |
+
load_btn = gr.Button("Load", variant="primary", scale=1)
|
| 745 |
+
status = gr.Markdown()
|
| 746 |
+
|
| 747 |
with gr.Row():
|
| 748 |
with gr.Column(scale=1, min_width=320):
|
| 749 |
query = gr.Textbox(label="Search task or episode", placeholder="e.g. napkin, pillowcase…")
|
|
|
|
| 756 |
prev_btn = gr.Button("← Prev")
|
| 757 |
next_btn = gr.Button("Next →")
|
| 758 |
gr.Markdown("---")
|
| 759 |
+
view = gr.Radio(camera_choices(3), value=0, label="Camera")
|
| 760 |
show_diff = gr.Checkbox(False, label="Add |GT − generated| difference panel")
|
| 761 |
+
fps = gr.Slider(2, 30, value=8, step=1, label="Playback fps")
|
| 762 |
info = gr.Markdown()
|
| 763 |
|
| 764 |
with gr.Column(scale=2):
|
|
|
|
| 772 |
elem_classes="window-video",
|
| 773 |
)
|
| 774 |
with gr.Tab("Frame stepper"):
|
| 775 |
+
frame_idx = gr.Slider(0, 32, value=16, step=1, label="Frame in window")
|
| 776 |
still = gr.Image(label="Frame comparison", elem_classes="window-still")
|
| 777 |
with gr.Tab("Driving actions"):
|
| 778 |
+
actions_plot = gr.Plot(label="Driving action chunk")
|
| 779 |
with gr.Tab("All windows"):
|
| 780 |
table = gr.Dataframe(
|
| 781 |
headers=["episode", "task", "edge", "arm", "crit PSNR", "full PSNR"],
|
|
|
|
| 785 |
max_height=650,
|
| 786 |
)
|
| 787 |
|
| 788 |
+
filter_inputs = [sid_state, query, edge, arm, sort]
|
| 789 |
filter_outputs = [picker, table, ids_state]
|
| 790 |
+
view_inputs = [sid_state, picker, view, show_diff, fps, frame_idx]
|
| 791 |
+
view_outputs = [video, still, info, actions_plot, frame_idx, view]
|
| 792 |
+
|
| 793 |
+
def wire_load(event):
|
| 794 |
+
return (
|
| 795 |
+
event(on_load, [repo_box, sid_state], [sid_state, status])
|
| 796 |
+
.then(on_filter, filter_inputs + [picker], filter_outputs)
|
| 797 |
+
.then(on_select, view_inputs, view_outputs)
|
| 798 |
+
)
|
| 799 |
|
| 800 |
+
wire_load(load_btn.click)
|
| 801 |
+
wire_load(repo_box.submit)
|
| 802 |
+
wire_load(demo.load)
|
| 803 |
+
|
| 804 |
+
for control in (query, edge, arm, sort):
|
| 805 |
control.change(on_filter, filter_inputs + [picker], filter_outputs).then(
|
| 806 |
on_select, view_inputs, view_outputs
|
| 807 |
)
|
|
|
|
| 809 |
picker.change(on_select, view_inputs, view_outputs)
|
| 810 |
for control in (view, show_diff, fps):
|
| 811 |
control.change(on_select, view_inputs, view_outputs)
|
| 812 |
+
frame_idx.change(on_frame, [sid_state, picker, view, show_diff, frame_idx], [still, actions_plot])
|
| 813 |
|
| 814 |
prev_btn.click(lambda i, c: step(i, c, -1), [ids_state, picker], picker)
|
| 815 |
next_btn.click(lambda i, c: step(i, c, +1), [ids_state, picker], picker)
|
|
|
|
| 819 |
|
| 820 |
table.select(on_table_select, ids_state, picker)
|
| 821 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 822 |
if __name__ == "__main__":
|
| 823 |
demo.queue(max_size=16).launch(css=CSS)
|