README.md

metadata

title: Rollout vs Ground Truth
emoji: 🎬
colorFrom: green
colorTo: purple
sdk: gradio
sdk_version: 6.22.0
python_version: '3.12'
app_file: app.py
pinned: false
short_description: Frame-locked viewer for rollouts vs. ground truth

Rollout vs. ground truth

Side-by-side viewer for forward-dynamics world-model rollouts. The page starts empty and assumes no dataset — type a dataset repo id into the box, press Load, and pick an episode.

Accepted forms: owner/name, owner/name@revision, or a pasted https://huggingface.co/datasets/… URL.

What it shows

Ground truth and the generated rollout are composed into a single video, so the two panels stay frame-locked; two independent players drift apart and cannot be scrubbed together. The strip carries in-frame captions, a frame counter, and a timeline marking the critical frame.

Sidebar controls: search, sort (critical-frame PSNR, task, manifest order), the episode list, a camera selector, an optional |GT − generated| difference panel, and playback fps — a 33-frame window is only ~1 s at native 30 fps. PSNR and MSE for the selected episode are shown underneath.

Expected dataset layout

windows_manifest.json                 # optional: list of window dicts (episode_id, task,
                                      #   edge_type, arm, start_frame, critical_frame, ...)
metrics_summary.json                  # optional: [{id, task, edge, arm, full_mse, full_psnr,
                                      #   crit_mse, crit_psnr}, ...]
windows/<episode_id>/gt.mp4           # required
windows/<episode_id>/generated.mp4    # required, same geometry and length as gt.mp4

Only the two mp4s are required. With no windows_manifest.json the episode list is discovered by listing windows/ one level deep. Videos may stack N cameras vertically — N is inferred from the frame aspect ratio and the camera selector adapts. Black letterbox bars are trimmed automatically, identically for both videos.

Storage model

Nothing is downloaded at build time and no repo is ever pulled whole, so datasets far larger than the Space still work.

• Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory listing when there is no manifest.
• An episode's mp4s are fetched the first time that episode is opened, never in bulk.
• Each browser session gets its own scratch directory. At most MAX_WINDOWS_ON_DISK episodes stay resident; beyond that the least recently used are evicted along with their rendered videos. Decoded frames are capped at 2 episodes in RAM and composed strips at 1.
• The directory is deleted when the session ends. A janitor also reaps sessions idle past SESSION_TTL and directories orphaned by an earlier process.

Configuration

Variable	Default	Purpose
HF_TOKEN	—	Required for private datasets. Set under Settings → Variables and secrets.
MAX_WINDOWS_ON_DISK	12	Episodes kept per session before LRU eviction.
MAX_DISCOVER	5000	Cap on episodes listed when a repo has no manifest.
SESSION_TTL	3600	Seconds of idleness before a session's storage is reclaimed.

Note: on a public Space the HF_TOKEN is used to read whatever repo a visitor types, so scope it to only the datasets you are willing to expose.

Running locally

pip install -r requirements.txt
export HF_TOKEN=hf_...          # if the dataset is private
python app.py