AdithyaSK's picture
AdithyaSK HF Staff
Harbor run viewer: Phase 0 eval, 15 harnesses x 50 tasks, pass@4
3d20eb8 verified
|
Raw
History Blame Contribute Delete
2.53 kB
metadata
title: Harbor Run Viewer
emoji: 🔭
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

Harbor Run Viewer

Train and eval runs from the OpenEnv × Harbor stack. The data lives in a Hugging Face bucket; this app only renders it.

The one design decision

A bucket mounted into a Space appears as an ordinary filesystem path inside the container, so the Space and a laptop run the same code against the same layout — DATA_DIR is /data in one case and ./data in the other. There is no local-vs-remote branch to keep in step, and no data in the image.

laptop:   DATA_DIR=./data          uvicorn app:app
Space:    DATA_DIR=/data           ← bucket mounted read-only or read-write
fallback: DATA_BUCKET=ns/bucket    ← read a bucket over HfFileSystem without mounting it

Run it locally

pip install -r requirements.txt
DATA_DIR=./data uvicorn app:app --port 8000     # http://localhost:8000

Publish runs to the bucket

hf buckets create <ns>/harbor-runs --exist-ok
hf sync ./data hf://buckets/<ns>/harbor-runs           # add --delete to mirror exactly

Point the Space at the bucket

hf spaces volumes set <ns>/harbor-run-viewer -v hf://buckets/<ns>/harbor-runs:/data

Only buckets support read-write mounts; models, datasets and Spaces are always read-only. A private bucket needs HF_TOKEN as a Space secret.

Then press ⟳ Refresh in the UI: it drops every cache and rescans. A mounted bucket already reflects new writes, so nothing needs rebuilding or restarting to see a run that finished a minute ago.

Layout

See CONTRACT.md. Briefly: one directory per run under runs/<run_id>/, identified by a run.json, with optional eval/summary.json and train/metrics.jsonl. A run with only training data renders as training; one with both shows both; nothing is required beyond run.json.

Two conventions in that contract carry real meaning rather than being formatting:

  • reward: null is not zero. It means the verifier never ran — a dead sandbox, no answer. Those attempts are excluded, and a task whose every attempt is null is counted as unmeasured. Averaging them in as zeros makes infrastructure failure look like a weak model.
  • reward_std sits next to reward. A step whose generations all score the same has zero advantage and teaches nothing, so the viewer reports steps with a gradient — a reward curve alone cannot distinguish learning from a flat run.