schema: rename model -> submission_name; seed with 3 baseline rows
Michael Rabinovich+1: Cursor
952dbca
Mirrors the cadgenbench-submissions schema change. LEADERBOARD_COLS
exposes submission_name as the lead identity column instead of model;
the handle_submit signature, the Submit-tab form Textbox, and the
meta.json example in the markdown all match.
Submit-tab markdown gets a short "Submission name" explainer making
explicit that the benchmark is system-agnostic - a submission may use
no LLM, one, or several. If the submitter wants to disclose their
stack they put it in submission_name or notes.
Local results.jsonl mirrors the 3 baseline seed rows from
cadgenbench-submissions so the offline fallback shows real data, not
the old fake demos.
Co-authored-by: Cursor <cursoragent@cursor.com>
step 5: leaderboard reads from submissions dataset on hub
Michael Rabinovich+1: Cursor
afed315
Replace the local-only JSONL read with hf_hub_download(force_download=True)
against michaelr27/cadgenbench-submissions. force_download avoids the
stale-snapshot trap that datasets.load_dataset would create per process.
Falls back to AI4Engineering/results.jsonl on any Hub error (broad except)
so the table stays up during a transient outage or a mid-migration rename.
HF_TOKEN secret already set on the Space via HfApi.add_space_secret to
read the private dataset. Secret becomes write-only at launch when the
dataset goes public; rotation TODO documented in space-setup/migration.md.
Co-authored-by: Cursor <cursoragent@cursor.com>
step 3: gradio leaderboard scaffold
Michael Rabinovich+1: Cursor
c040324
Replace the static placeholder index with a Gradio app: Leaderboard tab
backed by a hand-crafted results.jsonl, Submit tab (UI-only stub, real
eval wiring lands in step 6), About tab. Org slugs parameterised via
HF_ORG / HF_SUBMISSIONS_REPO / HF_DATA_REPO env vars so the future move
to the science org is a secret change, not a code change.
Files:
- app.py: gradio 6.14 app, three tabs
- requirements.txt: gradio, pandas, huggingface_hub, datasets
- results.jsonl: two dev seed rows
- README.md: sdk: static -> sdk: gradio
- legacy/: archive of the prototype nist comparison HTMLs
- .gitignore: __pycache__, .venv, .gradio
Schema: validity_rate (float [0,1]) instead of valid (bool); notes kept
in the row but hidden from the visible table. All copy uses "AI-driven"
rather than "LLM-driven".
Co-authored-by: Cursor <cursoragent@cursor.com>
