Spaces:
Running
Running
Ship redesigned PRIMO Space (rail nav, primo.css) reading PRIMOmics data
Browse files- README.md +10 -9
- app.py +131 -255
- assets/hugging-face.svg +9 -0
- assets/primo-mark.webp +0 -0
- assets/scienta-lab.svg +21 -0
- evaluator.py +1 -1
- fonts/dm-mono-regular.woff2 +0 -0
- fonts/funnel-display.woff2 +0 -0
- fonts/funnel-sans.woff2 +0 -0
- home.py +0 -213
- pages/about.md +3 -2
- pages/contribute.md +6 -6
- pages/submit.md +1 -1
- primo.css +216 -0
- quickstart.py +1 -1
- render.py +345 -0
- style.css +0 -444
README.md
CHANGED
|
@@ -22,12 +22,13 @@ per-category leaderboard. The datasets are opaque (`d001`, `d002`…) and you ne
|
|
| 22 |
see the disease, tissue, or target, which leaves you grading the *embedding*
|
| 23 |
itself with no room for per-task tuning.
|
| 24 |
|
| 25 |
-
|
| 26 |
-
at a time), **Tasks**, a **Submit** form (sign in with
|
| 27 |
-
**Contribute**, and **
|
| 28 |
-
`?board=rheumatology-bulk-rna`, and every
|
|
|
|
| 29 |
|
| 30 |
-
|
| 31 |
therapeutic areas PRIMO does not cover yet, each linking to Contribute. They are
|
| 32 |
declared in `boards.py` (`OPEN_BOARDS`) and drop out on their own once the
|
| 33 |
registry covers that slice. Task categories deliberately get none, because a
|
|
@@ -36,7 +37,7 @@ that does not exist.
|
|
| 36 |
|
| 37 |
🌐 Website: http://primomics.org/ ·
|
| 38 |
📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
|
| 39 |
-
📦 Data: https://huggingface.co/datasets/
|
| 40 |
|
| 41 |
## What's in the data
|
| 42 |
|
|
@@ -131,9 +132,9 @@ pushed by `benchmark/public_benchmark/baselines.py --score --publish`.
|
|
| 131 |
- **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
|
| 132 |
Hugging Face* button; submitting requires a logged-in HF account.
|
| 133 |
- Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
|
| 134 |
-
`
|
| 135 |
-
`
|
| 136 |
-
`<task_id>/labels.csv`), and **write** on `
|
| 137 |
persisted leaderboard).
|
| 138 |
- Results persist as one normalized `task_results.csv` (`model_name, task_id,
|
| 139 |
score, submitted_at, is_baseline, hf_username`) in the results dataset; the
|
|
|
|
| 22 |
see the disease, tissue, or target, which leaves you grading the *embedding*
|
| 23 |
itself with no room for per-task tuning.
|
| 24 |
|
| 25 |
+
A left **rail** navigates six pages: **Boards** (a grid of every board),
|
| 26 |
+
**Board** (one leaderboard at a time), **Tasks**, a **Submit** form (sign in with
|
| 27 |
+
Hugging Face), **Contribute**, and **Method**. Every board has its own URL,
|
| 28 |
+
`?board=rheumatology-bulk-rna`, and every page too, as `?tab=contribute`, so the
|
| 29 |
+
rail links and the open cards are shareable deep links.
|
| 30 |
|
| 31 |
+
Boards also shows **open boards**: greyed-out cards for the omics layers and the
|
| 32 |
therapeutic areas PRIMO does not cover yet, each linking to Contribute. They are
|
| 33 |
declared in `boards.py` (`OPEN_BOARDS`) and drop out on their own once the
|
| 34 |
registry covers that slice. Task categories deliberately get none, because a
|
|
|
|
| 37 |
|
| 38 |
🌐 Website: http://primomics.org/ ·
|
| 39 |
📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
|
| 40 |
+
📦 Data: https://huggingface.co/datasets/PRIMOmics/primo
|
| 41 |
|
| 42 |
## What's in the data
|
| 43 |
|
|
|
|
| 132 |
- **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
|
| 133 |
Hugging Face* button; submitting requires a logged-in HF account.
|
| 134 |
- Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
|
| 135 |
+
`PRIMOmics/primo` (the public `datasets.yaml` manifest) and
|
| 136 |
+
`PRIMOmics/primo-labels` (the private `tasks.yaml` registry +
|
| 137 |
+
`<task_id>/labels.csv`), and **write** on `PRIMOmics/primo-results` (the
|
| 138 |
persisted leaderboard).
|
| 139 |
- Results persist as one normalized `task_results.csv` (`model_name, task_id,
|
| 140 |
score, submitted_at, is_baseline, hf_username`) in the results dataset; the
|
app.py
CHANGED
|
@@ -1,13 +1,15 @@
|
|
| 1 |
"""Gradio front-end for the PRIMO public benchmark.
|
| 2 |
|
| 3 |
-
|
| 4 |
-
("what is actually being tested?"), Submit ("how
|
| 5 |
-
("what is missing, and how do I add it?"),
|
| 6 |
-
addressable
|
|
|
|
|
|
|
| 7 |
|
| 8 |
Upload one embedding file spanning every dataset (rows keyed by ``dataset_id``
|
| 9 |
+ ``sample_id``); a fixed linear probe scores each task (a dataset may carry
|
| 10 |
-
several hidden targets). Results roll up into
|
| 11 |
therapeutic area, one task family -- and each board ranks the models that
|
| 12 |
covered all of its tasks, one column per category in its native metric (AUROC or
|
| 13 |
Pearson), plus a ``Mean`` of those columns that orders the rows and is labelled
|
|
@@ -19,27 +21,21 @@ Disclosure policy -- what the public pages may show:
|
|
| 19 |
never study accessions, dataset_id -> cohort, hub keys, per-sample labels
|
| 20 |
Naming the accession behind ``d002`` would put every label one GEO download
|
| 21 |
away, so ``sources`` is collapsed to its archive (NCBI GEO / EMBL-EBI
|
| 22 |
-
ArrayExpress) and ``citation`` is never rendered at all.
|
|
|
|
| 23 |
|
| 24 |
Failures are surfaced by who owns them: a bad file tells the submitter exactly
|
| 25 |
what to fix; an evaluator-side failure says "our side, please retry" and logs
|
| 26 |
the traceback rather than blaming the submission.
|
| 27 |
|
| 28 |
A page load is ONE fetch: ``_init`` pulls the registry, the boards and the
|
| 29 |
-
persisted results once and threads that state into every
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
The board dropdown takes ``allow_custom_value``: its choices are only filled once
|
| 33 |
-
the registry loads, so a browser holding a value from an earlier version of the
|
| 34 |
-
page -- or any load where the registry is briefly unreachable -- would otherwise
|
| 35 |
-
be refused by Gradio's own preprocessing, before ``by_slug`` gets the chance to
|
| 36 |
-
fall back to the hero board.
|
| 37 |
"""
|
| 38 |
|
| 39 |
import os
|
| 40 |
import traceback
|
| 41 |
from datetime import datetime, timezone
|
| 42 |
-
from html import escape
|
| 43 |
from pathlib import Path
|
| 44 |
|
| 45 |
import gradio as gr
|
|
@@ -55,14 +51,8 @@ from evaluator import (
|
|
| 55 |
score_all,
|
| 56 |
scoreable_tasks,
|
| 57 |
)
|
| 58 |
-
from
|
| 59 |
-
from
|
| 60 |
-
RESERVED_COLUMNS,
|
| 61 |
-
per_task_table,
|
| 62 |
-
ranked_table,
|
| 63 |
-
source_repositories,
|
| 64 |
-
tasks_table,
|
| 65 |
-
)
|
| 66 |
from results import (
|
| 67 |
BASELINE_TAG,
|
| 68 |
IS_BASELINE,
|
|
@@ -74,74 +64,45 @@ from results import (
|
|
| 74 |
read_results,
|
| 75 |
)
|
| 76 |
|
| 77 |
-
|
| 78 |
-
|
|
|
|
|
|
|
| 79 |
|
| 80 |
TOKEN = os.environ.get("HF_TOKEN")
|
| 81 |
-
TAB_IDS = ("
|
| 82 |
-
|
| 83 |
-
DEEP_NAVY = "#050A3C"
|
| 84 |
-
PANEL = "#0A1049"
|
| 85 |
-
LINE = "#1E2775"
|
| 86 |
-
CYAN = "#16B3C0"
|
| 87 |
-
PAPER = "#F3F8F8"
|
| 88 |
-
MUTED = "#9FB3B7"
|
| 89 |
|
| 90 |
THEME = gr.themes.Base(
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
"sans-serif",
|
| 99 |
-
),
|
| 100 |
-
).set(
|
| 101 |
-
color_accent=CYAN,
|
| 102 |
-
border_color_accent=CYAN,
|
| 103 |
-
body_background_fill=DEEP_NAVY,
|
| 104 |
-
body_background_fill_dark=DEEP_NAVY,
|
| 105 |
-
background_fill_primary=DEEP_NAVY,
|
| 106 |
-
background_fill_primary_dark=DEEP_NAVY,
|
| 107 |
-
background_fill_secondary=PANEL,
|
| 108 |
-
background_fill_secondary_dark=PANEL,
|
| 109 |
-
block_background_fill=DEEP_NAVY,
|
| 110 |
-
block_background_fill_dark=DEEP_NAVY,
|
| 111 |
-
panel_background_fill=PANEL,
|
| 112 |
-
block_label_background_fill=PANEL,
|
| 113 |
-
block_label_text_color=MUTED,
|
| 114 |
-
body_text_color=PAPER,
|
| 115 |
-
body_text_color_dark=PAPER,
|
| 116 |
-
body_text_color_subdued=MUTED,
|
| 117 |
-
body_text_color_subdued_dark=MUTED,
|
| 118 |
-
border_color_primary=LINE,
|
| 119 |
-
border_color_primary_dark=LINE,
|
| 120 |
-
block_border_color=LINE,
|
| 121 |
-
input_background_fill=PANEL,
|
| 122 |
-
input_background_fill_dark=PANEL,
|
| 123 |
-
button_primary_background_fill=CYAN,
|
| 124 |
-
button_primary_text_color=DEEP_NAVY,
|
| 125 |
-
button_secondary_background_fill=PANEL,
|
| 126 |
-
table_border_color=LINE,
|
| 127 |
-
table_text_color=PAPER,
|
| 128 |
-
table_even_background_fill=DEEP_NAVY,
|
| 129 |
-
table_odd_background_fill=PANEL,
|
| 130 |
-
link_text_color=CYAN,
|
| 131 |
)
|
| 132 |
|
| 133 |
|
| 134 |
def _page_text(name: str) -> str:
|
| 135 |
"""One ``pages/<tab>.md`` per prose tab, named after the tab it fills.
|
| 136 |
|
| 137 |
-
Editing the site's words never means touching Python.
|
| 138 |
-
|
| 139 |
"""
|
| 140 |
return (PAGES_DIR / f"{name}.md").read_text()
|
| 141 |
|
| 142 |
|
| 143 |
def _registry_by_id() -> dict[str, dict]:
|
| 144 |
-
"""Scoreable tasks keyed by task_id (dataset present in the public manifest).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 145 |
datasets = manifest_ids(fetch_manifest(TOKEN))
|
| 146 |
registry = scoreable_tasks(fetch_tasks_registry(TOKEN), datasets)
|
| 147 |
return {_norm_id(task["task_id"]): task for task in registry}
|
|
@@ -154,7 +115,7 @@ def _page_state() -> PageState:
|
|
| 154 |
"""Registry, boards and persisted results -- one fetch per render.
|
| 155 |
|
| 156 |
A failed fetch yields empty structures so the page renders a "come back
|
| 157 |
-
later" state instead of a stack trace. Every
|
| 158 |
threaded through rather than refetched, so a page load is one round trip.
|
| 159 |
"""
|
| 160 |
try:
|
|
@@ -165,99 +126,6 @@ def _page_state() -> PageState:
|
|
| 165 |
return {}, [], pd.DataFrame(columns=RESULT_COLUMNS)
|
| 166 |
|
| 167 |
|
| 168 |
-
COLUMN_WIDTHS = {
|
| 169 |
-
"Rank": "70px",
|
| 170 |
-
"Model": "230px",
|
| 171 |
-
"Mean": "110px",
|
| 172 |
-
"Task": "300px",
|
| 173 |
-
"Family": "150px",
|
| 174 |
-
"Area": "170px",
|
| 175 |
-
"Metric": "100px",
|
| 176 |
-
"Best": "170px",
|
| 177 |
-
}
|
| 178 |
-
DEFAULT_WIDTH = "160px"
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
def _score_columns(df: pd.DataFrame) -> list[str]:
|
| 182 |
-
return [c for c in df.select_dtypes("number").columns if c != "Rank"]
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
def _styled(df: pd.DataFrame, label: str, axis: int, pinned: int) -> gr.DataFrame:
|
| 186 |
-
"""An MTEB-style table: best value in bold, ids pinned, searchable.
|
| 187 |
-
|
| 188 |
-
``axis=0`` bolds the best model per column (the ranked table); ``axis=1``
|
| 189 |
-
bolds the best model per row (the per-task table, read across). The label
|
| 190 |
-
names the table, so the copy file needs one block per tab, not per table.
|
| 191 |
-
"""
|
| 192 |
-
if df.empty:
|
| 193 |
-
return gr.DataFrame(df, label=label, interactive=False)
|
| 194 |
-
scores = _score_columns(df)
|
| 195 |
-
labels = [c for c in df.columns if c not in scores]
|
| 196 |
-
styler = df.style.format("{:.3f}", subset=scores, na_rep="n/a").highlight_max(
|
| 197 |
-
subset=scores, axis=axis, props="font-weight: 700"
|
| 198 |
-
)
|
| 199 |
-
if labels:
|
| 200 |
-
styler = styler.format(na_rep="n/a", subset=labels)
|
| 201 |
-
return gr.DataFrame(
|
| 202 |
-
styler,
|
| 203 |
-
label=label,
|
| 204 |
-
interactive=False,
|
| 205 |
-
wrap=True,
|
| 206 |
-
pinned_columns=pinned,
|
| 207 |
-
column_widths=[COLUMN_WIDTHS.get(c, DEFAULT_WIDTH) for c in df.columns],
|
| 208 |
-
show_search="filter",
|
| 209 |
-
show_copy_button=True,
|
| 210 |
-
show_fullscreen_button=True,
|
| 211 |
-
)
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
def _board_header(board: Board | None) -> str:
|
| 215 |
-
"""The board title strip. Registry strings are escaped: they are data, not code."""
|
| 216 |
-
if board is None:
|
| 217 |
-
return (
|
| 218 |
-
'<div class="primo-board-head"><h2>No board available</h2>'
|
| 219 |
-
"<p>The task registry could not be loaded. Please retry shortly.</p></div>"
|
| 220 |
-
)
|
| 221 |
-
return (
|
| 222 |
-
'<div class="primo-board-head primo-accent"><h2>'
|
| 223 |
-
f'<span class="primo-code primo-code-modality">{escape(board.code)}</span>'
|
| 224 |
-
f"{escape(board.name)}</h2>"
|
| 225 |
-
f"<p>{escape(board.blurb)} {board.n_tasks} tasks · "
|
| 226 |
-
f"{board.n_cohorts} cohorts · {board.n_patients:,} patients · "
|
| 227 |
-
f"{escape(board.modality)}</p></div>"
|
| 228 |
-
)
|
| 229 |
-
|
| 230 |
-
|
| 231 |
-
RANKED_LABEL = "🏆 Ranked"
|
| 232 |
-
PER_TASK_LABEL = "🔍 Per task"
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
def _board_page(slug: str | None, state: PageState | None = None):
|
| 236 |
-
"""Header + both tables for one board.
|
| 237 |
-
|
| 238 |
-
``state`` lets a caller that has already fetched pass it in; the first render
|
| 239 |
-
builds every tab from one fetch that way.
|
| 240 |
-
"""
|
| 241 |
-
by_id, boards, df = _page_state() if state is None else state
|
| 242 |
-
board = by_slug(boards, slug)
|
| 243 |
-
if board is None:
|
| 244 |
-
empty = pd.DataFrame()
|
| 245 |
-
return (
|
| 246 |
-
_board_header(None),
|
| 247 |
-
_styled(empty, RANKED_LABEL, 0, 0),
|
| 248 |
-
_styled(empty, PER_TASK_LABEL, 1, 0),
|
| 249 |
-
)
|
| 250 |
-
return (
|
| 251 |
-
_board_header(board),
|
| 252 |
-
_styled(ranked_table(df, by_id, board), RANKED_LABEL, 0, 2),
|
| 253 |
-
_styled(per_task_table(df, by_id, board), PER_TASK_LABEL, 1, 1),
|
| 254 |
-
)
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
def _board_choices(boards: list[Board]) -> list[tuple[str, str]]:
|
| 258 |
-
return [(f"{b.code} · {b.name} · {b.group}", b.slug) for b in boards]
|
| 259 |
-
|
| 260 |
-
|
| 261 |
def _about_text(by_id: dict[str, dict]) -> str:
|
| 262 |
"""Methodology + the archives the cohorts live in, never their accessions.
|
| 263 |
|
|
@@ -307,6 +175,12 @@ def _claimed_by(model: str) -> str:
|
|
| 307 |
return ""
|
| 308 |
|
| 309 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 310 |
def evaluate(
|
| 311 |
submission_path: str,
|
| 312 |
model_name: str,
|
|
@@ -318,7 +192,7 @@ def evaluate(
|
|
| 318 |
profile: gr.OAuthProfile | None,
|
| 319 |
):
|
| 320 |
def _refuse(message: str):
|
| 321 |
-
return
|
| 322 |
|
| 323 |
if profile is None:
|
| 324 |
return _refuse("Please sign in with Hugging Face to submit.")
|
|
@@ -387,122 +261,124 @@ def evaluate(
|
|
| 387 |
except Exception as error: # noqa: BLE001
|
| 388 |
traceback.print_exc()
|
| 389 |
summary += f"\n\n⚠️ scored, but the leaderboard was not saved: {error}"
|
| 390 |
-
return
|
| 391 |
|
| 392 |
|
| 393 |
def _landing_tab(params: dict, has_board: bool) -> str:
|
| 394 |
-
"""Which
|
| 395 |
|
| 396 |
-
An unknown ``?tab=`` falls through to
|
| 397 |
which would render the Space with every panel collapsed.
|
| 398 |
"""
|
| 399 |
tab = params.get("tab")
|
| 400 |
if tab in TAB_IDS:
|
| 401 |
return tab
|
| 402 |
-
return "
|
| 403 |
|
| 404 |
|
| 405 |
def _init(request: gr.Request):
|
| 406 |
-
"""Render every
|
| 407 |
-
|
| 408 |
-
by_id, boards, df = state
|
| 409 |
params = dict(request.query_params) if request else {}
|
| 410 |
board = by_slug(boards, params.get("board"))
|
| 411 |
selected = _landing_tab(params, bool(params.get("board") and board))
|
|
|
|
|
|
|
| 412 |
return (
|
| 413 |
gr.Tabs(selected=selected),
|
| 414 |
-
|
| 415 |
-
|
| 416 |
-
),
|
| 417 |
-
|
| 418 |
-
home_html(boards, df, by_id),
|
| 419 |
-
tasks_table(list(by_id.values())),
|
| 420 |
_about_text(by_id),
|
| 421 |
-
|
| 422 |
)
|
| 423 |
|
| 424 |
|
| 425 |
def build_demo() -> gr.Blocks:
|
| 426 |
with gr.Blocks(
|
| 427 |
-
title="PRIMO Benchmark", theme=THEME,
|
| 428 |
) as demo:
|
| 429 |
-
gr.
|
| 430 |
-
|
| 431 |
-
|
| 432 |
-
|
| 433 |
-
|
| 434 |
-
|
| 435 |
-
|
| 436 |
-
|
| 437 |
-
|
| 438 |
-
|
| 439 |
-
|
| 440 |
-
|
| 441 |
-
|
| 442 |
-
|
| 443 |
-
|
| 444 |
-
|
| 445 |
-
|
| 446 |
-
|
| 447 |
-
|
| 448 |
-
|
| 449 |
-
|
| 450 |
-
|
| 451 |
-
|
| 452 |
-
|
| 453 |
-
|
| 454 |
-
|
| 455 |
-
|
| 456 |
-
|
| 457 |
-
|
| 458 |
-
|
| 459 |
-
|
| 460 |
-
|
| 461 |
-
|
| 462 |
-
|
| 463 |
-
|
| 464 |
-
|
| 465 |
-
|
| 466 |
-
|
| 467 |
-
|
| 468 |
-
|
| 469 |
-
|
| 470 |
-
|
| 471 |
-
|
| 472 |
-
|
| 473 |
-
|
| 474 |
-
|
| 475 |
-
|
| 476 |
-
|
| 477 |
-
|
| 478 |
-
|
| 479 |
-
|
| 480 |
-
|
| 481 |
-
|
| 482 |
-
|
| 483 |
-
|
| 484 |
-
|
| 485 |
-
|
| 486 |
-
|
| 487 |
-
|
| 488 |
-
|
| 489 |
-
|
| 490 |
-
|
| 491 |
-
|
| 492 |
-
board_view = [board_head, ranked, per_task]
|
| 493 |
run_btn.click(
|
| 494 |
evaluate,
|
| 495 |
-
[file_in, model_tb, email_tb, paper_tb, hf_tb, notes_tb,
|
| 496 |
-
[result_md,
|
| 497 |
)
|
| 498 |
-
board_sel.change(_board_page, board_sel, board_view)
|
| 499 |
demo.load(
|
| 500 |
_init,
|
| 501 |
None,
|
| 502 |
-
[
|
| 503 |
)
|
| 504 |
return demo
|
| 505 |
|
| 506 |
|
| 507 |
if __name__ == "__main__":
|
| 508 |
-
build_demo().launch(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
"""Gradio front-end for the PRIMO public benchmark.
|
| 2 |
|
| 3 |
+
A left rail navigates six pages: Boards (a grid of every board), Board (one
|
| 4 |
+
leaderboard at a time), Tasks ("what is actually being tested?"), Submit ("how
|
| 5 |
+
do I enter?"), Contribute ("what is missing, and how do I add it?"), Method
|
| 6 |
+
("can I trust this?"). Every page is addressable -- ``?board=<slug>`` opens a
|
| 7 |
+
board and ``?tab=contribute`` opens a tab -- which is what the rail links and the
|
| 8 |
+
open cards use. The tab strip is hidden in CSS; the rail is the navigation.
|
| 9 |
|
| 10 |
Upload one embedding file spanning every dataset (rows keyed by ``dataset_id``
|
| 11 |
+ ``sample_id``); a fixed linear probe scores each task (a dataset may carry
|
| 12 |
+
several hidden targets). Results roll up into boards -- the whole modality, one
|
| 13 |
therapeutic area, one task family -- and each board ranks the models that
|
| 14 |
covered all of its tasks, one column per category in its native metric (AUROC or
|
| 15 |
Pearson), plus a ``Mean`` of those columns that orders the rows and is labelled
|
|
|
|
| 21 |
never study accessions, dataset_id -> cohort, hub keys, per-sample labels
|
| 22 |
Naming the accession behind ``d002`` would put every label one GEO download
|
| 23 |
away, so ``sources`` is collapsed to its archive (NCBI GEO / EMBL-EBI
|
| 24 |
+
ArrayExpress) and ``citation`` is never rendered at all. The submitter's Hugging
|
| 25 |
+
Face username is a name-ownership lock, kept private and never rendered.
|
| 26 |
|
| 27 |
Failures are surfaced by who owns them: a bad file tells the submitter exactly
|
| 28 |
what to fix; an evaluator-side failure says "our side, please retry" and logs
|
| 29 |
the traceback rather than blaming the submission.
|
| 30 |
|
| 31 |
A page load is ONE fetch: ``_init`` pulls the registry, the boards and the
|
| 32 |
+
persisted results once and threads that state into every page, rather than each
|
| 33 |
+
page fetching for itself.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
"""
|
| 35 |
|
| 36 |
import os
|
| 37 |
import traceback
|
| 38 |
from datetime import datetime, timezone
|
|
|
|
| 39 |
from pathlib import Path
|
| 40 |
|
| 41 |
import gradio as gr
|
|
|
|
| 51 |
score_all,
|
| 52 |
scoreable_tasks,
|
| 53 |
)
|
| 54 |
+
from leaderboard import RESERVED_COLUMNS, source_repositories
|
| 55 |
+
from render import rail_html, render_board, render_boards, render_tasks
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
from results import (
|
| 57 |
BASELINE_TAG,
|
| 58 |
IS_BASELINE,
|
|
|
|
| 64 |
read_results,
|
| 65 |
)
|
| 66 |
|
| 67 |
+
HERE = Path(__file__).parent
|
| 68 |
+
PAGES_DIR = HERE / "pages"
|
| 69 |
+
CSS_PATH = str(HERE / "primo.css")
|
| 70 |
+
ALLOWED_PATHS = [str(HERE / "assets"), str(HERE / "fonts")]
|
| 71 |
|
| 72 |
TOKEN = os.environ.get("HF_TOKEN")
|
| 73 |
+
TAB_IDS = ("boards", "board", "tasks", "submit", "contribute", "method")
|
| 74 |
+
FOOT_TABS = frozenset({"tasks", "submit", "contribute", "method"})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
THEME = gr.themes.Base(
|
| 77 |
+
font=["Funnel Sans", "sans-serif"], font_mono=["DM Mono", "monospace"]
|
| 78 |
+
)
|
| 79 |
+
|
| 80 |
+
SUBMIT_HEAD = (
|
| 81 |
+
'<div class="pm-head"><div><h1>Submit a model</h1>'
|
| 82 |
+
"<p>One embedding file. Partial coverage is fine — you are ranked on every "
|
| 83 |
+
"board you cover in full.</p></div></div>"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
)
|
| 85 |
|
| 86 |
|
| 87 |
def _page_text(name: str) -> str:
|
| 88 |
"""One ``pages/<tab>.md`` per prose tab, named after the tab it fills.
|
| 89 |
|
| 90 |
+
Editing the site's words never means touching Python. The data pages (Boards,
|
| 91 |
+
Board, Tasks) are generated markup instead.
|
| 92 |
"""
|
| 93 |
return (PAGES_DIR / f"{name}.md").read_text()
|
| 94 |
|
| 95 |
|
| 96 |
def _registry_by_id() -> dict[str, dict]:
|
| 97 |
+
"""Scoreable tasks keyed by task_id (dataset present in the public manifest).
|
| 98 |
+
|
| 99 |
+
``PRIMO_FIXTURE=1`` swaps the token-gated fetch for the test registry, so the
|
| 100 |
+
app runs offline for local smoke-testing.
|
| 101 |
+
"""
|
| 102 |
+
if os.environ.get("PRIMO_FIXTURE"):
|
| 103 |
+
from conftest import REGISTRY
|
| 104 |
+
|
| 105 |
+
return {task_id: dict(task) for task_id, task in REGISTRY.items()}
|
| 106 |
datasets = manifest_ids(fetch_manifest(TOKEN))
|
| 107 |
registry = scoreable_tasks(fetch_tasks_registry(TOKEN), datasets)
|
| 108 |
return {_norm_id(task["task_id"]): task for task in registry}
|
|
|
|
| 115 |
"""Registry, boards and persisted results -- one fetch per render.
|
| 116 |
|
| 117 |
A failed fetch yields empty structures so the page renders a "come back
|
| 118 |
+
later" state instead of a stack trace. Every page is built from one of these,
|
| 119 |
threaded through rather than refetched, so a page load is one round trip.
|
| 120 |
"""
|
| 121 |
try:
|
|
|
|
| 126 |
return {}, [], pd.DataFrame(columns=RESULT_COLUMNS)
|
| 127 |
|
| 128 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
def _about_text(by_id: dict[str, dict]) -> str:
|
| 130 |
"""Methodology + the archives the cohorts live in, never their accessions.
|
| 131 |
|
|
|
|
| 175 |
return ""
|
| 176 |
|
| 177 |
|
| 178 |
+
def _rendered_board(slug: str | None) -> str:
|
| 179 |
+
"""The Board page for ``slug`` -- re-rendered after a submission saves."""
|
| 180 |
+
by_id, boards, df = _page_state()
|
| 181 |
+
return render_board(by_slug(boards, slug), df, by_id)
|
| 182 |
+
|
| 183 |
+
|
| 184 |
def evaluate(
|
| 185 |
submission_path: str,
|
| 186 |
model_name: str,
|
|
|
|
| 192 |
profile: gr.OAuthProfile | None,
|
| 193 |
):
|
| 194 |
def _refuse(message: str):
|
| 195 |
+
return message, _rendered_board(slug)
|
| 196 |
|
| 197 |
if profile is None:
|
| 198 |
return _refuse("Please sign in with Hugging Face to submit.")
|
|
|
|
| 261 |
except Exception as error: # noqa: BLE001
|
| 262 |
traceback.print_exc()
|
| 263 |
summary += f"\n\n⚠️ scored, but the leaderboard was not saved: {error}"
|
| 264 |
+
return summary, _rendered_board(slug)
|
| 265 |
|
| 266 |
|
| 267 |
def _landing_tab(params: dict, has_board: bool) -> str:
|
| 268 |
+
"""Which page a visitor lands on: ``?tab=`` wins, then ``?board=``, else Boards.
|
| 269 |
|
| 270 |
+
An unknown ``?tab=`` falls through to Boards rather than selecting nothing,
|
| 271 |
which would render the Space with every panel collapsed.
|
| 272 |
"""
|
| 273 |
tab = params.get("tab")
|
| 274 |
if tab in TAB_IDS:
|
| 275 |
return tab
|
| 276 |
+
return "board" if has_board else "boards"
|
| 277 |
|
| 278 |
|
| 279 |
def _init(request: gr.Request):
|
| 280 |
+
"""Render every page from one fetch, landing where the query params ask."""
|
| 281 |
+
by_id, boards, df = _page_state()
|
|
|
|
| 282 |
params = dict(request.query_params) if request else {}
|
| 283 |
board = by_slug(boards, params.get("board"))
|
| 284 |
selected = _landing_tab(params, bool(params.get("board") and board))
|
| 285 |
+
active_slug = board.slug if selected == "board" and board else None
|
| 286 |
+
active_tab = selected if selected in FOOT_TABS else None
|
| 287 |
return (
|
| 288 |
gr.Tabs(selected=selected),
|
| 289 |
+
rail_html(boards, active_slug, active_tab),
|
| 290 |
+
board.slug if board else None,
|
| 291 |
+
render_boards(boards, df, by_id),
|
| 292 |
+
render_tasks(by_id),
|
|
|
|
|
|
|
| 293 |
_about_text(by_id),
|
| 294 |
+
render_board(board, df, by_id),
|
| 295 |
)
|
| 296 |
|
| 297 |
|
| 298 |
def build_demo() -> gr.Blocks:
|
| 299 |
with gr.Blocks(
|
| 300 |
+
title="PRIMO Benchmark", theme=THEME, css_paths=[CSS_PATH], fill_width=True
|
| 301 |
) as demo:
|
| 302 |
+
active_board = gr.State(None)
|
| 303 |
+
with gr.Row(elem_id="pm-shell"):
|
| 304 |
+
with gr.Column(elem_id="pm-rail"):
|
| 305 |
+
rail = gr.HTML()
|
| 306 |
+
with gr.Column(elem_id="pm-main"):
|
| 307 |
+
with gr.Tabs(elem_id="pm-pages") as pages:
|
| 308 |
+
with gr.Tab("Boards", id="boards"):
|
| 309 |
+
boards_html = gr.HTML()
|
| 310 |
+
with gr.Tab("Board", id="board"):
|
| 311 |
+
board_html = gr.HTML()
|
| 312 |
+
with gr.Tab("Tasks", id="tasks"):
|
| 313 |
+
tasks_html = gr.HTML()
|
| 314 |
+
with gr.Tab("Submit", id="submit"):
|
| 315 |
+
gr.HTML(SUBMIT_HEAD)
|
| 316 |
+
with gr.Column(elem_classes=["pm-body"]):
|
| 317 |
+
with gr.Row():
|
| 318 |
+
with gr.Column(scale=3):
|
| 319 |
+
gr.Markdown(_page_text("submit"))
|
| 320 |
+
with gr.Column(scale=2, elem_id="pm-form"):
|
| 321 |
+
gr.LoginButton()
|
| 322 |
+
model_tb = gr.Textbox(
|
| 323 |
+
label="Model name",
|
| 324 |
+
placeholder="e.g. eva-rna-v1",
|
| 325 |
+
info="Shown on the leaderboard.",
|
| 326 |
+
)
|
| 327 |
+
email_tb = gr.Textbox(
|
| 328 |
+
label="Email address",
|
| 329 |
+
placeholder="you@lab.org",
|
| 330 |
+
info="Contact for this submission, kept private.",
|
| 331 |
+
)
|
| 332 |
+
notes_tb = gr.Textbox(
|
| 333 |
+
label="Training data / notes (optional)",
|
| 334 |
+
placeholder="e.g. pretrained on atlas X",
|
| 335 |
+
info="About the model or its training data.",
|
| 336 |
+
)
|
| 337 |
+
paper_tb = gr.Textbox(
|
| 338 |
+
label="Paper link (optional)",
|
| 339 |
+
placeholder="https://arxiv.org/abs/...",
|
| 340 |
+
)
|
| 341 |
+
hf_tb = gr.Textbox(
|
| 342 |
+
label="Hugging Face model link (optional)",
|
| 343 |
+
placeholder="https://huggingface.co/...",
|
| 344 |
+
)
|
| 345 |
+
file_in = gr.File(
|
| 346 |
+
label="Submission (.csv / .tsv / .parquet / .npz)",
|
| 347 |
+
type="filepath",
|
| 348 |
+
)
|
| 349 |
+
run_btn = gr.Button(
|
| 350 |
+
"Evaluate",
|
| 351 |
+
elem_classes=[
|
| 352 |
+
"pm-btn",
|
| 353 |
+
"pm-btn--primary",
|
| 354 |
+
"pm-btn--block",
|
| 355 |
+
],
|
| 356 |
+
)
|
| 357 |
+
result_md = gr.Markdown()
|
| 358 |
+
with gr.Tab("Contribute", id="contribute"):
|
| 359 |
+
with gr.Column(elem_classes=["pm-body", "pm-prose"]):
|
| 360 |
+
gr.Markdown(_page_text("contribute"))
|
| 361 |
+
with gr.Tab("Method", id="method"):
|
| 362 |
+
with gr.Column(elem_classes=["pm-body", "pm-prose"]):
|
| 363 |
+
about_md = gr.Markdown()
|
| 364 |
+
|
|
|
|
| 365 |
run_btn.click(
|
| 366 |
evaluate,
|
| 367 |
+
[file_in, model_tb, email_tb, paper_tb, hf_tb, notes_tb, active_board],
|
| 368 |
+
[result_md, board_html],
|
| 369 |
)
|
|
|
|
| 370 |
demo.load(
|
| 371 |
_init,
|
| 372 |
None,
|
| 373 |
+
[pages, rail, active_board, boards_html, tasks_html, about_md, board_html],
|
| 374 |
)
|
| 375 |
return demo
|
| 376 |
|
| 377 |
|
| 378 |
if __name__ == "__main__":
|
| 379 |
+
build_demo().launch(
|
| 380 |
+
server_name="0.0.0.0",
|
| 381 |
+
server_port=7860,
|
| 382 |
+
ssr_mode=False,
|
| 383 |
+
allowed_paths=ALLOWED_PATHS,
|
| 384 |
+
)
|
assets/hugging-face.svg
ADDED
|
|
assets/primo-mark.webp
ADDED
|
assets/scienta-lab.svg
ADDED
|
|
evaluator.py
CHANGED
|
@@ -46,7 +46,7 @@ SPLIT = "split"
|
|
| 46 |
SPLIT_TRAIN = "train"
|
| 47 |
SPLIT_TEST = "test"
|
| 48 |
|
| 49 |
-
ORG = "
|
| 50 |
PUBLIC_REPO = f"{ORG}/primo"
|
| 51 |
LABELS_REPO = f"{ORG}/primo-labels"
|
| 52 |
RESULTS_REPO = f"{ORG}/primo-results"
|
|
|
|
| 46 |
SPLIT_TRAIN = "train"
|
| 47 |
SPLIT_TEST = "test"
|
| 48 |
|
| 49 |
+
ORG = "PRIMOmics"
|
| 50 |
PUBLIC_REPO = f"{ORG}/primo"
|
| 51 |
LABELS_REPO = f"{ORG}/primo-labels"
|
| 52 |
RESULTS_REPO = f"{ORG}/primo-results"
|
fonts/dm-mono-regular.woff2
ADDED
|
Binary file (20.6 kB). View file
|
|
|
fonts/funnel-display.woff2
ADDED
|
Binary file (17.7 kB). View file
|
|
|
fonts/funnel-sans.woff2
ADDED
|
Binary file (17.4 kB). View file
|
|
|
home.py
DELETED
|
@@ -1,213 +0,0 @@
|
|
| 1 |
-
"""The home page: boards -> a grid of cards, as one block of HTML.
|
| 2 |
-
|
| 3 |
-
Gradio has no card component, so the overview is rendered as markup and handed
|
| 4 |
-
to ``gr.HTML``. Each card is a plain ``<a href="?board=...">``: clicking it
|
| 5 |
-
reloads the Space on that board, which costs a page load but buys shareable
|
| 6 |
-
per-board URLs and needs no JavaScript.
|
| 7 |
-
|
| 8 |
-
Two things here are not leaderboards. The version strip states what v0 actually
|
| 9 |
-
covers, counted off the registry so it cannot go stale. The greyed-out OPEN cards
|
| 10 |
-
state what it does not cover, and link to Contribute instead of to a board: the
|
| 11 |
-
page is supposed to show its own gaps, not imply the registry is the territory.
|
| 12 |
-
|
| 13 |
-
Every string that comes from the registry goes through ``html.escape`` -- the
|
| 14 |
-
titles, blurbs and disease names are authored data, not constants.
|
| 15 |
-
"""
|
| 16 |
-
|
| 17 |
-
from html import escape
|
| 18 |
-
|
| 19 |
-
import pandas as pd
|
| 20 |
-
from boards import (
|
| 21 |
-
AREA_GROUP,
|
| 22 |
-
CATEGORY_GROUP,
|
| 23 |
-
GROUP_NOTE,
|
| 24 |
-
MODALITY_GROUP,
|
| 25 |
-
Board,
|
| 26 |
-
OpenBoard,
|
| 27 |
-
featured,
|
| 28 |
-
in_group,
|
| 29 |
-
metric_label,
|
| 30 |
-
open_in_group,
|
| 31 |
-
)
|
| 32 |
-
from leaderboard import TopModel, top_models
|
| 33 |
-
|
| 34 |
-
SECTIONS = (MODALITY_GROUP, AREA_GROUP, CATEGORY_GROUP)
|
| 35 |
-
N_TOP_MODELS = 3
|
| 36 |
-
GROUP_ACCENT = {
|
| 37 |
-
MODALITY_GROUP: "modality",
|
| 38 |
-
AREA_GROUP: "area",
|
| 39 |
-
CATEGORY_GROUP: "category",
|
| 40 |
-
}
|
| 41 |
-
VERSION = "v0"
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
def _plural(count: int, noun: str) -> str:
|
| 45 |
-
return f"{count} {noun}" if count == 1 else f"{count} {noun}s"
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
def banner_html(boards: list[Board]) -> str:
|
| 49 |
-
"""The version strip shown above every tab: what the benchmark covers today.
|
| 50 |
-
|
| 51 |
-
Counted off the registry rather than written by hand, so it can never claim a
|
| 52 |
-
scope that has since changed. An unavailable registry yields no strip at all
|
| 53 |
-
-- a version claim nobody can check is worse than none.
|
| 54 |
-
"""
|
| 55 |
-
modalities = in_group(boards, MODALITY_GROUP)
|
| 56 |
-
if not modalities:
|
| 57 |
-
return ""
|
| 58 |
-
areas = len({board.name for board in in_group(boards, AREA_GROUP)})
|
| 59 |
-
return (
|
| 60 |
-
f'<div class="primo-banner"><span class="primo-banner-tag">{VERSION}</span>'
|
| 61 |
-
f'<span>{escape(", ".join(b.name for b in modalities))} · '
|
| 62 |
-
f'{_plural(areas, "therapeutic area")} · '
|
| 63 |
-
f'{_plural(sum(b.n_tasks for b in modalities), "task")}. '
|
| 64 |
-
"The scope is incomplete by design, and the open boards on Home show where."
|
| 65 |
-
"</span></div>"
|
| 66 |
-
)
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
def _stat(name: str, value: object) -> str:
|
| 70 |
-
return (
|
| 71 |
-
f'<div class="primo-stat"><span class="primo-stat-label">{escape(name)}</span>'
|
| 72 |
-
f'<span class="primo-stat-value">{escape(f"{value:,}" if isinstance(value, int) else str(value))}</span></div>'
|
| 73 |
-
)
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
def _pills(board: Board) -> str:
|
| 77 |
-
pills = [
|
| 78 |
-
f'<span class="primo-pill primo-pill-modality">{escape(board.modality)}</span>'
|
| 79 |
-
]
|
| 80 |
-
pills += [
|
| 81 |
-
f'<span class="primo-pill">{escape(metric_label(m))}</span>'
|
| 82 |
-
for m in board.metrics
|
| 83 |
-
]
|
| 84 |
-
return f'<div class="primo-pills">{"".join(pills)}</div>'
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
def _top_row(row: TopModel) -> str:
|
| 88 |
-
tag = '<span class="primo-tag">baseline</span>' if row.is_baseline else ""
|
| 89 |
-
css = "primo-top-row primo-top-baseline" if row.is_baseline else "primo-top-row"
|
| 90 |
-
score = "n/a" if row.score is None else f"{row.score:.3f}"
|
| 91 |
-
return (
|
| 92 |
-
f'<div class="{css}"><span>{escape(row.name)}{tag}</span>'
|
| 93 |
-
f"<span>{score}</span></div>"
|
| 94 |
-
)
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
def _top_models(rows: list[TopModel]) -> str:
|
| 98 |
-
"""The card's mini-ranking.
|
| 99 |
-
|
| 100 |
-
A board whose only entries are our own baselines says so: "be the first" is
|
| 101 |
-
misleading once a PCA already holds a score somebody has to beat.
|
| 102 |
-
"""
|
| 103 |
-
if not rows:
|
| 104 |
-
return (
|
| 105 |
-
'<div class="primo-top primo-top-empty">No ranked model yet. '
|
| 106 |
-
"Be the first.</div>"
|
| 107 |
-
)
|
| 108 |
-
foot = (
|
| 109 |
-
'<div class="primo-top-foot">No submitted model yet. Beat the baseline.</div>'
|
| 110 |
-
if all(row.is_baseline for row in rows)
|
| 111 |
-
else ""
|
| 112 |
-
)
|
| 113 |
-
return (
|
| 114 |
-
'<div class="primo-top"><div class="primo-top-head">'
|
| 115 |
-
"<span>TOP MODELS</span><span>MEAN</span></div>"
|
| 116 |
-
f'{"".join(_top_row(row) for row in rows)}{foot}</div>'
|
| 117 |
-
)
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
def _card(board: Board, top: list[TopModel] | None) -> str:
|
| 121 |
-
accent = GROUP_ACCENT.get(board.group, "modality")
|
| 122 |
-
inner = [
|
| 123 |
-
'<div class="primo-card-head">'
|
| 124 |
-
f'<span class="primo-code primo-code-{accent}">{escape(board.code)}</span>'
|
| 125 |
-
f'<span class="primo-card-title">{escape(board.name)}</span></div>'
|
| 126 |
-
]
|
| 127 |
-
if top is None:
|
| 128 |
-
inner.append(f'<p class="primo-blurb">{escape(board.blurb)}</p>')
|
| 129 |
-
inner.append(
|
| 130 |
-
'<div class="primo-stats">'
|
| 131 |
-
+ _stat("TASKS", board.n_tasks)
|
| 132 |
-
+ _stat("COHORTS", board.n_cohorts)
|
| 133 |
-
+ _stat("PATIENTS", board.n_patients)
|
| 134 |
-
+ _stat("DISEASES", board.n_diseases)
|
| 135 |
-
+ "</div>"
|
| 136 |
-
)
|
| 137 |
-
else:
|
| 138 |
-
inner.append(
|
| 139 |
-
f'<div class="primo-card-sub">{board.n_tasks} tasks · '
|
| 140 |
-
f"{board.n_cohorts} cohorts · {board.n_patients:,} patients</div>"
|
| 141 |
-
)
|
| 142 |
-
inner.append(_top_models(top))
|
| 143 |
-
inner.append(_pills(board))
|
| 144 |
-
return (
|
| 145 |
-
f'<a class="primo-card primo-accent-{accent}" target="_self" '
|
| 146 |
-
f'href="?board={escape(board.slug)}">{"".join(inner)}</a>'
|
| 147 |
-
)
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
def _open_card(board: OpenBoard) -> str:
|
| 151 |
-
"""A greyed-out card for a slice nobody covers: no stats, one call to action."""
|
| 152 |
-
return (
|
| 153 |
-
'<a class="primo-card primo-card-open" target="_self" href="?tab=contribute">'
|
| 154 |
-
f'<div class="primo-card-head"><span class="primo-card-title">'
|
| 155 |
-
f'{escape(board.name)}</span><span class="primo-badge-open">OPEN</span></div>'
|
| 156 |
-
f'<p class="primo-blurb">{escape(board.blurb)}</p>'
|
| 157 |
-
'<div class="primo-open-cta">Propose a cohort →</div></a>'
|
| 158 |
-
)
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
def _section(group: str, boards: list[Board], opens: list[OpenBoard]) -> str:
|
| 162 |
-
open_count = (
|
| 163 |
-
f'<span class="primo-count-open">+{len(opens)} open</span>' if opens else ""
|
| 164 |
-
)
|
| 165 |
-
cards = "".join(_card(b, None) for b in boards)
|
| 166 |
-
cards += "".join(_open_card(b) for b in opens)
|
| 167 |
-
return (
|
| 168 |
-
'<section class="primo-section">'
|
| 169 |
-
'<div class="primo-section-head"><h3>'
|
| 170 |
-
f'{escape(group)} <span class="primo-count">{len(boards)}</span>'
|
| 171 |
-
f"{open_count}</h3>"
|
| 172 |
-
f'<span class="primo-section-note">{escape(GROUP_NOTE.get(group, ""))}</span>'
|
| 173 |
-
f'</div><div class="primo-grid">{cards}</div></section>'
|
| 174 |
-
)
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
def _sections(boards: list[Board]) -> str:
|
| 178 |
-
"""One block per facet: what is covered, then what is openly missing.
|
| 179 |
-
|
| 180 |
-
The Modality section is shown even when a single modality exists. A lone card
|
| 181 |
-
would read as a bug; a lone card beside the omics layers nobody has brought
|
| 182 |
-
yet reads as the point of the page.
|
| 183 |
-
"""
|
| 184 |
-
out = []
|
| 185 |
-
for group in SECTIONS:
|
| 186 |
-
cards, opens = in_group(boards, group), open_in_group(boards, group)
|
| 187 |
-
if not cards and not opens:
|
| 188 |
-
continue
|
| 189 |
-
out.append(_section(group, cards, opens))
|
| 190 |
-
return "".join(out)
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
def home_html(boards: list[Board], df: pd.DataFrame, by_id: dict[str, dict]) -> str:
|
| 194 |
-
"""The whole overview page: hero row, then one section per facet."""
|
| 195 |
-
if not boards:
|
| 196 |
-
return (
|
| 197 |
-
'<div class="primo-home"><p class="primo-empty">'
|
| 198 |
-
"The task registry is unavailable right now. Please retry in a moment."
|
| 199 |
-
"</p></div>"
|
| 200 |
-
)
|
| 201 |
-
heroes = "".join(
|
| 202 |
-
_card(b, top_models(df, by_id, b, N_TOP_MODELS)) for b in featured(boards)
|
| 203 |
-
)
|
| 204 |
-
sections = _sections(boards)
|
| 205 |
-
return (
|
| 206 |
-
'<div class="primo-home">'
|
| 207 |
-
'<div class="primo-title-row"><h2>Benchmark overview</h2>'
|
| 208 |
-
'<span class="primo-hint">Pick a board. Each one is its own leaderboard'
|
| 209 |
-
"</span></div>"
|
| 210 |
-
'<div class="primo-chip">FEATURED</div>'
|
| 211 |
-
f'<div class="primo-grid primo-grid-featured">{heroes}</div>'
|
| 212 |
-
f"{sections}</div>"
|
| 213 |
-
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
pages/about.md
CHANGED
|
@@ -41,7 +41,8 @@ columns.
|
|
| 41 |
## How the results are organised
|
| 42 |
|
| 43 |
A **board** is a self-contained leaderboard over a slice of the benchmark: the
|
| 44 |
-
whole modality, one therapeutic area, or one task family. Pick one
|
|
|
|
| 45 |
|
| 46 |
A board ranks only the models that covered **all of its tasks**. Skipping
|
| 47 |
Dermatology therefore costs you nothing on the Rheumatology board.
|
|
@@ -83,6 +84,6 @@ modality or therapeutic area, or work with us on the methodology? The
|
|
| 83 |
## Links
|
| 84 |
|
| 85 |
- 🌐 **Website**: [primomics.org](http://primomics.org/)
|
| 86 |
-
- 📦 **Data**: [
|
| 87 |
- 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 88 |
- 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
|
|
|
|
| 41 |
## How the results are organised
|
| 42 |
|
| 43 |
A **board** is a self-contained leaderboard over a slice of the benchmark: the
|
| 44 |
+
whole modality, one therapeutic area, or one task family. Pick one from the rail,
|
| 45 |
+
or on the **Boards** page.
|
| 46 |
|
| 47 |
A board ranks only the models that covered **all of its tasks**. Skipping
|
| 48 |
Dermatology therefore costs you nothing on the Rheumatology board.
|
|
|
|
| 84 |
## Links
|
| 85 |
|
| 86 |
- 🌐 **Website**: [primomics.org](http://primomics.org/)
|
| 87 |
+
- 📦 **Data**: [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo)
|
| 88 |
- 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 89 |
- 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
|
pages/contribute.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
-
PRIMO is an open benchmark, and it is **deliberately incomplete**. The
|
| 2 |
-
|
| 3 |
-
|
| 4 |
|
| 5 |
PRIMO was started by **Scienta Lab**. The intent is community governance; how
|
| 6 |
that actually works is part of what the first partners will define.
|
|
@@ -33,8 +33,8 @@ much as data does.
|
|
| 33 |
|
| 34 |
## Add a modality or a therapeutic area
|
| 35 |
|
| 36 |
-
The **OPEN** cards on
|
| 37 |
-
not reach yet. Read them as gaps waiting for someone rather than as a roadmap we
|
| 38 |
are already working through. Opening one takes a single cohort with clinical
|
| 39 |
labels, and not a whole atlas.
|
| 40 |
|
|
@@ -53,4 +53,4 @@ methodology, co-signature on what we publish, and early access to the results.
|
|
| 53 |
- **Question the method.** Open a thread in the Space's **Community** tab.
|
| 54 |
- **Submit a model.** Baselines are on the leaderboard already; beat them.
|
| 55 |
- **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 56 |
-
- **Look at the data**: [
|
|
|
|
| 1 |
+
PRIMO is an open benchmark, and it is **deliberately incomplete**. The **Boards**
|
| 2 |
+
page shows exactly how far it reaches today — the live boards — and names the
|
| 3 |
+
gaps as open cards. Everything past that still needs somebody to bring it.
|
| 4 |
|
| 5 |
PRIMO was started by **Scienta Lab**. The intent is community governance; how
|
| 6 |
that actually works is part of what the first partners will define.
|
|
|
|
| 33 |
|
| 34 |
## Add a modality or a therapeutic area
|
| 35 |
|
| 36 |
+
The **OPEN** cards on the Boards page mark the omics layers and clinical areas
|
| 37 |
+
PRIMO does not reach yet. Read them as gaps waiting for someone rather than as a roadmap we
|
| 38 |
are already working through. Opening one takes a single cohort with clinical
|
| 39 |
labels, and not a whole atlas.
|
| 40 |
|
|
|
|
| 53 |
- **Question the method.** Open a thread in the Space's **Community** tab.
|
| 54 |
- **Submit a model.** Baselines are on the leaderboard already; beat them.
|
| 55 |
- **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 56 |
+
- **Look at the data**: [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo)
|
pages/submit.md
CHANGED
|
@@ -15,7 +15,7 @@ python quickstart.py --out submission.parquet
|
|
| 15 |
|
| 16 |
Or do it by hand, in **three steps**:
|
| 17 |
|
| 18 |
-
1. **Get the data** → download the datasets from [
|
| 19 |
2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
|
| 20 |
3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
|
| 21 |
|
|
|
|
| 15 |
|
| 16 |
Or do it by hand, in **three steps**:
|
| 17 |
|
| 18 |
+
1. **Get the data** → download the datasets from [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo) (start with its `datasets.yaml`).
|
| 19 |
2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
|
| 20 |
3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
|
| 21 |
|
primo.css
ADDED
|
@@ -0,0 +1,216 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/* PRIMO benchmark platform — Gradio stylesheet
|
| 2 |
+
Derived from the Scienta design system + primomics.org.
|
| 3 |
+
Load with: gr.Blocks(css_paths=["primo.css"])
|
| 4 |
+
Every value here is a design-system token; do not invent new ones. */
|
| 5 |
+
|
| 6 |
+
/* ---------- fonts ---------- */
|
| 7 |
+
@font-face { font-family: "Funnel Display"; font-weight: 500 700; font-display: swap; src: url("/gradio_api/file=fonts/funnel-display.woff2") format("woff2"); }
|
| 8 |
+
@font-face { font-family: "Funnel Sans"; font-weight: 400 700; font-display: swap; src: url("/gradio_api/file=fonts/funnel-sans.woff2") format("woff2"); }
|
| 9 |
+
@font-face { font-family: "DM Mono"; font-weight: 400; font-display: swap; src: url("/gradio_api/file=fonts/dm-mono-regular.woff2") format("woff2"); }
|
| 10 |
+
|
| 11 |
+
/* ---------- tokens ---------- */
|
| 12 |
+
:root {
|
| 13 |
+
--marine-50: #e5e9fc; --marine-100: #bcc3ea; --marine-200: #929ed8;
|
| 14 |
+
--marine-400: #3f53b5; --marine-500: #162da3; --marine-700: #0d1b62;
|
| 15 |
+
--coral-400: #fd9878; --coral-500: #fd7e56; --coral-600: #d36745;
|
| 16 |
+
--n-100: #f5f5f5; --n-200: #e5e5e5; --n-300: #d4d4d4;
|
| 17 |
+
--n-400: #a3a3a3; --n-500: #737373; --n-600: #525252; --n-900: #171717;
|
| 18 |
+
|
| 19 |
+
--border: var(--n-200);
|
| 20 |
+
--text-body: var(--n-900);
|
| 21 |
+
--text-secondary: var(--n-600);
|
| 22 |
+
--text-muted: var(--n-500);
|
| 23 |
+
--surface-selected: var(--marine-50);
|
| 24 |
+
|
| 25 |
+
--sidebar: radial-gradient(ellipse 100% 5rem at 50% 100%, rgb(253 126 86 / .35), transparent 70%),
|
| 26 |
+
linear-gradient(45deg, var(--marine-500), var(--marine-700));
|
| 27 |
+
--sidebar-fg: rgb(255 255 255 / .9);
|
| 28 |
+
--sidebar-sub-fg: rgb(255 255 255 / .6);
|
| 29 |
+
--sidebar-accent: linear-gradient(90deg, rgb(253 152 120 / .15), rgb(63 83 181 / .2));
|
| 30 |
+
--sidebar-highlight: rgb(63 83 181 / .2);
|
| 31 |
+
--sidebar-border: var(--marine-700);
|
| 32 |
+
|
| 33 |
+
--font-sans: "Funnel Sans", sans-serif;
|
| 34 |
+
--font-title: "Funnel Display", sans-serif;
|
| 35 |
+
--font-mono: "DM Mono", ui-monospace, monospace;
|
| 36 |
+
|
| 37 |
+
--radius-sm: 4px; --radius-md: 6px; --radius-lg: 8px; --radius-xl: 12px;
|
| 38 |
+
--shadow-xs: 0 1px 2px rgb(9 18 65 / .05);
|
| 39 |
+
--shadow-sm: 0 1px 3px rgb(9 18 65 / .1), 0 1px 2px -1px rgb(9 18 65 / .1);
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
/* ---------- Gradio reset: get its chrome out of the way ---------- */
|
| 43 |
+
.gradio-container { max-width: 100% !important; padding: 0 !important; font-family: var(--font-sans) !important; }
|
| 44 |
+
.gradio-container .prose :where(h1, h2, h3) { font-family: var(--font-title); }
|
| 45 |
+
footer { display: none !important; }
|
| 46 |
+
/* the rail replaces the tab bar (Gradio renamed .tab-nav -> .tab-wrapper) */
|
| 47 |
+
#pm-pages > .tab-nav, #pm-pages > .tab-wrapper { display: none !important; }
|
| 48 |
+
#pm-pages > .tabitem { border: 0 !important; padding: 0 !important; }
|
| 49 |
+
#pm-shell { flex-wrap: nowrap !important; gap: 0 !important; align-items: stretch !important; }
|
| 50 |
+
|
| 51 |
+
/* ---------- rail ---------- */
|
| 52 |
+
#pm-rail {
|
| 53 |
+
width: 272px !important; min-width: 272px !important; flex: 0 0 272px !important;
|
| 54 |
+
background: var(--sidebar); color: var(--sidebar-fg); padding: 8px; gap: 0 !important;
|
| 55 |
+
}
|
| 56 |
+
.pm-brand { display: flex; align-items: center; gap: 8px; height: 3rem; padding: 8px; border-radius: var(--radius-md); color: #fff; text-decoration: none; }
|
| 57 |
+
.pm-brand:hover { background: var(--sidebar-highlight); }
|
| 58 |
+
.pm-brand img { height: 24px; width: auto; padding: 2px; background: #fff; border-radius: var(--radius-sm); object-fit: contain; }
|
| 59 |
+
.pm-brand b { font: 700 1.125rem/1 var(--font-title); letter-spacing: -.04em; }
|
| 60 |
+
.pm-brand span { font: 500 1.25rem/0 var(--font-title); letter-spacing: -.04em; color: var(--sidebar-sub-fg); }
|
| 61 |
+
|
| 62 |
+
.pm-rail-group { padding: 16px 16px 0; }
|
| 63 |
+
.pm-rail-label { display: flex; align-items: center; height: 2.5rem; padding: 8px;
|
| 64 |
+
font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--sidebar-sub-fg); }
|
| 65 |
+
|
| 66 |
+
/* board links are Gradio buttons, restyled */
|
| 67 |
+
#pm-rail .pm-link {
|
| 68 |
+
position: relative; display: flex !important; align-items: center; justify-content: space-between; gap: 8px;
|
| 69 |
+
width: 100%; height: 2rem; min-height: 2rem; padding: 0 8px; margin: 0 0 2px;
|
| 70 |
+
border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
|
| 71 |
+
font: 400 14px/1 var(--font-sans); color: var(--sidebar-sub-fg) !important; text-align: left;
|
| 72 |
+
transition: color .15s ease, background .15s ease;
|
| 73 |
+
}
|
| 74 |
+
#pm-rail .pm-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
|
| 75 |
+
#pm-rail .pm-link::before {
|
| 76 |
+
content: ""; position: absolute; top: 50%; left: 0; height: 50%; width: 2px;
|
| 77 |
+
translate: 0 -50%; background: var(--coral-500); opacity: 0; transition: opacity .15s ease;
|
| 78 |
+
}
|
| 79 |
+
#pm-rail .pm-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; font-weight: 500; }
|
| 80 |
+
#pm-rail .pm-link.pm-active::before { opacity: 1; }
|
| 81 |
+
.pm-link .pm-count { font: 400 12px/1 var(--font-mono); opacity: .75; }
|
| 82 |
+
.pm-link .pm-label { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
| 83 |
+
|
| 84 |
+
.pm-rail-foot { margin-top: auto; padding: 16px; border-top: 1px solid var(--sidebar-border); }
|
| 85 |
+
#pm-rail .pm-foot-link {
|
| 86 |
+
display: flex !important; align-items: center; width: 100%; height: 2.5rem; padding: 8px; margin: 0 0 2px;
|
| 87 |
+
border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
|
| 88 |
+
font: 500 14px/1 var(--font-sans); color: var(--sidebar-fg) !important; text-align: left;
|
| 89 |
+
}
|
| 90 |
+
#pm-rail .pm-foot-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
|
| 91 |
+
#pm-rail .pm-foot-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; }
|
| 92 |
+
|
| 93 |
+
/* ---------- page shell ---------- */
|
| 94 |
+
#pm-main { min-width: 0; background: #fff; }
|
| 95 |
+
.pm-head { display: flex; align-items: center; justify-content: space-between; gap: 24px; padding: 20px 32px; border-bottom: 1px solid var(--border); }
|
| 96 |
+
.pm-head h1 { margin: 0; font: 600 1.5rem/1.1 var(--font-title); letter-spacing: -.04em; color: var(--marine-700); }
|
| 97 |
+
.pm-head p { margin: 6px 0 0; font-size: 14px; color: var(--text-secondary); }
|
| 98 |
+
.pm-body { padding: 24px 32px 32px; }
|
| 99 |
+
.pm-prose { max-width: 860px; }
|
| 100 |
+
.pm-over { margin: 0; font: 400 10px/1.4 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted); }
|
| 101 |
+
.pm-over--marine { color: var(--marine-500); }
|
| 102 |
+
h2.pm-h2 { margin: 0; font: 600 1.125rem/1.2 var(--font-title); letter-spacing: -.03em; color: var(--marine-700); }
|
| 103 |
+
.pm-group { margin-bottom: 28px; }
|
| 104 |
+
.pm-group-head { display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 12px; }
|
| 105 |
+
.pm-group-head .pm-note { margin: 0; font-size: 12px; color: var(--text-muted); }
|
| 106 |
+
.pm-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px; }
|
| 107 |
+
|
| 108 |
+
/* ---------- cards ---------- */
|
| 109 |
+
.pm-card { display: flex; flex-direction: column; border: 1px solid var(--border); border-radius: var(--radius-xl);
|
| 110 |
+
background: #fff; padding: 20px; box-shadow: var(--shadow-xs); }
|
| 111 |
+
.pm-card--live { border-left: 4px solid var(--marine-500); }
|
| 112 |
+
.pm-card--open { border-style: dashed; border-color: var(--marine-200); box-shadow: none; }
|
| 113 |
+
.pm-card--note { background: var(--surface-selected); border-color: var(--marine-100); }
|
| 114 |
+
.pm-card h3 { margin: 0; font: 600 1.25rem/1.1 var(--font-title); letter-spacing: -.04em; color: var(--marine-700); }
|
| 115 |
+
.pm-card--open h3 { color: var(--text-secondary); }
|
| 116 |
+
.pm-card .pm-meta { margin: 10px 0 0; font: 400 12px/1.5 var(--font-mono); color: var(--text-muted); }
|
| 117 |
+
.pm-card .pm-asks { margin: 8px 0 0; font-size: 14px; color: var(--text-secondary); }
|
| 118 |
+
.pm-card .pm-leader { margin: 6px 0 0; font-size: 14px; color: var(--text-body); }
|
| 119 |
+
.pm-card .pm-leader--2 { margin-top: 2px; color: var(--text-muted); }
|
| 120 |
+
.pm-card p.pm-cta { margin: 14px 0 0; font-size: 14px; }
|
| 121 |
+
a { color: var(--marine-500); text-decoration: none; }
|
| 122 |
+
a:hover { color: var(--marine-700); }
|
| 123 |
+
|
| 124 |
+
/* ---------- badges ---------- */
|
| 125 |
+
.pm-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border: 1px solid transparent;
|
| 126 |
+
border-radius: var(--radius-md); font: 400 12px/1 var(--font-sans); white-space: nowrap; }
|
| 127 |
+
.pm-badge--marine { background: var(--marine-50); color: var(--marine-500); }
|
| 128 |
+
.pm-badge--quiet { background: var(--n-100); border-color: var(--border); color: var(--text-secondary); }
|
| 129 |
+
.pm-link .pm-badge--quiet { background: transparent; border-color: var(--sidebar-border); color: var(--sidebar-sub-fg); }
|
| 130 |
+
|
| 131 |
+
/* ---------- buttons (page actions) ---------- */
|
| 132 |
+
#pm-main .pm-btn { display: inline-flex !important; align-items: center; justify-content: center; height: 2rem; min-height: 2rem;
|
| 133 |
+
padding: 0 12px; border-radius: var(--radius-md); font: 500 12px/1 var(--font-sans); border: 1px solid transparent; box-shadow: none; }
|
| 134 |
+
#pm-main .pm-btn--primary { background: var(--marine-400) !important; color: #fff !important; }
|
| 135 |
+
#pm-main .pm-btn--primary:hover { background: var(--marine-500) !important; }
|
| 136 |
+
#pm-main .pm-btn--outline { background: #fff !important; border-color: var(--border) !important; color: var(--marine-700) !important; box-shadow: var(--shadow-xs); }
|
| 137 |
+
#pm-main .pm-btn--outline:hover { background: var(--n-100) !important; }
|
| 138 |
+
#pm-main .pm-btn--block { width: 100%; height: 2.25rem; font-size: 14px; }
|
| 139 |
+
|
| 140 |
+
/* ---------- toggle group (Ranked / Per task) ---------- */
|
| 141 |
+
#pm-main .pm-toggle { display: inline-flex; gap: 2px; padding: 3px; border-radius: var(--radius-lg); background: var(--n-100); margin-bottom: 16px; }
|
| 142 |
+
#pm-main .pm-toggle button { height: 1.75rem; min-height: 1.75rem; padding: 0 14px; border: 0 !important; border-radius: var(--radius-md);
|
| 143 |
+
background: transparent !important; box-shadow: none !important; font: 400 14px/1 var(--font-sans); color: var(--text-muted) !important; }
|
| 144 |
+
#pm-main .pm-toggle button.pm-active { background: #fff !important; color: var(--marine-400) !important; box-shadow: var(--shadow-sm) !important; font-weight: 500; }
|
| 145 |
+
|
| 146 |
+
/* ---------- tables ---------- */
|
| 147 |
+
.pm-table-wrap { width: 100%; overflow-x: auto; border: 1px solid var(--border); border-radius: var(--radius-md); }
|
| 148 |
+
.pm-table { width: 100%; border-collapse: collapse; table-layout: fixed; font-size: 14px; }
|
| 149 |
+
.pm-table thead { background: var(--n-100); }
|
| 150 |
+
.pm-table thead tr, .pm-table tbody tr { border-bottom: 1px solid var(--border); }
|
| 151 |
+
.pm-table tbody tr:last-child { border-bottom: 0; }
|
| 152 |
+
.pm-table th { height: 2.5rem; padding: 0 8px; text-align: left; vertical-align: middle;
|
| 153 |
+
font: 400 10px/1.3 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted); }
|
| 154 |
+
.pm-table td { padding: 8px; vertical-align: middle; color: var(--text-secondary); }
|
| 155 |
+
.pm-table th:first-child, .pm-table td:first-child { padding-left: 16px; }
|
| 156 |
+
.pm-table th:last-child, .pm-table td:last-child { padding-right: 16px; }
|
| 157 |
+
.pm-table td.pm-strong { color: var(--text-body); }
|
| 158 |
+
.pm-table td.pm-name { font-weight: 500; color: var(--marine-700); }
|
| 159 |
+
.pm-table .pm-num { text-align: right; font-family: var(--font-mono); font-size: 13px; color: var(--text-body); }
|
| 160 |
+
.pm-table .pm-num--dim { color: var(--n-400); }
|
| 161 |
+
.pm-table .pm-score { text-align: right; font-family: var(--font-mono); font-size: .9375rem; color: var(--marine-700); }
|
| 162 |
+
.pm-table .pm-metric { font-family: var(--font-mono); font-size: 12px; color: var(--marine-700); }
|
| 163 |
+
.pm-table .pm-groupline td { padding-top: 22px; border-bottom: 0;
|
| 164 |
+
font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--marine-700); }
|
| 165 |
+
.pm-table .pm-invite td { text-align: center; color: var(--text-muted); }
|
| 166 |
+
/* definition table (Method) */
|
| 167 |
+
.pm-table--def td:first-child { width: 190px; background: var(--n-100);
|
| 168 |
+
font: 400 11px/1.4 var(--font-mono); letter-spacing: .06em; text-transform: uppercase; color: var(--text-secondary); }
|
| 169 |
+
.pm-caption { margin: 14px 0 0; font-size: 14px; color: var(--text-muted); }
|
| 170 |
+
|
| 171 |
+
/* ---------- submit steps + code ---------- */
|
| 172 |
+
.pm-step { display: flex; gap: 16px; }
|
| 173 |
+
.pm-step-n { flex: none; width: 26px; height: 26px; display: grid; place-content: center;
|
| 174 |
+
border-radius: var(--radius-md); background: var(--marine-500); color: #fff; font: 400 12px/1 var(--font-mono); }
|
| 175 |
+
.pm-step p { max-width: 60ch; margin: 8px 0 0; font-size: 14px; line-height: 1.6; color: var(--text-secondary); }
|
| 176 |
+
.pm-code { margin: 14px 0 0; padding: 16px 18px; border-radius: var(--radius-md);
|
| 177 |
+
background: var(--n-900); color: var(--n-100); font: 400 12px/1.7 var(--font-mono); overflow-x: auto; }
|
| 178 |
+
code, .pm-inline-code { font-family: var(--font-mono); font-size: .95em; }
|
| 179 |
+
|
| 180 |
+
/* ---------- submit form (Gradio inputs) ---------- */
|
| 181 |
+
#pm-form { display: flex; flex-direction: column; gap: 14px; }
|
| 182 |
+
#pm-form .block { padding: 0 !important; border: 0 !important; background: transparent !important; }
|
| 183 |
+
#pm-form label > span { font-size: 13px !important; font-weight: 400 !important; color: var(--text-secondary) !important; }
|
| 184 |
+
#pm-form input[type="text"], #pm-form textarea {
|
| 185 |
+
border: 1px solid var(--border) !important; border-radius: var(--radius-md) !important;
|
| 186 |
+
box-shadow: var(--shadow-xs) !important; font-family: var(--font-sans) !important; font-size: 14px !important; }
|
| 187 |
+
#pm-form input[type="text"]:focus, #pm-form textarea:focus { outline: 3px solid rgb(63 83 181 / .5); outline-offset: 0; }
|
| 188 |
+
#pm-form .pm-hint { margin: 6px 0 0; font-size: 12px; color: var(--text-muted); }
|
| 189 |
+
#pm-form .upload-container, #pm-form .file-preview { border: 1px dashed var(--marine-200) !important;
|
| 190 |
+
border-radius: var(--radius-md) !important; background: #fff !important; }
|
| 191 |
+
|
| 192 |
+
/* ---------- routes (Contribute) ---------- */
|
| 193 |
+
.pm-route { display: block; padding: 20px; margin-bottom: 12px; border: 1px solid var(--border);
|
| 194 |
+
border-radius: var(--radius-xl); box-shadow: var(--shadow-xs); text-decoration: none; transition: border-color .15s ease, box-shadow .15s ease; }
|
| 195 |
+
.pm-route:hover { border-color: var(--marine-400); box-shadow: var(--shadow-sm); }
|
| 196 |
+
.pm-route-head { display: flex; align-items: baseline; justify-content: space-between; gap: 16px; }
|
| 197 |
+
.pm-route-cta { font: 400 12px/1 var(--font-mono); color: var(--coral-600); white-space: nowrap; }
|
| 198 |
+
.pm-route p { max-width: 70ch; margin: 8px 0 0; font-size: 14px; line-height: 1.6; color: var(--text-secondary); }
|
| 199 |
+
.pm-logos { display: flex; flex-wrap: wrap; gap: 10px; }
|
| 200 |
+
.pm-logo { height: 64px; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 3px;
|
| 201 |
+
padding: 0 26px; border: 1px solid var(--border); border-radius: var(--radius-lg); }
|
| 202 |
+
.pm-logo img { max-width: 132px; max-height: 22px; object-fit: contain; }
|
| 203 |
+
.pm-logo--hf img { max-height: 35px; }
|
| 204 |
+
.pm-logo--open { border-style: dashed; border-color: var(--marine-200); }
|
| 205 |
+
|
| 206 |
+
/* ---------- rail inner wrapper (keeps the foot links at the bottom) ---------- */
|
| 207 |
+
#pm-rail .pm-rail-inner { display: flex; flex-direction: column; min-height: 100%; }
|
| 208 |
+
.pm-leader .pm-num--dim { font-family: var(--font-mono); font-size: 12px; color: var(--text-muted); }
|
| 209 |
+
|
| 210 |
+
/* ---------- responsive: rail collapses under 900px ---------- */
|
| 211 |
+
@media (max-width: 900px) {
|
| 212 |
+
#pm-shell { flex-wrap: wrap !important; }
|
| 213 |
+
#pm-rail { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
| 214 |
+
.pm-grid { grid-template-columns: 1fr 1fr; }
|
| 215 |
+
.pm-head { flex-direction: column; align-items: flex-start; }
|
| 216 |
+
}
|
quickstart.py
CHANGED
|
@@ -22,7 +22,7 @@ import yaml
|
|
| 22 |
from huggingface_hub import snapshot_download
|
| 23 |
from sklearn.decomposition import PCA
|
| 24 |
|
| 25 |
-
PUBLIC_REPO = "
|
| 26 |
MANIFEST_FILENAME = "datasets.yaml"
|
| 27 |
|
| 28 |
DATASET_ID = "dataset_id"
|
|
|
|
| 22 |
from huggingface_hub import snapshot_download
|
| 23 |
from sklearn.decomposition import PCA
|
| 24 |
|
| 25 |
+
PUBLIC_REPO = "PRIMOmics/primo"
|
| 26 |
MANIFEST_FILENAME = "datasets.yaml"
|
| 27 |
|
| 28 |
DATASET_ID = "dataset_id"
|
render.py
ADDED
|
@@ -0,0 +1,345 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""The PRIMO pages, rendered as ``pm-*`` markup for ``gr.HTML`` blocks.
|
| 2 |
+
|
| 3 |
+
Gradio has no card or rich-table component, so every page here is a string of
|
| 4 |
+
HTML styled entirely by ``primo.css`` -- the rail, the board grid, the
|
| 5 |
+
leaderboard tables and the tasks table. The functions are pure: they take the
|
| 6 |
+
data the app already fetched (``Board`` objects, the results ``DataFrame``, the
|
| 7 |
+
registry) and return a string, so they unit-test without Gradio or a network.
|
| 8 |
+
|
| 9 |
+
Two rules hold everywhere:
|
| 10 |
+
|
| 11 |
+
* Every string that comes from the registry or from a submission goes through
|
| 12 |
+
``html.escape`` -- board names, blurbs, disease names and, above all, the
|
| 13 |
+
user-chosen model names that become table cells and column headers.
|
| 14 |
+
* The private ``hf_username`` is never rendered. It is a name-ownership lock in
|
| 15 |
+
the results dataset, not a public credit, so no table here carries a
|
| 16 |
+
"submitted by" column.
|
| 17 |
+
|
| 18 |
+
Navigation is reload-based: the rail is a column of ``<a href="?board=slug">``
|
| 19 |
+
and ``<a href="?tab=name">`` links, which costs a page load per click but buys
|
| 20 |
+
shareable per-board URLs and needs no JavaScript, matching how the board cards
|
| 21 |
+
have always worked.
|
| 22 |
+
"""
|
| 23 |
+
|
| 24 |
+
import math
|
| 25 |
+
import numbers
|
| 26 |
+
from html import escape
|
| 27 |
+
|
| 28 |
+
import pandas as pd
|
| 29 |
+
from boards import (
|
| 30 |
+
AREA_GROUP,
|
| 31 |
+
CATEGORY_GROUP,
|
| 32 |
+
GROUP_NOTE,
|
| 33 |
+
MODALITY_GROUP,
|
| 34 |
+
Board,
|
| 35 |
+
OpenBoard,
|
| 36 |
+
featured,
|
| 37 |
+
in_group,
|
| 38 |
+
metric_label,
|
| 39 |
+
open_in_group,
|
| 40 |
+
)
|
| 41 |
+
from leaderboard import per_task_table, ranked_table, tasks_table, top_models
|
| 42 |
+
|
| 43 |
+
SECTIONS = (MODALITY_GROUP, AREA_GROUP, CATEGORY_GROUP)
|
| 44 |
+
N_TOP_MODELS = 3
|
| 45 |
+
|
| 46 |
+
BRAND = (
|
| 47 |
+
'<a class="pm-brand" target="_self" href="?tab=boards">'
|
| 48 |
+
'<img src="/gradio_api/file=assets/primo-mark.webp" alt="">'
|
| 49 |
+
"<b>PRIMO</b><span>benchmark</span></a>"
|
| 50 |
+
)
|
| 51 |
+
|
| 52 |
+
FOOT_LINKS = (
|
| 53 |
+
("tasks", "Tasks"),
|
| 54 |
+
("submit", "Submit a model"),
|
| 55 |
+
("contribute", "Contribute"),
|
| 56 |
+
("method", "Method"),
|
| 57 |
+
)
|
| 58 |
+
|
| 59 |
+
NAME_COLUMNS = frozenset({"Model", "Best"})
|
| 60 |
+
STRONG_COLUMNS = frozenset({"Task"})
|
| 61 |
+
METRIC_COLUMNS = frozenset({"Metric"})
|
| 62 |
+
RIGHT_COLUMNS = frozenset({"Rank", "Patients"})
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
# --------------------------------------------------------------------- rail
|
| 66 |
+
def rail_html(boards: list[Board], active_slug: str | None, active_tab: str | None) -> str:
|
| 67 |
+
"""The left navigation: brand, one link per board grouped by facet, foot links.
|
| 68 |
+
|
| 69 |
+
``active_slug`` highlights the board a visitor is on; ``active_tab`` highlights
|
| 70 |
+
a foot link. Open boards link to Contribute, mirroring the overview cards.
|
| 71 |
+
"""
|
| 72 |
+
parts = ['<div class="pm-rail-inner">', BRAND]
|
| 73 |
+
for group in SECTIONS:
|
| 74 |
+
cards, opens = in_group(boards, group), open_in_group(boards, group)
|
| 75 |
+
if not cards and not opens:
|
| 76 |
+
continue
|
| 77 |
+
parts.append(
|
| 78 |
+
f'<div class="pm-rail-group"><p class="pm-rail-label">{escape(group)}</p></div>'
|
| 79 |
+
)
|
| 80 |
+
for board in cards:
|
| 81 |
+
cls = "pm-link pm-active" if board.slug == active_slug else "pm-link"
|
| 82 |
+
parts.append(
|
| 83 |
+
f'<a class="{cls}" target="_self" href="?board={escape(board.slug)}">'
|
| 84 |
+
f'<span class="pm-label">{escape(board.name)}</span>'
|
| 85 |
+
f'<span class="pm-count">{board.n_tasks}</span></a>'
|
| 86 |
+
)
|
| 87 |
+
for board in opens:
|
| 88 |
+
parts.append(
|
| 89 |
+
'<a class="pm-link" target="_self" href="?tab=contribute">'
|
| 90 |
+
f'<span class="pm-label">{escape(board.name)}</span>'
|
| 91 |
+
'<span class="pm-badge pm-badge--quiet">open</span></a>'
|
| 92 |
+
)
|
| 93 |
+
parts.append('<div class="pm-rail-foot">')
|
| 94 |
+
for tab, label in FOOT_LINKS:
|
| 95 |
+
cls = "pm-foot-link pm-active" if tab == active_tab else "pm-foot-link"
|
| 96 |
+
parts.append(
|
| 97 |
+
f'<a class="{cls}" target="_self" href="?tab={tab}">{escape(label)}</a>'
|
| 98 |
+
)
|
| 99 |
+
parts.append("</div></div>")
|
| 100 |
+
return "".join(parts)
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
# ------------------------------------------------------------------ boards
|
| 104 |
+
def _leading(top) -> str:
|
| 105 |
+
"""The card's mini-ranking, with the same empty/baseline states as before.
|
| 106 |
+
|
| 107 |
+
An empty board says "be the first"; a board held only by our baselines says
|
| 108 |
+
"beat the baseline" instead, because "be the first" misleads once a PCA
|
| 109 |
+
already holds a score.
|
| 110 |
+
"""
|
| 111 |
+
parts = ['<p class="pm-over" style="margin-top:14px">Leading</p>']
|
| 112 |
+
if not top:
|
| 113 |
+
parts.append(
|
| 114 |
+
'<p class="pm-leader pm-leader--2">No ranked model yet. Be the first.</p>'
|
| 115 |
+
)
|
| 116 |
+
return "".join(parts)
|
| 117 |
+
for row in top:
|
| 118 |
+
badge = (
|
| 119 |
+
' <span class="pm-badge pm-badge--marine">baseline</span>'
|
| 120 |
+
if row.is_baseline
|
| 121 |
+
else ""
|
| 122 |
+
)
|
| 123 |
+
score = "n/a" if row.score is None else f"{row.score:.3f}"
|
| 124 |
+
parts.append(
|
| 125 |
+
f'<p class="pm-leader">{escape(row.name)}{badge}'
|
| 126 |
+
f' <span class="pm-num--dim">{score}</span></p>'
|
| 127 |
+
)
|
| 128 |
+
if all(row.is_baseline for row in top):
|
| 129 |
+
parts.append(
|
| 130 |
+
'<p class="pm-leader pm-leader--2">No submitted model yet. '
|
| 131 |
+
"Beat the baseline.</p>"
|
| 132 |
+
)
|
| 133 |
+
return "".join(parts)
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
def _live_card(board: Board, df: pd.DataFrame, by_id: dict[str, dict]) -> str:
|
| 137 |
+
top = top_models(df, by_id, board, N_TOP_MODELS)
|
| 138 |
+
return (
|
| 139 |
+
f'<a class="pm-card pm-card--live" target="_self" href="?board={escape(board.slug)}">'
|
| 140 |
+
f"<h3>{escape(board.name)}</h3>"
|
| 141 |
+
f'<p class="pm-meta">{board.n_tasks} tasks · {board.n_cohorts} cohorts<br>'
|
| 142 |
+
f"{board.n_patients:,} patients · {board.n_diseases} diseases</p>"
|
| 143 |
+
f'<p class="pm-asks">{escape(board.blurb)}</p>'
|
| 144 |
+
f"{_leading(top)}</a>"
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
def _open_card(board: OpenBoard) -> str:
|
| 149 |
+
return (
|
| 150 |
+
'<a class="pm-card pm-card--open" target="_self" href="?tab=contribute">'
|
| 151 |
+
f"<h3>{escape(board.name)}</h3>"
|
| 152 |
+
'<p class="pm-meta"><span class="pm-badge pm-badge--quiet">OPEN · no cohort yet</span></p>'
|
| 153 |
+
f'<p class="pm-asks">{escape(board.blurb)}</p>'
|
| 154 |
+
'<p class="pm-cta">Propose a cohort →</p></a>'
|
| 155 |
+
)
|
| 156 |
+
|
| 157 |
+
|
| 158 |
+
def render_boards(boards: list[Board], df: pd.DataFrame, by_id: dict[str, dict]) -> str:
|
| 159 |
+
"""The Boards overview: every board as a card, grouped by facet.
|
| 160 |
+
|
| 161 |
+
Live cards link to their board and teaser their leaders; open cards state a
|
| 162 |
+
gap and link to Contribute. A section counts its open slices so the page
|
| 163 |
+
reads as showing its own gaps, not the registry as the whole territory.
|
| 164 |
+
"""
|
| 165 |
+
if not boards:
|
| 166 |
+
return (
|
| 167 |
+
'<div class="pm-body"><p class="pm-caption">The task registry is '
|
| 168 |
+
"unavailable right now. Please retry in a moment.</p></div>"
|
| 169 |
+
)
|
| 170 |
+
n_live = len(boards)
|
| 171 |
+
n_open = sum(len(open_in_group(boards, group)) for group in SECTIONS)
|
| 172 |
+
out = [
|
| 173 |
+
'<div class="pm-head"><div><h1>Boards</h1>',
|
| 174 |
+
f"<p>{n_live} live · {n_open} open. Every board is scored by the same "
|
| 175 |
+
"fixed linear probe on labels we never release.</p></div></div>",
|
| 176 |
+
'<div class="pm-body">',
|
| 177 |
+
]
|
| 178 |
+
for group in SECTIONS:
|
| 179 |
+
cards, opens = in_group(boards, group), open_in_group(boards, group)
|
| 180 |
+
if not cards and not opens:
|
| 181 |
+
continue
|
| 182 |
+
note = escape(GROUP_NOTE.get(group, ""))
|
| 183 |
+
counter = f' · +{len(opens)} open' if opens else ""
|
| 184 |
+
out.append(
|
| 185 |
+
'<div class="pm-group"><div class="pm-group-head">'
|
| 186 |
+
f'<p class="pm-over">{escape(group)}</p>'
|
| 187 |
+
f'<p class="pm-note">{note}{counter}</p></div><div class="pm-grid">'
|
| 188 |
+
)
|
| 189 |
+
out += [_live_card(b, df, by_id) for b in cards]
|
| 190 |
+
out += [_open_card(b) for b in opens]
|
| 191 |
+
out.append("</div></div>")
|
| 192 |
+
out.append("</div>")
|
| 193 |
+
return "".join(out)
|
| 194 |
+
|
| 195 |
+
|
| 196 |
+
# ------------------------------------------------------------- html tables
|
| 197 |
+
def _fmt_value(value, is_score: bool) -> str | None:
|
| 198 |
+
"""Display text for one cell; ``None`` marks a blank (``n/a``) cell."""
|
| 199 |
+
try:
|
| 200 |
+
if pd.isna(value):
|
| 201 |
+
return None
|
| 202 |
+
except (TypeError, ValueError):
|
| 203 |
+
pass
|
| 204 |
+
if isinstance(value, str):
|
| 205 |
+
return value
|
| 206 |
+
if isinstance(value, numbers.Integral):
|
| 207 |
+
return str(int(value))
|
| 208 |
+
if isinstance(value, numbers.Real):
|
| 209 |
+
if not math.isfinite(float(value)):
|
| 210 |
+
return None
|
| 211 |
+
return f"{float(value):.3f}" if is_score else str(value)
|
| 212 |
+
return str(value)
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
def _score_columns(df: pd.DataFrame) -> list[str]:
|
| 216 |
+
"""Numeric columns to format and bold -- ``Rank`` is an index, not a score."""
|
| 217 |
+
return [c for c in df.select_dtypes("number").columns if c not in RIGHT_COLUMNS]
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
def _bold_cells(df: pd.DataFrame, score_cols: list[str], axis: int) -> set:
|
| 221 |
+
"""Which ``(row, col)`` cells hold the best value.
|
| 222 |
+
|
| 223 |
+
``axis=0`` bolds the best model per column (the ranked table, read down);
|
| 224 |
+
``axis=1`` bolds the best model per row (the per-task table, read across).
|
| 225 |
+
"""
|
| 226 |
+
bold = set()
|
| 227 |
+
if not score_cols:
|
| 228 |
+
return bold
|
| 229 |
+
if axis == 0:
|
| 230 |
+
for col in score_cols:
|
| 231 |
+
best = df[col].max(skipna=True)
|
| 232 |
+
if pd.notna(best):
|
| 233 |
+
for idx, value in df[col].items():
|
| 234 |
+
if pd.notna(value) and value == best:
|
| 235 |
+
bold.add((idx, col))
|
| 236 |
+
else:
|
| 237 |
+
for idx, row in df.iterrows():
|
| 238 |
+
present = {c: row[c] for c in score_cols if pd.notna(row[c])}
|
| 239 |
+
if present:
|
| 240 |
+
best = max(present.values())
|
| 241 |
+
bold.update((idx, c) for c, v in present.items() if v == best)
|
| 242 |
+
return bold
|
| 243 |
+
|
| 244 |
+
|
| 245 |
+
def _cell_class(col: str, is_score: bool, blank: bool) -> str:
|
| 246 |
+
if col in NAME_COLUMNS:
|
| 247 |
+
return "pm-name"
|
| 248 |
+
if col in STRONG_COLUMNS:
|
| 249 |
+
return "pm-strong"
|
| 250 |
+
if col in METRIC_COLUMNS:
|
| 251 |
+
return "pm-metric"
|
| 252 |
+
if is_score or col in RIGHT_COLUMNS:
|
| 253 |
+
return "pm-num pm-num--dim" if blank else "pm-num"
|
| 254 |
+
return ""
|
| 255 |
+
|
| 256 |
+
|
| 257 |
+
def _df_to_table(df: pd.DataFrame, bold_axis: int | None, empty: str) -> str:
|
| 258 |
+
"""Render a DataFrame as a ``pm-table``, escaping every header and cell."""
|
| 259 |
+
if df.empty:
|
| 260 |
+
return f'<p class="pm-caption">{escape(empty)}</p>'
|
| 261 |
+
score_cols = _score_columns(df)
|
| 262 |
+
bold = _bold_cells(df, score_cols, bold_axis) if bold_axis is not None else set()
|
| 263 |
+
|
| 264 |
+
head = []
|
| 265 |
+
for col in df.columns:
|
| 266 |
+
align = ' style="text-align:right"' if col in score_cols or col in RIGHT_COLUMNS else ""
|
| 267 |
+
head.append(f"<th{align}>{escape(str(col))}</th>")
|
| 268 |
+
body = []
|
| 269 |
+
for idx, row in df.iterrows():
|
| 270 |
+
cells = []
|
| 271 |
+
for col in df.columns:
|
| 272 |
+
is_score = col in score_cols
|
| 273 |
+
text = _fmt_value(row[col], is_score)
|
| 274 |
+
cls = _cell_class(col, is_score, text is None)
|
| 275 |
+
weight = ' style="font-weight:700"' if (idx, col) in bold else ""
|
| 276 |
+
shown = "n/a" if text is None else escape(text)
|
| 277 |
+
cells.append(f'<td class="{cls}"{weight}>{shown}</td>')
|
| 278 |
+
body.append(f"<tr>{''.join(cells)}</tr>")
|
| 279 |
+
return (
|
| 280 |
+
'<div class="pm-table-wrap"><table class="pm-table" style="table-layout:auto">'
|
| 281 |
+
f"<thead><tr>{''.join(head)}</tr></thead>"
|
| 282 |
+
f"<tbody>{''.join(body)}</tbody></table></div>"
|
| 283 |
+
)
|
| 284 |
+
|
| 285 |
+
|
| 286 |
+
# ------------------------------------------------------------------- board
|
| 287 |
+
def _board_meta(board: Board) -> str:
|
| 288 |
+
metrics = " · ".join(metric_label(m) for m in board.metrics)
|
| 289 |
+
tail = f" · {metrics}" if metrics else ""
|
| 290 |
+
return (
|
| 291 |
+
f"{escape(board.blurb)} · {board.n_tasks} tasks · {board.n_cohorts} cohorts · "
|
| 292 |
+
f"{board.n_patients:,} patients · {escape(board.modality)}{tail}"
|
| 293 |
+
)
|
| 294 |
+
|
| 295 |
+
|
| 296 |
+
def render_board(board: Board | None, df: pd.DataFrame, by_id: dict[str, dict]) -> str:
|
| 297 |
+
"""One board: title strip, the ranked leaderboard, then the per-task table."""
|
| 298 |
+
if board is None:
|
| 299 |
+
return (
|
| 300 |
+
'<div class="pm-head"><div><h1>No board available</h1>'
|
| 301 |
+
"<p>The task registry could not be loaded. Please retry shortly.</p>"
|
| 302 |
+
"</div></div>"
|
| 303 |
+
)
|
| 304 |
+
ranked = _df_to_table(
|
| 305 |
+
ranked_table(df, by_id, board),
|
| 306 |
+
bold_axis=0,
|
| 307 |
+
empty="No model has covered every task of this board yet. Be the first to submit.",
|
| 308 |
+
)
|
| 309 |
+
per_task = _df_to_table(
|
| 310 |
+
per_task_table(df, by_id, board),
|
| 311 |
+
bold_axis=1,
|
| 312 |
+
empty="No submission has scored on this board yet.",
|
| 313 |
+
)
|
| 314 |
+
return (
|
| 315 |
+
'<div class="pm-head"><div><p class="pm-over">Board</p>'
|
| 316 |
+
f"<h1>{escape(board.name)}</h1><p>{_board_meta(board)}</p></div></div>"
|
| 317 |
+
'<div class="pm-body">'
|
| 318 |
+
'<div class="pm-group"><div class="pm-group-head">'
|
| 319 |
+
'<p class="pm-over pm-over--marine">Ranked</p>'
|
| 320 |
+
f'<p class="pm-note">Only models that covered all {board.n_tasks} tasks are '
|
| 321 |
+
"ranked. Mean averages the family columns and mixes metrics — a tie-break, "
|
| 322 |
+
"not a score.</p>"
|
| 323 |
+
f"</div>{ranked}</div>"
|
| 324 |
+
'<div class="pm-group"><div class="pm-group-head">'
|
| 325 |
+
'<p class="pm-over pm-over--marine">Per task</p>'
|
| 326 |
+
'<p class="pm-note">Every submission, partial ones included — read across a row.</p>'
|
| 327 |
+
f"</div>{per_task}</div>"
|
| 328 |
+
"</div>"
|
| 329 |
+
)
|
| 330 |
+
|
| 331 |
+
|
| 332 |
+
# ------------------------------------------------------------------- tasks
|
| 333 |
+
def render_tasks(by_id: dict[str, dict]) -> str:
|
| 334 |
+
"""The Tasks page: one scannable row per hidden target. Provenance is never shown."""
|
| 335 |
+
df = tasks_table(list(by_id.values()))
|
| 336 |
+
table = _df_to_table(
|
| 337 |
+
df, bold_axis=None, empty="The task registry is unavailable right now."
|
| 338 |
+
)
|
| 339 |
+
return (
|
| 340 |
+
'<div class="pm-head"><div><h1>Tasks</h1>'
|
| 341 |
+
f"<p>{len(df)} hidden clinical targets. One fixed linear probe reads each "
|
| 342 |
+
"one out of your embedding; the cohorts stay anonymous, the biology does "
|
| 343 |
+
"not.</p></div></div>"
|
| 344 |
+
f'<div class="pm-body">{table}</div>'
|
| 345 |
+
)
|
style.css
DELETED
|
@@ -1,444 +0,0 @@
|
|
| 1 |
-
@import url("https://fonts.googleapis.com/css2?family=Funnel+Display:wght@400..800&display=swap");
|
| 2 |
-
|
| 3 |
-
/* PRIMO charter, dark. Deep Navy is the charter's own dark value, so the Space
|
| 4 |
-
stays on-brand while reading like a leaderboard should. Slate and Cyan swap
|
| 5 |
-
roles from the light build: on navy, raw Cyan clears 7:1 and carries text,
|
| 6 |
-
while Slate is too dark to read and is lightened for muted copy. */
|
| 7 |
-
:root {
|
| 8 |
-
--primo-navy: #080f5f;
|
| 9 |
-
--primo-deep-navy: #050a3c;
|
| 10 |
-
--primo-cyan: #16b3c0;
|
| 11 |
-
--primo-paper: #f3f8f8;
|
| 12 |
-
|
| 13 |
-
--primo-bg: var(--primo-deep-navy);
|
| 14 |
-
--primo-card: #0a1049;
|
| 15 |
-
--primo-card-hover: #0e1663;
|
| 16 |
-
--primo-line: #1e2775;
|
| 17 |
-
--primo-text: var(--primo-paper);
|
| 18 |
-
--primo-muted: #9fb3b7;
|
| 19 |
-
--primo-dim: #74898e;
|
| 20 |
-
--primo-modality: var(--primo-cyan);
|
| 21 |
-
--primo-area: #8b99f7;
|
| 22 |
-
--primo-category: #e0b341;
|
| 23 |
-
--primo-featured: var(--primo-cyan);
|
| 24 |
-
--primo-wash: rgba(243, 248, 248, 0.06);
|
| 25 |
-
}
|
| 26 |
-
|
| 27 |
-
.primo-home h2,
|
| 28 |
-
.primo-home h3,
|
| 29 |
-
.primo-board-head h2,
|
| 30 |
-
.prose h1,
|
| 31 |
-
.prose h2,
|
| 32 |
-
.prose h3 {
|
| 33 |
-
font-family: "Funnel Display", "Funnel Sans", ui-sans-serif, system-ui, sans-serif;
|
| 34 |
-
letter-spacing: -0.015em;
|
| 35 |
-
}
|
| 36 |
-
|
| 37 |
-
.primo-banner {
|
| 38 |
-
display: flex;
|
| 39 |
-
align-items: center;
|
| 40 |
-
gap: 10px;
|
| 41 |
-
flex-wrap: wrap;
|
| 42 |
-
font-size: 0.82rem;
|
| 43 |
-
color: var(--primo-muted);
|
| 44 |
-
border: 1px solid var(--primo-line);
|
| 45 |
-
border-radius: 10px;
|
| 46 |
-
padding: 9px 14px;
|
| 47 |
-
margin-bottom: 4px;
|
| 48 |
-
}
|
| 49 |
-
|
| 50 |
-
.primo-banner-tag {
|
| 51 |
-
font-size: 0.7rem;
|
| 52 |
-
font-weight: 700;
|
| 53 |
-
letter-spacing: 0.06em;
|
| 54 |
-
text-transform: uppercase;
|
| 55 |
-
color: var(--primo-featured);
|
| 56 |
-
background: var(--primo-wash);
|
| 57 |
-
border: 1px solid rgba(22, 179, 192, 0.3);
|
| 58 |
-
border-radius: 999px;
|
| 59 |
-
padding: 2px 9px;
|
| 60 |
-
}
|
| 61 |
-
|
| 62 |
-
.primo-home {
|
| 63 |
-
max-width: 1400px;
|
| 64 |
-
margin: 0 auto;
|
| 65 |
-
padding: 8px 4px 32px;
|
| 66 |
-
color: var(--primo-text);
|
| 67 |
-
}
|
| 68 |
-
|
| 69 |
-
.primo-title-row {
|
| 70 |
-
display: flex;
|
| 71 |
-
align-items: baseline;
|
| 72 |
-
justify-content: space-between;
|
| 73 |
-
gap: 16px;
|
| 74 |
-
flex-wrap: wrap;
|
| 75 |
-
margin-bottom: 20px;
|
| 76 |
-
}
|
| 77 |
-
|
| 78 |
-
.primo-title-row h2 {
|
| 79 |
-
font-size: 2rem;
|
| 80 |
-
font-weight: 700;
|
| 81 |
-
letter-spacing: -0.02em;
|
| 82 |
-
margin: 0;
|
| 83 |
-
color: var(--primo-text);
|
| 84 |
-
}
|
| 85 |
-
|
| 86 |
-
.primo-hint {
|
| 87 |
-
font-size: 0.85rem;
|
| 88 |
-
color: var(--primo-muted);
|
| 89 |
-
}
|
| 90 |
-
|
| 91 |
-
.primo-chip {
|
| 92 |
-
display: inline-block;
|
| 93 |
-
font-size: 0.7rem;
|
| 94 |
-
font-weight: 600;
|
| 95 |
-
letter-spacing: 0.08em;
|
| 96 |
-
color: var(--primo-featured);
|
| 97 |
-
background: var(--primo-wash);
|
| 98 |
-
border: 1px solid rgba(22, 179, 192, 0.3);
|
| 99 |
-
border-radius: 999px;
|
| 100 |
-
padding: 4px 12px;
|
| 101 |
-
margin-bottom: 12px;
|
| 102 |
-
}
|
| 103 |
-
|
| 104 |
-
.primo-grid {
|
| 105 |
-
display: grid;
|
| 106 |
-
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
|
| 107 |
-
gap: 16px;
|
| 108 |
-
}
|
| 109 |
-
|
| 110 |
-
.primo-grid-featured {
|
| 111 |
-
grid-template-columns: repeat(auto-fit, minmax(320px, 1fr));
|
| 112 |
-
margin-bottom: 32px;
|
| 113 |
-
}
|
| 114 |
-
|
| 115 |
-
.primo-card {
|
| 116 |
-
display: block;
|
| 117 |
-
position: relative;
|
| 118 |
-
background: var(--primo-card);
|
| 119 |
-
border: 1px solid var(--primo-line);
|
| 120 |
-
border-left: 3px solid var(--primo-line);
|
| 121 |
-
border-radius: 12px;
|
| 122 |
-
padding: 16px 18px;
|
| 123 |
-
text-decoration: none !important;
|
| 124 |
-
color: inherit;
|
| 125 |
-
transition: box-shadow 0.15s ease, border-color 0.15s ease, transform 0.15s ease;
|
| 126 |
-
}
|
| 127 |
-
|
| 128 |
-
.primo-card:hover {
|
| 129 |
-
background: var(--primo-card-hover);
|
| 130 |
-
transform: translateY(-2px);
|
| 131 |
-
box-shadow: 0 8px 22px rgba(0, 0, 0, 0.45);
|
| 132 |
-
}
|
| 133 |
-
|
| 134 |
-
.primo-accent-modality {
|
| 135 |
-
border-left-color: var(--primo-modality);
|
| 136 |
-
}
|
| 137 |
-
.primo-accent-area {
|
| 138 |
-
border-left-color: var(--primo-area);
|
| 139 |
-
}
|
| 140 |
-
.primo-accent-category {
|
| 141 |
-
border-left-color: var(--primo-category);
|
| 142 |
-
}
|
| 143 |
-
.primo-accent-modality:hover {
|
| 144 |
-
border-color: var(--primo-modality);
|
| 145 |
-
}
|
| 146 |
-
.primo-accent-area:hover {
|
| 147 |
-
border-color: var(--primo-area);
|
| 148 |
-
}
|
| 149 |
-
.primo-accent-category:hover {
|
| 150 |
-
border-color: var(--primo-category);
|
| 151 |
-
}
|
| 152 |
-
|
| 153 |
-
/* ``align-self`` so an open card sizes to its own text: stretched to match a
|
| 154 |
-
full board it becomes a large empty rectangle, which reads as broken. */
|
| 155 |
-
.primo-card-open {
|
| 156 |
-
align-self: start;
|
| 157 |
-
background: transparent;
|
| 158 |
-
border-style: dashed;
|
| 159 |
-
border-left-color: var(--primo-line);
|
| 160 |
-
opacity: 0.72;
|
| 161 |
-
}
|
| 162 |
-
|
| 163 |
-
.primo-card-open:hover {
|
| 164 |
-
opacity: 1;
|
| 165 |
-
background: var(--primo-card);
|
| 166 |
-
border-color: var(--primo-dim);
|
| 167 |
-
}
|
| 168 |
-
|
| 169 |
-
.primo-card-open .primo-card-title {
|
| 170 |
-
color: var(--primo-muted);
|
| 171 |
-
}
|
| 172 |
-
|
| 173 |
-
.primo-badge-open {
|
| 174 |
-
margin-left: auto;
|
| 175 |
-
font-size: 0.6rem;
|
| 176 |
-
font-weight: 700;
|
| 177 |
-
letter-spacing: 0.08em;
|
| 178 |
-
color: var(--primo-dim);
|
| 179 |
-
border: 1px solid var(--primo-line);
|
| 180 |
-
border-radius: 999px;
|
| 181 |
-
padding: 2px 8px;
|
| 182 |
-
}
|
| 183 |
-
|
| 184 |
-
.primo-open-cta {
|
| 185 |
-
font-size: 0.8rem;
|
| 186 |
-
font-weight: 600;
|
| 187 |
-
color: var(--primo-area);
|
| 188 |
-
}
|
| 189 |
-
|
| 190 |
-
.primo-count-open {
|
| 191 |
-
font-size: 0.7rem;
|
| 192 |
-
font-weight: 600;
|
| 193 |
-
color: var(--primo-dim);
|
| 194 |
-
margin-left: 6px;
|
| 195 |
-
}
|
| 196 |
-
|
| 197 |
-
.primo-card-head {
|
| 198 |
-
display: flex;
|
| 199 |
-
align-items: center;
|
| 200 |
-
gap: 10px;
|
| 201 |
-
margin-bottom: 8px;
|
| 202 |
-
}
|
| 203 |
-
|
| 204 |
-
/* Replaces the per-board emoji: letters scale to any modality anyone adds, and
|
| 205 |
-
the colour is the group's, so a new board needs no design decision. */
|
| 206 |
-
.primo-code {
|
| 207 |
-
flex: none;
|
| 208 |
-
font-size: 0.66rem;
|
| 209 |
-
font-weight: 700;
|
| 210 |
-
letter-spacing: 0.07em;
|
| 211 |
-
line-height: 1;
|
| 212 |
-
color: var(--primo-deep-navy);
|
| 213 |
-
background: var(--primo-modality);
|
| 214 |
-
border-radius: 5px;
|
| 215 |
-
padding: 5px 6px;
|
| 216 |
-
}
|
| 217 |
-
|
| 218 |
-
.primo-code-area {
|
| 219 |
-
background: var(--primo-area);
|
| 220 |
-
}
|
| 221 |
-
|
| 222 |
-
.primo-code-category {
|
| 223 |
-
background: var(--primo-category);
|
| 224 |
-
}
|
| 225 |
-
|
| 226 |
-
.primo-board-head .primo-code {
|
| 227 |
-
vertical-align: middle;
|
| 228 |
-
display: inline-block;
|
| 229 |
-
margin-right: 9px;
|
| 230 |
-
}
|
| 231 |
-
|
| 232 |
-
.primo-card-title {
|
| 233 |
-
font-size: 1.05rem;
|
| 234 |
-
font-weight: 650;
|
| 235 |
-
color: var(--primo-text);
|
| 236 |
-
}
|
| 237 |
-
|
| 238 |
-
.primo-card-sub {
|
| 239 |
-
font-size: 0.8rem;
|
| 240 |
-
color: var(--primo-muted);
|
| 241 |
-
margin-bottom: 14px;
|
| 242 |
-
}
|
| 243 |
-
|
| 244 |
-
.primo-blurb {
|
| 245 |
-
font-size: 0.85rem;
|
| 246 |
-
line-height: 1.45;
|
| 247 |
-
color: var(--primo-muted);
|
| 248 |
-
margin: 0 0 14px;
|
| 249 |
-
min-height: 2.5em;
|
| 250 |
-
}
|
| 251 |
-
|
| 252 |
-
.primo-stats {
|
| 253 |
-
display: grid;
|
| 254 |
-
grid-template-columns: 1fr 1fr;
|
| 255 |
-
gap: 10px 12px;
|
| 256 |
-
margin-bottom: 14px;
|
| 257 |
-
}
|
| 258 |
-
|
| 259 |
-
.primo-stat {
|
| 260 |
-
display: flex;
|
| 261 |
-
flex-direction: column;
|
| 262 |
-
gap: 2px;
|
| 263 |
-
}
|
| 264 |
-
|
| 265 |
-
.primo-stat-label {
|
| 266 |
-
font-size: 0.65rem;
|
| 267 |
-
letter-spacing: 0.06em;
|
| 268 |
-
color: var(--primo-dim);
|
| 269 |
-
}
|
| 270 |
-
|
| 271 |
-
.primo-stat-value {
|
| 272 |
-
font-size: 1.15rem;
|
| 273 |
-
font-weight: 600;
|
| 274 |
-
color: var(--primo-text);
|
| 275 |
-
font-variant-numeric: tabular-nums;
|
| 276 |
-
}
|
| 277 |
-
|
| 278 |
-
.primo-top {
|
| 279 |
-
border-top: 1px solid var(--primo-line);
|
| 280 |
-
padding-top: 10px;
|
| 281 |
-
margin-bottom: 14px;
|
| 282 |
-
}
|
| 283 |
-
|
| 284 |
-
.primo-top-head,
|
| 285 |
-
.primo-top-row {
|
| 286 |
-
display: flex;
|
| 287 |
-
justify-content: space-between;
|
| 288 |
-
gap: 12px;
|
| 289 |
-
padding: 5px 0;
|
| 290 |
-
}
|
| 291 |
-
|
| 292 |
-
.primo-top-head {
|
| 293 |
-
font-size: 0.65rem;
|
| 294 |
-
letter-spacing: 0.06em;
|
| 295 |
-
color: var(--primo-dim);
|
| 296 |
-
}
|
| 297 |
-
|
| 298 |
-
.primo-top-row {
|
| 299 |
-
font-size: 0.85rem;
|
| 300 |
-
border-top: 1px solid var(--primo-line);
|
| 301 |
-
font-variant-numeric: tabular-nums;
|
| 302 |
-
}
|
| 303 |
-
|
| 304 |
-
.primo-top-row span:first-child {
|
| 305 |
-
overflow: hidden;
|
| 306 |
-
text-overflow: ellipsis;
|
| 307 |
-
white-space: nowrap;
|
| 308 |
-
}
|
| 309 |
-
|
| 310 |
-
.primo-top-empty {
|
| 311 |
-
font-size: 0.82rem;
|
| 312 |
-
color: var(--primo-dim);
|
| 313 |
-
font-style: italic;
|
| 314 |
-
}
|
| 315 |
-
|
| 316 |
-
.primo-top-baseline span:first-child {
|
| 317 |
-
color: var(--primo-muted);
|
| 318 |
-
}
|
| 319 |
-
|
| 320 |
-
.primo-tag {
|
| 321 |
-
font-size: 0.6rem;
|
| 322 |
-
letter-spacing: 0.06em;
|
| 323 |
-
text-transform: uppercase;
|
| 324 |
-
color: var(--primo-dim);
|
| 325 |
-
border: 1px solid var(--primo-line);
|
| 326 |
-
border-radius: 999px;
|
| 327 |
-
padding: 1px 6px;
|
| 328 |
-
margin-left: 7px;
|
| 329 |
-
}
|
| 330 |
-
|
| 331 |
-
.primo-top-foot {
|
| 332 |
-
font-size: 0.75rem;
|
| 333 |
-
color: var(--primo-dim);
|
| 334 |
-
font-style: italic;
|
| 335 |
-
border-top: 1px solid var(--primo-line);
|
| 336 |
-
padding-top: 7px;
|
| 337 |
-
}
|
| 338 |
-
|
| 339 |
-
.primo-pills {
|
| 340 |
-
display: flex;
|
| 341 |
-
flex-wrap: wrap;
|
| 342 |
-
gap: 6px;
|
| 343 |
-
}
|
| 344 |
-
|
| 345 |
-
.primo-pill {
|
| 346 |
-
font-size: 0.72rem;
|
| 347 |
-
color: var(--primo-muted);
|
| 348 |
-
background: var(--primo-wash);
|
| 349 |
-
border: 1px solid var(--primo-line);
|
| 350 |
-
border-radius: 999px;
|
| 351 |
-
padding: 2px 10px;
|
| 352 |
-
}
|
| 353 |
-
|
| 354 |
-
.primo-pill-modality {
|
| 355 |
-
color: var(--primo-modality);
|
| 356 |
-
border-color: rgba(22, 179, 192, 0.32);
|
| 357 |
-
background: var(--primo-wash);
|
| 358 |
-
}
|
| 359 |
-
|
| 360 |
-
/* No fill: the cards already lift off the page, and a second surface behind
|
| 361 |
-
them only muddies the contrast they rely on. */
|
| 362 |
-
.primo-section {
|
| 363 |
-
border: 1px solid var(--primo-line);
|
| 364 |
-
border-radius: 14px;
|
| 365 |
-
padding: 0 0 18px;
|
| 366 |
-
margin-bottom: 20px;
|
| 367 |
-
background: transparent;
|
| 368 |
-
}
|
| 369 |
-
|
| 370 |
-
.primo-section-head {
|
| 371 |
-
display: flex;
|
| 372 |
-
align-items: center;
|
| 373 |
-
justify-content: space-between;
|
| 374 |
-
gap: 16px;
|
| 375 |
-
flex-wrap: wrap;
|
| 376 |
-
padding: 16px 20px;
|
| 377 |
-
border-bottom: 1px solid var(--primo-line);
|
| 378 |
-
margin-bottom: 18px;
|
| 379 |
-
}
|
| 380 |
-
|
| 381 |
-
.primo-section-head h3 {
|
| 382 |
-
font-size: 1.1rem;
|
| 383 |
-
font-weight: 650;
|
| 384 |
-
margin: 0;
|
| 385 |
-
color: var(--primo-text);
|
| 386 |
-
}
|
| 387 |
-
|
| 388 |
-
.primo-count {
|
| 389 |
-
font-size: 0.75rem;
|
| 390 |
-
font-weight: 600;
|
| 391 |
-
color: var(--primo-muted);
|
| 392 |
-
background: var(--primo-wash);
|
| 393 |
-
border-radius: 999px;
|
| 394 |
-
padding: 2px 9px;
|
| 395 |
-
margin-left: 6px;
|
| 396 |
-
}
|
| 397 |
-
|
| 398 |
-
.primo-section-note {
|
| 399 |
-
font-size: 0.82rem;
|
| 400 |
-
color: var(--primo-dim);
|
| 401 |
-
}
|
| 402 |
-
|
| 403 |
-
.primo-section .primo-grid {
|
| 404 |
-
padding: 0 20px;
|
| 405 |
-
}
|
| 406 |
-
|
| 407 |
-
.primo-empty {
|
| 408 |
-
color: var(--primo-muted);
|
| 409 |
-
font-style: italic;
|
| 410 |
-
}
|
| 411 |
-
|
| 412 |
-
.primo-board-head {
|
| 413 |
-
border-left: 3px solid var(--primo-modality);
|
| 414 |
-
padding: 2px 0 2px 14px;
|
| 415 |
-
margin-bottom: 4px;
|
| 416 |
-
}
|
| 417 |
-
|
| 418 |
-
.primo-board-head h2 {
|
| 419 |
-
margin: 0 0 4px;
|
| 420 |
-
font-size: 1.5rem;
|
| 421 |
-
font-weight: 700;
|
| 422 |
-
}
|
| 423 |
-
|
| 424 |
-
.primo-board-head p {
|
| 425 |
-
margin: 0;
|
| 426 |
-
color: var(--primo-muted);
|
| 427 |
-
font-size: 0.9rem;
|
| 428 |
-
}
|
| 429 |
-
|
| 430 |
-
/* Gradio's default inline-code chip is light-on-light, unreadable on this
|
| 431 |
-
theme. Scoped to :not(pre) so fenced blocks keep their own styling. */
|
| 432 |
-
.md :not(pre) > code,
|
| 433 |
-
.prose :not(pre) > code {
|
| 434 |
-
background: rgba(22, 179, 192, 0.1) !important;
|
| 435 |
-
color: #6fe3ec !important;
|
| 436 |
-
border: 1px solid rgba(22, 179, 192, 0.3);
|
| 437 |
-
border-radius: 5px;
|
| 438 |
-
padding: 1px 6px;
|
| 439 |
-
font-size: 0.9em;
|
| 440 |
-
}
|
| 441 |
-
|
| 442 |
-
footer {
|
| 443 |
-
display: none !important;
|
| 444 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|