HungryTorch commited on
Commit
c9d0ebc
·
verified ·
1 Parent(s): fd100fa

Ship redesigned PRIMO Space (rail nav, primo.css) reading PRIMOmics data

Browse files
README.md CHANGED
@@ -22,12 +22,13 @@ per-category leaderboard. The datasets are opaque (`d001`, `d002`…) and you ne
22
  see the disease, tissue, or target, which leaves you grading the *embedding*
23
  itself with no room for per-task tuning.
24
 
25
- The Space has six tabs: **Home** (a grid of boards), **Leaderboard** (one board
26
- at a time), **Tasks**, a **Submit** form (sign in with Hugging Face),
27
- **Contribute**, and **About**. Every board has its own URL,
28
- `?board=rheumatology-bulk-rna`, and every tab too, as `?tab=contribute`.
 
29
 
30
- Home also shows **open boards**: greyed-out cards for the omics layers and the
31
  therapeutic areas PRIMO does not cover yet, each linking to Contribute. They are
32
  declared in `boards.py` (`OPEN_BOARDS`) and drop out on their own once the
33
  registry covers that slice. Task categories deliberately get none, because a
@@ -36,7 +37,7 @@ that does not exist.
36
 
37
  🌐 Website: http://primomics.org/ ·
38
  📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
39
- 📦 Data: https://huggingface.co/datasets/ScientaLab/primo
40
 
41
  ## What's in the data
42
 
@@ -131,9 +132,9 @@ pushed by `benchmark/public_benchmark/baselines.py --score --publish`.
131
  - **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
132
  Hugging Face* button; submitting requires a logged-in HF account.
133
  - Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
134
- `ScientaLab/primo` (the public `datasets.yaml` manifest) and
135
- `ScientaLab/primo-labels` (the private `tasks.yaml` registry +
136
- `<task_id>/labels.csv`), and **write** on `ScientaLab/primo-results` (the
137
  persisted leaderboard).
138
  - Results persist as one normalized `task_results.csv` (`model_name, task_id,
139
  score, submitted_at, is_baseline, hf_username`) in the results dataset; the
 
22
  see the disease, tissue, or target, which leaves you grading the *embedding*
23
  itself with no room for per-task tuning.
24
 
25
+ A left **rail** navigates six pages: **Boards** (a grid of every board),
26
+ **Board** (one leaderboard at a time), **Tasks**, a **Submit** form (sign in with
27
+ Hugging Face), **Contribute**, and **Method**. Every board has its own URL,
28
+ `?board=rheumatology-bulk-rna`, and every page too, as `?tab=contribute`, so the
29
+ rail links and the open cards are shareable deep links.
30
 
31
+ Boards also shows **open boards**: greyed-out cards for the omics layers and the
32
  therapeutic areas PRIMO does not cover yet, each linking to Contribute. They are
33
  declared in `boards.py` (`OPEN_BOARDS`) and drop out on their own once the
34
  registry covers that slice. Task categories deliberately get none, because a
 
37
 
38
  🌐 Website: http://primomics.org/ ·
39
  📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
40
+ 📦 Data: https://huggingface.co/datasets/PRIMOmics/primo
41
 
42
  ## What's in the data
43
 
 
132
  - **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
133
  Hugging Face* button; submitting requires a logged-in HF account.
134
  - Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
135
+ `PRIMOmics/primo` (the public `datasets.yaml` manifest) and
136
+ `PRIMOmics/primo-labels` (the private `tasks.yaml` registry +
137
+ `<task_id>/labels.csv`), and **write** on `PRIMOmics/primo-results` (the
138
  persisted leaderboard).
139
  - Results persist as one normalized `task_results.csv` (`model_name, task_id,
140
  score, submitted_at, is_baseline, hf_username`) in the results dataset; the
app.py CHANGED
@@ -1,13 +1,15 @@
1
  """Gradio front-end for the PRIMO public benchmark.
2
 
3
- Six pages: Home (a grid of boards), Leaderboard (one board at a time), Tasks
4
- ("what is actually being tested?"), Submit ("how do I enter?"), Contribute
5
- ("what is missing, and how do I add it?"), About ("can I trust this?"). A tab is
6
- addressable as ``?tab=contribute``, which is what the open cards on Home link to.
 
 
7
 
8
  Upload one embedding file spanning every dataset (rows keyed by ``dataset_id``
9
  + ``sample_id``); a fixed linear probe scores each task (a dataset may carry
10
- several hidden targets). Results roll up into BOARDS -- the whole modality, one
11
  therapeutic area, one task family -- and each board ranks the models that
12
  covered all of its tasks, one column per category in its native metric (AUROC or
13
  Pearson), plus a ``Mean`` of those columns that orders the rows and is labelled
@@ -19,27 +21,21 @@ Disclosure policy -- what the public pages may show:
19
  never study accessions, dataset_id -> cohort, hub keys, per-sample labels
20
  Naming the accession behind ``d002`` would put every label one GEO download
21
  away, so ``sources`` is collapsed to its archive (NCBI GEO / EMBL-EBI
22
- ArrayExpress) and ``citation`` is never rendered at all.
 
23
 
24
  Failures are surfaced by who owns them: a bad file tells the submitter exactly
25
  what to fix; an evaluator-side failure says "our side, please retry" and logs
26
  the traceback rather than blaming the submission.
27
 
28
  A page load is ONE fetch: ``_init`` pulls the registry, the boards and the
29
- persisted results once and threads that state into every tab, rather than each
30
- tab fetching for itself.
31
-
32
- The board dropdown takes ``allow_custom_value``: its choices are only filled once
33
- the registry loads, so a browser holding a value from an earlier version of the
34
- page -- or any load where the registry is briefly unreachable -- would otherwise
35
- be refused by Gradio's own preprocessing, before ``by_slug`` gets the chance to
36
- fall back to the hero board.
37
  """
38
 
39
  import os
40
  import traceback
41
  from datetime import datetime, timezone
42
- from html import escape
43
  from pathlib import Path
44
 
45
  import gradio as gr
@@ -55,14 +51,8 @@ from evaluator import (
55
  score_all,
56
  scoreable_tasks,
57
  )
58
- from home import banner_html, home_html
59
- from leaderboard import (
60
- RESERVED_COLUMNS,
61
- per_task_table,
62
- ranked_table,
63
- source_repositories,
64
- tasks_table,
65
- )
66
  from results import (
67
  BASELINE_TAG,
68
  IS_BASELINE,
@@ -74,74 +64,45 @@ from results import (
74
  read_results,
75
  )
76
 
77
- PAGES_DIR = Path(__file__).parent / "pages"
78
- STYLE = (Path(__file__).parent / "style.css").read_text()
 
 
79
 
80
  TOKEN = os.environ.get("HF_TOKEN")
81
- TAB_IDS = ("home", "leaderboard", "tasks", "submit", "contribute", "about")
82
-
83
- DEEP_NAVY = "#050A3C"
84
- PANEL = "#0A1049"
85
- LINE = "#1E2775"
86
- CYAN = "#16B3C0"
87
- PAPER = "#F3F8F8"
88
- MUTED = "#9FB3B7"
89
 
90
  THEME = gr.themes.Base(
91
- primary_hue=gr.themes.colors.cyan,
92
- secondary_hue=gr.themes.colors.blue,
93
- neutral_hue=gr.themes.colors.slate,
94
- font=(
95
- gr.themes.GoogleFont("Funnel Sans"),
96
- "ui-sans-serif",
97
- "system-ui",
98
- "sans-serif",
99
- ),
100
- ).set(
101
- color_accent=CYAN,
102
- border_color_accent=CYAN,
103
- body_background_fill=DEEP_NAVY,
104
- body_background_fill_dark=DEEP_NAVY,
105
- background_fill_primary=DEEP_NAVY,
106
- background_fill_primary_dark=DEEP_NAVY,
107
- background_fill_secondary=PANEL,
108
- background_fill_secondary_dark=PANEL,
109
- block_background_fill=DEEP_NAVY,
110
- block_background_fill_dark=DEEP_NAVY,
111
- panel_background_fill=PANEL,
112
- block_label_background_fill=PANEL,
113
- block_label_text_color=MUTED,
114
- body_text_color=PAPER,
115
- body_text_color_dark=PAPER,
116
- body_text_color_subdued=MUTED,
117
- body_text_color_subdued_dark=MUTED,
118
- border_color_primary=LINE,
119
- border_color_primary_dark=LINE,
120
- block_border_color=LINE,
121
- input_background_fill=PANEL,
122
- input_background_fill_dark=PANEL,
123
- button_primary_background_fill=CYAN,
124
- button_primary_text_color=DEEP_NAVY,
125
- button_secondary_background_fill=PANEL,
126
- table_border_color=LINE,
127
- table_text_color=PAPER,
128
- table_even_background_fill=DEEP_NAVY,
129
- table_odd_background_fill=PANEL,
130
- link_text_color=CYAN,
131
  )
132
 
133
 
134
  def _page_text(name: str) -> str:
135
  """One ``pages/<tab>.md`` per prose tab, named after the tab it fills.
136
 
137
- Editing the site's words never means touching Python. Home has no file --
138
- it is the generated board grid.
139
  """
140
  return (PAGES_DIR / f"{name}.md").read_text()
141
 
142
 
143
  def _registry_by_id() -> dict[str, dict]:
144
- """Scoreable tasks keyed by task_id (dataset present in the public manifest)."""
 
 
 
 
 
 
 
 
145
  datasets = manifest_ids(fetch_manifest(TOKEN))
146
  registry = scoreable_tasks(fetch_tasks_registry(TOKEN), datasets)
147
  return {_norm_id(task["task_id"]): task for task in registry}
@@ -154,7 +115,7 @@ def _page_state() -> PageState:
154
  """Registry, boards and persisted results -- one fetch per render.
155
 
156
  A failed fetch yields empty structures so the page renders a "come back
157
- later" state instead of a stack trace. Every tab is built from one of these,
158
  threaded through rather than refetched, so a page load is one round trip.
159
  """
160
  try:
@@ -165,99 +126,6 @@ def _page_state() -> PageState:
165
  return {}, [], pd.DataFrame(columns=RESULT_COLUMNS)
166
 
167
 
168
- COLUMN_WIDTHS = {
169
- "Rank": "70px",
170
- "Model": "230px",
171
- "Mean": "110px",
172
- "Task": "300px",
173
- "Family": "150px",
174
- "Area": "170px",
175
- "Metric": "100px",
176
- "Best": "170px",
177
- }
178
- DEFAULT_WIDTH = "160px"
179
-
180
-
181
- def _score_columns(df: pd.DataFrame) -> list[str]:
182
- return [c for c in df.select_dtypes("number").columns if c != "Rank"]
183
-
184
-
185
- def _styled(df: pd.DataFrame, label: str, axis: int, pinned: int) -> gr.DataFrame:
186
- """An MTEB-style table: best value in bold, ids pinned, searchable.
187
-
188
- ``axis=0`` bolds the best model per column (the ranked table); ``axis=1``
189
- bolds the best model per row (the per-task table, read across). The label
190
- names the table, so the copy file needs one block per tab, not per table.
191
- """
192
- if df.empty:
193
- return gr.DataFrame(df, label=label, interactive=False)
194
- scores = _score_columns(df)
195
- labels = [c for c in df.columns if c not in scores]
196
- styler = df.style.format("{:.3f}", subset=scores, na_rep="n/a").highlight_max(
197
- subset=scores, axis=axis, props="font-weight: 700"
198
- )
199
- if labels:
200
- styler = styler.format(na_rep="n/a", subset=labels)
201
- return gr.DataFrame(
202
- styler,
203
- label=label,
204
- interactive=False,
205
- wrap=True,
206
- pinned_columns=pinned,
207
- column_widths=[COLUMN_WIDTHS.get(c, DEFAULT_WIDTH) for c in df.columns],
208
- show_search="filter",
209
- show_copy_button=True,
210
- show_fullscreen_button=True,
211
- )
212
-
213
-
214
- def _board_header(board: Board | None) -> str:
215
- """The board title strip. Registry strings are escaped: they are data, not code."""
216
- if board is None:
217
- return (
218
- '<div class="primo-board-head"><h2>No board available</h2>'
219
- "<p>The task registry could not be loaded. Please retry shortly.</p></div>"
220
- )
221
- return (
222
- '<div class="primo-board-head primo-accent"><h2>'
223
- f'<span class="primo-code primo-code-modality">{escape(board.code)}</span>'
224
- f"{escape(board.name)}</h2>"
225
- f"<p>{escape(board.blurb)} {board.n_tasks} tasks · "
226
- f"{board.n_cohorts} cohorts · {board.n_patients:,} patients · "
227
- f"{escape(board.modality)}</p></div>"
228
- )
229
-
230
-
231
- RANKED_LABEL = "🏆 Ranked"
232
- PER_TASK_LABEL = "🔍 Per task"
233
-
234
-
235
- def _board_page(slug: str | None, state: PageState | None = None):
236
- """Header + both tables for one board.
237
-
238
- ``state`` lets a caller that has already fetched pass it in; the first render
239
- builds every tab from one fetch that way.
240
- """
241
- by_id, boards, df = _page_state() if state is None else state
242
- board = by_slug(boards, slug)
243
- if board is None:
244
- empty = pd.DataFrame()
245
- return (
246
- _board_header(None),
247
- _styled(empty, RANKED_LABEL, 0, 0),
248
- _styled(empty, PER_TASK_LABEL, 1, 0),
249
- )
250
- return (
251
- _board_header(board),
252
- _styled(ranked_table(df, by_id, board), RANKED_LABEL, 0, 2),
253
- _styled(per_task_table(df, by_id, board), PER_TASK_LABEL, 1, 1),
254
- )
255
-
256
-
257
- def _board_choices(boards: list[Board]) -> list[tuple[str, str]]:
258
- return [(f"{b.code} · {b.name} · {b.group}", b.slug) for b in boards]
259
-
260
-
261
  def _about_text(by_id: dict[str, dict]) -> str:
262
  """Methodology + the archives the cohorts live in, never their accessions.
263
 
@@ -307,6 +175,12 @@ def _claimed_by(model: str) -> str:
307
  return ""
308
 
309
 
 
 
 
 
 
 
310
  def evaluate(
311
  submission_path: str,
312
  model_name: str,
@@ -318,7 +192,7 @@ def evaluate(
318
  profile: gr.OAuthProfile | None,
319
  ):
320
  def _refuse(message: str):
321
- return (message, *_board_page(slug))
322
 
323
  if profile is None:
324
  return _refuse("Please sign in with Hugging Face to submit.")
@@ -387,122 +261,124 @@ def evaluate(
387
  except Exception as error: # noqa: BLE001
388
  traceback.print_exc()
389
  summary += f"\n\n⚠️ scored, but the leaderboard was not saved: {error}"
390
- return (summary, *_board_page(slug))
391
 
392
 
393
  def _landing_tab(params: dict, has_board: bool) -> str:
394
- """Which tab a visitor lands on: ``?tab=`` wins, then ``?board=``, else Home.
395
 
396
- An unknown ``?tab=`` falls through to Home rather than selecting nothing,
397
  which would render the Space with every panel collapsed.
398
  """
399
  tab = params.get("tab")
400
  if tab in TAB_IDS:
401
  return tab
402
- return "leaderboard" if has_board else "home"
403
 
404
 
405
  def _init(request: gr.Request):
406
- """Render every tab from one fetch, landing where the query params ask."""
407
- state = _page_state()
408
- by_id, boards, df = state
409
  params = dict(request.query_params) if request else {}
410
  board = by_slug(boards, params.get("board"))
411
  selected = _landing_tab(params, bool(params.get("board") and board))
 
 
412
  return (
413
  gr.Tabs(selected=selected),
414
- gr.Dropdown(
415
- choices=_board_choices(boards), value=board.slug if board else None
416
- ),
417
- banner_html(boards),
418
- home_html(boards, df, by_id),
419
- tasks_table(list(by_id.values())),
420
  _about_text(by_id),
421
- *_board_page(board.slug if board else None, state),
422
  )
423
 
424
 
425
  def build_demo() -> gr.Blocks:
426
  with gr.Blocks(
427
- title="PRIMO Benchmark", theme=THEME, css=STYLE, fill_width=True
428
  ) as demo:
429
- gr.Markdown(
430
- "# 🧬 PRIMO: Patient Representations in Multi-Omics\n\n"
431
- "**Does your model's patient embedding predict a real clinical "
432
- "outcome?** We score it on cohorts whose labels you never see."
433
- )
434
- banner = gr.HTML()
435
- with gr.Tabs() as tabs:
436
- with gr.Tab("Home", id="home"):
437
- home = gr.HTML()
438
- with gr.Tab("Leaderboard", id="leaderboard"):
439
- board_sel = gr.Dropdown(
440
- label="Board",
441
- choices=[],
442
- allow_custom_value=True,
443
- info="A slice of the benchmark: one modality, area or task family.",
444
- )
445
- board_head = gr.HTML()
446
- gr.Markdown(_page_text("leaderboard"))
447
- ranked = gr.DataFrame(label=RANKED_LABEL, interactive=False)
448
- per_task = gr.DataFrame(label=PER_TASK_LABEL, interactive=False)
449
- with gr.Tab("Tasks", id="tasks"):
450
- gr.Markdown(_page_text("tasks"))
451
- tasks_df = gr.DataFrame(interactive=False, wrap=True)
452
- with gr.Tab("Submit", id="submit"):
453
- gr.Markdown(_page_text("submit"))
454
- gr.LoginButton()
455
- with gr.Row():
456
- with gr.Column():
457
- model_tb = gr.Textbox(
458
- label="Model name",
459
- placeholder="e.g. eva-rna-v1",
460
- info="Shown on the leaderboard.",
461
- )
462
- email_tb = gr.Textbox(
463
- label="Email address",
464
- placeholder="you@lab.org",
465
- info="Contact for this submission, kept private.",
466
- )
467
- notes_tb = gr.Textbox(
468
- label="Training data / notes (optional)",
469
- placeholder="e.g. pretrained on atlas X",
470
- info="About the model or its training data.",
471
- )
472
- with gr.Column():
473
- paper_tb = gr.Textbox(
474
- label="Paper link (optional)",
475
- placeholder="https://arxiv.org/abs/...",
476
- )
477
- hf_tb = gr.Textbox(
478
- label="Hugging Face model link (optional)",
479
- placeholder="https://huggingface.co/...",
480
- )
481
- file_in = gr.File(
482
- label="Submission (.csv / .tsv / .parquet / .npz)",
483
- type="filepath",
484
- )
485
- run_btn = gr.Button("Evaluate", variant="primary")
486
- result_md = gr.Markdown()
487
- with gr.Tab("Contribute", id="contribute"):
488
- gr.Markdown(_page_text("contribute"))
489
- with gr.Tab("About", id="about"):
490
- about_md = gr.Markdown()
491
-
492
- board_view = [board_head, ranked, per_task]
493
  run_btn.click(
494
  evaluate,
495
- [file_in, model_tb, email_tb, paper_tb, hf_tb, notes_tb, board_sel],
496
- [result_md, *board_view],
497
  )
498
- board_sel.change(_board_page, board_sel, board_view)
499
  demo.load(
500
  _init,
501
  None,
502
- [tabs, board_sel, banner, home, tasks_df, about_md, *board_view],
503
  )
504
  return demo
505
 
506
 
507
  if __name__ == "__main__":
508
- build_demo().launch(server_name="0.0.0.0", server_port=7860, ssr_mode=False)
 
 
 
 
 
 
1
  """Gradio front-end for the PRIMO public benchmark.
2
 
3
+ A left rail navigates six pages: Boards (a grid of every board), Board (one
4
+ leaderboard at a time), Tasks ("what is actually being tested?"), Submit ("how
5
+ do I enter?"), Contribute ("what is missing, and how do I add it?"), Method
6
+ ("can I trust this?"). Every page is addressable -- ``?board=<slug>`` opens a
7
+ board and ``?tab=contribute`` opens a tab -- which is what the rail links and the
8
+ open cards use. The tab strip is hidden in CSS; the rail is the navigation.
9
 
10
  Upload one embedding file spanning every dataset (rows keyed by ``dataset_id``
11
  + ``sample_id``); a fixed linear probe scores each task (a dataset may carry
12
+ several hidden targets). Results roll up into boards -- the whole modality, one
13
  therapeutic area, one task family -- and each board ranks the models that
14
  covered all of its tasks, one column per category in its native metric (AUROC or
15
  Pearson), plus a ``Mean`` of those columns that orders the rows and is labelled
 
21
  never study accessions, dataset_id -> cohort, hub keys, per-sample labels
22
  Naming the accession behind ``d002`` would put every label one GEO download
23
  away, so ``sources`` is collapsed to its archive (NCBI GEO / EMBL-EBI
24
+ ArrayExpress) and ``citation`` is never rendered at all. The submitter's Hugging
25
+ Face username is a name-ownership lock, kept private and never rendered.
26
 
27
  Failures are surfaced by who owns them: a bad file tells the submitter exactly
28
  what to fix; an evaluator-side failure says "our side, please retry" and logs
29
  the traceback rather than blaming the submission.
30
 
31
  A page load is ONE fetch: ``_init`` pulls the registry, the boards and the
32
+ persisted results once and threads that state into every page, rather than each
33
+ page fetching for itself.
 
 
 
 
 
 
34
  """
35
 
36
  import os
37
  import traceback
38
  from datetime import datetime, timezone
 
39
  from pathlib import Path
40
 
41
  import gradio as gr
 
51
  score_all,
52
  scoreable_tasks,
53
  )
54
+ from leaderboard import RESERVED_COLUMNS, source_repositories
55
+ from render import rail_html, render_board, render_boards, render_tasks
 
 
 
 
 
 
56
  from results import (
57
  BASELINE_TAG,
58
  IS_BASELINE,
 
64
  read_results,
65
  )
66
 
67
+ HERE = Path(__file__).parent
68
+ PAGES_DIR = HERE / "pages"
69
+ CSS_PATH = str(HERE / "primo.css")
70
+ ALLOWED_PATHS = [str(HERE / "assets"), str(HERE / "fonts")]
71
 
72
  TOKEN = os.environ.get("HF_TOKEN")
73
+ TAB_IDS = ("boards", "board", "tasks", "submit", "contribute", "method")
74
+ FOOT_TABS = frozenset({"tasks", "submit", "contribute", "method"})
 
 
 
 
 
 
75
 
76
  THEME = gr.themes.Base(
77
+ font=["Funnel Sans", "sans-serif"], font_mono=["DM Mono", "monospace"]
78
+ )
79
+
80
+ SUBMIT_HEAD = (
81
+ '<div class="pm-head"><div><h1>Submit a model</h1>'
82
+ "<p>One embedding file. Partial coverage is fine — you are ranked on every "
83
+ "board you cover in full.</p></div></div>"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
  )
85
 
86
 
87
  def _page_text(name: str) -> str:
88
  """One ``pages/<tab>.md`` per prose tab, named after the tab it fills.
89
 
90
+ Editing the site's words never means touching Python. The data pages (Boards,
91
+ Board, Tasks) are generated markup instead.
92
  """
93
  return (PAGES_DIR / f"{name}.md").read_text()
94
 
95
 
96
  def _registry_by_id() -> dict[str, dict]:
97
+ """Scoreable tasks keyed by task_id (dataset present in the public manifest).
98
+
99
+ ``PRIMO_FIXTURE=1`` swaps the token-gated fetch for the test registry, so the
100
+ app runs offline for local smoke-testing.
101
+ """
102
+ if os.environ.get("PRIMO_FIXTURE"):
103
+ from conftest import REGISTRY
104
+
105
+ return {task_id: dict(task) for task_id, task in REGISTRY.items()}
106
  datasets = manifest_ids(fetch_manifest(TOKEN))
107
  registry = scoreable_tasks(fetch_tasks_registry(TOKEN), datasets)
108
  return {_norm_id(task["task_id"]): task for task in registry}
 
115
  """Registry, boards and persisted results -- one fetch per render.
116
 
117
  A failed fetch yields empty structures so the page renders a "come back
118
+ later" state instead of a stack trace. Every page is built from one of these,
119
  threaded through rather than refetched, so a page load is one round trip.
120
  """
121
  try:
 
126
  return {}, [], pd.DataFrame(columns=RESULT_COLUMNS)
127
 
128
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
129
  def _about_text(by_id: dict[str, dict]) -> str:
130
  """Methodology + the archives the cohorts live in, never their accessions.
131
 
 
175
  return ""
176
 
177
 
178
+ def _rendered_board(slug: str | None) -> str:
179
+ """The Board page for ``slug`` -- re-rendered after a submission saves."""
180
+ by_id, boards, df = _page_state()
181
+ return render_board(by_slug(boards, slug), df, by_id)
182
+
183
+
184
  def evaluate(
185
  submission_path: str,
186
  model_name: str,
 
192
  profile: gr.OAuthProfile | None,
193
  ):
194
  def _refuse(message: str):
195
+ return message, _rendered_board(slug)
196
 
197
  if profile is None:
198
  return _refuse("Please sign in with Hugging Face to submit.")
 
261
  except Exception as error: # noqa: BLE001
262
  traceback.print_exc()
263
  summary += f"\n\n⚠️ scored, but the leaderboard was not saved: {error}"
264
+ return summary, _rendered_board(slug)
265
 
266
 
267
  def _landing_tab(params: dict, has_board: bool) -> str:
268
+ """Which page a visitor lands on: ``?tab=`` wins, then ``?board=``, else Boards.
269
 
270
+ An unknown ``?tab=`` falls through to Boards rather than selecting nothing,
271
  which would render the Space with every panel collapsed.
272
  """
273
  tab = params.get("tab")
274
  if tab in TAB_IDS:
275
  return tab
276
+ return "board" if has_board else "boards"
277
 
278
 
279
  def _init(request: gr.Request):
280
+ """Render every page from one fetch, landing where the query params ask."""
281
+ by_id, boards, df = _page_state()
 
282
  params = dict(request.query_params) if request else {}
283
  board = by_slug(boards, params.get("board"))
284
  selected = _landing_tab(params, bool(params.get("board") and board))
285
+ active_slug = board.slug if selected == "board" and board else None
286
+ active_tab = selected if selected in FOOT_TABS else None
287
  return (
288
  gr.Tabs(selected=selected),
289
+ rail_html(boards, active_slug, active_tab),
290
+ board.slug if board else None,
291
+ render_boards(boards, df, by_id),
292
+ render_tasks(by_id),
 
 
293
  _about_text(by_id),
294
+ render_board(board, df, by_id),
295
  )
296
 
297
 
298
  def build_demo() -> gr.Blocks:
299
  with gr.Blocks(
300
+ title="PRIMO Benchmark", theme=THEME, css_paths=[CSS_PATH], fill_width=True
301
  ) as demo:
302
+ active_board = gr.State(None)
303
+ with gr.Row(elem_id="pm-shell"):
304
+ with gr.Column(elem_id="pm-rail"):
305
+ rail = gr.HTML()
306
+ with gr.Column(elem_id="pm-main"):
307
+ with gr.Tabs(elem_id="pm-pages") as pages:
308
+ with gr.Tab("Boards", id="boards"):
309
+ boards_html = gr.HTML()
310
+ with gr.Tab("Board", id="board"):
311
+ board_html = gr.HTML()
312
+ with gr.Tab("Tasks", id="tasks"):
313
+ tasks_html = gr.HTML()
314
+ with gr.Tab("Submit", id="submit"):
315
+ gr.HTML(SUBMIT_HEAD)
316
+ with gr.Column(elem_classes=["pm-body"]):
317
+ with gr.Row():
318
+ with gr.Column(scale=3):
319
+ gr.Markdown(_page_text("submit"))
320
+ with gr.Column(scale=2, elem_id="pm-form"):
321
+ gr.LoginButton()
322
+ model_tb = gr.Textbox(
323
+ label="Model name",
324
+ placeholder="e.g. eva-rna-v1",
325
+ info="Shown on the leaderboard.",
326
+ )
327
+ email_tb = gr.Textbox(
328
+ label="Email address",
329
+ placeholder="you@lab.org",
330
+ info="Contact for this submission, kept private.",
331
+ )
332
+ notes_tb = gr.Textbox(
333
+ label="Training data / notes (optional)",
334
+ placeholder="e.g. pretrained on atlas X",
335
+ info="About the model or its training data.",
336
+ )
337
+ paper_tb = gr.Textbox(
338
+ label="Paper link (optional)",
339
+ placeholder="https://arxiv.org/abs/...",
340
+ )
341
+ hf_tb = gr.Textbox(
342
+ label="Hugging Face model link (optional)",
343
+ placeholder="https://huggingface.co/...",
344
+ )
345
+ file_in = gr.File(
346
+ label="Submission (.csv / .tsv / .parquet / .npz)",
347
+ type="filepath",
348
+ )
349
+ run_btn = gr.Button(
350
+ "Evaluate",
351
+ elem_classes=[
352
+ "pm-btn",
353
+ "pm-btn--primary",
354
+ "pm-btn--block",
355
+ ],
356
+ )
357
+ result_md = gr.Markdown()
358
+ with gr.Tab("Contribute", id="contribute"):
359
+ with gr.Column(elem_classes=["pm-body", "pm-prose"]):
360
+ gr.Markdown(_page_text("contribute"))
361
+ with gr.Tab("Method", id="method"):
362
+ with gr.Column(elem_classes=["pm-body", "pm-prose"]):
363
+ about_md = gr.Markdown()
364
+
 
365
  run_btn.click(
366
  evaluate,
367
+ [file_in, model_tb, email_tb, paper_tb, hf_tb, notes_tb, active_board],
368
+ [result_md, board_html],
369
  )
 
370
  demo.load(
371
  _init,
372
  None,
373
+ [pages, rail, active_board, boards_html, tasks_html, about_md, board_html],
374
  )
375
  return demo
376
 
377
 
378
  if __name__ == "__main__":
379
+ build_demo().launch(
380
+ server_name="0.0.0.0",
381
+ server_port=7860,
382
+ ssr_mode=False,
383
+ allowed_paths=ALLOWED_PATHS,
384
+ )
assets/hugging-face.svg ADDED
assets/primo-mark.webp ADDED
assets/scienta-lab.svg ADDED
evaluator.py CHANGED
@@ -46,7 +46,7 @@ SPLIT = "split"
46
  SPLIT_TRAIN = "train"
47
  SPLIT_TEST = "test"
48
 
49
- ORG = "ScientaLab"
50
  PUBLIC_REPO = f"{ORG}/primo"
51
  LABELS_REPO = f"{ORG}/primo-labels"
52
  RESULTS_REPO = f"{ORG}/primo-results"
 
46
  SPLIT_TRAIN = "train"
47
  SPLIT_TEST = "test"
48
 
49
+ ORG = "PRIMOmics"
50
  PUBLIC_REPO = f"{ORG}/primo"
51
  LABELS_REPO = f"{ORG}/primo-labels"
52
  RESULTS_REPO = f"{ORG}/primo-results"
fonts/dm-mono-regular.woff2 ADDED
Binary file (20.6 kB). View file
 
fonts/funnel-display.woff2 ADDED
Binary file (17.7 kB). View file
 
fonts/funnel-sans.woff2 ADDED
Binary file (17.4 kB). View file
 
home.py DELETED
@@ -1,213 +0,0 @@
1
- """The home page: boards -> a grid of cards, as one block of HTML.
2
-
3
- Gradio has no card component, so the overview is rendered as markup and handed
4
- to ``gr.HTML``. Each card is a plain ``<a href="?board=...">``: clicking it
5
- reloads the Space on that board, which costs a page load but buys shareable
6
- per-board URLs and needs no JavaScript.
7
-
8
- Two things here are not leaderboards. The version strip states what v0 actually
9
- covers, counted off the registry so it cannot go stale. The greyed-out OPEN cards
10
- state what it does not cover, and link to Contribute instead of to a board: the
11
- page is supposed to show its own gaps, not imply the registry is the territory.
12
-
13
- Every string that comes from the registry goes through ``html.escape`` -- the
14
- titles, blurbs and disease names are authored data, not constants.
15
- """
16
-
17
- from html import escape
18
-
19
- import pandas as pd
20
- from boards import (
21
- AREA_GROUP,
22
- CATEGORY_GROUP,
23
- GROUP_NOTE,
24
- MODALITY_GROUP,
25
- Board,
26
- OpenBoard,
27
- featured,
28
- in_group,
29
- metric_label,
30
- open_in_group,
31
- )
32
- from leaderboard import TopModel, top_models
33
-
34
- SECTIONS = (MODALITY_GROUP, AREA_GROUP, CATEGORY_GROUP)
35
- N_TOP_MODELS = 3
36
- GROUP_ACCENT = {
37
- MODALITY_GROUP: "modality",
38
- AREA_GROUP: "area",
39
- CATEGORY_GROUP: "category",
40
- }
41
- VERSION = "v0"
42
-
43
-
44
- def _plural(count: int, noun: str) -> str:
45
- return f"{count} {noun}" if count == 1 else f"{count} {noun}s"
46
-
47
-
48
- def banner_html(boards: list[Board]) -> str:
49
- """The version strip shown above every tab: what the benchmark covers today.
50
-
51
- Counted off the registry rather than written by hand, so it can never claim a
52
- scope that has since changed. An unavailable registry yields no strip at all
53
- -- a version claim nobody can check is worse than none.
54
- """
55
- modalities = in_group(boards, MODALITY_GROUP)
56
- if not modalities:
57
- return ""
58
- areas = len({board.name for board in in_group(boards, AREA_GROUP)})
59
- return (
60
- f'<div class="primo-banner"><span class="primo-banner-tag">{VERSION}</span>'
61
- f'<span>{escape(", ".join(b.name for b in modalities))} · '
62
- f'{_plural(areas, "therapeutic area")} · '
63
- f'{_plural(sum(b.n_tasks for b in modalities), "task")}. '
64
- "The scope is incomplete by design, and the open boards on Home show where."
65
- "</span></div>"
66
- )
67
-
68
-
69
- def _stat(name: str, value: object) -> str:
70
- return (
71
- f'<div class="primo-stat"><span class="primo-stat-label">{escape(name)}</span>'
72
- f'<span class="primo-stat-value">{escape(f"{value:,}" if isinstance(value, int) else str(value))}</span></div>'
73
- )
74
-
75
-
76
- def _pills(board: Board) -> str:
77
- pills = [
78
- f'<span class="primo-pill primo-pill-modality">{escape(board.modality)}</span>'
79
- ]
80
- pills += [
81
- f'<span class="primo-pill">{escape(metric_label(m))}</span>'
82
- for m in board.metrics
83
- ]
84
- return f'<div class="primo-pills">{"".join(pills)}</div>'
85
-
86
-
87
- def _top_row(row: TopModel) -> str:
88
- tag = '<span class="primo-tag">baseline</span>' if row.is_baseline else ""
89
- css = "primo-top-row primo-top-baseline" if row.is_baseline else "primo-top-row"
90
- score = "n/a" if row.score is None else f"{row.score:.3f}"
91
- return (
92
- f'<div class="{css}"><span>{escape(row.name)}{tag}</span>'
93
- f"<span>{score}</span></div>"
94
- )
95
-
96
-
97
- def _top_models(rows: list[TopModel]) -> str:
98
- """The card's mini-ranking.
99
-
100
- A board whose only entries are our own baselines says so: "be the first" is
101
- misleading once a PCA already holds a score somebody has to beat.
102
- """
103
- if not rows:
104
- return (
105
- '<div class="primo-top primo-top-empty">No ranked model yet. '
106
- "Be the first.</div>"
107
- )
108
- foot = (
109
- '<div class="primo-top-foot">No submitted model yet. Beat the baseline.</div>'
110
- if all(row.is_baseline for row in rows)
111
- else ""
112
- )
113
- return (
114
- '<div class="primo-top"><div class="primo-top-head">'
115
- "<span>TOP MODELS</span><span>MEAN</span></div>"
116
- f'{"".join(_top_row(row) for row in rows)}{foot}</div>'
117
- )
118
-
119
-
120
- def _card(board: Board, top: list[TopModel] | None) -> str:
121
- accent = GROUP_ACCENT.get(board.group, "modality")
122
- inner = [
123
- '<div class="primo-card-head">'
124
- f'<span class="primo-code primo-code-{accent}">{escape(board.code)}</span>'
125
- f'<span class="primo-card-title">{escape(board.name)}</span></div>'
126
- ]
127
- if top is None:
128
- inner.append(f'<p class="primo-blurb">{escape(board.blurb)}</p>')
129
- inner.append(
130
- '<div class="primo-stats">'
131
- + _stat("TASKS", board.n_tasks)
132
- + _stat("COHORTS", board.n_cohorts)
133
- + _stat("PATIENTS", board.n_patients)
134
- + _stat("DISEASES", board.n_diseases)
135
- + "</div>"
136
- )
137
- else:
138
- inner.append(
139
- f'<div class="primo-card-sub">{board.n_tasks} tasks · '
140
- f"{board.n_cohorts} cohorts · {board.n_patients:,} patients</div>"
141
- )
142
- inner.append(_top_models(top))
143
- inner.append(_pills(board))
144
- return (
145
- f'<a class="primo-card primo-accent-{accent}" target="_self" '
146
- f'href="?board={escape(board.slug)}">{"".join(inner)}</a>'
147
- )
148
-
149
-
150
- def _open_card(board: OpenBoard) -> str:
151
- """A greyed-out card for a slice nobody covers: no stats, one call to action."""
152
- return (
153
- '<a class="primo-card primo-card-open" target="_self" href="?tab=contribute">'
154
- f'<div class="primo-card-head"><span class="primo-card-title">'
155
- f'{escape(board.name)}</span><span class="primo-badge-open">OPEN</span></div>'
156
- f'<p class="primo-blurb">{escape(board.blurb)}</p>'
157
- '<div class="primo-open-cta">Propose a cohort →</div></a>'
158
- )
159
-
160
-
161
- def _section(group: str, boards: list[Board], opens: list[OpenBoard]) -> str:
162
- open_count = (
163
- f'<span class="primo-count-open">+{len(opens)} open</span>' if opens else ""
164
- )
165
- cards = "".join(_card(b, None) for b in boards)
166
- cards += "".join(_open_card(b) for b in opens)
167
- return (
168
- '<section class="primo-section">'
169
- '<div class="primo-section-head"><h3>'
170
- f'{escape(group)} <span class="primo-count">{len(boards)}</span>'
171
- f"{open_count}</h3>"
172
- f'<span class="primo-section-note">{escape(GROUP_NOTE.get(group, ""))}</span>'
173
- f'</div><div class="primo-grid">{cards}</div></section>'
174
- )
175
-
176
-
177
- def _sections(boards: list[Board]) -> str:
178
- """One block per facet: what is covered, then what is openly missing.
179
-
180
- The Modality section is shown even when a single modality exists. A lone card
181
- would read as a bug; a lone card beside the omics layers nobody has brought
182
- yet reads as the point of the page.
183
- """
184
- out = []
185
- for group in SECTIONS:
186
- cards, opens = in_group(boards, group), open_in_group(boards, group)
187
- if not cards and not opens:
188
- continue
189
- out.append(_section(group, cards, opens))
190
- return "".join(out)
191
-
192
-
193
- def home_html(boards: list[Board], df: pd.DataFrame, by_id: dict[str, dict]) -> str:
194
- """The whole overview page: hero row, then one section per facet."""
195
- if not boards:
196
- return (
197
- '<div class="primo-home"><p class="primo-empty">'
198
- "The task registry is unavailable right now. Please retry in a moment."
199
- "</p></div>"
200
- )
201
- heroes = "".join(
202
- _card(b, top_models(df, by_id, b, N_TOP_MODELS)) for b in featured(boards)
203
- )
204
- sections = _sections(boards)
205
- return (
206
- '<div class="primo-home">'
207
- '<div class="primo-title-row"><h2>Benchmark overview</h2>'
208
- '<span class="primo-hint">Pick a board. Each one is its own leaderboard'
209
- "</span></div>"
210
- '<div class="primo-chip">FEATURED</div>'
211
- f'<div class="primo-grid primo-grid-featured">{heroes}</div>'
212
- f"{sections}</div>"
213
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
pages/about.md CHANGED
@@ -41,7 +41,8 @@ columns.
41
  ## How the results are organised
42
 
43
  A **board** is a self-contained leaderboard over a slice of the benchmark: the
44
- whole modality, one therapeutic area, or one task family. Pick one on **Home**.
 
45
 
46
  A board ranks only the models that covered **all of its tasks**. Skipping
47
  Dermatology therefore costs you nothing on the Rheumatology board.
@@ -83,6 +84,6 @@ modality or therapeutic area, or work with us on the methodology? The
83
  ## Links
84
 
85
  - 🌐 **Website**: [primomics.org](http://primomics.org/)
86
- - 📦 **Data**: [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo)
87
  - 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
88
  - 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
 
41
  ## How the results are organised
42
 
43
  A **board** is a self-contained leaderboard over a slice of the benchmark: the
44
+ whole modality, one therapeutic area, or one task family. Pick one from the rail,
45
+ or on the **Boards** page.
46
 
47
  A board ranks only the models that covered **all of its tasks**. Skipping
48
  Dermatology therefore costs you nothing on the Rheumatology board.
 
84
  ## Links
85
 
86
  - 🌐 **Website**: [primomics.org](http://primomics.org/)
87
+ - 📦 **Data**: [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo)
88
  - 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
89
  - 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
pages/contribute.md CHANGED
@@ -1,6 +1,6 @@
1
- PRIMO is an open benchmark, and it is **deliberately incomplete**. The strip at
2
- the top of this page says exactly how far it reaches today. Everything past that
3
- line still needs somebody to bring it.
4
 
5
  PRIMO was started by **Scienta Lab**. The intent is community governance; how
6
  that actually works is part of what the first partners will define.
@@ -33,8 +33,8 @@ much as data does.
33
 
34
  ## Add a modality or a therapeutic area
35
 
36
- The **OPEN** cards on Home mark the omics layers and clinical areas PRIMO does
37
- not reach yet. Read them as gaps waiting for someone rather than as a roadmap we
38
  are already working through. Opening one takes a single cohort with clinical
39
  labels, and not a whole atlas.
40
 
@@ -53,4 +53,4 @@ methodology, co-signature on what we publish, and early access to the results.
53
  - **Question the method.** Open a thread in the Space's **Community** tab.
54
  - **Submit a model.** Baselines are on the leaderboard already; beat them.
55
  - **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
56
- - **Look at the data**: [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo)
 
1
+ PRIMO is an open benchmark, and it is **deliberately incomplete**. The **Boards**
2
+ page shows exactly how far it reaches today the live boards — and names the
3
+ gaps as open cards. Everything past that still needs somebody to bring it.
4
 
5
  PRIMO was started by **Scienta Lab**. The intent is community governance; how
6
  that actually works is part of what the first partners will define.
 
33
 
34
  ## Add a modality or a therapeutic area
35
 
36
+ The **OPEN** cards on the Boards page mark the omics layers and clinical areas
37
+ PRIMO does not reach yet. Read them as gaps waiting for someone rather than as a roadmap we
38
  are already working through. Opening one takes a single cohort with clinical
39
  labels, and not a whole atlas.
40
 
 
53
  - **Question the method.** Open a thread in the Space's **Community** tab.
54
  - **Submit a model.** Baselines are on the leaderboard already; beat them.
55
  - **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
56
+ - **Look at the data**: [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo)
pages/submit.md CHANGED
@@ -15,7 +15,7 @@ python quickstart.py --out submission.parquet
15
 
16
  Or do it by hand, in **three steps**:
17
 
18
- 1. **Get the data** → download the datasets from [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo) (start with its `datasets.yaml`).
19
  2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
20
  3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
21
 
 
15
 
16
  Or do it by hand, in **three steps**:
17
 
18
+ 1. **Get the data** → download the datasets from [PRIMOmics/primo](https://huggingface.co/datasets/PRIMOmics/primo) (start with its `datasets.yaml`).
19
  2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
20
  3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
21
 
primo.css ADDED
@@ -0,0 +1,216 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* PRIMO benchmark platform — Gradio stylesheet
2
+ Derived from the Scienta design system + primomics.org.
3
+ Load with: gr.Blocks(css_paths=["primo.css"])
4
+ Every value here is a design-system token; do not invent new ones. */
5
+
6
+ /* ---------- fonts ---------- */
7
+ @font-face { font-family: "Funnel Display"; font-weight: 500 700; font-display: swap; src: url("/gradio_api/file=fonts/funnel-display.woff2") format("woff2"); }
8
+ @font-face { font-family: "Funnel Sans"; font-weight: 400 700; font-display: swap; src: url("/gradio_api/file=fonts/funnel-sans.woff2") format("woff2"); }
9
+ @font-face { font-family: "DM Mono"; font-weight: 400; font-display: swap; src: url("/gradio_api/file=fonts/dm-mono-regular.woff2") format("woff2"); }
10
+
11
+ /* ---------- tokens ---------- */
12
+ :root {
13
+ --marine-50: #e5e9fc; --marine-100: #bcc3ea; --marine-200: #929ed8;
14
+ --marine-400: #3f53b5; --marine-500: #162da3; --marine-700: #0d1b62;
15
+ --coral-400: #fd9878; --coral-500: #fd7e56; --coral-600: #d36745;
16
+ --n-100: #f5f5f5; --n-200: #e5e5e5; --n-300: #d4d4d4;
17
+ --n-400: #a3a3a3; --n-500: #737373; --n-600: #525252; --n-900: #171717;
18
+
19
+ --border: var(--n-200);
20
+ --text-body: var(--n-900);
21
+ --text-secondary: var(--n-600);
22
+ --text-muted: var(--n-500);
23
+ --surface-selected: var(--marine-50);
24
+
25
+ --sidebar: radial-gradient(ellipse 100% 5rem at 50% 100%, rgb(253 126 86 / .35), transparent 70%),
26
+ linear-gradient(45deg, var(--marine-500), var(--marine-700));
27
+ --sidebar-fg: rgb(255 255 255 / .9);
28
+ --sidebar-sub-fg: rgb(255 255 255 / .6);
29
+ --sidebar-accent: linear-gradient(90deg, rgb(253 152 120 / .15), rgb(63 83 181 / .2));
30
+ --sidebar-highlight: rgb(63 83 181 / .2);
31
+ --sidebar-border: var(--marine-700);
32
+
33
+ --font-sans: "Funnel Sans", sans-serif;
34
+ --font-title: "Funnel Display", sans-serif;
35
+ --font-mono: "DM Mono", ui-monospace, monospace;
36
+
37
+ --radius-sm: 4px; --radius-md: 6px; --radius-lg: 8px; --radius-xl: 12px;
38
+ --shadow-xs: 0 1px 2px rgb(9 18 65 / .05);
39
+ --shadow-sm: 0 1px 3px rgb(9 18 65 / .1), 0 1px 2px -1px rgb(9 18 65 / .1);
40
+ }
41
+
42
+ /* ---------- Gradio reset: get its chrome out of the way ---------- */
43
+ .gradio-container { max-width: 100% !important; padding: 0 !important; font-family: var(--font-sans) !important; }
44
+ .gradio-container .prose :where(h1, h2, h3) { font-family: var(--font-title); }
45
+ footer { display: none !important; }
46
+ /* the rail replaces the tab bar (Gradio renamed .tab-nav -> .tab-wrapper) */
47
+ #pm-pages > .tab-nav, #pm-pages > .tab-wrapper { display: none !important; }
48
+ #pm-pages > .tabitem { border: 0 !important; padding: 0 !important; }
49
+ #pm-shell { flex-wrap: nowrap !important; gap: 0 !important; align-items: stretch !important; }
50
+
51
+ /* ---------- rail ---------- */
52
+ #pm-rail {
53
+ width: 272px !important; min-width: 272px !important; flex: 0 0 272px !important;
54
+ background: var(--sidebar); color: var(--sidebar-fg); padding: 8px; gap: 0 !important;
55
+ }
56
+ .pm-brand { display: flex; align-items: center; gap: 8px; height: 3rem; padding: 8px; border-radius: var(--radius-md); color: #fff; text-decoration: none; }
57
+ .pm-brand:hover { background: var(--sidebar-highlight); }
58
+ .pm-brand img { height: 24px; width: auto; padding: 2px; background: #fff; border-radius: var(--radius-sm); object-fit: contain; }
59
+ .pm-brand b { font: 700 1.125rem/1 var(--font-title); letter-spacing: -.04em; }
60
+ .pm-brand span { font: 500 1.25rem/0 var(--font-title); letter-spacing: -.04em; color: var(--sidebar-sub-fg); }
61
+
62
+ .pm-rail-group { padding: 16px 16px 0; }
63
+ .pm-rail-label { display: flex; align-items: center; height: 2.5rem; padding: 8px;
64
+ font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--sidebar-sub-fg); }
65
+
66
+ /* board links are Gradio buttons, restyled */
67
+ #pm-rail .pm-link {
68
+ position: relative; display: flex !important; align-items: center; justify-content: space-between; gap: 8px;
69
+ width: 100%; height: 2rem; min-height: 2rem; padding: 0 8px; margin: 0 0 2px;
70
+ border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
71
+ font: 400 14px/1 var(--font-sans); color: var(--sidebar-sub-fg) !important; text-align: left;
72
+ transition: color .15s ease, background .15s ease;
73
+ }
74
+ #pm-rail .pm-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
75
+ #pm-rail .pm-link::before {
76
+ content: ""; position: absolute; top: 50%; left: 0; height: 50%; width: 2px;
77
+ translate: 0 -50%; background: var(--coral-500); opacity: 0; transition: opacity .15s ease;
78
+ }
79
+ #pm-rail .pm-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; font-weight: 500; }
80
+ #pm-rail .pm-link.pm-active::before { opacity: 1; }
81
+ .pm-link .pm-count { font: 400 12px/1 var(--font-mono); opacity: .75; }
82
+ .pm-link .pm-label { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
83
+
84
+ .pm-rail-foot { margin-top: auto; padding: 16px; border-top: 1px solid var(--sidebar-border); }
85
+ #pm-rail .pm-foot-link {
86
+ display: flex !important; align-items: center; width: 100%; height: 2.5rem; padding: 8px; margin: 0 0 2px;
87
+ border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
88
+ font: 500 14px/1 var(--font-sans); color: var(--sidebar-fg) !important; text-align: left;
89
+ }
90
+ #pm-rail .pm-foot-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
91
+ #pm-rail .pm-foot-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; }
92
+
93
+ /* ---------- page shell ---------- */
94
+ #pm-main { min-width: 0; background: #fff; }
95
+ .pm-head { display: flex; align-items: center; justify-content: space-between; gap: 24px; padding: 20px 32px; border-bottom: 1px solid var(--border); }
96
+ .pm-head h1 { margin: 0; font: 600 1.5rem/1.1 var(--font-title); letter-spacing: -.04em; color: var(--marine-700); }
97
+ .pm-head p { margin: 6px 0 0; font-size: 14px; color: var(--text-secondary); }
98
+ .pm-body { padding: 24px 32px 32px; }
99
+ .pm-prose { max-width: 860px; }
100
+ .pm-over { margin: 0; font: 400 10px/1.4 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted); }
101
+ .pm-over--marine { color: var(--marine-500); }
102
+ h2.pm-h2 { margin: 0; font: 600 1.125rem/1.2 var(--font-title); letter-spacing: -.03em; color: var(--marine-700); }
103
+ .pm-group { margin-bottom: 28px; }
104
+ .pm-group-head { display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 12px; }
105
+ .pm-group-head .pm-note { margin: 0; font-size: 12px; color: var(--text-muted); }
106
+ .pm-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px; }
107
+
108
+ /* ---------- cards ---------- */
109
+ .pm-card { display: flex; flex-direction: column; border: 1px solid var(--border); border-radius: var(--radius-xl);
110
+ background: #fff; padding: 20px; box-shadow: var(--shadow-xs); }
111
+ .pm-card--live { border-left: 4px solid var(--marine-500); }
112
+ .pm-card--open { border-style: dashed; border-color: var(--marine-200); box-shadow: none; }
113
+ .pm-card--note { background: var(--surface-selected); border-color: var(--marine-100); }
114
+ .pm-card h3 { margin: 0; font: 600 1.25rem/1.1 var(--font-title); letter-spacing: -.04em; color: var(--marine-700); }
115
+ .pm-card--open h3 { color: var(--text-secondary); }
116
+ .pm-card .pm-meta { margin: 10px 0 0; font: 400 12px/1.5 var(--font-mono); color: var(--text-muted); }
117
+ .pm-card .pm-asks { margin: 8px 0 0; font-size: 14px; color: var(--text-secondary); }
118
+ .pm-card .pm-leader { margin: 6px 0 0; font-size: 14px; color: var(--text-body); }
119
+ .pm-card .pm-leader--2 { margin-top: 2px; color: var(--text-muted); }
120
+ .pm-card p.pm-cta { margin: 14px 0 0; font-size: 14px; }
121
+ a { color: var(--marine-500); text-decoration: none; }
122
+ a:hover { color: var(--marine-700); }
123
+
124
+ /* ---------- badges ---------- */
125
+ .pm-badge { display: inline-flex; align-items: center; height: 20px; padding: 0 6px; border: 1px solid transparent;
126
+ border-radius: var(--radius-md); font: 400 12px/1 var(--font-sans); white-space: nowrap; }
127
+ .pm-badge--marine { background: var(--marine-50); color: var(--marine-500); }
128
+ .pm-badge--quiet { background: var(--n-100); border-color: var(--border); color: var(--text-secondary); }
129
+ .pm-link .pm-badge--quiet { background: transparent; border-color: var(--sidebar-border); color: var(--sidebar-sub-fg); }
130
+
131
+ /* ---------- buttons (page actions) ---------- */
132
+ #pm-main .pm-btn { display: inline-flex !important; align-items: center; justify-content: center; height: 2rem; min-height: 2rem;
133
+ padding: 0 12px; border-radius: var(--radius-md); font: 500 12px/1 var(--font-sans); border: 1px solid transparent; box-shadow: none; }
134
+ #pm-main .pm-btn--primary { background: var(--marine-400) !important; color: #fff !important; }
135
+ #pm-main .pm-btn--primary:hover { background: var(--marine-500) !important; }
136
+ #pm-main .pm-btn--outline { background: #fff !important; border-color: var(--border) !important; color: var(--marine-700) !important; box-shadow: var(--shadow-xs); }
137
+ #pm-main .pm-btn--outline:hover { background: var(--n-100) !important; }
138
+ #pm-main .pm-btn--block { width: 100%; height: 2.25rem; font-size: 14px; }
139
+
140
+ /* ---------- toggle group (Ranked / Per task) ---------- */
141
+ #pm-main .pm-toggle { display: inline-flex; gap: 2px; padding: 3px; border-radius: var(--radius-lg); background: var(--n-100); margin-bottom: 16px; }
142
+ #pm-main .pm-toggle button { height: 1.75rem; min-height: 1.75rem; padding: 0 14px; border: 0 !important; border-radius: var(--radius-md);
143
+ background: transparent !important; box-shadow: none !important; font: 400 14px/1 var(--font-sans); color: var(--text-muted) !important; }
144
+ #pm-main .pm-toggle button.pm-active { background: #fff !important; color: var(--marine-400) !important; box-shadow: var(--shadow-sm) !important; font-weight: 500; }
145
+
146
+ /* ---------- tables ---------- */
147
+ .pm-table-wrap { width: 100%; overflow-x: auto; border: 1px solid var(--border); border-radius: var(--radius-md); }
148
+ .pm-table { width: 100%; border-collapse: collapse; table-layout: fixed; font-size: 14px; }
149
+ .pm-table thead { background: var(--n-100); }
150
+ .pm-table thead tr, .pm-table tbody tr { border-bottom: 1px solid var(--border); }
151
+ .pm-table tbody tr:last-child { border-bottom: 0; }
152
+ .pm-table th { height: 2.5rem; padding: 0 8px; text-align: left; vertical-align: middle;
153
+ font: 400 10px/1.3 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted); }
154
+ .pm-table td { padding: 8px; vertical-align: middle; color: var(--text-secondary); }
155
+ .pm-table th:first-child, .pm-table td:first-child { padding-left: 16px; }
156
+ .pm-table th:last-child, .pm-table td:last-child { padding-right: 16px; }
157
+ .pm-table td.pm-strong { color: var(--text-body); }
158
+ .pm-table td.pm-name { font-weight: 500; color: var(--marine-700); }
159
+ .pm-table .pm-num { text-align: right; font-family: var(--font-mono); font-size: 13px; color: var(--text-body); }
160
+ .pm-table .pm-num--dim { color: var(--n-400); }
161
+ .pm-table .pm-score { text-align: right; font-family: var(--font-mono); font-size: .9375rem; color: var(--marine-700); }
162
+ .pm-table .pm-metric { font-family: var(--font-mono); font-size: 12px; color: var(--marine-700); }
163
+ .pm-table .pm-groupline td { padding-top: 22px; border-bottom: 0;
164
+ font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--marine-700); }
165
+ .pm-table .pm-invite td { text-align: center; color: var(--text-muted); }
166
+ /* definition table (Method) */
167
+ .pm-table--def td:first-child { width: 190px; background: var(--n-100);
168
+ font: 400 11px/1.4 var(--font-mono); letter-spacing: .06em; text-transform: uppercase; color: var(--text-secondary); }
169
+ .pm-caption { margin: 14px 0 0; font-size: 14px; color: var(--text-muted); }
170
+
171
+ /* ---------- submit steps + code ---------- */
172
+ .pm-step { display: flex; gap: 16px; }
173
+ .pm-step-n { flex: none; width: 26px; height: 26px; display: grid; place-content: center;
174
+ border-radius: var(--radius-md); background: var(--marine-500); color: #fff; font: 400 12px/1 var(--font-mono); }
175
+ .pm-step p { max-width: 60ch; margin: 8px 0 0; font-size: 14px; line-height: 1.6; color: var(--text-secondary); }
176
+ .pm-code { margin: 14px 0 0; padding: 16px 18px; border-radius: var(--radius-md);
177
+ background: var(--n-900); color: var(--n-100); font: 400 12px/1.7 var(--font-mono); overflow-x: auto; }
178
+ code, .pm-inline-code { font-family: var(--font-mono); font-size: .95em; }
179
+
180
+ /* ---------- submit form (Gradio inputs) ---------- */
181
+ #pm-form { display: flex; flex-direction: column; gap: 14px; }
182
+ #pm-form .block { padding: 0 !important; border: 0 !important; background: transparent !important; }
183
+ #pm-form label > span { font-size: 13px !important; font-weight: 400 !important; color: var(--text-secondary) !important; }
184
+ #pm-form input[type="text"], #pm-form textarea {
185
+ border: 1px solid var(--border) !important; border-radius: var(--radius-md) !important;
186
+ box-shadow: var(--shadow-xs) !important; font-family: var(--font-sans) !important; font-size: 14px !important; }
187
+ #pm-form input[type="text"]:focus, #pm-form textarea:focus { outline: 3px solid rgb(63 83 181 / .5); outline-offset: 0; }
188
+ #pm-form .pm-hint { margin: 6px 0 0; font-size: 12px; color: var(--text-muted); }
189
+ #pm-form .upload-container, #pm-form .file-preview { border: 1px dashed var(--marine-200) !important;
190
+ border-radius: var(--radius-md) !important; background: #fff !important; }
191
+
192
+ /* ---------- routes (Contribute) ---------- */
193
+ .pm-route { display: block; padding: 20px; margin-bottom: 12px; border: 1px solid var(--border);
194
+ border-radius: var(--radius-xl); box-shadow: var(--shadow-xs); text-decoration: none; transition: border-color .15s ease, box-shadow .15s ease; }
195
+ .pm-route:hover { border-color: var(--marine-400); box-shadow: var(--shadow-sm); }
196
+ .pm-route-head { display: flex; align-items: baseline; justify-content: space-between; gap: 16px; }
197
+ .pm-route-cta { font: 400 12px/1 var(--font-mono); color: var(--coral-600); white-space: nowrap; }
198
+ .pm-route p { max-width: 70ch; margin: 8px 0 0; font-size: 14px; line-height: 1.6; color: var(--text-secondary); }
199
+ .pm-logos { display: flex; flex-wrap: wrap; gap: 10px; }
200
+ .pm-logo { height: 64px; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 3px;
201
+ padding: 0 26px; border: 1px solid var(--border); border-radius: var(--radius-lg); }
202
+ .pm-logo img { max-width: 132px; max-height: 22px; object-fit: contain; }
203
+ .pm-logo--hf img { max-height: 35px; }
204
+ .pm-logo--open { border-style: dashed; border-color: var(--marine-200); }
205
+
206
+ /* ---------- rail inner wrapper (keeps the foot links at the bottom) ---------- */
207
+ #pm-rail .pm-rail-inner { display: flex; flex-direction: column; min-height: 100%; }
208
+ .pm-leader .pm-num--dim { font-family: var(--font-mono); font-size: 12px; color: var(--text-muted); }
209
+
210
+ /* ---------- responsive: rail collapses under 900px ---------- */
211
+ @media (max-width: 900px) {
212
+ #pm-shell { flex-wrap: wrap !important; }
213
+ #pm-rail { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
214
+ .pm-grid { grid-template-columns: 1fr 1fr; }
215
+ .pm-head { flex-direction: column; align-items: flex-start; }
216
+ }
quickstart.py CHANGED
@@ -22,7 +22,7 @@ import yaml
22
  from huggingface_hub import snapshot_download
23
  from sklearn.decomposition import PCA
24
 
25
- PUBLIC_REPO = "ScientaLab/primo"
26
  MANIFEST_FILENAME = "datasets.yaml"
27
 
28
  DATASET_ID = "dataset_id"
 
22
  from huggingface_hub import snapshot_download
23
  from sklearn.decomposition import PCA
24
 
25
+ PUBLIC_REPO = "PRIMOmics/primo"
26
  MANIFEST_FILENAME = "datasets.yaml"
27
 
28
  DATASET_ID = "dataset_id"
render.py ADDED
@@ -0,0 +1,345 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The PRIMO pages, rendered as ``pm-*`` markup for ``gr.HTML`` blocks.
2
+
3
+ Gradio has no card or rich-table component, so every page here is a string of
4
+ HTML styled entirely by ``primo.css`` -- the rail, the board grid, the
5
+ leaderboard tables and the tasks table. The functions are pure: they take the
6
+ data the app already fetched (``Board`` objects, the results ``DataFrame``, the
7
+ registry) and return a string, so they unit-test without Gradio or a network.
8
+
9
+ Two rules hold everywhere:
10
+
11
+ * Every string that comes from the registry or from a submission goes through
12
+ ``html.escape`` -- board names, blurbs, disease names and, above all, the
13
+ user-chosen model names that become table cells and column headers.
14
+ * The private ``hf_username`` is never rendered. It is a name-ownership lock in
15
+ the results dataset, not a public credit, so no table here carries a
16
+ "submitted by" column.
17
+
18
+ Navigation is reload-based: the rail is a column of ``<a href="?board=slug">``
19
+ and ``<a href="?tab=name">`` links, which costs a page load per click but buys
20
+ shareable per-board URLs and needs no JavaScript, matching how the board cards
21
+ have always worked.
22
+ """
23
+
24
+ import math
25
+ import numbers
26
+ from html import escape
27
+
28
+ import pandas as pd
29
+ from boards import (
30
+ AREA_GROUP,
31
+ CATEGORY_GROUP,
32
+ GROUP_NOTE,
33
+ MODALITY_GROUP,
34
+ Board,
35
+ OpenBoard,
36
+ featured,
37
+ in_group,
38
+ metric_label,
39
+ open_in_group,
40
+ )
41
+ from leaderboard import per_task_table, ranked_table, tasks_table, top_models
42
+
43
+ SECTIONS = (MODALITY_GROUP, AREA_GROUP, CATEGORY_GROUP)
44
+ N_TOP_MODELS = 3
45
+
46
+ BRAND = (
47
+ '<a class="pm-brand" target="_self" href="?tab=boards">'
48
+ '<img src="/gradio_api/file=assets/primo-mark.webp" alt="">'
49
+ "<b>PRIMO</b><span>benchmark</span></a>"
50
+ )
51
+
52
+ FOOT_LINKS = (
53
+ ("tasks", "Tasks"),
54
+ ("submit", "Submit a model"),
55
+ ("contribute", "Contribute"),
56
+ ("method", "Method"),
57
+ )
58
+
59
+ NAME_COLUMNS = frozenset({"Model", "Best"})
60
+ STRONG_COLUMNS = frozenset({"Task"})
61
+ METRIC_COLUMNS = frozenset({"Metric"})
62
+ RIGHT_COLUMNS = frozenset({"Rank", "Patients"})
63
+
64
+
65
+ # --------------------------------------------------------------------- rail
66
+ def rail_html(boards: list[Board], active_slug: str | None, active_tab: str | None) -> str:
67
+ """The left navigation: brand, one link per board grouped by facet, foot links.
68
+
69
+ ``active_slug`` highlights the board a visitor is on; ``active_tab`` highlights
70
+ a foot link. Open boards link to Contribute, mirroring the overview cards.
71
+ """
72
+ parts = ['<div class="pm-rail-inner">', BRAND]
73
+ for group in SECTIONS:
74
+ cards, opens = in_group(boards, group), open_in_group(boards, group)
75
+ if not cards and not opens:
76
+ continue
77
+ parts.append(
78
+ f'<div class="pm-rail-group"><p class="pm-rail-label">{escape(group)}</p></div>'
79
+ )
80
+ for board in cards:
81
+ cls = "pm-link pm-active" if board.slug == active_slug else "pm-link"
82
+ parts.append(
83
+ f'<a class="{cls}" target="_self" href="?board={escape(board.slug)}">'
84
+ f'<span class="pm-label">{escape(board.name)}</span>'
85
+ f'<span class="pm-count">{board.n_tasks}</span></a>'
86
+ )
87
+ for board in opens:
88
+ parts.append(
89
+ '<a class="pm-link" target="_self" href="?tab=contribute">'
90
+ f'<span class="pm-label">{escape(board.name)}</span>'
91
+ '<span class="pm-badge pm-badge--quiet">open</span></a>'
92
+ )
93
+ parts.append('<div class="pm-rail-foot">')
94
+ for tab, label in FOOT_LINKS:
95
+ cls = "pm-foot-link pm-active" if tab == active_tab else "pm-foot-link"
96
+ parts.append(
97
+ f'<a class="{cls}" target="_self" href="?tab={tab}">{escape(label)}</a>'
98
+ )
99
+ parts.append("</div></div>")
100
+ return "".join(parts)
101
+
102
+
103
+ # ------------------------------------------------------------------ boards
104
+ def _leading(top) -> str:
105
+ """The card's mini-ranking, with the same empty/baseline states as before.
106
+
107
+ An empty board says "be the first"; a board held only by our baselines says
108
+ "beat the baseline" instead, because "be the first" misleads once a PCA
109
+ already holds a score.
110
+ """
111
+ parts = ['<p class="pm-over" style="margin-top:14px">Leading</p>']
112
+ if not top:
113
+ parts.append(
114
+ '<p class="pm-leader pm-leader--2">No ranked model yet. Be the first.</p>'
115
+ )
116
+ return "".join(parts)
117
+ for row in top:
118
+ badge = (
119
+ ' <span class="pm-badge pm-badge--marine">baseline</span>'
120
+ if row.is_baseline
121
+ else ""
122
+ )
123
+ score = "n/a" if row.score is None else f"{row.score:.3f}"
124
+ parts.append(
125
+ f'<p class="pm-leader">{escape(row.name)}{badge}'
126
+ f' <span class="pm-num--dim">{score}</span></p>'
127
+ )
128
+ if all(row.is_baseline for row in top):
129
+ parts.append(
130
+ '<p class="pm-leader pm-leader--2">No submitted model yet. '
131
+ "Beat the baseline.</p>"
132
+ )
133
+ return "".join(parts)
134
+
135
+
136
+ def _live_card(board: Board, df: pd.DataFrame, by_id: dict[str, dict]) -> str:
137
+ top = top_models(df, by_id, board, N_TOP_MODELS)
138
+ return (
139
+ f'<a class="pm-card pm-card--live" target="_self" href="?board={escape(board.slug)}">'
140
+ f"<h3>{escape(board.name)}</h3>"
141
+ f'<p class="pm-meta">{board.n_tasks} tasks · {board.n_cohorts} cohorts<br>'
142
+ f"{board.n_patients:,} patients · {board.n_diseases} diseases</p>"
143
+ f'<p class="pm-asks">{escape(board.blurb)}</p>'
144
+ f"{_leading(top)}</a>"
145
+ )
146
+
147
+
148
+ def _open_card(board: OpenBoard) -> str:
149
+ return (
150
+ '<a class="pm-card pm-card--open" target="_self" href="?tab=contribute">'
151
+ f"<h3>{escape(board.name)}</h3>"
152
+ '<p class="pm-meta"><span class="pm-badge pm-badge--quiet">OPEN · no cohort yet</span></p>'
153
+ f'<p class="pm-asks">{escape(board.blurb)}</p>'
154
+ '<p class="pm-cta">Propose a cohort →</p></a>'
155
+ )
156
+
157
+
158
+ def render_boards(boards: list[Board], df: pd.DataFrame, by_id: dict[str, dict]) -> str:
159
+ """The Boards overview: every board as a card, grouped by facet.
160
+
161
+ Live cards link to their board and teaser their leaders; open cards state a
162
+ gap and link to Contribute. A section counts its open slices so the page
163
+ reads as showing its own gaps, not the registry as the whole territory.
164
+ """
165
+ if not boards:
166
+ return (
167
+ '<div class="pm-body"><p class="pm-caption">The task registry is '
168
+ "unavailable right now. Please retry in a moment.</p></div>"
169
+ )
170
+ n_live = len(boards)
171
+ n_open = sum(len(open_in_group(boards, group)) for group in SECTIONS)
172
+ out = [
173
+ '<div class="pm-head"><div><h1>Boards</h1>',
174
+ f"<p>{n_live} live · {n_open} open. Every board is scored by the same "
175
+ "fixed linear probe on labels we never release.</p></div></div>",
176
+ '<div class="pm-body">',
177
+ ]
178
+ for group in SECTIONS:
179
+ cards, opens = in_group(boards, group), open_in_group(boards, group)
180
+ if not cards and not opens:
181
+ continue
182
+ note = escape(GROUP_NOTE.get(group, ""))
183
+ counter = f' · +{len(opens)} open' if opens else ""
184
+ out.append(
185
+ '<div class="pm-group"><div class="pm-group-head">'
186
+ f'<p class="pm-over">{escape(group)}</p>'
187
+ f'<p class="pm-note">{note}{counter}</p></div><div class="pm-grid">'
188
+ )
189
+ out += [_live_card(b, df, by_id) for b in cards]
190
+ out += [_open_card(b) for b in opens]
191
+ out.append("</div></div>")
192
+ out.append("</div>")
193
+ return "".join(out)
194
+
195
+
196
+ # ------------------------------------------------------------- html tables
197
+ def _fmt_value(value, is_score: bool) -> str | None:
198
+ """Display text for one cell; ``None`` marks a blank (``n/a``) cell."""
199
+ try:
200
+ if pd.isna(value):
201
+ return None
202
+ except (TypeError, ValueError):
203
+ pass
204
+ if isinstance(value, str):
205
+ return value
206
+ if isinstance(value, numbers.Integral):
207
+ return str(int(value))
208
+ if isinstance(value, numbers.Real):
209
+ if not math.isfinite(float(value)):
210
+ return None
211
+ return f"{float(value):.3f}" if is_score else str(value)
212
+ return str(value)
213
+
214
+
215
+ def _score_columns(df: pd.DataFrame) -> list[str]:
216
+ """Numeric columns to format and bold -- ``Rank`` is an index, not a score."""
217
+ return [c for c in df.select_dtypes("number").columns if c not in RIGHT_COLUMNS]
218
+
219
+
220
+ def _bold_cells(df: pd.DataFrame, score_cols: list[str], axis: int) -> set:
221
+ """Which ``(row, col)`` cells hold the best value.
222
+
223
+ ``axis=0`` bolds the best model per column (the ranked table, read down);
224
+ ``axis=1`` bolds the best model per row (the per-task table, read across).
225
+ """
226
+ bold = set()
227
+ if not score_cols:
228
+ return bold
229
+ if axis == 0:
230
+ for col in score_cols:
231
+ best = df[col].max(skipna=True)
232
+ if pd.notna(best):
233
+ for idx, value in df[col].items():
234
+ if pd.notna(value) and value == best:
235
+ bold.add((idx, col))
236
+ else:
237
+ for idx, row in df.iterrows():
238
+ present = {c: row[c] for c in score_cols if pd.notna(row[c])}
239
+ if present:
240
+ best = max(present.values())
241
+ bold.update((idx, c) for c, v in present.items() if v == best)
242
+ return bold
243
+
244
+
245
+ def _cell_class(col: str, is_score: bool, blank: bool) -> str:
246
+ if col in NAME_COLUMNS:
247
+ return "pm-name"
248
+ if col in STRONG_COLUMNS:
249
+ return "pm-strong"
250
+ if col in METRIC_COLUMNS:
251
+ return "pm-metric"
252
+ if is_score or col in RIGHT_COLUMNS:
253
+ return "pm-num pm-num--dim" if blank else "pm-num"
254
+ return ""
255
+
256
+
257
+ def _df_to_table(df: pd.DataFrame, bold_axis: int | None, empty: str) -> str:
258
+ """Render a DataFrame as a ``pm-table``, escaping every header and cell."""
259
+ if df.empty:
260
+ return f'<p class="pm-caption">{escape(empty)}</p>'
261
+ score_cols = _score_columns(df)
262
+ bold = _bold_cells(df, score_cols, bold_axis) if bold_axis is not None else set()
263
+
264
+ head = []
265
+ for col in df.columns:
266
+ align = ' style="text-align:right"' if col in score_cols or col in RIGHT_COLUMNS else ""
267
+ head.append(f"<th{align}>{escape(str(col))}</th>")
268
+ body = []
269
+ for idx, row in df.iterrows():
270
+ cells = []
271
+ for col in df.columns:
272
+ is_score = col in score_cols
273
+ text = _fmt_value(row[col], is_score)
274
+ cls = _cell_class(col, is_score, text is None)
275
+ weight = ' style="font-weight:700"' if (idx, col) in bold else ""
276
+ shown = "n/a" if text is None else escape(text)
277
+ cells.append(f'<td class="{cls}"{weight}>{shown}</td>')
278
+ body.append(f"<tr>{''.join(cells)}</tr>")
279
+ return (
280
+ '<div class="pm-table-wrap"><table class="pm-table" style="table-layout:auto">'
281
+ f"<thead><tr>{''.join(head)}</tr></thead>"
282
+ f"<tbody>{''.join(body)}</tbody></table></div>"
283
+ )
284
+
285
+
286
+ # ------------------------------------------------------------------- board
287
+ def _board_meta(board: Board) -> str:
288
+ metrics = " · ".join(metric_label(m) for m in board.metrics)
289
+ tail = f" · {metrics}" if metrics else ""
290
+ return (
291
+ f"{escape(board.blurb)} · {board.n_tasks} tasks · {board.n_cohorts} cohorts · "
292
+ f"{board.n_patients:,} patients · {escape(board.modality)}{tail}"
293
+ )
294
+
295
+
296
+ def render_board(board: Board | None, df: pd.DataFrame, by_id: dict[str, dict]) -> str:
297
+ """One board: title strip, the ranked leaderboard, then the per-task table."""
298
+ if board is None:
299
+ return (
300
+ '<div class="pm-head"><div><h1>No board available</h1>'
301
+ "<p>The task registry could not be loaded. Please retry shortly.</p>"
302
+ "</div></div>"
303
+ )
304
+ ranked = _df_to_table(
305
+ ranked_table(df, by_id, board),
306
+ bold_axis=0,
307
+ empty="No model has covered every task of this board yet. Be the first to submit.",
308
+ )
309
+ per_task = _df_to_table(
310
+ per_task_table(df, by_id, board),
311
+ bold_axis=1,
312
+ empty="No submission has scored on this board yet.",
313
+ )
314
+ return (
315
+ '<div class="pm-head"><div><p class="pm-over">Board</p>'
316
+ f"<h1>{escape(board.name)}</h1><p>{_board_meta(board)}</p></div></div>"
317
+ '<div class="pm-body">'
318
+ '<div class="pm-group"><div class="pm-group-head">'
319
+ '<p class="pm-over pm-over--marine">Ranked</p>'
320
+ f'<p class="pm-note">Only models that covered all {board.n_tasks} tasks are '
321
+ "ranked. Mean averages the family columns and mixes metrics — a tie-break, "
322
+ "not a score.</p>"
323
+ f"</div>{ranked}</div>"
324
+ '<div class="pm-group"><div class="pm-group-head">'
325
+ '<p class="pm-over pm-over--marine">Per task</p>'
326
+ '<p class="pm-note">Every submission, partial ones included — read across a row.</p>'
327
+ f"</div>{per_task}</div>"
328
+ "</div>"
329
+ )
330
+
331
+
332
+ # ------------------------------------------------------------------- tasks
333
+ def render_tasks(by_id: dict[str, dict]) -> str:
334
+ """The Tasks page: one scannable row per hidden target. Provenance is never shown."""
335
+ df = tasks_table(list(by_id.values()))
336
+ table = _df_to_table(
337
+ df, bold_axis=None, empty="The task registry is unavailable right now."
338
+ )
339
+ return (
340
+ '<div class="pm-head"><div><h1>Tasks</h1>'
341
+ f"<p>{len(df)} hidden clinical targets. One fixed linear probe reads each "
342
+ "one out of your embedding; the cohorts stay anonymous, the biology does "
343
+ "not.</p></div></div>"
344
+ f'<div class="pm-body">{table}</div>'
345
+ )
style.css DELETED
@@ -1,444 +0,0 @@
1
- @import url("https://fonts.googleapis.com/css2?family=Funnel+Display:wght@400..800&display=swap");
2
-
3
- /* PRIMO charter, dark. Deep Navy is the charter's own dark value, so the Space
4
- stays on-brand while reading like a leaderboard should. Slate and Cyan swap
5
- roles from the light build: on navy, raw Cyan clears 7:1 and carries text,
6
- while Slate is too dark to read and is lightened for muted copy. */
7
- :root {
8
- --primo-navy: #080f5f;
9
- --primo-deep-navy: #050a3c;
10
- --primo-cyan: #16b3c0;
11
- --primo-paper: #f3f8f8;
12
-
13
- --primo-bg: var(--primo-deep-navy);
14
- --primo-card: #0a1049;
15
- --primo-card-hover: #0e1663;
16
- --primo-line: #1e2775;
17
- --primo-text: var(--primo-paper);
18
- --primo-muted: #9fb3b7;
19
- --primo-dim: #74898e;
20
- --primo-modality: var(--primo-cyan);
21
- --primo-area: #8b99f7;
22
- --primo-category: #e0b341;
23
- --primo-featured: var(--primo-cyan);
24
- --primo-wash: rgba(243, 248, 248, 0.06);
25
- }
26
-
27
- .primo-home h2,
28
- .primo-home h3,
29
- .primo-board-head h2,
30
- .prose h1,
31
- .prose h2,
32
- .prose h3 {
33
- font-family: "Funnel Display", "Funnel Sans", ui-sans-serif, system-ui, sans-serif;
34
- letter-spacing: -0.015em;
35
- }
36
-
37
- .primo-banner {
38
- display: flex;
39
- align-items: center;
40
- gap: 10px;
41
- flex-wrap: wrap;
42
- font-size: 0.82rem;
43
- color: var(--primo-muted);
44
- border: 1px solid var(--primo-line);
45
- border-radius: 10px;
46
- padding: 9px 14px;
47
- margin-bottom: 4px;
48
- }
49
-
50
- .primo-banner-tag {
51
- font-size: 0.7rem;
52
- font-weight: 700;
53
- letter-spacing: 0.06em;
54
- text-transform: uppercase;
55
- color: var(--primo-featured);
56
- background: var(--primo-wash);
57
- border: 1px solid rgba(22, 179, 192, 0.3);
58
- border-radius: 999px;
59
- padding: 2px 9px;
60
- }
61
-
62
- .primo-home {
63
- max-width: 1400px;
64
- margin: 0 auto;
65
- padding: 8px 4px 32px;
66
- color: var(--primo-text);
67
- }
68
-
69
- .primo-title-row {
70
- display: flex;
71
- align-items: baseline;
72
- justify-content: space-between;
73
- gap: 16px;
74
- flex-wrap: wrap;
75
- margin-bottom: 20px;
76
- }
77
-
78
- .primo-title-row h2 {
79
- font-size: 2rem;
80
- font-weight: 700;
81
- letter-spacing: -0.02em;
82
- margin: 0;
83
- color: var(--primo-text);
84
- }
85
-
86
- .primo-hint {
87
- font-size: 0.85rem;
88
- color: var(--primo-muted);
89
- }
90
-
91
- .primo-chip {
92
- display: inline-block;
93
- font-size: 0.7rem;
94
- font-weight: 600;
95
- letter-spacing: 0.08em;
96
- color: var(--primo-featured);
97
- background: var(--primo-wash);
98
- border: 1px solid rgba(22, 179, 192, 0.3);
99
- border-radius: 999px;
100
- padding: 4px 12px;
101
- margin-bottom: 12px;
102
- }
103
-
104
- .primo-grid {
105
- display: grid;
106
- grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
107
- gap: 16px;
108
- }
109
-
110
- .primo-grid-featured {
111
- grid-template-columns: repeat(auto-fit, minmax(320px, 1fr));
112
- margin-bottom: 32px;
113
- }
114
-
115
- .primo-card {
116
- display: block;
117
- position: relative;
118
- background: var(--primo-card);
119
- border: 1px solid var(--primo-line);
120
- border-left: 3px solid var(--primo-line);
121
- border-radius: 12px;
122
- padding: 16px 18px;
123
- text-decoration: none !important;
124
- color: inherit;
125
- transition: box-shadow 0.15s ease, border-color 0.15s ease, transform 0.15s ease;
126
- }
127
-
128
- .primo-card:hover {
129
- background: var(--primo-card-hover);
130
- transform: translateY(-2px);
131
- box-shadow: 0 8px 22px rgba(0, 0, 0, 0.45);
132
- }
133
-
134
- .primo-accent-modality {
135
- border-left-color: var(--primo-modality);
136
- }
137
- .primo-accent-area {
138
- border-left-color: var(--primo-area);
139
- }
140
- .primo-accent-category {
141
- border-left-color: var(--primo-category);
142
- }
143
- .primo-accent-modality:hover {
144
- border-color: var(--primo-modality);
145
- }
146
- .primo-accent-area:hover {
147
- border-color: var(--primo-area);
148
- }
149
- .primo-accent-category:hover {
150
- border-color: var(--primo-category);
151
- }
152
-
153
- /* ``align-self`` so an open card sizes to its own text: stretched to match a
154
- full board it becomes a large empty rectangle, which reads as broken. */
155
- .primo-card-open {
156
- align-self: start;
157
- background: transparent;
158
- border-style: dashed;
159
- border-left-color: var(--primo-line);
160
- opacity: 0.72;
161
- }
162
-
163
- .primo-card-open:hover {
164
- opacity: 1;
165
- background: var(--primo-card);
166
- border-color: var(--primo-dim);
167
- }
168
-
169
- .primo-card-open .primo-card-title {
170
- color: var(--primo-muted);
171
- }
172
-
173
- .primo-badge-open {
174
- margin-left: auto;
175
- font-size: 0.6rem;
176
- font-weight: 700;
177
- letter-spacing: 0.08em;
178
- color: var(--primo-dim);
179
- border: 1px solid var(--primo-line);
180
- border-radius: 999px;
181
- padding: 2px 8px;
182
- }
183
-
184
- .primo-open-cta {
185
- font-size: 0.8rem;
186
- font-weight: 600;
187
- color: var(--primo-area);
188
- }
189
-
190
- .primo-count-open {
191
- font-size: 0.7rem;
192
- font-weight: 600;
193
- color: var(--primo-dim);
194
- margin-left: 6px;
195
- }
196
-
197
- .primo-card-head {
198
- display: flex;
199
- align-items: center;
200
- gap: 10px;
201
- margin-bottom: 8px;
202
- }
203
-
204
- /* Replaces the per-board emoji: letters scale to any modality anyone adds, and
205
- the colour is the group's, so a new board needs no design decision. */
206
- .primo-code {
207
- flex: none;
208
- font-size: 0.66rem;
209
- font-weight: 700;
210
- letter-spacing: 0.07em;
211
- line-height: 1;
212
- color: var(--primo-deep-navy);
213
- background: var(--primo-modality);
214
- border-radius: 5px;
215
- padding: 5px 6px;
216
- }
217
-
218
- .primo-code-area {
219
- background: var(--primo-area);
220
- }
221
-
222
- .primo-code-category {
223
- background: var(--primo-category);
224
- }
225
-
226
- .primo-board-head .primo-code {
227
- vertical-align: middle;
228
- display: inline-block;
229
- margin-right: 9px;
230
- }
231
-
232
- .primo-card-title {
233
- font-size: 1.05rem;
234
- font-weight: 650;
235
- color: var(--primo-text);
236
- }
237
-
238
- .primo-card-sub {
239
- font-size: 0.8rem;
240
- color: var(--primo-muted);
241
- margin-bottom: 14px;
242
- }
243
-
244
- .primo-blurb {
245
- font-size: 0.85rem;
246
- line-height: 1.45;
247
- color: var(--primo-muted);
248
- margin: 0 0 14px;
249
- min-height: 2.5em;
250
- }
251
-
252
- .primo-stats {
253
- display: grid;
254
- grid-template-columns: 1fr 1fr;
255
- gap: 10px 12px;
256
- margin-bottom: 14px;
257
- }
258
-
259
- .primo-stat {
260
- display: flex;
261
- flex-direction: column;
262
- gap: 2px;
263
- }
264
-
265
- .primo-stat-label {
266
- font-size: 0.65rem;
267
- letter-spacing: 0.06em;
268
- color: var(--primo-dim);
269
- }
270
-
271
- .primo-stat-value {
272
- font-size: 1.15rem;
273
- font-weight: 600;
274
- color: var(--primo-text);
275
- font-variant-numeric: tabular-nums;
276
- }
277
-
278
- .primo-top {
279
- border-top: 1px solid var(--primo-line);
280
- padding-top: 10px;
281
- margin-bottom: 14px;
282
- }
283
-
284
- .primo-top-head,
285
- .primo-top-row {
286
- display: flex;
287
- justify-content: space-between;
288
- gap: 12px;
289
- padding: 5px 0;
290
- }
291
-
292
- .primo-top-head {
293
- font-size: 0.65rem;
294
- letter-spacing: 0.06em;
295
- color: var(--primo-dim);
296
- }
297
-
298
- .primo-top-row {
299
- font-size: 0.85rem;
300
- border-top: 1px solid var(--primo-line);
301
- font-variant-numeric: tabular-nums;
302
- }
303
-
304
- .primo-top-row span:first-child {
305
- overflow: hidden;
306
- text-overflow: ellipsis;
307
- white-space: nowrap;
308
- }
309
-
310
- .primo-top-empty {
311
- font-size: 0.82rem;
312
- color: var(--primo-dim);
313
- font-style: italic;
314
- }
315
-
316
- .primo-top-baseline span:first-child {
317
- color: var(--primo-muted);
318
- }
319
-
320
- .primo-tag {
321
- font-size: 0.6rem;
322
- letter-spacing: 0.06em;
323
- text-transform: uppercase;
324
- color: var(--primo-dim);
325
- border: 1px solid var(--primo-line);
326
- border-radius: 999px;
327
- padding: 1px 6px;
328
- margin-left: 7px;
329
- }
330
-
331
- .primo-top-foot {
332
- font-size: 0.75rem;
333
- color: var(--primo-dim);
334
- font-style: italic;
335
- border-top: 1px solid var(--primo-line);
336
- padding-top: 7px;
337
- }
338
-
339
- .primo-pills {
340
- display: flex;
341
- flex-wrap: wrap;
342
- gap: 6px;
343
- }
344
-
345
- .primo-pill {
346
- font-size: 0.72rem;
347
- color: var(--primo-muted);
348
- background: var(--primo-wash);
349
- border: 1px solid var(--primo-line);
350
- border-radius: 999px;
351
- padding: 2px 10px;
352
- }
353
-
354
- .primo-pill-modality {
355
- color: var(--primo-modality);
356
- border-color: rgba(22, 179, 192, 0.32);
357
- background: var(--primo-wash);
358
- }
359
-
360
- /* No fill: the cards already lift off the page, and a second surface behind
361
- them only muddies the contrast they rely on. */
362
- .primo-section {
363
- border: 1px solid var(--primo-line);
364
- border-radius: 14px;
365
- padding: 0 0 18px;
366
- margin-bottom: 20px;
367
- background: transparent;
368
- }
369
-
370
- .primo-section-head {
371
- display: flex;
372
- align-items: center;
373
- justify-content: space-between;
374
- gap: 16px;
375
- flex-wrap: wrap;
376
- padding: 16px 20px;
377
- border-bottom: 1px solid var(--primo-line);
378
- margin-bottom: 18px;
379
- }
380
-
381
- .primo-section-head h3 {
382
- font-size: 1.1rem;
383
- font-weight: 650;
384
- margin: 0;
385
- color: var(--primo-text);
386
- }
387
-
388
- .primo-count {
389
- font-size: 0.75rem;
390
- font-weight: 600;
391
- color: var(--primo-muted);
392
- background: var(--primo-wash);
393
- border-radius: 999px;
394
- padding: 2px 9px;
395
- margin-left: 6px;
396
- }
397
-
398
- .primo-section-note {
399
- font-size: 0.82rem;
400
- color: var(--primo-dim);
401
- }
402
-
403
- .primo-section .primo-grid {
404
- padding: 0 20px;
405
- }
406
-
407
- .primo-empty {
408
- color: var(--primo-muted);
409
- font-style: italic;
410
- }
411
-
412
- .primo-board-head {
413
- border-left: 3px solid var(--primo-modality);
414
- padding: 2px 0 2px 14px;
415
- margin-bottom: 4px;
416
- }
417
-
418
- .primo-board-head h2 {
419
- margin: 0 0 4px;
420
- font-size: 1.5rem;
421
- font-weight: 700;
422
- }
423
-
424
- .primo-board-head p {
425
- margin: 0;
426
- color: var(--primo-muted);
427
- font-size: 0.9rem;
428
- }
429
-
430
- /* Gradio's default inline-code chip is light-on-light, unreadable on this
431
- theme. Scoped to :not(pre) so fenced blocks keep their own styling. */
432
- .md :not(pre) > code,
433
- .prose :not(pre) > code {
434
- background: rgba(22, 179, 192, 0.1) !important;
435
- color: #6fe3ec !important;
436
- border: 1px solid rgba(22, 179, 192, 0.3);
437
- border-radius: 5px;
438
- padding: 1px 6px;
439
- font-size: 0.9em;
440
- }
441
-
442
- footer {
443
- display: none !important;
444
- }