oncodsl / Read docs /PROMPT_stability_panel.md
govindbalki's picture
Upload folder using huggingface_hub
0fff343 verified
|
Raw
History Blame Contribute Delete
6.03 kB
# Prompt β€” "Stability across seeds" panel in the Lab (HNSC/HPV)
Add a multi-seed stability panel to the Lab that reruns the HPV search across
several seeds and shows (a) how tightly the result holds and (b) which genes
recur across independent searches. **Frontend-only** β€” reuse the existing run,
transfer, reveal, and full-rank endpoints. No engine / API / airgap change.
`tsc` after.
Gated to `dataset === "hnsc" && target === "hpv"` (hidden otherwise). Everything
this needs already exists on the wire; do not add a backend endpoint.
---
## What it does
For each seed in a list (default `[1, 3, 7, 11, 13, 17, 23, 29]`), fire a normal
HPV run with that seed, wait for it to finish, then pull its held-out result and
its independent-cohort transfer. Aggregate across seeds into one panel.
## Data flow (all existing endpoints)
- Fire a run: `postRun({ objective_spec: OBJECTIVE_PRESETS.hpv, params: {
...params, seed }, engine: "v2", dataset, coherence, diversity, ...ratesDiff })`
β€” i.e. the SAME body `launchRun()` builds, only `params.seed` overridden per
seed. Use the user's current Parameters / coherence / diversity / DSL rates so
the sweep matches their single-run config.
- Wait for completion **without disturbing the main Live view / Result state**.
Write a dedicated helper `awaitRunDone(runId): Promise<RunResult>` that opens
an `EventSource(${API_URL}/runs/${runId}/stream)`, resolves on the `done`
event (parse `RunResult`), rejects on `error`, and closes the stream β€” it must
NOT call `setGenerations` / `setResult` / `setStatus` (those belong to the
main single-run UI). (Polling `getRunStatus` until done then `getRunResult` is
an acceptable alternative.)
- Per seed collect: **`winning.holdout_score`** (this is the field the Result
panel's "Held-out AUROC" card and the verdict display β€” use it, NOT
`holdout_auroc`, so the stability dots match the numbers already on screen) and
`winning.permutation_p` (from the RunResult), then `getRunTransfer(runId)` β†’
`auroc`, `p`, `n`, `found_symbols`, `missing_symbols`. If transfer fails (e.g.
GSE65858 not built), store null for the transfer fields and keep the held-out
row β€” never crash the sweep.
- After all seeds: call `getFullRankDiagnostic(dataset, target)` once and build a
`symbol β†’ set_name` map from `reference_marks` (p16 / cell_cycle) to tag genes.
## Recurrence (the key output)
- A seed's winner symbols come from `postReveal(winning.gene_ids)` β€” a bounded
reveal of just that winner's genes (same discipline `/transfer` uses). Source
it from the winner, not the transfer payload, so recurrence still works even if
a seed's transfer call failed. (`found_symbols βˆͺ missing_symbols` from the
transfer is an equivalent set when transfer succeeds.)
- Tally: for every symbol, count in how many seeds it appears. Sort desc. Tag
each with its reference set via the map (else "alternate"/"β€”"). Genes in only
1 seed collapse into a single "passengers Β· 1 seed each" row (show the count of
such genes).
## UI (new `<StabilityPanel>` β€” match the approved mockup)
Reuse `<SectionCard>`, `<InfoTip>`, `fmtFit`, and the palette tokens so it looks
native. Place it as its own SectionCard in the Result area, after the
independent-cohort validation panel. Gated to hnsc/hpv.
- Header + its own **"Run stability check"** button (independent of the main Run
button) + a seed-list input (comma or space separated, default the 8 above).
While sweeping: disable both this button and the main Run button, show
"Running seed {k} of {N}…", and offer a **Stop** button that sets a cancel
flag checked between seeds (finish the current seed, then halt).
- **Section 1 β€” "Does the result hold across seeds?"** Two horizontal dot-strips
on a 0.5 β†’ 1.0 axis (label "0.5 coin-flip" left, "1.0" right): Held-out AUROC
(accent-teal dots) and Independent AUROC (amber dots), one dot per completed
seed, with a light min–max band behind the dots. Caption with the two ranges
(`min–max`, via `fmtFit`) and a one-line plain read ("tight + high = stable,
not a lucky seed; the independent cohort costs a little, as expected").
- **Section 2 β€” "Do the same genes keep coming back?"** Recurrence bars: symbol
| bar (width = seeds-appeared / N) | "{k} / {N} Β· {set}". Colour by tag:
cell_cycle amber, p16 gold, alternate teal, passengers grey. Caption: recurring
cell-cycle / proliferation genes = real signal; one-off passengers = noise.
- **Section 3 β€” per-seed table**: seed | held-out AUROC | p | independent AUROC |
p | genes found/total. Route every number through `fmtFit` ("β€”" for null).
- Panel `?` (InfoTip): what multi-seed proves β€” that the result isn't one lucky
random start (spread), and that the same biology recurs across independent
searches (recurrence); note each seed is a full run so the sweep takes a while.
## CONSTRAINTS
- Frontend-only. No engine/API/airgap change. Reuse `postRun`, the SSE stream,
`getRunResult`, `getRunTransfer`, `getFullRankDiagnostic`. The only gene names
shown are the winner's revealed symbols (already bounded via `/transfer`) and
the reference marks (bounded known set) β€” no new reveal surface.
- The sweep must not overwrite the main single-run Result / Live view; it keeps
its own state (`stabilityRows`, `stabilityStatus`, `stabilityProgress`,
`stabilityCancel`).
- Runs are sequential (the backend runs one at a time); show progress, allow
Stop. Graceful nulls if a transfer fails.
## Checkpoint
- On an HNSC/HPV run, a "Stability across seeds" panel appears with a "Run
stability check" button; clicking it sweeps the seeds, showing live progress.
- When done: two dot-strips (held-out + independent AUROC, one dot per seed with
the range), a gene-recurrence bar list tagged by reference set, and a per-seed
table. Hidden for colorectal.
- Main single-run Result/Live view is untouched while the sweep runs.
- `tsc` clean; no behaviour change to existing runs; airgap untouched.