oncodsl / Read docs /PROMPT_stability_panel.md
govindbalki's picture
Upload folder using huggingface_hub
0fff343 verified
|
Raw
History Blame Contribute Delete
6.03 kB

Prompt β€” "Stability across seeds" panel in the Lab (HNSC/HPV)

Add a multi-seed stability panel to the Lab that reruns the HPV search across several seeds and shows (a) how tightly the result holds and (b) which genes recur across independent searches. Frontend-only β€” reuse the existing run, transfer, reveal, and full-rank endpoints. No engine / API / airgap change. tsc after.

Gated to dataset === "hnsc" && target === "hpv" (hidden otherwise). Everything this needs already exists on the wire; do not add a backend endpoint.


What it does

For each seed in a list (default [1, 3, 7, 11, 13, 17, 23, 29]), fire a normal HPV run with that seed, wait for it to finish, then pull its held-out result and its independent-cohort transfer. Aggregate across seeds into one panel.

Data flow (all existing endpoints)

  • Fire a run: postRun({ objective_spec: OBJECTIVE_PRESETS.hpv, params: { ...params, seed }, engine: "v2", dataset, coherence, diversity, ...ratesDiff }) β€” i.e. the SAME body launchRun() builds, only params.seed overridden per seed. Use the user's current Parameters / coherence / diversity / DSL rates so the sweep matches their single-run config.
  • Wait for completion without disturbing the main Live view / Result state. Write a dedicated helper awaitRunDone(runId): Promise<RunResult> that opens an EventSource(${API_URL}/runs/${runId}/stream), resolves on the done event (parse RunResult), rejects on error, and closes the stream β€” it must NOT call setGenerations / setResult / setStatus (those belong to the main single-run UI). (Polling getRunStatus until done then getRunResult is an acceptable alternative.)
  • Per seed collect: winning.holdout_score (this is the field the Result panel's "Held-out AUROC" card and the verdict display β€” use it, NOT holdout_auroc, so the stability dots match the numbers already on screen) and winning.permutation_p (from the RunResult), then getRunTransfer(runId) β†’ auroc, p, n, found_symbols, missing_symbols. If transfer fails (e.g. GSE65858 not built), store null for the transfer fields and keep the held-out row β€” never crash the sweep.
  • After all seeds: call getFullRankDiagnostic(dataset, target) once and build a symbol β†’ set_name map from reference_marks (p16 / cell_cycle) to tag genes.

Recurrence (the key output)

  • A seed's winner symbols come from postReveal(winning.gene_ids) β€” a bounded reveal of just that winner's genes (same discipline /transfer uses). Source it from the winner, not the transfer payload, so recurrence still works even if a seed's transfer call failed. (found_symbols βˆͺ missing_symbols from the transfer is an equivalent set when transfer succeeds.)
  • Tally: for every symbol, count in how many seeds it appears. Sort desc. Tag each with its reference set via the map (else "alternate"/"β€”"). Genes in only 1 seed collapse into a single "passengers Β· 1 seed each" row (show the count of such genes).

UI (new <StabilityPanel> β€” match the approved mockup)

Reuse <SectionCard>, <InfoTip>, fmtFit, and the palette tokens so it looks native. Place it as its own SectionCard in the Result area, after the independent-cohort validation panel. Gated to hnsc/hpv.

  • Header + its own "Run stability check" button (independent of the main Run button) + a seed-list input (comma or space separated, default the 8 above). While sweeping: disable both this button and the main Run button, show "Running seed {k} of {N}…", and offer a Stop button that sets a cancel flag checked between seeds (finish the current seed, then halt).
  • Section 1 β€” "Does the result hold across seeds?" Two horizontal dot-strips on a 0.5 β†’ 1.0 axis (label "0.5 coin-flip" left, "1.0" right): Held-out AUROC (accent-teal dots) and Independent AUROC (amber dots), one dot per completed seed, with a light min–max band behind the dots. Caption with the two ranges (min–max, via fmtFit) and a one-line plain read ("tight + high = stable, not a lucky seed; the independent cohort costs a little, as expected").
  • Section 2 β€” "Do the same genes keep coming back?" Recurrence bars: symbol | bar (width = seeds-appeared / N) | "{k} / {N} Β· {set}". Colour by tag: cell_cycle amber, p16 gold, alternate teal, passengers grey. Caption: recurring cell-cycle / proliferation genes = real signal; one-off passengers = noise.
  • Section 3 β€” per-seed table: seed | held-out AUROC | p | independent AUROC | p | genes found/total. Route every number through fmtFit ("β€”" for null).
  • Panel ? (InfoTip): what multi-seed proves β€” that the result isn't one lucky random start (spread), and that the same biology recurs across independent searches (recurrence); note each seed is a full run so the sweep takes a while.

CONSTRAINTS

  • Frontend-only. No engine/API/airgap change. Reuse postRun, the SSE stream, getRunResult, getRunTransfer, getFullRankDiagnostic. The only gene names shown are the winner's revealed symbols (already bounded via /transfer) and the reference marks (bounded known set) β€” no new reveal surface.
  • The sweep must not overwrite the main single-run Result / Live view; it keeps its own state (stabilityRows, stabilityStatus, stabilityProgress, stabilityCancel).
  • Runs are sequential (the backend runs one at a time); show progress, allow Stop. Graceful nulls if a transfer fails.

Checkpoint

  • On an HNSC/HPV run, a "Stability across seeds" panel appears with a "Run stability check" button; clicking it sweeps the seeds, showing live progress.
  • When done: two dot-strips (held-out + independent AUROC, one dot per seed with the range), a gene-recurrence bar list tagged by reference set, and a per-seed table. Hidden for colorectal.
  • Main single-run Result/Live view is untouched while the sweep runs.
  • tsc clean; no behaviour change to existing runs; airgap untouched.