oncodsl / Read docs /PROMPT_liveview_objectives.md
govindbalki's picture
Upload folder using huggingface_hub
0fff343 verified
|
Raw
History Blame Contribute Delete
8.06 kB
# Prompt β€” Live-view fixes + objective-modal clarity (run this one)
Five fixes, all targeting existing code (don't rebuild what's there). Read the named spots first. Keep MSI/TMB behaviour + airgap tests intact.
## 1. Strengthen the unsupervised metric (engine_v2/fitness.py, `_score_silhouette`, ~lines 104–130)
It z-scores the output (line 117) and rejects clusters < 10% (line 123). A `protected_div`-by-self program makes ~1 for most patients and a few huge values β†’ a "perfect" outlier split (held-out silhouette 1.0 that aligns with nothing).
- WINSORIZE first: clip the score to its 2.5–97.5 percentile range. Do NOT rank-transform (it erases genuine bimodal gaps).
- ORDERING MATTERS: the existing `std == 0` guard (line 114) runs on the RAW score β€” the self-division case has LARGE raw std, so it slips through. After winsorizing, recompute mean/std on the WINSORIZED score and z-score with those; if the winsorized std < 1e-9 β†’ return `worst_score()`. (Clipping collapses the blow-up to near-constant, so this is what actually catches it.)
- Raise the minimum cluster size from 10% to ~30%: `min_cluster = max(10, int(np.ceil(0.30 * n)))`.
Re-run should no longer return ~1.0 from a self-dividing program.
## 2. Shorten the TMB y-axis label (Lab.tsx `FITNESS_LABEL_BY_TARGET`, ~line 217)
Change `tmb` from "negative association (signed βˆ’spearman with TMB)" to "neg. association with TMB". Leave `msi` ("separation (AUROC)") as-is; optionally make `none` read "cluster separation (0–1)". Ensure the rotated title is fully visible / vertically centred. (The label is ALREADY objective-aware β€” don't rebuild that.)
## 3. Make the fitness-curve TOOLTIP objective-aware (the real bug)
`TIPS.fitnessCurve` (~line 85) is a SINGLE, TMB-centric string (used by the InfoTip at ~line 871). The y-axis already selects by the active objective via `fitnessLabel` (line 890) β€” make the tooltip do the same: build a per-target record and select with the same target value (e.g. `FITNESS_TIP_BY_TARGET[target]`), replacing the single `TIPS.fitnessCurve` usage. Three verbatim texts:
- msi: "This shows how well the best program SEPARATES the two subtypes (MSI-H vs MSS), measured by AUROC: 0.5 = coin-flip (no separation), 1.0 = perfect; higher = a cleaner split. We use AUROC because MSI-H is only ~15% of patients, so it isn't fooled by always guessing the majority."
- tmb: "This shows how strongly the program's score moves OPPOSITE to mutation burden β€” low score where mutations are high (the broken-spell-checker pattern). 0 = no relationship; higher = a stronger opposite-direction link; the top = score and mutation count move almost perfectly oppositely. The underlying correlation is negative (e.g. βˆ’0.49); we plot its strength (0.49) so up = better."
- none: "There is no target here. This measures how cleanly the program's score splits patients into TWO groups: 0 β‰ˆ no real split, 1 = two clean, well-separated clusters. After the run we check what the split lines up with (MSI? TMB?) β€” that's the post-hoc alignment in the Result panel."
## 4. Add tooltips to the NODES and GENES stat cards
`StatCard` already takes a `tip` prop (~line 1054) β€” just pass it:
- NODES: "How many operators (Select / Reduce / Combine / …) the program is built from β€” its size/complexity."
- GENES: "How many distinct genes the program uses."
## 5. Rewrite the objective `?` modals for clarity (web/app/paramHelpContent.tsx)
### 5a. Fix the shared `ObjectiveIntro` (~line 462) β€” it currently says "a target column", which is FALSE for unsupervised.
Replace its text with (verbatim): "An objective is the rule that scores every program β€” the fitness the engine maximises. It's computed only from the program's per-patient output (and, for the supervised objectives, a target column such as MSI or TMB β€” never gene names). It sets what 'good' means; the engine then composes DSL programs to satisfy it."
### 5b. The shared `ObjectiveFooter` (~line 474) is target-centric β€” KEEP it for MSI and TMB, but do NOT render it for unsupervised (unsupervised has no target / no Associate/Effect). Give unsupervised its own footer (see 5e).
### 5c. obj_msi.detailed β€” rewrite the Items to (verbatim, keep the <ObjectiveIntro/> … <ObjectiveFooter/> wrapper):
- What it optimises: "a score that ranks MSI-H patients above MSS β€” i.e. tells the two subtypes apart."
- How it's scored (AUROC): "pick one random MSI-H and one random MSS patient; AUROC is the chance the score puts the MSI-H one higher. 0.5 = coin-flip (no separation), 1.0 = perfect. We use AUROC, not plain accuracy, because MSI-H is only ~15% of patients β€” 'always guess MSS' would look 85% accurate while separating nothing."
- Represented as: `{ target: msi, metric: AUROC }` β€” "the engine sees only the score + the MSI label, never gene names."
- The catch: "it rewards ANY separator. MSI-H and MSS differ in thousands of genes, so it usually grabs easy 'shortcut' genes (consequences or coincidences), not the causal MMR genes."
### 5d. obj_tmb.detailed β€” rewrite the Items to (verbatim, keep wrapper + <ObjectiveFooter/>):
- What it optimises: "a score that goes DOWN as mutation count goes UP β€” the fingerprint of a broken repair gene (switch it off β†’ mutations pile up). The repair genes are the MMR set: MLH1, MSH2, MSH6, PMS2."
- Represented as: `{ target: tmb, metric: correlation, direction: negative }` β€” "the engine sees only the score + the TMB numbers."
- Why 'negative' (not just 'related'): "we reward the score being NEGATIVELY correlated with TMB. Rewarding any correlation would also pick genes that RISE with mutations β€” the opposite of the repair signature."
- Correlation isn't causation (honest): "this is a mechanism-shaped association β€” a much better proxy for the cause than predicting the label, but it doesn't prove causation. Genes silenced alongside the repair genes can mimic the same low-expression↔high-TMB pattern."
- Toward causal: "the program can choose to ADJUST for confounders (age, stage) via the Effect operator β€” the most causal move the observational data honestly allows. The engine decides whether it helps; we don't hardcode it."
### 5e. obj_unsupervised.detailed β€” replace the single paragraph + shared footer with these Items (verbatim) and a CUSTOM footer (NOT <ObjectiveFooter/>):
- No target at all: "Unlike the other objectives, this one is given NO target β€” only the gene numbers. The search never sees MSI, TMB, or any label."
- What it optimises: "a score that divides patients into two as-cleanly-separated-as-possible groups (measured by cluster separation). It is NOT told what the groups should be."
- Shape vs aim: "it always produces a two-group split, but it doesn't aim at MSS/dMMR β€” or anything. It finds whatever the strongest natural division in the data is."
- How we read it: "after the run we check what the discovered split lines up with β€” MSI? TMB? β€” the 'post-hoc alignment' in the Result panel."
- The win: "if the strongest natural split turns out to BE the MSS-vs-dMMR divide, it aligns with MSI at high AUROC β†’ the engine rediscovered the subtype without ever being told it exists."
- Honest expectation: "it may instead land on a different dominant axis (e.g. immune hot vs cold) that only partly overlaps MSI. MSI-H is only ~15% of patients, so a clean recovery isn't guaranteed β€” the result is HOW MUCH the blind split overlaps MSS/dMMR."
- Custom footer (italic, muted, in place of ObjectiveFooter): "The only thing outside the DSL here is 'find the cleanest split, judged honestly on held-out data' β€” there is no target, and the program can't see one. That's exactly what makes a match with MSI meaningful: the engine wasn't told to look for it."
## Confirm
Unsupervised no longer returns a degenerate ~1.0 silhouette; the fitness-curve tooltip matches the active objective; TMB y-axis label fits; NODES & GENES have tooltips; the three objective modals read clearly; and the unsupervised modal no longer shows the target-centric intro/footer.