Add dataset selection heuristics to system prompt

Pass survival_columns through to the per-dataset prompt context and add a
"## Dataset Selection Heuristics" section covering survival/prognostic,
tumor-vs-normal matched-pair, subtype (Bailey/Moffitt/Puleo), and
no-preference/robust-cohort questions. Efficiency Rule #1 now directs the
agent to apply these heuristics, state its chosen dataset_id and rationale,
and proceed when the user doesn't name a dataset.

prompts.yaml

@@ -128,6 +128,22 @@ The ONLY datasets registered in this system are listed below with everything\
   \ endfor %}\n{% endif %}\n{% endfor %}\n\
   \nCRITICAL: Do not claim access to any dataset not listed above. Do not infer additional\
   \ datasets from training knowledge.\n
+## Dataset Selection Heuristics\n\nWhen the user's request does not name a specific dataset,\
+  \ apply these heuristics to choose one. State your chosen dataset_id and a one-sentence\
+  \ rationale tied to the heuristic before proceeding with any tool calls.\n\n- **Survival or\
+  \ prognostic questions:** prefer datasets with non-empty survival_columns — tcga_paad\
+  \ (curated subset), puleo_2018, paca_au_rnaseq, gse28735_pdac, gse50827_nones, gse57495,\
+  \ cptac_pda.\n- **Tumor vs. normal or matched-pair questions:** prefer paired tissue designs\
+  \ — gse16515_mayo, gse28735_pdac, gse15471_badea (note: gse15471_badea's \"normal\"\
+  \ samples are adjacent non-tumor tissue, per its limitations).\n- **Subtype questions:**\
+  \ Bailey 4-subtype (squamous, pancreatic progenitor, immunogenic, ADEX) — only\
+  \ paca_au_rnaseq or paca_au_array have membership.ordered, use one of these. Classical/basal\
+  \ (Moffitt) — gse71729_moffitt. 5-subtype (Puleo) — puleo_2018.\n- **No-preference,\
+  \ robust, or large-cohort questions:** prefer larger curated cohorts — tcga_paad,\
+  \ puleo_2018, paca_ca_rnaseq, paca_au_rnaseq. Pick one dataset and proceed; only run multiple\
+  \ cohorts if the user explicitly asks for cross-cohort validation.\n- **Always:** when you\
+  \ select a dataset using these heuristics (i.e. the user did not name one), state the chosen\
+  \ dataset_id and your rationale before making any tool calls.\n
 ## Handling Dataset Limitations and Refusals\n\nIf the user's requested analysis matches a\
   \ REFUSE rule or Known limitation for the dataset they specified (e.g. \"compare Bailey\
   \ subtypes in gse28735_pdac\" when gse28735_pdac has no Bailey labels, or \"tumor vs normal\
@@ -142,7 +158,9 @@ The ONLY datasets registered in this system are listed below with everything\
   \ on every request.\n\n1. **Skip `dataset_list_available` when the dataset is already identified.** If the user's message\
   \ names a specific dataset (e.g. \"Moffitt\", \"GSE71729\", \"gse71729_moffitt\") or any GSE accession number, do not call\
   \ `dataset_list_available`. The dataset is already known — proceed directly to `dataset_describe` or the analysis tool sequence.\
-  \ Only call `dataset_list_available` when the user asks what datasets exist or when no specific dataset is mentioned.\n\n\
+  \ Only call `dataset_list_available` when the user asks what datasets exist or when no specific dataset is mentioned.\
+  \ In the latter case, apply ## Dataset Selection Heuristics to choose a dataset, state your choice and rationale, then\
+  \ proceed.\n\n\
   2. **Never call `decoupler_check_limma_available` during analysis.** This tool is for diagnostics only. During any analysis\
   \ workflow, call `decoupler_differential_expression` directly with `method=\"limma\"`. If limma is unavailable, the tool\
   \ handles the fallback internally and reports which method was used.\n\n3. **Never call `dataset_validate_manifest_against_data`\

src/agent.py

@@ -66,6 +66,7 @@ def get_system_prompt(
                 "requires_collapse": _feat.get("requires_collapse", False),
                 "group_columns":     _raw.get("group_columns", []),
                 "default_contrasts": _raw.get("default_contrasts", []),
+                "survival_columns":  _raw.get("survival_columns") or {},
                 "refusal_rules":     _raw.get("refusal_rules", []),
                 "limitations":       _raw.get("limitations", []),
                 "reporting_rules":   _raw.get("reporting_rules", []),
@@ -148,6 +149,7 @@ class CodeAgent:
                 "requires_collapse": _feat.get("requires_collapse", False),
                 "group_columns":     _raw.get("group_columns", []),
                 "default_contrasts": _raw.get("default_contrasts", []),
+                "survival_columns":  _raw.get("survival_columns") or {},
                 "refusal_rules":     _raw.get("refusal_rules", []),
                 "limitations":       _raw.get("limitations", []),
                 "reporting_rules":   _raw.get("reporting_rules", []),