avoigt1121 Claude Opus 5 commited on
Commit
80b5e01
·
2 Parent(s): 736c0d296d0c56

merge: cohort survey + denominator fix into main (ADR-0006/0007)

Browse files

origin/main had moved 10 commits ahead (ADR-0006 join contract, ADR-0007
re-curation cadence + weekly freshness job, machine-API auth diagnosis).

Conflicts, and how they were resolved:

- variant_status.py — both sides added fields to StatusMatrix. Additive, so
both kept: genes_assayed + profiled (this branch) alongside curation
(ADR-0007). Fields regrouped ahead of the methods; a field declared after a
method is legal but reads as an accident.
- TODO.md / memory.md — both appended dated 2026-07-28 entries. Kept both in
chronological order. Dropped main's placeholder "curate more PDAC studies"
and this branch's stale "re-curation cadence" placeholder, each superseded
by the other side's completed item.

ALL SIX ARTIFACTS RE-CURATED, not merged textually. The four pre-existing ones
had been overwritten by this branch (dropping ADR-0007's source_import_date)
and then textually auto-merged back by git. That produced plausible JSON, but
a semantically-merged data file nobody generated is not evidence of anything —
and the two NEW cohorts genuinely lacked source_import_date, which would have
left them permanently drift_comparable:false, i.e. silently exempt from the
freshness check they most needed. Regenerating on the merged code is the only
way the artifacts provably match it.

Verified: all six carry schema v3 + profiled + source_import_date, and
`python -m src.curate --check` now reports six cohorts up_to_date and
drift-comparable (exit 0). ADR-0007's freshness machinery works over
list_curated(), so the two new cohorts were picked up with no change to it.

Left ADR-0007's "Four cohorts are curated today" untouched — that is a
point-in-time observation in an accepted ADR's Context, not a live list.

Tests: 153 pass (136 from this branch + 17 from main).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.github/workflows/curation-freshness.yml ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-0007 — weekly curated-artifact freshness check.
2
+ #
3
+ # Runs `python -m src.curate --check`: one `/studies/<id>` metadata request per curated study,
4
+ # comparing upstream's `importDate` against the `source_import_date` recorded at curation. It
5
+ # REPORTS; it never re-curates. Re-curation stays a reviewed commit — a job that "fixed" what it
6
+ # found would put the cBioPortal API back on an unattended path, which is what ADR-0005 C4 removed.
7
+ #
8
+ # ---------------------------------------------------------------------------------------------
9
+ # READ THIS BEFORE TRUSTING THIS FILE: it does NOT run today.
10
+ #
11
+ # This repo's `origin` IS the HuggingFace Space (`git push origin main` builds the Space), and
12
+ # HuggingFace does not execute GitHub Actions. Unlike its siblings — `DecoupleRpy_Agent`,
13
+ # `pdac-subtype-agent`, `pdac-analysis-orchestrator`, which each carry a second `github` remote —
14
+ # this repo has **no GitHub mirror**, so nothing schedules this workflow. It is committed anyway
15
+ # for the same reason `DecoupleRpy_Agent/.github/workflows/security.yml` is: adding the mirror
16
+ # becomes zero extra work, and the check's definition never diverges between the two paths.
17
+ #
18
+ # To make it actually run, add the mirror (matching the sibling naming):
19
+ # git remote add github https://github.com/Anne-Voigt/pdac-genomics-agent.git
20
+ # git push github main
21
+ #
22
+ # Until then, the check is a manual command — `python -m src.curate --check` — or a local
23
+ # scheduler; see `docs/adr/ADR-0007-recuration-cadence.md` § "Running the check".
24
+ # ---------------------------------------------------------------------------------------------
25
+ name: curation-freshness
26
+
27
+ on:
28
+ schedule:
29
+ # Weekly, Monday 07:00 UTC — an hour after the sibling security scan so the two unattended
30
+ # jobs don't report at once. Weekly is the right granularity for a signal that moved twice
31
+ # in the last year: it is four metadata requests, and drift is never urgent (a stale cohort
32
+ # keeps answering, and says its age).
33
+ - cron: "0 7 * * 1"
34
+ workflow_dispatch:
35
+
36
+ permissions:
37
+ contents: read
38
+
39
+ jobs:
40
+ check:
41
+ runs-on: ubuntu-latest
42
+ steps:
43
+ - uses: actions/checkout@v7
44
+
45
+ - uses: actions/setup-python@v6
46
+ with:
47
+ python-version: "3.11" # matches `python_version` in the README front-matter
48
+
49
+ - name: Install deps
50
+ # The whole pinned set rather than a hand-picked subset: `src.curate` imports the
51
+ # curation chain (pandas, scipy), and a second dependency list is a second thing to
52
+ # drift.
53
+ run: |
54
+ python -m pip install --upgrade pip
55
+ pip install -r requirements.txt
56
+
57
+ - name: Check curated-artifact freshness
58
+ # Exits 1 on upstream drift or an artifact past the 365-day horizon. That failure is the
59
+ # notification — it means "a human should look and decide", never "the Space is broken".
60
+ # The deployed Space is unaffected either way: it serves artifacts, not this job.
61
+ run: python -m src.curate --check
CLAUDE.md CHANGED
@@ -136,6 +136,9 @@ tests/
136
  - **ADR-0005** — public cBioPortal instance + **curate-and-cache** for v1 (no self-host); its
137
  conditions C1–C4 are the standing obligations on the source (attribution, per-study license,
138
  polite client, no live API on the request path).
 
 
 
139
  - Originating decision: `pdac-subtype-agent` **ADR-0019** (scope, data model, sources, panel) +
140
  `SPIKE-dna-panel-feasibility.md` (cBioPortal proven).
141
 
 
136
  - **ADR-0005** — public cBioPortal instance + **curate-and-cache** for v1 (no self-host); its
137
  conditions C1–C4 are the standing obligations on the source (attribution, per-study license,
138
  polite client, no live API on the request path).
139
+ - **ADR-0007** — re-curation cadence: drift is detected from upstream's `importDate`
140
+ (`python -m src.curate --check`, one metadata request per study, never writes), artifact age
141
+ rides in every answer's `citation.curation` block, and staleness is **visible, never a refusal**.
142
  - Originating decision: `pdac-subtype-agent` **ADR-0019** (scope, data model, sources, panel) +
143
  `SPIKE-dna-panel-feasibility.md` (cBioPortal proven).
144
 
TODO.md CHANGED
@@ -73,27 +73,58 @@ the design doc's M0–M4 is the intent behind it — see that doc's §0).
73
  **`ACCESS_CONTROL=enforce` flipped 2026-07-27** (set as a *variable*, not a secret: it is not
74
  sensitive, and an auditor should be able to see the gate is on without revealing a secret).
75
  Verified live: the machine API now returns `denied` — fail-closed, as designed.
76
- - [ ] **`ALLOWED_IDS` is still unset.** Right now **only `anne-voigt` can use the Space.**
77
- Needs the HF usernames of the lab members who should have access — a membership
78
- decision, and guessing usernames would either lock out real people or grant access to
79
- accounts that are not theirs.
80
- - [ ] **Verify the OAuth UI path while signed in.** The machine-API denial is confirmed; the
81
- *allowed* path is not. Sign in to the Space and run one query. If the owner is denied,
82
- roll back by setting `ACCESS_CONTROL` to empty (gate off) and investigate.
83
- - [ ] **Machine API is now closed to the orchestrator** until it sends a custom
84
- `x-orchestrator-token` whose identity is allow-listed HF edge strips the `x-hf-*`
85
- header `gradio_client` sends automatically. Expected, not a regression.
 
 
 
 
 
 
 
 
 
 
 
 
86
  - [x] **Confirm cBioPortal ToS for automated use; public instance vs self-host** ✅ (2026-07-24) →
87
  `docs/cbioportal-terms-and-deployability.md` + **ADR-0005**. Verdict: usable with conditions,
88
  no ToS bar on automated access, **public instance + curate-and-cache, no self-host**.
89
  Remaining is *implementation* of conditions C1–C4 (attribution, per-study license flag, polite
90
  client, no live API on the request path) — see "Pre-deploy gates" below.
91
- - [ ] Merge the registration descriptor into `pdac-analysis-orchestrator`'s `agents.yaml`.
 
 
 
 
92
 
93
  ## Later
94
 
95
- - [ ] **Re-curation cadence** artifacts are a point-in-time snapshot (`curated_at` in each file).
96
- Decide when they get refreshed and how a stale one is noticed; nothing warns today.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
97
  - [x] **Tumor cohorts beyond cell lines** ✅ (2026-07-28) — full survey of all 538 cBioPortal
98
  studies; **+2 curated, 6 total**: `pancreas_cptac_gdc` (183, mut+CNV+expr, GRCh38) and
99
  `pdac_msk_2024` (2,336, mut+CNV, the largest PDAC cohort available). Driver frequencies
@@ -164,8 +195,12 @@ the design doc's M0–M4 is the intent behind it — see that doc's §0).
164
  an uncurated study is a **refusal**, never a live fetch. Four cohorts curated and
165
  committed (paad_tcga, ccle_broad_2019, paad_qcmg_uq_2016, paad_utsw_2015). Proven by
166
  a test that sabotages the HTTP layer and still answers.
167
- - [ ] Write the **variant×subtype join-contract ADR** (see `RESERVED.md`) — the label-join half is
168
- built; the orchestrator-side contract is undocumented.
 
 
 
 
169
 
170
  ## Open questions
171
 
@@ -173,7 +208,11 @@ the design doc's M0–M4 is the intent behind it — see that doc's §0).
173
  (private, empty, no deploy; `origin` not yet pointed at it — wire up at first deploy).
174
  - ~~cBioPortal public instance vs self-host~~ **RESOLVED 2026-07-24** — public + curate-and-cache,
175
  no self-host (ADR-0005).
176
- - Does the orchestrator align sample ids across two Spaces cleanly, or does the join need a shared id map?
 
 
 
 
177
  - Does the orchestrator cache a fetched subtype label per session, or re-call PurIST per join?
178
  (Belongs to `pdac-analysis-orchestrator`, not here.)
179
  - Should this agent ever route through `biodata-registry`, or stay a direct-source agent? The
 
73
  **`ACCESS_CONTROL=enforce` flipped 2026-07-27** (set as a *variable*, not a secret: it is not
74
  sensitive, and an auditor should be able to see the gate is on without revealing a secret).
75
  Verified live: the machine API now returns `denied` — fail-closed, as designed.
76
+ - [x] **`ALLOWED_IDS` set** 2026-07-27 `anne-voigt,cpelz741` (Carl), mirroring
77
+ `pdac-subtype-agent` prod rather than guessing an id.
78
+ - [x] **OAuth UI path VERIFIED signed in** 2026-07-27 signed in as `anne-voigt`, ran a
79
+ CCLE query in the deployed UI: chart rendered (KRAS ~84% of 57 pancreatic lines) and the
80
+ licence caution displayed. Signed-out the same query correctly returns
81
+ `{"status":"denied"}`. Both halves of the gate now confirmed in the UI.
82
+ - [x] **Machine API DIAGNOSED this Space was never broken** ✅ (2026-07-28). Probed the
83
+ deployed Space directly: a valid token in `x-orchestrator-token` `status: ok`;
84
+ a bogus one denied; none denied. The gate resolves `whoami` correctly from
85
+ inside the Space. **The earlier "server-side identity resolution" diagnosis was
86
+ wrong.** Root cause is caller-side: `router._build_client_kwargs()` attaches the
87
+ header **only if `HF_TOKEN` is set**, so an unset secret on the *orchestrator's*
88
+ Space sends no header at all — which this Space reported as "Please sign in",
89
+ pointing the investigation the wrong way.
90
+ - [ ] ⚠️ **ACTION (human, on the ORCHESTRATOR Space, not this one): set `HF_TOKEN`** in
91
+ `pdac-analysis-orchestrator` prod Space secrets. Nothing to change here.
92
+ - [x] **Denial diagnosis added** ✅ (2026-07-28) — machine denials now carry
93
+ `machine_auth: no_token_header | token_unresolved | not_allowlisted` with a message
94
+ naming the side that must act (never "sign in" to a machine). 4 regression tests.
95
+ - [ ] Open design question, unchanged: should the gate depend on an outbound `whoami` per
96
+ request at all (vs a shared secret)? Same request-path-coupling class as ADR-0005 C4.
97
+ Not urgent — it works; it is a latency/failure-mode question.
98
  - [x] **Confirm cBioPortal ToS for automated use; public instance vs self-host** ✅ (2026-07-24) →
99
  `docs/cbioportal-terms-and-deployability.md` + **ADR-0005**. Verdict: usable with conditions,
100
  no ToS bar on automated access, **public instance + curate-and-cache, no self-host**.
101
  Remaining is *implementation* of conditions C1–C4 (attribution, per-study license flag, polite
102
  client, no live API on the request path) — see "Pre-deploy gates" below.
103
+ - [~] Merge the registration descriptor into `pdac-analysis-orchestrator`'s `agents.yaml`
104
+ **spun off and in progress** in that repo (`dispatch_kind: tool_server`, modelled on its
105
+ existing `pdac_subtype_agent` entry). Its auth is already in place: that repo sends
106
+ `x-orchestrator-token` (its `05bda30`) resolving to `anne-voigt`, who is allow-listed here,
107
+ and the pattern is verified working against prod.
108
 
109
  ## Later
110
 
111
+ - [x] **Re-curation cadence** 2026-07-28 **ADR-0007**. Cadence is driven by upstream's
112
+ `importDate`, not a calendar: `python -m src.curate --check` costs one metadata request per
113
+ study, reports drift + age, exits non-zero for CI, and **never writes** (re-curation stays a
114
+ reviewed commit). Artifact age rides in every answer (`citation.curation`) and on the page
115
+ past 180d/365d horizons. **Stale is visible, never a refusal** — old is not wrong. Confirmed
116
+ the premise while building: all four cohorts last imported upstream Jan 2026, curated Jul —
117
+ a monthly refresh would have been pure churn. Follow-ups below.
118
+ - [~] **Weekly `--check` job** — written 2026-07-28: `.github/workflows/curation-freshness.yml`
119
+ (Mondays 07:00 UTC + `workflow_dispatch`, report-only, exits non-zero on drift/stale; a test
120
+ asserts it can never gain a writing `src.curate` step). **It does not run yet** and the file
121
+ says so: `origin` IS the HF Space and HuggingFace does not execute GitHub Actions, and unlike
122
+ every sibling this repo has **no `github` remote**. Committed anyway on the
123
+ `DecoupleRpy_Agent/security.yml` precedent, so adding the mirror is the only step left.
124
+ - [ ] **Add the GitHub mirror** — the one ops action that turns the weekly check on (and would give
125
+ this repo CI at all, which it has never had):
126
+ `git remote add github https://github.com/Anne-Voigt/pdac-genomics-agent.git && git push github main`.
127
+ Needs a human: creating the repo under the org. Until then `--check` is a manual command.
128
  - [x] **Tumor cohorts beyond cell lines** ✅ (2026-07-28) — full survey of all 538 cBioPortal
129
  studies; **+2 curated, 6 total**: `pancreas_cptac_gdc` (183, mut+CNV+expr, GRCh38) and
130
  `pdac_msk_2024` (2,336, mut+CNV, the largest PDAC cohort available). Driver frequencies
 
195
  an uncurated study is a **refusal**, never a live fetch. Four cohorts curated and
196
  committed (paad_tcga, ccle_broad_2019, paad_qcmg_uq_2016, paad_utsw_2015). Proven by
197
  a test that sabotages the HTTP layer and still answers.
198
+ - [x] Write the **variant×subtype join-contract ADR** (2026-07-28)
199
+ `docs/adr/ADR-0006-variant-subtype-join-contract.md`. Settles the split (label-join here /
200
+ live-PurIST via the orchestrator), the routing contract + the orchestrator's obligations,
201
+ subtype-label VALUE validation against a closed vocabulary (the CCLE histology trap), the
202
+ statistical-honesty stance (no cross-gene correction, `testable=false` + reason, small-n
203
+ caveats), and metadata-label vs live-PurIST provenance. No behaviour change.
204
 
205
  ## Open questions
206
 
 
208
  (private, empty, no deploy; `origin` not yet pointed at it — wire up at first deploy).
209
  - ~~cBioPortal public instance vs self-host~~ **RESOLVED 2026-07-24** — public + curate-and-cache,
210
  no self-host (ADR-0005).
211
+ - Does the orchestrator align sample ids across two Spaces cleanly, or does the join need a shared id
212
+ map? **Still open — deliberately** (ADR-0006 §2). The spike's 145/146 was *within* one cBioPortal
213
+ cohort, not the cross-Space case that would need a map. Interim rule is written down: exact
214
+ sample-id string equality, report join coverage, no fuzzy/prefix matching, and a collapsed join is
215
+ a refusal with the numbers shown. Resolving it amends ADR-0006 §2; it does not reopen the split.
216
  - Does the orchestrator cache a fetched subtype label per session, or re-call PurIST per join?
217
  (Belongs to `pdac-analysis-orchestrator`, not here.)
218
  - Should this agent ever route through `biodata-registry`, or stay a direct-source agent? The
docs/adr/ADR-0005-cbioportal-public-vs-selfhost.md CHANGED
@@ -61,6 +61,10 @@ exponential backoff + jitter on `429`/`5xx`, and a descriptive `User-Agent` with
61
  change on their side can break an ingest run (fail loud, don't serve partial).
62
  - Cached matrices can drift from upstream until we deliberately re-curate (acceptable — variant
63
  *status* over a driver panel is stable; invalidation is a deliberate re-pull, not a poll).
 
 
 
 
64
 
65
  **Neutral**
66
  - No change to the tool contracts or the data model; this is purely a source-dependency decision.
 
61
  change on their side can break an ingest run (fail loud, don't serve partial).
62
  - Cached matrices can drift from upstream until we deliberately re-curate (acceptable — variant
63
  *status* over a driver panel is stable; invalidation is a deliberate re-pull, not a poll).
64
+ **Superseded in part by ADR-0007 (2026-07-28):** "a deliberate re-pull" was a mechanism, not a
65
+ policy — nothing said *when*, and nothing read `curated_at`. ADR-0007 fixes the cadence to
66
+ upstream's `importDate` (`--check`), surfaces artifact age in every answer, and rules that
67
+ staleness is visible but never a refusal. The re-pull stays deliberate and reviewed.
68
 
69
  **Neutral**
70
  - No change to the tool contracts or the data model; this is purely a source-dependency decision.
docs/adr/ADR-0006-variant-subtype-join-contract.md ADDED
@@ -0,0 +1,260 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-0006 — The variant×subtype join contract: label-join here, live-PurIST join in the orchestrator
2
+
3
+ **Status:** Accepted · **Date:** 2026-07-28 · **Deciders:** Annie + Carl (source author)
4
+
5
+ **Relates to:** ADR-0001 (this agent is a stateless specialist; the cross-agent join is orchestration)
6
+ · ADR-0002 (the gene×sample status matrix this joins from) · ADR-0003 (BYOD is `grounded=false` and
7
+ never label-joined) · ADR-0005 C4 (the request path reads curated artifacts, never the live API).
8
+ **Sibling:** `pdac-subtype-agent` **ADR-0020** — variant×subtype is a cross-*modality* instance of
9
+ its alignment-by-sample-id primitive, and inherits its "align, never pool" and mandatory-caveat
10
+ stances. **Originating:** `pdac-subtype-agent` ADR-0019 (scope) and
11
+ `docs/design/DESIGN-pdac-genomics-agent.md` §4.
12
+
13
+ > **Why this ADR is late.** The label-join half shipped in M2 (2026-07-24) and the routing result was
14
+ > already exposed, but the contract lived only as prose in `TODO.md`, the design doc §4, and module
15
+ > docstrings. It is this agent's differentiator *and* the one decision that spans two Spaces — the
16
+ > only major decision without an ADR. This writes it down; it does not change the built behaviour,
17
+ > with the exception of §3, which records a correctness fix already made in code.
18
+
19
+ ## Context
20
+
21
+ The question Carl actually wants answered is **"does a gene's variant status line up with molecular
22
+ subtype?"** — KRAS/TP53/MYC/GATA6 alteration against basal-like vs classical. It is the reason a DNA
23
+ specialist earns its place next to `pdac-subtype-agent`: neither agent can answer it alone.
24
+
25
+ Three facts constrain any design:
26
+
27
+ 1. **It needs two modalities that live in two Spaces.** Variant status is here; the PurIST subtype
28
+ call is in `pdac-subtype-agent`. ADR-0001/ADR-0011 fixed the family's Option A: specialists are
29
+ **stateless siblings that never call each other**; cross-agent composition is the orchestrator's
30
+ job. A `variant_by_subtype` that reached out to a sibling would violate that outright.
31
+ 2. **But not every cohort needs a live PurIST call.** Several registered cBioPortal cohorts already
32
+ carry a molecular-subtype label as a clinical attribute (`paad_tcga` → `MOFFITT_SUBTYPE` and
33
+ friends). For those, the whole join is local: the label is already in the curated artifact next to
34
+ the status matrix, and no sibling is involved.
35
+ 3. **The join key is the sample id**, the same primitive `pdac-subtype-agent` ADR-0020 uses for
36
+ multi-dataset alignment. The feasibility spike measured 145/146 join coverage on `paad_tcga`
37
+ between the DNA and expression modalities of the same cohort.
38
+
39
+ So the design question is not "who owns the join" but **where the seam falls** — and what shape this
40
+ agent hands the orchestrator when the join is not local.
41
+
42
+ ## Decision
43
+
44
+ ### 1. The split: the label-join is local, the live-PurIST join is orchestration — DECIDED
45
+
46
+ `variant_by_subtype(genes, source, …)` implements **only** the half a stateless agent can do alone:
47
+
48
+ - **Local (label-join).** The cohort carries a validated subtype label in its curated clinical
49
+ attributes → this agent inner-joins per-gene variant status to that label **on sample id**, builds
50
+ the subtype × {altered, WT} contingency table, and runs the association test itself. It returns
51
+ `join_available: true` with `subtype_source: "clinical_attribute:<ATTR>"`.
52
+ - **Routed (live-PurIST).** No in-metadata label, or a non-registered (BYOD) source → this agent
53
+ computes nothing and returns `{join_available: false, route: "orchestrator", reason, …}`.
54
+
55
+ `altered` is defined per modality: mutation = `{missense, truncating, hotspot}` (i.e. non-`WT`),
56
+ CNV = `{deep_del, loss, gain, amp}` (i.e. non-`neutral`). The test is **Fisher's exact for a 2×2**
57
+ table and **chi-square for R×2** (R>2 subtype classes) — Fisher because subtype cohorts are small
58
+ and asymptotic χ² is unreliable at low expected counts, chi-square above 2×2 because exact R×2 is
59
+ not worth the cost at these n.
60
+
61
+ The per-study modality-coverage gate and the pancreatic-lineage filter apply **before** the join, not
62
+ after: a cohort with no CNV data returns `route: "unavailable_modality"`, never a null column read as
63
+ "no alteration".
64
+
65
+ ### 2. The routing contract: exact shape, orchestrator obligation, and the sample-id question — DECIDED
66
+
67
+ **The shape this agent guarantees.** Both branches are the *same tool's* return value, discriminated
68
+ by `join_available`. The orchestrator must switch on it, never on the presence of `genes`:
69
+
70
+ ```jsonc
71
+ // local — the join happened here
72
+ { "join_available": true, "source": "cbioportal:paad_tcga", "grounded": true,
73
+ "subtype_source": "clinical_attribute:MOFFITT_SUBTYPE", "modality": "mutation",
74
+ "n_labeled": 145,
75
+ "genes": { "KRAS": { "contingency": {"basal-like": {"altered": 41, "WT": 3}, …},
76
+ "n": 145, "testable": true,
77
+ "test": {"name": "fisher_exact", "odds_ratio": …, "p_value": …} } },
78
+ "caveats": [...], "citation": {...} }
79
+
80
+ // routed — the orchestrator must complete the join
81
+ { "join_available": false, "route": "orchestrator", "reason": "<machine-stable reason>",
82
+ "source": "cbioportal:<study>", "note": "<human-readable restatement>" }
83
+ ```
84
+
85
+ **What the orchestrator must do with `route: "orchestrator"`.** Three steps, in this order:
86
+
87
+ 1. Call `query_variant_status(genes, source)` **here** for the per-sample status matrix.
88
+ 2. Call `pdac-subtype-agent` for the PurIST label over the *same cohort*.
89
+ 3. Join the two on **sample id** and run the association test on its side, stamping
90
+ `subtype_source: "purist:<version>"` (see §5).
91
+
92
+ The orchestrator **must not** synthesize a label and call back into `variant_by_subtype`: the
93
+ `subtype_attribute` parameter exists for a *curator* naming a real clinical attribute, not as a
94
+ back-door for injecting externally computed labels. Nor may it re-run this agent's test on a mixed
95
+ provenance set. Composition is the orchestrator's; **this agent never calls a sibling**, which is why
96
+ `route: "orchestrator"` is a terminal answer here rather than an internal retry.
97
+
98
+ **Sample-id alignment across two Spaces — the open question, and the interim rule.** Whether ids
99
+ align cleanly across Spaces is recorded as open in `TODO.md` and is **not resolved by this ADR**,
100
+ because the evidence does not yet exist to resolve it: the spike measured alignment *within* one
101
+ cBioPortal cohort (145/146 on `paad_tcga`), not between a cBioPortal cohort here and a GEO/registry
102
+ matrix there, which is the case that would actually need a mapping. The interim rule is the honest
103
+ one, and it is the same refuse-rather-than-guess posture the family uses elsewhere:
104
+
105
+ > The orchestrator joins on **exact sample-id string equality** and **reports the join coverage**
106
+ > (`n_joined` / `n_variant` / `n_subtype`). It performs **no fuzzy matching, no normalization, no
107
+ > truncation-to-a-prefix** (e.g. TCGA barcode trimming) unless a *declared, reviewed* id map says so.
108
+ > A join whose coverage collapses is a **refusal with the coverage numbers shown**, not a quiet
109
+ > analysis of whatever overlapped.
110
+
111
+ A silent low-coverage join is the dangerous failure here: it yields a small, biased, confident-looking
112
+ table. Building a shared id map is deferred until a real cross-Space cohort pair demonstrates it is
113
+ needed — and it would be a **declared artifact**, not inference at join time.
114
+
115
+ ### 3. Subtype-label validation: attribute id is not enough — the VALUES must speak PDAC subtype — DECIDED
116
+
117
+ `find_subtype_labels` auto-detects a label from a known ordered attribute list (`MOFFITT_SUBTYPE`,
118
+ `MRNA_SUBTYPE`, `SUBTYPE`, …). **Attribute-id matching alone is unsafe and is now insufficient by
119
+ contract.** `SUBTYPE` is a generic id that cohorts fill differently: in `ccle_broad_2019` it holds
120
+ **histology** — "Adenocarcinoma", "Melanoma", "Small Cell Lung Cancer" — so id-matching produced a
121
+ confident, well-formatted **variant × histology** association across a pan-cancer panel, presented as
122
+ variant × *molecular subtype*. That is precisely the "plausible-looking wrong answer" the family's
123
+ gates exist to prevent, and it is worse than no answer.
124
+
125
+ Therefore, on the **auto-detect** path, a candidate attribute is accepted only if its **values** pass
126
+ a closed PDAC molecular-subtype vocabulary check (`basal`, `classical`, `squamous`, `progenitor`,
127
+ `immunogenic`, `adex`, `quasimesenchymal`, `exocrine`, `mesenchymal`), with a **≥0.5** fraction of
128
+ non-blank values matching — below 1.0 because real cohorts carry stray `NA`/`Other`/unclassified
129
+ entries. A candidate that fails is **skipped**, the probe continues, and if nothing passes the cohort
130
+ routes to the orchestrator. The vocabulary is deliberately **closed**: a label set that does not
131
+ speak it is not a PDAC subtype call, whatever the attribute is named. Widening it is a deliberate
132
+ edit, and each addition must be a real published PDAC subtype term.
133
+
134
+ **An explicitly named `subtype_attribute` bypasses the vocabulary check.** The caller has named a
135
+ specific attribute and taken responsibility for it; this is the escape hatch for a legitimately
136
+ oddly-vocabularied cohort, and it keeps the guard from becoming a wall. The bypass is visible in the
137
+ output — `subtype_source` always names the attribute used, so any answer is traceable to its label
138
+ source.
139
+
140
+ ### 4. Statistical honesty: no multiple-testing correction, explicit `testable=false`, standing small-n caveat — DECIDED
141
+
142
+ **No multiple-testing correction across genes, and this stays for v1.** The tool is a *descriptive
143
+ screen* over a 19-gene driver panel the user chose, not a discovery scan over a genome; the family's
144
+ consumers are non-coding scientists asking about specific genes. Correcting silently would be worse
145
+ than not correcting: it would imply a family-wise hypothesis the user did not pose, and the correct
146
+ denominator depends on how many genes they actually asked about across the session — something a
147
+ stateless tool cannot know. Instead the p-values are shipped **uncorrected and labelled as such** via
148
+ a mandatory caveat (`"Association is descriptive (no multiple-testing correction across genes
149
+ here)."`), and the caveat block is not optional output. If a caller wants FDR across a panel, that is
150
+ an orchestrator-side decision made with the full set of tests in hand.
151
+
152
+ **`testable: false` with a machine-readable `reason` is the answer when a table cannot support a
153
+ test** — rather than a p-value of 1.0, a `NaN`, or an exception. Two cases:
154
+ - `"only one subtype class present"` — fewer than 2 subtype classes after the join.
155
+ - `"no variation in alteration status (all-altered or all-WT)"` — a degenerate column.
156
+
157
+ Both still return the **contingency table and `n`**, because the counts are the useful answer even
158
+ when the test is undefined. All-altered is not hypothetical: KRAS is altered in ~90%+ of PDAC, so
159
+ the panel's flagship gene is exactly the one that degenerates most often — the honest report is "45/45
160
+ basal, 98/100 classical, no test possible", not a spurious statistic.
161
+
162
+ **Small-n caveat discipline.** PDAC subtype cohorts are small and get smaller after the lineage
163
+ filter and the label inner-join. `n_labeled` and the per-gene `n` are always returned so the reader
164
+ can see how thin the table is, and the caveat block always states that the label is a pre-existing
165
+ cohort attribute rather than a live call. Fisher's exact is used at 2×2 specifically so small counts
166
+ do not get an unreliable asymptotic p-value. This ADR does **not** impose a hard minimum-n refusal:
167
+ below-threshold cells still carry real information as *counts*, and a hard cut would silently drop
168
+ cohorts the user explicitly asked about. Show the n, show the table, let the caveat do its work.
169
+
170
+ ### 5. A live-PurIST label and a metadata label are NEVER interchangeable — different provenance, reported as such — DECIDED
171
+
172
+ They are not the same measurement and must never be pooled, averaged, or silently substituted:
173
+
174
+ - A **metadata label** is a historical call by the cohort's authors, using their classifier, their
175
+ data version, and their thresholds (Moffitt 2015, Bailey 2016, Collisson 2011 — *different
176
+ taxonomies with different class counts*).
177
+ - A **live PurIST call** is our classifier run now, on the expression matrix we hold, at a known
178
+ model version.
179
+
180
+ Therefore `subtype_source` is a **required** field on every joined result and is the provenance
181
+ discriminator: `"clinical_attribute:<ATTR>"` versus `"purist:<version>"`. The rules that follow:
182
+
183
+ - **Never mix within one contingency table.** A cohort is joined against one label source, not a
184
+ fill-in-the-gaps blend of a metadata label plus PurIST for the unlabelled samples. That would put
185
+ two classifiers' calls in the same table and attribute the disagreement to biology.
186
+ - **Cross-source comparison is a concordance question, not a join.** "Does the metadata label agree
187
+ with PurIST?" is a legitimate and interesting question — but it is *its own* analysis with its own
188
+ output (agreement rate, confusion matrix), owned by the orchestrator, and it is **not** licence to
189
+ treat the two as one variable afterwards.
190
+ - **Two results with different `subtype_source` are not comparable head-to-head** and must be
191
+ reported with their provenance attached, in the same spirit as ADR-0020's mandatory comparability
192
+ banner. A downstream renderer may not drop `subtype_source` from a variant×subtype result.
193
+
194
+ ## Consequences
195
+
196
+ **Positive**
197
+ - The differentiator is finally specified end-to-end: the orchestrator has a written contract with a
198
+ discriminated return shape, three ordered steps, and an explicit prohibition list.
199
+ - Statelessness (ADR-0001/0011) survives contact with the one genuinely cross-Space question, and the
200
+ common case (`paad_tcga`) still answers in one hop with zero sibling traffic and no API call
201
+ (ADR-0005 C4).
202
+ - The vocabulary check closes a live, demonstrated wrong-answer path (CCLE histology-as-subtype)
203
+ rather than a theoretical one.
204
+ - Degenerate tables and small cohorts produce honest, machine-readable non-answers instead of
205
+ statistics that read as findings.
206
+
207
+ **Negative**
208
+ - The orchestrator carries real work this ADR only *specifies*: the routed branch, the id join with
209
+ coverage reporting, and its own association test. Until that lands, unlabelled cohorts have no
210
+ end-to-end answer — a routed refusal, not a result.
211
+ - Two association-test implementations will exist (here and orchestrator-side). They must agree on
212
+ the `altered` vocabulary and the Fisher/χ² rule, or the same cohort answers differently by route.
213
+ Mitigation: this ADR is the single normative statement of both, and the orchestrator implementation
214
+ should be reviewed against §1.
215
+ - The closed vocabulary will reject a genuinely-labelled cohort using unusual terms; the named-attribute
216
+ bypass is the intended (manual) remedy.
217
+ - Uncorrected p-values can still be over-read by a non-statistician reader, caveat or not.
218
+
219
+ **Neutral**
220
+ - No change to the status-matrix model (ADR-0002) or the curation pipeline; this is a contract over
221
+ existing artifacts.
222
+ - Sample-id alignment across Spaces remains open, with a conservative interim rule; resolving it will
223
+ amend §2 rather than reopen the split.
224
+
225
+ ## Alternatives considered
226
+
227
+ 1. **This agent calls `pdac-subtype-agent` directly to get PurIST and completes every join itself.**
228
+ Rejected — it is the exact coupling ADR-0001/0011 rejected family-wide: specialists become clients
229
+ of each other, gain a second Space as a runtime dependency, and lose statelessness. It would also
230
+ duplicate the orchestrator's routing and auth surface in a specialist.
231
+ 2. **Push the label-join to the orchestrator too, so there is exactly one join implementation.**
232
+ Rejected — for a cohort whose label sits in its own curated artifact, this would force two Space
233
+ round-trips and a sibling call to do something purely local, on the most common path
234
+ (`paad_tcga`). The seam belongs where the data is: local label → local join.
235
+ 3. **Auto-detect subtype labels by attribute id only (the pre-fix behaviour).** Rejected on evidence
236
+ — `ccle_broad_2019`'s `SUBTYPE` is histology, and id-matching turned that into a confident
237
+ variant×histology association labelled as subtype.
238
+ 4. **Free-text/heuristic label discovery over all clinical attributes.** Rejected — a wider net finds
239
+ more wrong labels, not more right ones. The known-attribute list plus a closed value vocabulary
240
+ keeps every join traceable to a named, recognized label source.
241
+ 5. **Apply Benjamini–Hochberg across the requested gene panel inside the tool.** Rejected for v1 —
242
+ imposes a family-wise hypothesis the user did not pose, and the correct denominator is
243
+ session-level knowledge a stateless tool does not have. Available to the orchestrator, which does.
244
+ 6. **Refuse below a minimum n (e.g. n<20) instead of caveating.** Rejected — the counts are still the
245
+ answer at small n, and a hard cut would silently drop cohorts the user named. Report n, caveat,
246
+ and refuse only the genuinely *undefined* (`testable=false`).
247
+ 7. **Treat a metadata label and a PurIST call as one interchangeable "subtype" variable.** Rejected —
248
+ different classifiers and, for Bailey/Collisson, different taxonomies entirely; pooling them
249
+ converts classifier disagreement into apparent biology.
250
+
251
+ ## Revisit triggers
252
+
253
+ - A real cross-Space cohort pair shows sample-id alignment **is not** clean → resolve §2's open
254
+ question with a declared id map (amendment, not a re-split).
255
+ - The orchestrator-side live-PurIST join lands → confirm it matches §1's `altered`/test rules and
256
+ stamps `subtype_source: "purist:<version>"` per §5.
257
+ - A registered cohort is rejected by the §3 vocabulary despite carrying real molecular subtypes →
258
+ widen the closed list deliberately (and record the term's source).
259
+ - Panel-wide screening becomes a routine ask (rather than gene-specific questions) → revisit §4's
260
+ no-correction stance at the orchestrator layer.
docs/adr/ADR-0007-recuration-cadence.md ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-0007 — Re-curation cadence: upstream-signal staleness, visible age, no refusal
2
+
3
+ **Status:** Accepted · **Date:** 2026-07-28 · **Renumbered:** 2026-07-28 · **Deciders:** Annie
4
+
5
+ **Numbering.** Claimed and written on this lane as **ADR-0006**, renumbered to **0007** before
6
+ merge. A concurrent lane had claimed 0006 for the variant×subtype join contract (`a792086`) and
7
+ published it (`64d7f04`) — both already on `main`. Per `RESERVED.md` protocol rule 4 the earlier
8
+ claim keeps the number and the later one moves; this ADR had not been merged or cited outside its
9
+ branch, so the rename is safe.
10
+
11
+ **Third occurrence of the same root cause, and it is worth naming.** This lane's worktree held its
12
+ own copy of `RESERVED.md` and was branched before the other lane's claim landed, so the
13
+ one-shared-row conflict could not fire at claim time — exactly the failure recorded for the 0003
14
+ collision below. The claim was committed alone and pushed, but to the *feature branch*, not to
15
+ `main` (rule 5), because `origin/main` here IS the live Space and pushing it is a deploy. That is a
16
+ real tension in the protocol on this repo: **the collision was caught by reading
17
+ `SHOWCASE_STATUS.md` before updating it, not by the ledger.** Until rule 5 has a deploy-safe
18
+ mechanism here, `git fetch origin main && git show origin/main:docs/adr/RESERVED.md` before
19
+ claiming is the check that actually works.
20
+
21
+ **Follows:** ADR-0005 (public cBioPortal + curate-and-cache, condition C4) and ADR-0002 (the
22
+ curation contract). ADR-0005 made the API a curation-time dependency; this ADR handles the
23
+ consequence it named and left open.
24
+
25
+ ## Context
26
+
27
+ ADR-0005's own Consequences section wrote the gap down and moved on:
28
+
29
+ > Cached matrices can drift from upstream until we deliberately re-curate (acceptable — variant
30
+ > *status* over a driver panel is stable; invalidation is a deliberate re-pull, not a poll).
31
+
32
+ "A deliberate re-pull" is a mechanism, not a policy. In practice nothing told anyone *when* to
33
+ pull. Each artifact carried a `curated_at` timestamp from the beginning, but **nothing read it,
34
+ nothing warned, and no threshold existed** — so a cohort could sit unrefreshed for years and every
35
+ answer would look exactly as confident as one curated yesterday. Four cohorts are curated today
36
+ (`paad_tcga`, `ccle_broad_2019`, `paad_qcmg_uq_2016`, `paad_utsw_2015`).
37
+
38
+ Three facts shape the answer:
39
+
40
+ 1. **cBioPortal studies are versioned releases, not a live feed.** A published cohort like
41
+ `paad_tcga` can be byte-stable for years; `ccle_broad_2019` gets periodic refreshes. A blanket
42
+ "re-curate monthly" would spend the condition-C3 politeness budget rewriting identical files.
43
+ 2. **Upstream publishes the signal we need.** `/studies/<id>` carries `importDate` — one cheap
44
+ request per study answers "has upstream re-imported since we curated?" without re-pulling
45
+ anything.
46
+ 3. **Artifacts are committed and reviewable by design** (ADR-0005). That reviewability is the
47
+ reason a stale artifact is a governance problem at all, and it is also the constraint: nothing
48
+ may refresh an artifact automatically at runtime, or C4's decoupling comes back as an
49
+ intermittent one.
50
+
51
+ ## Decision
52
+
53
+ **Drive re-curation off upstream's `importDate`, make artifact age visible in every answer, and
54
+ never refuse for staleness alone.** Three parts:
55
+
56
+ ### 1. Cadence — event-driven, with a time-based backstop
57
+
58
+ `python -m src.curate --check` fetches **only** `/studies/<id>` for each curated study and compares
59
+ upstream's `importDate` against the one recorded in the artifact. Drift → re-curate; no drift →
60
+ do nothing, regardless of age. Curation now records `source_import_date` in the payload so the
61
+ comparison exists at all.
62
+
63
+ Two time horizons act as a backstop for the case where nobody ran the check either:
64
+ `REVIEW_AFTER_DAYS = 180` ("a human should look") and `STALE_AFTER_DAYS = 365` ("say it in the
65
+ answer"). They are advisory in both states.
66
+
67
+ `--check` exits non-zero on drift or a stale artifact, so CI or a periodic job can fail on it.
68
+ **It never writes.** A checker that repaired what it found would re-establish the automatic
69
+ API path C4 removed, and would do so unreviewed.
70
+
71
+ ### 2. Visibility — in the answer, not just the log
72
+
73
+ Every grounded tool payload's `citation` block now carries a `curation` sub-block:
74
+ `curated_at`, `age_days`, `level` (`current` / `review_due` / `stale` / `unknown`),
75
+ `source_import_date`, `drift_comparable`, and a plain-language `note`. Past the review horizon the
76
+ Gradio page also renders a visible line, next to the existing licence caution.
77
+
78
+ This follows the repo's standing stance, already applied to `terms` in `licenses.citation_block`
79
+ and to the licence caution in the UI: provenance that stops at the storage layer is provenance
80
+ nobody honours. A maintainer-only `--check` would have satisfied the letter of the gap and left
81
+ the person acting on a frequency none the wiser.
82
+
83
+ ### 3. Staleness never refuses
84
+
85
+ A stale-but-valid cohort is **old, not wrong**. Its status calls were correct for the upstream
86
+ release they were curated from, and that release still exists and is still citable. Refusing it
87
+ would withhold a correct answer because a maintainer missed a review window — trading the user's
88
+ answer for our maintenance signal.
89
+
90
+ Refusal stays reserved for gates where the answer would be *incorrect*: genome-build mismatch,
91
+ species, absent per-study modality, and controlled access (C2). Age is not in that class, and
92
+ putting it there would dilute what a refusal from this agent means.
93
+
94
+ `test_stale_artifact_still_answers_and_says_so` pins this: it ages an artifact past the horizon and
95
+ asserts the answer is intact *and* labelled.
96
+
97
+ ## Running the check
98
+
99
+ A check nobody runs is the same as the `curated_at` nobody read, so the schedule is part of the
100
+ decision rather than a follow-up.
101
+
102
+ **Weekly, unattended:** `.github/workflows/curation-freshness.yml` — Monday 07:00 UTC, plus
103
+ `workflow_dispatch`. Weekly is right for a signal that moved twice in the last year: four metadata
104
+ requests, and drift is never urgent, because a stale cohort keeps answering and says its age.
105
+
106
+ **It does not run yet, and that is stated in the file rather than assumed away.** `origin` here IS
107
+ the HuggingFace Space, and HuggingFace does not execute GitHub Actions. Every sibling
108
+ (`DecoupleRpy_Agent`, `pdac-subtype-agent`, `pdac-analysis-orchestrator`) carries a second `github`
109
+ remote; **this repo has none**, so nothing schedules the workflow. It is committed anyway on the
110
+ precedent of `DecoupleRpy_Agent/.github/workflows/security.yml`: adding the mirror later is then
111
+ zero work, and the check's definition never forks between the two paths. Adding that mirror is the
112
+ one ops step that turns this on.
113
+
114
+ **Until then**, the check is `python -m src.curate --check` — run by hand, or from a local
115
+ scheduler. A launchd agent on the maintainer's machine is the low-ceremony option and is
116
+ deliberately *not* installed by this repo: a job that runs on one laptop is invisible to everyone
117
+ else, and pretending otherwise is how the original gap happened.
118
+
119
+ The job is report-only in every form. Nothing in this cadence writes an artifact.
120
+
121
+ ## Consequences
122
+
123
+ **Positive**
124
+ - Re-curation is triggered by evidence rather than a calendar; the polite-client budget is spent
125
+ only where upstream actually moved.
126
+ - A reader of any answer can see the snapshot's age without asking a maintainer.
127
+ - `--check` is CI-attachable, so drift can become a failing build rather than a memory.
128
+ - No new runtime dependency and no runtime writes — C4 holds exactly as before.
129
+
130
+ **Negative**
131
+ - The four artifacts curated before this ADR carried no `source_import_date`. Rather than leave
132
+ them `drift_unknown` until each happened to be re-curated, the field was **backfilled once**,
133
+ in this ADR's commit, from a live `/studies/<id>` read of each — a metadata-only addition that
134
+ touches no status call. Any *future* artifact arriving without the field still reports
135
+ `drift_unknown`, which is the honest state and not "no drift". (The backfill also confirmed the
136
+ premise empirically: all four were last imported upstream in **January 2026** and curated in
137
+ July — six months of a published cohort not moving, which is exactly why a monthly re-pull would
138
+ have been churn.)
139
+ - `importDate` is trusted as upstream's release stamp. If cBioPortal re-imports without moving it,
140
+ the check misses the drift — the time horizons are the only backstop for that.
141
+ - Freshness metadata is now part of the response contract, so consumers may start depending on it.
142
+
143
+ **Neutral**
144
+ - **No schema-version bump.** `source_import_date` is optional and its absence degrades to an
145
+ honest "not comparable". Bumping to v3 would refuse all four existing artifacts and force
146
+ precisely the blanket re-curation this cadence exists to avoid.
147
+
148
+ ## Alternatives considered
149
+
150
+ 1. **Fixed calendar refresh (e.g. monthly/quarterly re-curate everything).** Rejected — busywork
151
+ against versioned releases that rarely change; burns the C3 budget re-fetching identical data,
152
+ and produces no-op commits that train reviewers to skim artifact diffs.
153
+ 2. **Refuse to serve past a staleness threshold.** Rejected — disproportionate. See part 3. It also
154
+ converts a maintenance lapse into a user-visible outage of a *correct* answer.
155
+ 3. **A background job that re-curates on drift.** Rejected — artifacts are committed and reviewed
156
+ on purpose (ADR-0005); an automatic writer both bypasses that review and reintroduces the
157
+ request-path API coupling C4 removed.
158
+ 4. **`--check` only, no age in the answer.** Rejected — it fixes the maintainer's blindness and
159
+ leaves the reader's intact, against the repo's own stance on provenance.
160
+ 5. **Hash the upstream payload instead of trusting `importDate`.** Rejected for v1 — that is a full
161
+ re-fetch, i.e. re-curation, which is the cost this design is avoiding. Revisit if `importDate`
162
+ proves unreliable.
163
+
164
+ ## Revisit triggers
165
+
166
+ - `importDate` observed to be stale or unmoved across a real upstream change → move to a
167
+ content-hash check (alternative 5) or shorten the horizons.
168
+ - A cohort where drift is *frequent* (a live-updating source rather than a versioned release) →
169
+ that source needs its own cadence, not this one.
170
+ - The panel or the cohort count grows enough that a serial `--check` stops being cheap.
docs/adr/RESERVED.md CHANGED
@@ -33,10 +33,12 @@ use) · `DROPPED` (burned).
33
  | 0003 | PUBLISHED | BYOD MAF/segment upload contract (`grounded=false`, refusals) | `ADR-0003-byod-contract.md` |
34
  | 0004 | PUBLISHED | Auth gate — OAuth allow-list + `ACCESS_CONTROL` dark-launch | `ADR-0004-auth-gate.md` |
35
  | 0005 | PUBLISHED | Public cBioPortal instance + curate-and-cache for v1 (no self-host) | `ADR-0005-cbioportal-public-vs-selfhost.md` |
 
 
36
  | **0014** | **EXTERNAL** | Shared security-scan standard (pip-audit + bandit + gitleaks + trivy) — authored in `DecoupleRpy_Agent`. **Do not use;** adopt it, don't renumber it. Adopted here as `security/scan.sh`. | *(sibling repo)* |
37
  | **0015** | **EXTERNAL** | `data_level` / registry `modality` semantics — authored in `biodata-registry`. **Do not use.** A DNA `modality` extension would be an amendment *there*, under that owner. | *(sibling repo)* |
38
 
39
- **Next free number: 0006.**
40
 
41
  ## Resolved: the 0003 collision (2026-07-24 → fixed 2026-07-25)
42
 
@@ -58,9 +60,6 @@ outside the repo. Both branches are now merged into `main`.
58
 
59
  ## Known upcoming ADRs (claim a number when you start)
60
 
61
- - **variant×subtype join contract** — the orchestrator-owned cross-agent join (subtype from a live
62
- PurIST call vs a metadata label), the association test, and small-n honesty. Sibling of
63
- `pdac-subtype-agent` ADR-0020. The label-join half is built (M2); the contract is unwritten.
64
  - **Does this agent use `biodata-registry` at all?** The founding design assumed a registry `modality`
65
  extension as a hard blocker; the build bypassed it and reads cBioPortal directly (ADR-0001
66
  amendment, 2026-07-25). Reopening that route needs this ADR — but nothing is blocked meanwhile.
 
33
  | 0003 | PUBLISHED | BYOD MAF/segment upload contract (`grounded=false`, refusals) | `ADR-0003-byod-contract.md` |
34
  | 0004 | PUBLISHED | Auth gate — OAuth allow-list + `ACCESS_CONTROL` dark-launch | `ADR-0004-auth-gate.md` |
35
  | 0005 | PUBLISHED | Public cBioPortal instance + curate-and-cache for v1 (no self-host) | `ADR-0005-cbioportal-public-vs-selfhost.md` |
36
+ | 0006 | PUBLISHED | variant×subtype join contract — label-join here / live-PurIST via the orchestrator | `ADR-0006-variant-subtype-join-contract.md` |
37
+ | 0007 | PUBLISHED | Re-curation cadence — upstream-signal staleness, visible age, no refusal | `ADR-0007-recuration-cadence.md` |
38
  | **0014** | **EXTERNAL** | Shared security-scan standard (pip-audit + bandit + gitleaks + trivy) — authored in `DecoupleRpy_Agent`. **Do not use;** adopt it, don't renumber it. Adopted here as `security/scan.sh`. | *(sibling repo)* |
39
  | **0015** | **EXTERNAL** | `data_level` / registry `modality` semantics — authored in `biodata-registry`. **Do not use.** A DNA `modality` extension would be an amendment *there*, under that owner. | *(sibling repo)* |
40
 
41
+ **Next free number: 0008.**
42
 
43
  ## Resolved: the 0003 collision (2026-07-24 → fixed 2026-07-25)
44
 
 
60
 
61
  ## Known upcoming ADRs (claim a number when you start)
62
 
 
 
 
63
  - **Does this agent use `biodata-registry` at all?** The founding design assumed a registry `modality`
64
  extension as a hard blocker; the build bypassed it and reads cBioPortal directly (ADR-0001
65
  amendment, 2026-07-25). Reopening that route needs this ADR — but nothing is blocked meanwhile.
docs/cbioportal-terms-and-deployability.md CHANGED
@@ -183,6 +183,12 @@ Sources:
183
  - [x] **C4 — Curate-and-cache** ✅ 2026-07-25: `python -m src.curate` is the only caller that touches
184
  the API; the request path reads committed artifacts and refuses an uncurated study. Verified by
185
  a test that sabotages the HTTP layer and still answers.
 
 
 
 
 
 
186
  - [x] **v1 = public instance** — ratified in ADR-0005; self-host deferred with the re-visit triggers
187
  above.
188
  - [x] Only **open-access** studies curated for v1 — enforced in code (C2), not just intended; a test
 
183
  - [x] **C4 — Curate-and-cache** ✅ 2026-07-25: `python -m src.curate` is the only caller that touches
184
  the API; the request path reads committed artifacts and refuses an uncurated study. Verified by
185
  a test that sabotages the HTTP layer and still answers.
186
+ **Cadence (ADR-0007, 2026-07-28):** C4's own consequence — a cached artifact drifting
187
+ unnoticed — is now handled. `python -m src.curate --check` compares upstream's `importDate`
188
+ against the one recorded at curation (one metadata request per study, no re-pull, C3-cheap),
189
+ every grounded answer carries the artifact's age in `citation.curation`, and staleness is
190
+ **visible, never a refusal**. `--check` deliberately does not write: re-curation stays a
191
+ reviewed commit, because an automatic repairer would put the API back on an unattended path.
192
  - [x] **v1 = public instance** — ratified in ADR-0005; self-host deferred with the re-visit triggers
193
  above.
194
  - [x] Only **open-access** studies curated for v1 — enforced in code (C2), not just intended; a test
gradio_ui.py CHANGED
@@ -190,6 +190,24 @@ def _licence_caution(result) -> str:
190
  return ""
191
 
192
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
193
  def _status_caution_md(result):
194
  """Surface the per-study coverage gate and any licence constraint — say it, loudly."""
195
  if not isinstance(result, dict):
@@ -203,6 +221,7 @@ def _status_caution_md(result):
203
  "'no alterations found'."
204
  )
205
  parts.append(_licence_caution(result))
 
206
  text = "".join(p for p in parts if p).strip()
207
  if not text:
208
  return gr.update(value="", visible=False)
@@ -223,9 +242,12 @@ def _subtype_caution_md(result):
223
  visible=True,
224
  )
225
  caveats = result.get("caveats") or []
226
- if not caveats:
 
 
 
227
  return gr.update(value="", visible=False)
228
- return gr.update(value="⚠️ **Caveats:** " + " ".join(caveats), visible=True)
229
 
230
 
231
  # --------------------------------------------------------------------------- #
@@ -247,6 +269,25 @@ def _subtype_caution_md(result):
247
  _MACHINE_TOKEN_HEADERS = ("x-orchestrator-token", "x-hf-authorization")
248
  _WHOAMI_CACHE: dict[str, str | None] = {}
249
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
250
 
251
  def _header_value(headers, name: str) -> str | None:
252
  if not headers:
@@ -278,28 +319,55 @@ def _resolve_token_identity(token: str) -> str | None:
278
  return identity
279
 
280
 
281
- def _machine_caller_identity(request) -> str | None:
282
- """Identity behind a machine call, or None (fail-closed)."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
283
  headers = getattr(request, "headers", None) if request else None
 
284
  for header in _MACHINE_TOKEN_HEADERS:
285
  raw = _header_value(headers, header)
286
  if not raw:
287
  continue
288
  token = raw[7:].strip() if raw.lower().startswith("bearer ") else raw.strip()
289
  if token:
 
290
  identity = _resolve_token_identity(token)
291
  if identity:
292
- return identity
293
- return None
294
 
295
 
296
  def _machine_gate(tool: str, source: dict, request) -> tuple[str | None, str | None]:
297
  """(identity, denial_json). A non-None denial must be returned to the caller verbatim."""
298
- username = _machine_caller_identity(request)
299
  allowed, denial = check_access(username)
300
  if not allowed:
 
 
301
  record_run(tool, username=username, source=source, status="denied")
302
- return username, json.dumps({"status": "denied", "reason": denial}, indent=2)
 
 
 
 
 
 
 
303
  return username, None
304
 
305
 
 
190
  return ""
191
 
192
 
193
+ def _freshness_caution(result) -> str:
194
+ """A visible line once a cohort's artifact is past the review horizon (ADR-0007).
195
+
196
+ Same reasoning as `_licence_caution`: the freshness block is already in the JSON, but a
197
+ snapshot date nobody reads is a snapshot date nobody accounts for. Deliberately NOT a
198
+ refusal — the answer is old, not wrong, and it still renders in full.
199
+ """
200
+ curation = (result or {}).get("citation", {}).get("curation") if isinstance(result, dict) else None
201
+ if not curation:
202
+ return ""
203
+ level = curation.get("level")
204
+ if level == "stale":
205
+ return f" 🕰️ **Snapshot age:** {curation.get('note')}"
206
+ if level in ("review_due", "unknown"):
207
+ return f" ℹ️ **Snapshot age:** {curation.get('note')}"
208
+ return ""
209
+
210
+
211
  def _status_caution_md(result):
212
  """Surface the per-study coverage gate and any licence constraint — say it, loudly."""
213
  if not isinstance(result, dict):
 
221
  "'no alterations found'."
222
  )
223
  parts.append(_licence_caution(result))
224
+ parts.append(_freshness_caution(result))
225
  text = "".join(p for p in parts if p).strip()
226
  if not text:
227
  return gr.update(value="", visible=False)
 
242
  visible=True,
243
  )
244
  caveats = result.get("caveats") or []
245
+ parts = ["⚠️ **Caveats:** " + " ".join(caveats) if caveats else ""]
246
+ parts.append(_freshness_caution(result))
247
+ text = "".join(p for p in parts if p).strip()
248
+ if not text:
249
  return gr.update(value="", visible=False)
250
+ return gr.update(value=text, visible=True)
251
 
252
 
253
  # --------------------------------------------------------------------------- #
 
269
  _MACHINE_TOKEN_HEADERS = ("x-orchestrator-token", "x-hf-authorization")
270
  _WHOAMI_CACHE: dict[str, str | None] = {}
271
 
272
+ # Machine-caller denial text, per diagnosis. Deliberately NOT the UI's "please sign in": a
273
+ # machine caller cannot sign in, and telling it to is what made this look like a Space-side
274
+ # auth bug. Each message names the side that has to act.
275
+ _MACHINE_DENIAL_REASONS = {
276
+ "no_token_header": (
277
+ "🔒 No machine credential presented: neither 'x-orchestrator-token' nor "
278
+ "'x-hf-authorization' reached this Space. The caller is misconfigured — the "
279
+ "orchestrator attaches that header only when its own HF_TOKEN is set, so check that "
280
+ "secret on the CALLING Space. (Note HF edge infra strips the reserved 'x-hf-*' "
281
+ "namespace, so only the custom header survives in a deployed Space.)"
282
+ ),
283
+ "token_unresolved": (
284
+ "🔒 A machine credential was presented but could not be resolved to a HuggingFace "
285
+ "identity — the token is invalid/expired, or the identity lookup failed upstream. "
286
+ "This is fail-closed by design; retry once before treating it as a rejection."
287
+ ),
288
+ "not_allowlisted": None, # check_access already says who was refused and why — keep it.
289
+ }
290
+
291
 
292
  def _header_value(headers, name: str) -> str | None:
293
  if not headers:
 
319
  return identity
320
 
321
 
322
+ def _machine_caller_identity(request) -> tuple[str | None, str]:
323
+ """`(identity, diagnosis)` behind a machine call. Identity None ⇒ denied (fail-closed).
324
+
325
+ The ``diagnosis`` is why, and it exists because the three failure modes are operationally
326
+ very different but used to be indistinguishable in the response:
327
+
328
+ - ``no_token_header`` — the caller sent neither machine header. **The caller is
329
+ misconfigured, not unauthorized**: the orchestrator only attaches
330
+ ``x-orchestrator-token`` when its own ``HF_TOKEN`` is set, so an unset secret on *its*
331
+ Space silently produces this. (Cost a live debugging session, 2026-07-27→28: this state
332
+ returned "Please sign in", which reads as a *this*-Space auth bug and sent the
333
+ investigation server-side, where nothing was wrong.)
334
+ - ``token_unresolved`` — a token arrived but ``whoami`` would not resolve it (bad/expired
335
+ token, or an upstream blip — the ambiguity ADR-0005 C4's reasoning warns about, since
336
+ this is an outbound call on the request path).
337
+ - ``not_allowlisted`` — identity resolved fine; it is simply not on ``ALLOWED_IDS``. The
338
+ only one of the three that is a genuine authorization decision.
339
+ """
340
  headers = getattr(request, "headers", None) if request else None
341
+ presented = False
342
  for header in _MACHINE_TOKEN_HEADERS:
343
  raw = _header_value(headers, header)
344
  if not raw:
345
  continue
346
  token = raw[7:].strip() if raw.lower().startswith("bearer ") else raw.strip()
347
  if token:
348
+ presented = True
349
  identity = _resolve_token_identity(token)
350
  if identity:
351
+ return identity, "resolved"
352
+ return None, ("token_unresolved" if presented else "no_token_header")
353
 
354
 
355
  def _machine_gate(tool: str, source: dict, request) -> tuple[str | None, str | None]:
356
  """(identity, denial_json). A non-None denial must be returned to the caller verbatim."""
357
+ username, diagnosis = _machine_caller_identity(request)
358
  allowed, denial = check_access(username)
359
  if not allowed:
360
+ if username:
361
+ diagnosis = "not_allowlisted"
362
  record_run(tool, username=username, source=source, status="denied")
363
+ return username, json.dumps(
364
+ {
365
+ "status": "denied",
366
+ "reason": _MACHINE_DENIAL_REASONS.get(diagnosis) or denial,
367
+ "machine_auth": diagnosis,
368
+ },
369
+ indent=2,
370
+ )
371
  return username, None
372
 
373
 
licenses.py CHANGED
@@ -291,7 +291,9 @@ def study_citation(study_id: str | None, terms: dict | None = None) -> str:
291
  return CBIOPORTAL_STUDY_UNKNOWN
292
 
293
 
294
- def citation_block(source: str | None, terms: dict | None = None) -> dict:
 
 
295
  """The attribution + terms block stamped onto every tool payload (C1 + C2).
296
 
297
  `source` is the tool's source string — ``"cbioportal:<study>"`` for a grounded cohort, or
@@ -301,6 +303,11 @@ def citation_block(source: str | None, terms: dict | None = None) -> dict:
301
  `terms` is the curated per-study license block (C2). It rides in the *answer*, not just in
302
  the artifact, because the person acting on the number is the one who needs to know the
303
  cohort's license — provenance that stops at the storage layer is provenance nobody reads.
 
 
 
 
 
304
  """
305
  src = (source or "").strip()
306
  if not src.startswith("cbioportal:"):
@@ -314,4 +321,6 @@ def citation_block(source: str | None, terms: dict | None = None) -> dict:
314
  }
315
  if terms:
316
  block["study_terms"] = terms
 
 
317
  return block
 
291
  return CBIOPORTAL_STUDY_UNKNOWN
292
 
293
 
294
+ def citation_block(
295
+ source: str | None, terms: dict | None = None, curation: dict | None = None
296
+ ) -> dict:
297
  """The attribution + terms block stamped onto every tool payload (C1 + C2).
298
 
299
  `source` is the tool's source string — ``"cbioportal:<study>"`` for a grounded cohort, or
 
303
  `terms` is the curated per-study license block (C2). It rides in the *answer*, not just in
304
  the artifact, because the person acting on the number is the one who needs to know the
305
  cohort's license — provenance that stops at the storage layer is provenance nobody reads.
306
+
307
+ `curation` is the artifact's freshness block (ADR-0007), carried for exactly the same
308
+ reason and computed by `curated_store.freshness_from_payload`. It states how old the
309
+ snapshot is and which upstream release it came from; it never changes whether an answer is
310
+ produced. An answer that is old is still an answer — it just has to say so.
311
  """
312
  src = (source or "").strip()
313
  if not src.startswith("cbioportal:"):
 
321
  }
322
  if terms:
323
  block["study_terms"] = terms
324
+ if curation:
325
+ block["curation"] = curation
326
  return block
memory.md CHANGED
@@ -500,6 +500,222 @@ failure.
500
 
501
  ---
502
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
503
  ## 2026-07-28 — Cohort survey; +2 curated (6 total); per-GENE coverage gate added
504
 
505
  Surveyed all 538 cBioPortal studies for pancreatic cohorts and curated the two that earned it.
 
500
 
501
  ---
502
 
503
+ ## 2026-07-27 — Carl allow-listed; UI gate verified BOTH ways; machine auth still failing
504
+
505
+ **`ALLOWED_IDS=anne-voigt,cpelz741`.** Carl's HF id was taken from `pdac-subtype-agent` prod
506
+ (`SHOWCASE_STATUS.md` records its live config) rather than guessed — a wrong username would either
507
+ lock out a colleague or hand access to an unrelated account that owns that name.
508
+
509
+ **The OAuth UI path is now VERIFIED, which was the last open deploy question.** Signed in as
510
+ `anne-voigt` in the deployed Space: a CCLE query rendered the frequency chart (KRAS ~84% across the
511
+ 57 lineage-filtered pancreatic lines) *and* displayed the licence caution — "commercial use of this
512
+ cohort is restricted under Broad DepMap/CCLE Portal Terms". Signed out, the same query returns
513
+ `{"status":"denied"}`. Both halves of the gate confirmed end-to-end in the real UI.
514
+
515
+ Incidental: the Space **does not render inside the huggingface.co page frame** (blank), but works at
516
+ the direct host `https://anne-voigt-pdac-genomics-agent.hf.space`. Use the direct host for testing.
517
+ The cohort dropdown renders its value only on focus — a Gradio display quirk, not an empty default.
518
+
519
+ **Broad terms read directly** (see previous entry): confirmed, and the *Continuity of Terms* clause
520
+ means rehosting requires reposting them in full — done in `CCLE_BROAD_TERMS.md`.
521
+
522
+ **Open and now spun off: the machine API denies the orchestrator despite correct configuration.**
523
+ The orchestrator already sends `x-orchestrator-token` (its commit `05bda30`, built for the subtype
524
+ agent's ADR-0018 gate) and that token resolves to `anne-voigt`, which is allow-listed here. It is
525
+ still denied. What is ruled out: the header is *not* being dropped — a local probe confirmed
526
+ `x-orchestrator-token` arrives in `gr.Request.headers` alongside `x-hf-authorization`. So the
527
+ failure is **server-side identity resolution**: `_resolve_token_identity()` calls
528
+ `huggingface_hub.whoami()` at request time from inside the Space and returns None on any failure.
529
+ It fails closed, which is right, but silently, which is why this took a live test to notice.
530
+
531
+ The design question worth answering before patching: **should a gate depend on an outbound network
532
+ call per request at all?** That is a third-party dependency on the request path — the same class of
533
+ coupling ADR-0005 C4 removed for cBioPortal. A shared-secret comparison would have neither the
534
+ latency nor the failure mode.
535
+
536
+ ---
537
+
538
+ ## 2026-07-28 — ADR-0006: the variant×subtype join contract (written down at last)
539
+
540
+ The agent's differentiator was the only major decision without an ADR — it lived as prose in
541
+ `TODO.md`, `DESIGN-pdac-genomics-agent.md` §4, and module docstrings, despite being the one part of
542
+ the design that **spans two Spaces**. Now `docs/adr/ADR-0006-variant-subtype-join-contract.md`.
543
+
544
+ **No behaviour changed.** This documents what M2 already ships (plus the vocabulary fix already in
545
+ code) and, crucially, states the orchestrator's obligations normatively so the other repo has a
546
+ contract to build against rather than a docstring to interpret.
547
+
548
+ What it settles:
549
+
550
+ 1. **The split.** Label-join is local (cohort carries a validated subtype clinical attribute → join
551
+ on sample id + test here); the live-PurIST join is orchestration (ADR-0001/0011 — stateless
552
+ siblings never call each other), returning `{join_available: false, route: "orchestrator"}`.
553
+ `altered` vocabularies and the Fisher-2×2 / χ²-R×2 rule are pinned here as the single normative
554
+ statement, because the orchestrator will implement the *same* test on its side.
555
+ 2. **The routing contract** — the discriminated return shape (switch on `join_available`, never on
556
+ the presence of `genes`), the orchestrator's three ordered steps, and an explicit prohibition:
557
+ it may **not** synthesize a label and call back through `subtype_attribute`.
558
+ **Sample-id alignment across Spaces stays OPEN** (the spike measured 145/146 *within* one
559
+ cBioPortal cohort — not the cross-Space case that would actually need a map). Interim rule:
560
+ exact string equality, **report join coverage**, no fuzzy/prefix matching, and a collapsed join
561
+ is a refusal **with the numbers shown**. A silent low-coverage join is the dangerous failure.
562
+ 3. **Label validation is load-bearing.** Attribute-id matching alone is unsafe *on evidence*:
563
+ `ccle_broad_2019`'s `SUBTYPE` holds histology ("Adenocarcinoma", "Melanoma"), which produced a
564
+ confident variant×histology association dressed as variant×subtype. Auto-detect now validates
565
+ VALUES against a **closed** PDAC subtype vocabulary (≥0.5 of non-blank values); an explicitly
566
+ named attribute bypasses it (caller took responsibility) and is always visible in `subtype_source`.
567
+ 4. **Statistical honesty — kept as-is, deliberately.** No multiple-testing correction across genes
568
+ (descriptive screen over a user-chosen panel; the right denominator is session-level knowledge a
569
+ stateless tool lacks — FDR is the orchestrator's call), mandatory caveat saying so, and
570
+ `testable=false` + machine-readable reason instead of a bogus p-value. Contingency table and `n`
571
+ are returned even when untestable — KRAS is altered in ~90%+ of PDAC, so the flagship gene is the
572
+ one that degenerates most often. No hard minimum-n refusal; show n, caveat, refuse only the
573
+ genuinely undefined.
574
+ 5. **Metadata label ≠ live PurIST call.** Never interchangeable, never mixed in one contingency
575
+ table, never blended to fill gaps. `subtype_source` is required on every joined result
576
+ (`clinical_attribute:<ATTR>` vs `purist:<version>`) and may not be dropped downstream.
577
+ "Do they agree?" is a *concordance analysis*, not a licence to pool.
578
+
579
+ Cross-references `pdac-subtype-agent` **ADR-0020**, whose alignment-by-sample-id primitive this
580
+ reuses — variant×subtype is a cross-*modality* instance of the same cross-source alignment pattern,
581
+ and inherits its align-never-pool + mandatory-caveat stance.
582
+
583
+ **Ledger protocol followed to the letter**: 0006 claimed with status `CLAIMED`, committed **alone**,
584
+ pushed to `main` (`a792086`) *before* the ADR was written — the step that exists because of the real
585
+ 0003 collision. Flipped to `PUBLISHED` after.
586
+
587
+ **Known follow-up (not a blocker):** two association-test implementations will exist (here and
588
+ orchestrator-side) and must agree on §1's rules, or the same cohort answers differently by route.
589
+
590
+ ---
591
+
592
+ ## 2026-07-28 — Machine-API auth: the Space was never broken; the bug is caller-side
593
+
594
+ **The 2026-07-27 diagnosis in the entry above is WRONG and is corrected here.** It concluded the
595
+ failure was "server-side identity resolution — `_resolve_token_identity()` calls `whoami` per
596
+ request and returns None". Probing the deployed Space directly disproves that:
597
+
598
+ | probe (against prod, `ACCESS_CONTROL=enforce`) | result |
599
+ |---|---|
600
+ | valid HF token in `x-orchestrator-token` | **`status: ok`** — full panel returned |
601
+ | bogus token in `x-orchestrator-token` | denied |
602
+ | header absent entirely | denied |
603
+
604
+ So `whoami` resolves fine from inside the Space and the gate discriminates exactly as designed.
605
+ **Root cause is the caller:** `pdac-analysis-orchestrator`'s `router._build_client_kwargs()`
606
+ attaches `x-orchestrator-token` **only when its own `HF_TOKEN` is set** — an unset secret on *that*
607
+ Space sends no header at all. The fix is a secret on the orchestrator Space; **no code change was
608
+ needed here.**
609
+
610
+ **Why it was misdiagnosed, and the actual fix shipped.** All three failure modes returned one
611
+ message — *"🔒 Please sign in with your HuggingFace account"* — which reads as a this-Space auth
612
+ bug and sent the investigation server-side, where nothing was wrong. A machine caller cannot sign
613
+ in; telling it to is a category error. Machine denials now carry a `machine_auth` diagnosis:
614
+
615
+ - `no_token_header` — **caller misconfigured**, message names the orchestrator's `HF_TOKEN`.
616
+ - `token_unresolved` — a token arrived but did not resolve (bad/expired, or an upstream blip).
617
+ - `not_allowlisted` — resolved fine, simply not on `ALLOWED_IDS`. The *only* one of the three that
618
+ is a real authorization decision; still names who was refused.
619
+
620
+ 4 regression tests pin the three states apart (`tests/test_machine_api_auth.py`). **113 green.**
621
+
622
+ **Method note worth keeping:** the earlier conclusion came from reading the code and a local
623
+ header probe; it took ~4 curl calls against the deployed Space to overturn it. When a gate is
624
+ "failing", probe the deployed thing with a known-good credential *before* theorising about why the
625
+ resolution path is broken — the cheap experiment discriminates caller-side from server-side in one
626
+ step. The lookalike messages are what made theorising feel productive.
627
+
628
+ **Still open (unchanged, not urgent):** should an auth gate depend on an outbound `whoami` on the
629
+ request path at all, versus a shared-secret comparison? Same coupling class ADR-0005 C4 removed for
630
+ cBioPortal. It works today; this is a latency/failure-mode question, not a correctness one.
631
+
632
+ ---
633
+
634
+ ## 2026-07-28 — Re-curation cadence (ADR-0007): drift, visibility, and no refusal
635
+
636
+ Closed the gap ADR-0005 C4 created and left open. `curated_at` had been written into every
637
+ artifact since commit one and **read by nothing** — no threshold, no warning, no policy. A cohort
638
+ could drift from upstream for years and every answer would look equally confident.
639
+
640
+ **Cadence is upstream-driven, not calendar-driven.** `python -m src.curate --check` reads one
641
+ `/studies/<id>` per curated study and compares upstream's `importDate` against the stamp now
642
+ recorded at curation (`source_import_date`). Drift → re-curate; no drift → do nothing, however old
643
+ the file is. **The empirical confirmation is the point:** all four cohorts were last imported
644
+ upstream in **January 2026** and curated in July — six months of a published cohort not moving.
645
+ A "refresh monthly" policy would have rewritten four byte-identical files and spent the C3
646
+ politeness budget for zero new data. `REVIEW_AFTER_DAYS=180` / `STALE_AFTER_DAYS=365` are only a
647
+ backstop for when nobody runs the check.
648
+
649
+ **Age is visible to the reader, not just the maintainer.** Every grounded answer's `citation` block
650
+ now carries `curation` (`curated_at`, `age_days`, `level`, `source_import_date`, `drift_comparable`,
651
+ plain-language `note`); the UI renders a line past the horizon, next to the existing licence
652
+ caution. Same stance that put `terms` in the response contract — provenance that stops at the
653
+ storage layer is provenance nobody honours.
654
+
655
+ **Stale never refuses.** A stale-but-valid cohort is old, not wrong: its calls were correct for the
656
+ release they were curated from. Refusing would withhold a correct answer over a maintenance lapse
657
+ and would dilute what a refusal from this agent means. Refusal stays for the *incorrectness* gates
658
+ (build, species, absent modality, access tier). Pinned by
659
+ `test_stale_artifact_still_answers_and_says_so`.
660
+
661
+ **`--check` never writes**, deliberately — re-curation stays a reviewed commit. A checker that
662
+ repaired what it found would put the API back on an unattended path, i.e. undo C4 quietly.
663
+
664
+ Two implementation notes worth keeping:
665
+
666
+ - **No schema bump.** `source_import_date` is optional; absence degrades to an honest
667
+ `drift_comparable: false`. Bumping to v3 would have refused all four existing artifacts and
668
+ forced exactly the blanket re-curation this cadence exists to avoid.
669
+ - The four pre-existing artifacts had their `source_import_date` **backfilled once** in the same
670
+ commit from a live metadata read (metadata only — no status call touched), so `--check` is useful
671
+ today rather than "unknown until each happens to be re-curated".
672
+
673
+ 121 tests pass (11 new, `tests/test_recuration_cadence.py`). `--check` verified live against the
674
+ public instance: all four `up_to_date`. **Not deployed** — the branch
675
+ `claude/infallible-kowalevski-2d4648` is pushed to origin and rebased onto `main`, but
676
+ `origin/main` IS the live Space, so promoting is a deliberate deploy decision.
677
+
678
+ **Renumbered 0006 → 0007, and the near-miss is the lesson again.** A concurrent lane had already
679
+ claimed *and published* 0006 (the variant×subtype join contract) on `main`. This lane's worktree
680
+ held its own copy of `RESERVED.md` and predated that claim, so the one-shared-row conflict could
681
+ not fire — the **third** time that root cause has bitten this family. The claim here was committed
682
+ alone and pushed, but to the feature branch rather than `main` (protocol rule 5), because
683
+ `origin/main` IS the live Space and pushing it is a deploy: rule 5 and the deploy model are in
684
+ genuine tension on this repo. **What actually caught it was reading `SHOWCASE_STATUS.md` before
685
+ updating it.** Practical check until rule 5 has a deploy-safe form here:
686
+ `git fetch origin main && git show origin/main:docs/adr/RESERVED.md` *before* claiming.
687
+
688
+ **Weekly job added the same day** (`.github/workflows/curation-freshness.yml`, Mondays 07:00 UTC +
689
+ `workflow_dispatch`, report-only). **It does not run, and the file says so in its own header rather
690
+ than looking green.** `origin` IS the HF Space and HuggingFace does not execute GitHub Actions —
691
+ and unlike **every** sibling (`DecoupleRpy_Agent`, `pdac-subtype-agent`,
692
+ `pdac-analysis-orchestrator`, each of which carries a second `github` remote),
693
+ **this repo has no GitHub mirror at all**, so it has never had CI of any kind. Committed anyway on
694
+ the `DecoupleRpy_Agent/.github/workflows/security.yml` precedent: the definition never forks, and
695
+ adding the mirror later is then zero work. **The one remaining ops step is creating
696
+ `Anne-Voigt/pdac-genomics-agent` on GitHub and `git push github main`** — that turns the check on
697
+ and gives the repo CI. A local launchd agent was deliberately NOT installed: a job that runs on one
698
+ laptop is invisible to everyone else, which is how the original gap happened. A test
699
+ (`test_weekly_job_is_report_only`) asserts the job can never gain a writing `src.curate` step —
700
+ that would re-establish the unattended API path C4 removed, in a file nobody reads once it is
701
+ green. 122 tests.
702
+
703
+ **Deployed 2026-07-28** (`6f23703` → live; a third lane's `4b8cb52` landed on top minutes later and
704
+ the Space is RUNNING at that sha, which contains ours). Rebased onto `fdfbfb3` (the machine-auth
705
+ diagnosis lane) before pushing — **126 tests green** with both lanes' changes together, up from our
706
+ 122. **Verified in prod, not just RUNNING:** `paad_tcga` → 186 samples, KRAS 136, TP53 104 —
707
+ identical to local and to the original spike — and `citation.curation` serves
708
+ `level: current, age_days: 2, source_import_date: 2026-01-12, drift_comparable: true`, confirming
709
+ the backfill landed in the artifacts the Space actually reads.
710
+
711
+ Not a bug, but worth knowing before someone loses an hour to it: the machine endpoints take
712
+ `genes` as a **comma-separated string** (`_split_genes`), not a list. Passing a Python list returns
713
+ `{"status":"error","reason":"AttributeError: 'list' object has no attribute 'split'"}` — an
714
+ unhelpful message for what is a caller-side mistake, and the same class of lookalike-error problem
715
+ the machine-auth lane just fixed for denials.
716
+
717
+ ---
718
+
719
  ## 2026-07-28 — Cohort survey; +2 curated (6 total); per-GENE coverage gate added
720
 
721
  Surveyed all 538 cBioPortal studies for pancreatic cohorts and curated the two that earned it.
src/curate.py CHANGED
@@ -5,6 +5,7 @@ Run by a developer, on purpose, rarely::
5
  python -m src.curate paad_tcga
6
  python -m src.curate paad_tcga ccle_broad_2019 --dry-run
7
  python -m src.curate --list
 
8
 
9
  It fetches a study's mutation + discrete-CNV calls plus the clinical attributes the request
10
  path needs, classifies everything through the same `variant_status` rules the live path always
@@ -18,6 +19,11 @@ someone looked at.
18
 
19
  **Politeness (condition C3).** Studies are curated serially with a pause between them, and the
20
  client self-throttles. Curation is a handful of requests per study, not a crawl.
 
 
 
 
 
21
  """
22
 
23
  from __future__ import annotations
@@ -101,7 +107,11 @@ def curate(study_id: str, *, dry_run: bool = False) -> dict:
101
  )
102
  clinical = fetch_clinical(study_id)
103
  payload = curated_store.build_payload(
104
- matrix, study_id=study_id, clinical=clinical, terms=terms
 
 
 
 
105
  )
106
  counts = payload["counts"]
107
  print(
@@ -121,6 +131,88 @@ def curate(study_id: str, *, dry_run: bool = False) -> dict:
121
  return payload
122
 
123
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
  def main(argv: list[str] | None = None) -> int:
125
  parser = argparse.ArgumentParser(
126
  prog="python -m src.curate", description=__doc__.splitlines()[0]
@@ -128,6 +220,12 @@ def main(argv: list[str] | None = None) -> int:
128
  parser.add_argument("studies", nargs="*", help="cBioPortal study ids, e.g. paad_tcga")
129
  parser.add_argument("--list", action="store_true", help="list already-curated studies")
130
  parser.add_argument("--dry-run", action="store_true", help="fetch and report, do not write")
 
 
 
 
 
 
131
  args = parser.parse_args(argv)
132
 
133
  if args.list:
@@ -135,8 +233,12 @@ def main(argv: list[str] | None = None) -> int:
135
  print("\n".join(curated) if curated else "(no curated studies)")
136
  return 0
137
 
 
 
 
 
138
  if not args.studies:
139
- parser.error("give at least one study id, or --list")
140
 
141
  # C3: show what we are identifying ourselves as, so an operator can see it in one place.
142
  # The +URL in the User-Agent is the contact channel; no extra config is required.
 
5
  python -m src.curate paad_tcga
6
  python -m src.curate paad_tcga ccle_broad_2019 --dry-run
7
  python -m src.curate --list
8
+ python -m src.curate --check # drift/age report, no re-curation, exit 1 if stale
9
 
10
  It fetches a study's mutation + discrete-CNV calls plus the clinical attributes the request
11
  path needs, classifies everything through the same `variant_status` rules the live path always
 
19
 
20
  **Politeness (condition C3).** Studies are curated serially with a pause between them, and the
21
  client self-throttles. Curation is a handful of requests per study, not a crawl.
22
+
23
+ **Cadence (ADR-0007).** Re-curation is event-driven, not scheduled: `--check` costs one metadata
24
+ request per study and tells you whether upstream re-imported the study since we curated it. Only
25
+ then is a re-pull worth making. `--check` never writes — refreshing an artifact stays a reviewed
26
+ commit, because that reviewability is the whole reason the artifacts are in git.
27
  """
28
 
29
  from __future__ import annotations
 
107
  )
108
  clinical = fetch_clinical(study_id)
109
  payload = curated_store.build_payload(
110
+ matrix,
111
+ study_id=study_id,
112
+ clinical=clinical,
113
+ terms=terms,
114
+ import_date=metadata.get("importDate"),
115
  )
116
  counts = payload["counts"]
117
  print(
 
131
  return payload
132
 
133
 
134
+ def check(study_ids: list[str] | None = None) -> tuple[list[dict], int]:
135
+ """Staleness CHECK — report drift without re-curating anything (ADR-0007).
136
+
137
+ One `/studies/<id>` request per study: compare upstream's `importDate` against the one
138
+ recorded when we curated, and report the artifact's age. That is the whole point of driving
139
+ off an upstream signal rather than a calendar — a cohort that upstream has not re-imported
140
+ needs no refresh no matter how old our file is, and re-pulling it on a monthly timer would
141
+ spend the C3 politeness budget to rewrite a byte-identical artifact.
142
+
143
+ This is a maintainer/CI tool, not a request-path one: it *reads* artifacts and never writes.
144
+ Re-curation stays a deliberate, reviewed commit — a checker that repaired what it found
145
+ would put the API back on an automatic path, which is exactly what C4 removed.
146
+
147
+ Returns `(rows, exit_code)`. Exit code 1 on any drift or `stale` artifact, so CI can fail.
148
+ """
149
+ studies = list(study_ids) if study_ids else curated_store.list_curated()
150
+ if not studies:
151
+ print("(no curated studies)")
152
+ return [], 0
153
+
154
+ rows: list[dict] = []
155
+ for i, study in enumerate(studies):
156
+ if i:
157
+ time.sleep(INTER_STUDY_PAUSE_S)
158
+ fresh = curated_store.freshness(study)
159
+ if fresh is None:
160
+ rows.append({"study": study, "state": "not_curated"})
161
+ continue
162
+ row = {
163
+ "study": study,
164
+ "age_days": fresh["age_days"],
165
+ "level": fresh["level"],
166
+ "curated_import_date": fresh["source_import_date"],
167
+ }
168
+ try:
169
+ row["upstream_import_date"] = cbioportal_io.study_metadata(study).get("importDate")
170
+ except Exception as exc: # noqa: BLE001 — an unreachable API is a check failure, not drift
171
+ row["state"] = "check_failed"
172
+ row["error"] = f"{type(exc).__name__}: {exc}"
173
+ rows.append(row)
174
+ continue
175
+ if not fresh["drift_comparable"]:
176
+ # Pre-ADR-0007 artifact: no recorded importDate to compare against. Say so — an
177
+ # unknown is not a "no drift".
178
+ row["state"] = "drift_unknown"
179
+ elif row["upstream_import_date"] != row["curated_import_date"]:
180
+ row["state"] = "drift"
181
+ else:
182
+ row["state"] = "up_to_date"
183
+ rows.append(row)
184
+
185
+ print(f"{'study':<24} {'state':<14} {'age':>6} upstream importDate")
186
+ for row in rows:
187
+ age = "?" if row.get("age_days") is None else f"{row['age_days']}d"
188
+ print(
189
+ f"{row['study']:<24} {row.get('state', '?'):<14} {age:>6} "
190
+ f"{row.get('upstream_import_date') or '-'}"
191
+ + (f" (curated from {row['curated_import_date']})" if row.get("state") == "drift" else "")
192
+ + (f" {row.get('error')}" if row.get("error") else "")
193
+ )
194
+
195
+ drifted = [r["study"] for r in rows if r.get("state") == "drift"]
196
+ stale = [r["study"] for r in rows if r.get("level") == "stale"]
197
+ unknown = [r["study"] for r in rows if r.get("state") == "drift_unknown"]
198
+ if unknown:
199
+ print(
200
+ f"\nNo recorded upstream importDate for: {', '.join(unknown)} — curated before "
201
+ "ADR-0007. Drift cannot be established for these until they are next re-curated."
202
+ )
203
+ if drifted:
204
+ print(f"\nUpstream has re-imported since we curated: {', '.join(drifted)}")
205
+ print(f"Review and re-curate deliberately: python -m src.curate {' '.join(drifted)}")
206
+ if stale:
207
+ print(
208
+ f"\nPast the {curated_store.STALE_AFTER_DAYS}-day horizon: {', '.join(stale)} "
209
+ "(answers still serve, and say their age)."
210
+ )
211
+ if not drifted and not stale:
212
+ print("\nAll curated artifacts are within the review horizon and match upstream.")
213
+ return rows, (1 if (drifted or stale) else 0)
214
+
215
+
216
  def main(argv: list[str] | None = None) -> int:
217
  parser = argparse.ArgumentParser(
218
  prog="python -m src.curate", description=__doc__.splitlines()[0]
 
220
  parser.add_argument("studies", nargs="*", help="cBioPortal study ids, e.g. paad_tcga")
221
  parser.add_argument("--list", action="store_true", help="list already-curated studies")
222
  parser.add_argument("--dry-run", action="store_true", help="fetch and report, do not write")
223
+ parser.add_argument(
224
+ "--check",
225
+ action="store_true",
226
+ help="report artifact age + upstream drift without re-curating; exit 1 if any drifted "
227
+ "or is past the stale horizon (CI-friendly)",
228
+ )
229
  args = parser.parse_args(argv)
230
 
231
  if args.list:
 
233
  print("\n".join(curated) if curated else "(no curated studies)")
234
  return 0
235
 
236
+ if args.check:
237
+ print(f"[check] User-Agent: {cbioportal_io.user_agent()}\n")
238
+ return check(args.studies)[1]
239
+
240
  if not args.studies:
241
+ parser.error("give at least one study id, --check, or --list")
242
 
243
  # C3: show what we are identifying ourselves as, so an operator can see it in one place.
244
  # The +URL in the User-Agent is the contact channel; no extra config is required.
src/resources/curated/ccle_broad_2019.json CHANGED
@@ -7328,7 +7328,7 @@
7328
  "n_mutated_cells": 3485,
7329
  "n_samples": 1739
7330
  },
7331
- "curated_at": "2026-07-28T19:28:30.612841+00:00",
7332
  "genes_assayed": null,
7333
  "modalities": {
7334
  "cnv": true,
@@ -14401,7 +14401,7 @@
14401
  "ELAVL1\tDJM1_SKIN": "ELAVL1 E6K",
14402
  "ELAVL1\tDV90_LUNG": "ELAVL1 G58*",
14403
  "ELAVL1\tEN_ENDOMETRIUM": "ELAVL1 X58_splice",
14404
- "ELAVL1\tGP2D_LARGE_INTESTINE": "ELAVL1 T143A",
14405
  "ELAVL1\tGP5D_LARGE_INTESTINE": "ELAVL1 R115W",
14406
  "ELAVL1\tHCC1428_BREAST": "ELAVL1 N280S",
14407
  "ELAVL1\tHEC108_ENDOMETRIUM": "ELAVL1 A240T",
@@ -15943,7 +15943,7 @@
15943
  "TGFBR2\tHEC6_ENDOMETRIUM": "TGFBR2 G586S",
15944
  "TGFBR2\tHRT18_LARGE_INTESTINE": "TGFBR2 L477P",
15945
  "TGFBR2\tHT115_LARGE_INTESTINE": "TGFBR2 R522*",
15946
- "TGFBR2\tKM12_LARGE_INTESTINE": "TGFBR2 L479P",
15947
  "TGFBR2\tLN235_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 F304L",
15948
  "TGFBR2\tLNZ308_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 R232W",
15949
  "TGFBR2\tLNZTA3WT4_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 R232W",
@@ -18717,6 +18717,7 @@
18717
  ],
18718
  "schema_version": 3,
18719
  "source": "cbioportal:ccle_broad_2019",
 
18720
  "study": "ccle_broad_2019",
18721
  "terms": {
18722
  "access_tier": "open",
 
7328
  "n_mutated_cells": 3485,
7329
  "n_samples": 1739
7330
  },
7331
+ "curated_at": "2026-07-28T23:39:38.203680+00:00",
7332
  "genes_assayed": null,
7333
  "modalities": {
7334
  "cnv": true,
 
14401
  "ELAVL1\tDJM1_SKIN": "ELAVL1 E6K",
14402
  "ELAVL1\tDV90_LUNG": "ELAVL1 G58*",
14403
  "ELAVL1\tEN_ENDOMETRIUM": "ELAVL1 X58_splice",
14404
+ "ELAVL1\tGP2D_LARGE_INTESTINE": "ELAVL1 R115W",
14405
  "ELAVL1\tGP5D_LARGE_INTESTINE": "ELAVL1 R115W",
14406
  "ELAVL1\tHCC1428_BREAST": "ELAVL1 N280S",
14407
  "ELAVL1\tHEC108_ENDOMETRIUM": "ELAVL1 A240T",
 
15943
  "TGFBR2\tHEC6_ENDOMETRIUM": "TGFBR2 G586S",
15944
  "TGFBR2\tHRT18_LARGE_INTESTINE": "TGFBR2 L477P",
15945
  "TGFBR2\tHT115_LARGE_INTESTINE": "TGFBR2 R522*",
15946
+ "TGFBR2\tKM12_LARGE_INTESTINE": "TGFBR2 S587F",
15947
  "TGFBR2\tLN235_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 F304L",
15948
  "TGFBR2\tLNZ308_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 R232W",
15949
  "TGFBR2\tLNZTA3WT4_CENTRAL_NERVOUS_SYSTEM": "TGFBR2 R232W",
 
18717
  ],
18718
  "schema_version": 3,
18719
  "source": "cbioportal:ccle_broad_2019",
18720
+ "source_import_date": "2026-01-05 21:05:38",
18721
  "study": "ccle_broad_2019",
18722
  "terms": {
18723
  "access_tier": "open",
src/resources/curated/paad_qcmg_uq_2016.json CHANGED
@@ -924,7 +924,7 @@
924
  "n_mutated_cells": 932,
925
  "n_samples": 456
926
  },
927
- "curated_at": "2026-07-28T19:28:38.629322+00:00",
928
  "genes_assayed": null,
929
  "modalities": {
930
  "cnv": false,
@@ -3705,6 +3705,7 @@
3705
  ],
3706
  "schema_version": 3,
3707
  "source": "cbioportal:paad_qcmg_uq_2016",
 
3708
  "study": "paad_qcmg_uq_2016",
3709
  "terms": {
3710
  "access_tier": "open",
 
924
  "n_mutated_cells": 932,
925
  "n_samples": 456
926
  },
927
+ "curated_at": "2026-07-28T23:39:47.011537+00:00",
928
  "genes_assayed": null,
929
  "modalities": {
930
  "cnv": false,
 
3705
  ],
3706
  "schema_version": 3,
3707
  "source": "cbioportal:paad_qcmg_uq_2016",
3708
+ "source_import_date": "2026-01-12 13:30:07",
3709
  "study": "paad_qcmg_uq_2016",
3710
  "terms": {
3711
  "access_tier": "open",
src/resources/curated/paad_tcga.json CHANGED
@@ -1672,7 +1672,7 @@
1672
  "n_mutated_cells": 350,
1673
  "n_samples": 186
1674
  },
1675
- "curated_at": "2026-07-28T19:28:08.341270+00:00",
1676
  "genes_assayed": null,
1677
  "modalities": {
1678
  "cnv": true,
@@ -2964,6 +2964,7 @@
2964
  ],
2965
  "schema_version": 3,
2966
  "source": "cbioportal:paad_tcga",
 
2967
  "study": "paad_tcga",
2968
  "terms": {
2969
  "access_tier": "open",
 
1672
  "n_mutated_cells": 350,
1673
  "n_samples": 186
1674
  },
1675
+ "curated_at": "2026-07-28T23:39:26.922919+00:00",
1676
  "genes_assayed": null,
1677
  "modalities": {
1678
  "cnv": true,
 
2964
  ],
2965
  "schema_version": 3,
2966
  "source": "cbioportal:paad_tcga",
2967
+ "source_import_date": "2026-01-12 13:01:54",
2968
  "study": "paad_tcga",
2969
  "terms": {
2970
  "access_tier": "open",
src/resources/curated/paad_utsw_2015.json CHANGED
@@ -1576,7 +1576,7 @@
1576
  "n_mutated_cells": 222,
1577
  "n_samples": 109
1578
  },
1579
- "curated_at": "2026-07-28T19:28:46.756019+00:00",
1580
  "genes_assayed": null,
1581
  "modalities": {
1582
  "cnv": true,
@@ -2203,6 +2203,7 @@
2203
  ],
2204
  "schema_version": 3,
2205
  "source": "cbioportal:paad_utsw_2015",
 
2206
  "study": "paad_utsw_2015",
2207
  "terms": {
2208
  "access_tier": "open",
 
1576
  "n_mutated_cells": 222,
1577
  "n_samples": 109
1578
  },
1579
+ "curated_at": "2026-07-28T23:39:56.327392+00:00",
1580
  "genes_assayed": null,
1581
  "modalities": {
1582
  "cnv": true,
 
2203
  ],
2204
  "schema_version": 3,
2205
  "source": "cbioportal:paad_utsw_2015",
2206
+ "source_import_date": "2026-01-12 13:08:04",
2207
  "study": "paad_utsw_2015",
2208
  "terms": {
2209
  "access_tier": "open",
src/resources/curated/pancreas_cptac_gdc.json CHANGED
@@ -926,7 +926,7 @@
926
  "n_mutated_cells": 411,
927
  "n_samples": 183
928
  },
929
- "curated_at": "2026-07-28T19:28:54.461154+00:00",
930
  "genes_assayed": null,
931
  "modalities": {
932
  "cnv": true,
@@ -2339,6 +2339,7 @@
2339
  ],
2340
  "schema_version": 3,
2341
  "source": "cbioportal:pancreas_cptac_gdc",
 
2342
  "study": "pancreas_cptac_gdc",
2343
  "terms": {
2344
  "access_tier": "open",
 
926
  "n_mutated_cells": 411,
927
  "n_samples": 183
928
  },
929
+ "curated_at": "2026-07-28T23:40:06.073894+00:00",
930
  "genes_assayed": null,
931
  "modalities": {
932
  "cnv": true,
 
2339
  ],
2340
  "schema_version": 3,
2341
  "source": "cbioportal:pancreas_cptac_gdc",
2342
+ "source_import_date": "2026-01-14 11:11:19",
2343
  "study": "pancreas_cptac_gdc",
2344
  "terms": {
2345
  "access_tier": "open",
src/resources/curated/pdac_msk_2024.json CHANGED
@@ -5373,7 +5373,7 @@
5373
  "n_mutated_cells": 5970,
5374
  "n_samples": 2336
5375
  },
5376
- "curated_at": "2026-07-28T19:29:05.834992+00:00",
5377
  "genes_assayed": [
5378
  "KRAS",
5379
  "TP53",
@@ -19738,6 +19738,7 @@
19738
  ],
19739
  "schema_version": 3,
19740
  "source": "cbioportal:pdac_msk_2024",
 
19741
  "study": "pdac_msk_2024",
19742
  "terms": {
19743
  "access_tier": "open",
 
5373
  "n_mutated_cells": 5970,
5374
  "n_samples": 2336
5375
  },
5376
+ "curated_at": "2026-07-28T23:40:19.051177+00:00",
5377
  "genes_assayed": [
5378
  "KRAS",
5379
  "TP53",
 
19738
  ],
19739
  "schema_version": 3,
19740
  "source": "cbioportal:pdac_msk_2024",
19741
+ "source_import_date": "2026-01-13 10:24:41",
19742
  "study": "pdac_msk_2024",
19743
  "terms": {
19744
  "access_tier": "open",
src/tools/query_variant_status.py CHANGED
@@ -131,7 +131,9 @@ def query_variant_status(genes: list[str], source: str) -> dict:
131
  "unassayed_genes": unassayed, # explicit, so a caller can SAY what was not measured
132
  "genes": out_genes,
133
  # C1 (ADR-0005 / cBioPortal terms): attribution rides the response contract itself.
134
- "citation": citation_block(sm.source, getattr(sm, "terms", None)),
 
 
135
  }
136
 
137
 
 
131
  "unassayed_genes": unassayed, # explicit, so a caller can SAY what was not measured
132
  "genes": out_genes,
133
  # C1 (ADR-0005 / cBioPortal terms): attribution rides the response contract itself.
134
+ "citation": citation_block(
135
+ sm.source, getattr(sm, "terms", None), getattr(sm, "curation", None)
136
+ ),
137
  }
138
 
139
 
src/tools/variant_by_subtype.py CHANGED
@@ -115,7 +115,9 @@ def variant_by_subtype(
115
  "Subtype label is a pre-existing cohort clinical attribute, not a live PurIST call.",
116
  "Association is descriptive (no multiple-testing correction across genes here).",
117
  ],
118
- "citation": citation_block(sm.source, getattr(sm, "terms", None)),
 
 
119
  }
120
 
121
 
 
115
  "Subtype label is a pre-existing cohort clinical attribute, not a live PurIST call.",
116
  "Association is descriptive (no multiple-testing correction across genes here).",
117
  ],
118
+ "citation": citation_block(
119
+ sm.source, getattr(sm, "terms", None), getattr(sm, "curation", None)
120
+ ),
121
  }
122
 
123
 
src/workflows/curated_store.py CHANGED
@@ -41,6 +41,25 @@ SCHEMA_VERSION = 3
41
 
42
  ARTIFACT_DIR = Path(__file__).resolve().parents[1] / "resources" / "curated"
43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  # Clinical attributes an artifact must carry for the request path to stay offline:
45
  # lineage (pan-cancer subsetting) + every subtype label the join knows how to use.
46
  LINEAGE_ATTRIBUTES = ["CANCER_TYPE", "ONCOTREE_CODE"]
@@ -89,12 +108,21 @@ def write(payload: dict) -> Path:
89
  return path
90
 
91
 
92
- def build_payload(matrix, *, study_id: str, clinical: dict, terms: dict) -> dict:
 
 
93
  """Serialize a live-built `StatusMatrix` + its clinical attributes into an artifact.
94
 
95
  Sparse: only cells that differ from the implied baseline (`WT` / `neutral`) are kept.
96
  `terms` is the per-study license/access block (condition C2), stored so the serve path can
97
  re-check it without the API and so every answer can carry it.
 
 
 
 
 
 
 
98
  """
99
  def _sparse(frame, baseline: str) -> dict:
100
  if frame is None:
@@ -112,6 +140,7 @@ def build_payload(matrix, *, study_id: str, clinical: dict, terms: dict) -> dict
112
  "terms": terms,
113
  "source": matrix.source,
114
  "curated_at": datetime.now(UTC).isoformat(),
 
115
  "panel_genes": list(matrix.genes),
116
  # Per-GENE assay coverage for targeted cohorts (null = genome/exome-wide, all covered).
117
  # Not a schema bump: every artifact written before this key existed came from a
@@ -142,6 +171,84 @@ def build_payload(matrix, *, study_id: str, clinical: dict, terms: dict) -> dict
142
  }
143
 
144
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
145
  def study_terms(study_id: str) -> dict | None:
146
  """The cached per-study license/access block (C2), or None if the study is not curated."""
147
  if not is_curated(study_id):
 
41
 
42
  ARTIFACT_DIR = Path(__file__).resolve().parents[1] / "resources" / "curated"
43
 
44
+ # --------------------------------------------------------------------------- #
45
+ # Re-curation cadence (ADR-0007)
46
+ # --------------------------------------------------------------------------- #
47
+ # An artifact is a point-in-time snapshot. It carried `curated_at` from the start, but nothing
48
+ # read it — so a cohort could drift from upstream for years and every answer would look equally
49
+ # confident. These two horizons make age *visible*; they never make an answer refuse.
50
+ #
51
+ # The horizons are a BACKSTOP, not the primary signal. The primary signal is upstream's own
52
+ # `importDate` (`python -m src.curate --check`), because cBioPortal studies are versioned
53
+ # releases that change rarely: a published cohort like `paad_tcga` can be correct for years, and
54
+ # re-curating it on a calendar is churn that spends the C3 politeness budget for no new data.
55
+ # Time-based horizons exist only to catch the case where nobody ran the check either.
56
+ #
57
+ # 180 days = "a human should look" (roughly one release cycle for the refreshed cohorts, e.g.
58
+ # CCLE). 365 days = "say so in the answer itself" — a year is long enough that a reader acting
59
+ # on the number deserves to be told, whether or not anything upstream actually moved.
60
+ REVIEW_AFTER_DAYS = 180
61
+ STALE_AFTER_DAYS = 365
62
+
63
  # Clinical attributes an artifact must carry for the request path to stay offline:
64
  # lineage (pan-cancer subsetting) + every subtype label the join knows how to use.
65
  LINEAGE_ATTRIBUTES = ["CANCER_TYPE", "ONCOTREE_CODE"]
 
108
  return path
109
 
110
 
111
+ def build_payload(
112
+ matrix, *, study_id: str, clinical: dict, terms: dict, import_date: str | None = None
113
+ ) -> dict:
114
  """Serialize a live-built `StatusMatrix` + its clinical attributes into an artifact.
115
 
116
  Sparse: only cells that differ from the implied baseline (`WT` / `neutral`) are kept.
117
  `terms` is the per-study license/access block (condition C2), stored so the serve path can
118
  re-check it without the API and so every answer can carry it.
119
+
120
+ `import_date` is upstream's own `importDate` for the study *as of this curation* (ADR-0007).
121
+ It is what makes drift detectable without re-curating: `--check` fetches one metadata record
122
+ and compares. **Deliberately NOT a schema bump** — the four artifacts curated before the
123
+ cadence existed simply lack the field, and refusing them would force exactly the blanket
124
+ re-curation this cadence is designed to avoid. Absence degrades to "drift not comparable",
125
+ which the check reports honestly instead of guessing.
126
  """
127
  def _sparse(frame, baseline: str) -> dict:
128
  if frame is None:
 
140
  "terms": terms,
141
  "source": matrix.source,
142
  "curated_at": datetime.now(UTC).isoformat(),
143
+ "source_import_date": import_date,
144
  "panel_genes": list(matrix.genes),
145
  # Per-GENE assay coverage for targeted cohorts (null = genome/exome-wide, all covered).
146
  # Not a schema bump: every artifact written before this key existed came from a
 
171
  }
172
 
173
 
174
+ def _parse_ts(value: str | None) -> datetime | None:
175
+ """Parse an ISO timestamp, tolerating a missing/naive/garbage value.
176
+
177
+ A freshness *display* must never be the thing that breaks an answer, so an unparseable
178
+ `curated_at` reports "unknown age" rather than raising on the request path.
179
+ """
180
+ if not value:
181
+ return None
182
+ try:
183
+ parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))
184
+ except ValueError:
185
+ return None
186
+ return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
187
+
188
+
189
+ def freshness_from_payload(payload: dict, *, now: datetime | None = None) -> dict:
190
+ """How old this artifact is, and whether that is worth saying out loud (ADR-0007).
191
+
192
+ Returned with every grounded answer via `licenses.citation_block`, because the person
193
+ acting on a frequency is the one who needs to know the snapshot is from 2026. Provenance
194
+ that stops at the storage layer is provenance nobody honours — the same argument that put
195
+ `terms` in the response contract.
196
+
197
+ `level` is advisory in all three states. **Nothing here refuses.** A stale-but-valid cohort
198
+ is old, not wrong: the numbers were correct for the release they were curated from, and
199
+ withholding a correct answer because a maintainer missed a review window would trade a real
200
+ answer for a maintenance signal. Refusal is reserved for the gates where the answer would
201
+ be *incorrect* — genome build, species, absent modality, controlled access.
202
+ """
203
+ now = now or datetime.now(UTC)
204
+ curated_at = payload.get("curated_at")
205
+ parsed = _parse_ts(curated_at)
206
+ age_days = None if parsed is None else max(0, (now - parsed).days)
207
+
208
+ if age_days is None:
209
+ level = "unknown"
210
+ note = (
211
+ "This cohort's artifact carries no readable curation date, so its age cannot be "
212
+ "stated. Re-curate it to restore the provenance."
213
+ )
214
+ elif age_days >= STALE_AFTER_DAYS:
215
+ level = "stale"
216
+ note = (
217
+ f"This cohort was curated {age_days} days ago and has not been refreshed since. "
218
+ "The status calls were correct for the upstream release they were curated from; "
219
+ "check whether that release has been superseded before relying on them."
220
+ )
221
+ elif age_days >= REVIEW_AFTER_DAYS:
222
+ level = "review_due"
223
+ note = (
224
+ f"This cohort was curated {age_days} days ago and is due a re-curation review "
225
+ "(`python -m src.curate --check`)."
226
+ )
227
+ else:
228
+ level = "current"
229
+ note = f"Curated {age_days} days ago from the upstream release named below."
230
+
231
+ return {
232
+ "curated_at": curated_at,
233
+ "age_days": age_days,
234
+ "level": level,
235
+ "note": note,
236
+ # Upstream's release stamp as of curation. `None` on artifacts predating ADR-0007 —
237
+ # reported as such rather than silently treated as "no drift".
238
+ "source_import_date": payload.get("source_import_date"),
239
+ "drift_comparable": payload.get("source_import_date") is not None,
240
+ "review_after_days": REVIEW_AFTER_DAYS,
241
+ "stale_after_days": STALE_AFTER_DAYS,
242
+ }
243
+
244
+
245
+ def freshness(study_id: str, *, now: datetime | None = None) -> dict | None:
246
+ """`freshness_from_payload` for a curated study id; None if it is not curated."""
247
+ if not is_curated(study_id):
248
+ return None
249
+ return freshness_from_payload(read(study_id), now=now)
250
+
251
+
252
  def study_terms(study_id: str) -> dict | None:
253
  """The cached per-study license/access block (C2), or None if the study is not curated."""
254
  if not is_curated(study_id):
src/workflows/variant_status.py CHANGED
@@ -132,6 +132,9 @@ class StatusMatrix:
132
  # full cohort silently counts it as one (paad_tcga: 186 samples, 150 sequenced -> KRAS
133
  # reads 73% instead of its true 91%).
134
  profiled: dict = None # {"mutation": [ids] | None, "cnv": [ids] | None}
 
 
 
135
 
136
  def __post_init__(self):
137
  if self.profiled is None:
@@ -252,6 +255,7 @@ def _from_artifact(payload: dict, *, restrict: list[str] | None) -> StatusMatrix
252
  terms=payload.get("terms"),
253
  genes_assayed=payload.get("genes_assayed"),
254
  profiled=payload.get("profiled") or {},
 
255
  )
256
 
257
 
 
132
  # full cohort silently counts it as one (paad_tcga: 186 samples, 150 sequenced -> KRAS
133
  # reads 73% instead of its true 91%).
134
  profiled: dict = None # {"mutation": [ids] | None, "cnv": [ids] | None}
135
+ # Artifact age / upstream-release provenance (ADR-0007). None for BYOD and for a
136
+ # freshly-curated in-memory matrix — an artifact that was never written has no age.
137
+ curation: dict | None = None
138
 
139
  def __post_init__(self):
140
  if self.profiled is None:
 
255
  terms=payload.get("terms"),
256
  genes_assayed=payload.get("genes_assayed"),
257
  profiled=payload.get("profiled") or {},
258
+ curation=curated_store.freshness_from_payload(payload),
259
  )
260
 
261
 
tests/test_machine_api_auth.py CHANGED
@@ -110,3 +110,60 @@ def test_empty_genes_falls_back_to_full_panel():
110
  """The machine contract sends a comma-separated string; empty means 'the whole panel'."""
111
  assert gradio_ui._split_genes("") == list(gradio_ui.PANEL)
112
  assert gradio_ui._split_genes("kras, tp53") == ["KRAS", "TP53"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
110
  """The machine contract sends a comma-separated string; empty means 'the whole panel'."""
111
  assert gradio_ui._split_genes("") == list(gradio_ui.PANEL)
112
  assert gradio_ui._split_genes("kras, tp53") == ["KRAS", "TP53"]
113
+
114
+
115
+ # --------------------------------------------------------------------------- #
116
+ # Denial DIAGNOSIS — the three failure modes must be distinguishable
117
+ # --------------------------------------------------------------------------- #
118
+ # Regression guard for a real misdiagnosis (2026-07-27→28). The orchestrator was denied and
119
+ # the response said "Please sign in with your HuggingFace account", so the investigation went
120
+ # server-side — probing headers, suspecting `whoami`. Nothing was wrong here: the caller had
121
+ # simply sent no header at all (its own HF_TOKEN was unset). These three states are
122
+ # operationally different and must never again be one indistinguishable message.
123
+
124
+
125
+ def test_no_token_header_is_diagnosed_as_caller_misconfiguration(monkeypatch):
126
+ monkeypatch.setattr(gradio_ui, "_query_variant_status", _explode)
127
+ body = json.loads(gradio_ui.query_variant_status(source="x", request=_Request()))
128
+
129
+ assert body["status"] == "denied"
130
+ assert body["machine_auth"] == "no_token_header"
131
+ # names the calling Space's secret, and does NOT tell a machine to sign in
132
+ assert "HF_TOKEN" in body["reason"]
133
+ assert "sign in" not in body["reason"].lower()
134
+
135
+
136
+ def test_unresolvable_token_is_diagnosed_separately_from_a_missing_one(monkeypatch):
137
+ monkeypatch.setattr(gradio_ui, "_resolve_token_identity", lambda token: None)
138
+ monkeypatch.setattr(gradio_ui, "_query_variant_status", _explode)
139
+
140
+ body = json.loads(
141
+ gradio_ui.query_variant_status(
142
+ source="x", request=_Request({"x-orchestrator-token": "bad-token"})
143
+ )
144
+ )
145
+
146
+ assert body["machine_auth"] == "token_unresolved"
147
+
148
+
149
+ def test_resolved_but_unlisted_identity_is_the_only_real_authorization_denial(monkeypatch):
150
+ """Identity resolved fine — this one IS an allow-list decision, and still names who."""
151
+ monkeypatch.setattr(gradio_ui, "_resolve_token_identity", lambda token: "stranger-bot")
152
+ monkeypatch.setattr(gradio_ui, "_query_variant_status", _explode)
153
+
154
+ body = json.loads(
155
+ gradio_ui.query_variant_status(
156
+ source="x", request=_Request({"x-orchestrator-token": "good-token"})
157
+ )
158
+ )
159
+
160
+ assert body["machine_auth"] == "not_allowlisted"
161
+ assert "stranger-bot" in body["reason"]
162
+
163
+
164
+ def test_identity_resolution_reports_resolved_on_success(monkeypatch):
165
+ monkeypatch.setattr(gradio_ui, "_resolve_token_identity", lambda token: "orchestrator-bot")
166
+ identity, diagnosis = gradio_ui._machine_caller_identity(
167
+ _Request({"x-orchestrator-token": "good-token"})
168
+ )
169
+ assert (identity, diagnosis) == ("orchestrator-bot", "resolved")
tests/test_recuration_cadence.py ADDED
@@ -0,0 +1,203 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Re-curation cadence — artifact age is visible, and never a refusal (ADR-0007).
2
+
3
+ Two claims are load-bearing here:
4
+
5
+ 1. `test_stale_artifact_still_answers` — the whole decision. A stale cohort is old, not wrong,
6
+ so the answer must still be produced *and* must say its age. A future change that turns
7
+ staleness into a refusal breaks this test, which is the point.
8
+ 2. `test_check_makes_one_metadata_call_per_study` — the cadence is driven by upstream's
9
+ `importDate`, not a calendar, and checking costs one request per study rather than a re-pull.
10
+ """
11
+
12
+ from __future__ import annotations
13
+
14
+ import json
15
+ from datetime import UTC, datetime, timedelta
16
+
17
+ import pytest
18
+
19
+ from src import curate
20
+ from src.tools.query_variant_status import query_variant_status
21
+ from src.workflows import cbioportal_io, curated_store
22
+
23
+
24
+ def _age_artifact(study: str, days: int, *, import_date: str | None = "2026-01-01 00:00:00"):
25
+ """Rewrite a curated artifact's dates in place (test-local; curation is normally the writer)."""
26
+ payload = curated_store.read(study)
27
+ payload["curated_at"] = (datetime.now(UTC) - timedelta(days=days)).isoformat()
28
+ payload["source_import_date"] = import_date
29
+ curated_store.write(payload)
30
+ return payload
31
+
32
+
33
+ # --------------------------------------------------------------------------- #
34
+ # Freshness levels
35
+ # --------------------------------------------------------------------------- #
36
+ @pytest.mark.parametrize(
37
+ ("days", "expected"),
38
+ [
39
+ (1, "current"),
40
+ (curated_store.REVIEW_AFTER_DAYS + 1, "review_due"),
41
+ (curated_store.STALE_AFTER_DAYS + 1, "stale"),
42
+ ],
43
+ )
44
+ def test_freshness_levels(patched_cbio, days, expected):
45
+ _age_artifact("paad_tcga", days)
46
+ assert curated_store.freshness("paad_tcga")["level"] == expected
47
+
48
+
49
+ def test_unparseable_curated_at_reports_unknown_not_crash(patched_cbio):
50
+ """A broken date must degrade to 'unknown age', never take an answer down with it."""
51
+ payload = curated_store.read("paad_tcga")
52
+ payload["curated_at"] = "not-a-date"
53
+ curated_store.write(payload)
54
+
55
+ fresh = curated_store.freshness("paad_tcga")
56
+ assert fresh["level"] == "unknown"
57
+ assert fresh["age_days"] is None
58
+ # and the tool still answers
59
+ assert query_variant_status(["KRAS"], "cbioportal:paad_tcga")["n_samples"] == 5
60
+
61
+
62
+ def test_missing_import_date_is_not_reported_as_no_drift(patched_cbio):
63
+ """Artifacts predating ADR-0007 carry no importDate — that is an unknown, not a match."""
64
+ _age_artifact("paad_tcga", 10, import_date=None)
65
+ assert curated_store.freshness("paad_tcga")["drift_comparable"] is False
66
+
67
+
68
+ # --------------------------------------------------------------------------- #
69
+ # Visibility: the age rides in the answer
70
+ # --------------------------------------------------------------------------- #
71
+ def test_answer_carries_the_curation_block(patched_cbio):
72
+ """Every grounded answer states its snapshot age, next to the citation and terms."""
73
+ result = query_variant_status(["KRAS"], "cbioportal:paad_tcga")
74
+ curation = result["citation"]["curation"]
75
+
76
+ assert curation["level"] == "current"
77
+ assert curation["age_days"] is not None
78
+ assert curation["curated_at"]
79
+
80
+
81
+ def test_stale_artifact_still_answers_and_says_so(patched_cbio):
82
+ """THE decision (ADR-0007): stale is visible, not refused.
83
+
84
+ A stale-but-valid cohort was correct for the upstream release it was curated from. Refusing
85
+ it would withhold a correct answer over a maintenance lapse — refusal stays reserved for the
86
+ gates where the answer would be *wrong* (build, species, modality, access tier).
87
+ """
88
+ _age_artifact("paad_tcga", curated_store.STALE_AFTER_DAYS + 30)
89
+
90
+ result = query_variant_status(["KRAS"], "cbioportal:paad_tcga")
91
+
92
+ assert result["n_samples"] == 5 # the answer is intact
93
+ assert "KRAS" in result["genes"]
94
+ curation = result["citation"]["curation"]
95
+ assert curation["level"] == "stale"
96
+ assert str(curation["age_days"]) in curation["note"]
97
+
98
+
99
+ def test_ui_surfaces_stale_note(patched_cbio):
100
+ """Past the horizon the page says so — a date only in the JSON is a date nobody reads."""
101
+ from gradio_ui import _freshness_caution
102
+
103
+ _age_artifact("paad_tcga", curated_store.STALE_AFTER_DAYS + 5)
104
+ stale = query_variant_status(["KRAS"], "cbioportal:paad_tcga")
105
+ assert "Snapshot age" in _freshness_caution(stale)
106
+
107
+ _age_artifact("paad_tcga", 5)
108
+ fresh = query_variant_status(["KRAS"], "cbioportal:paad_tcga")
109
+ assert _freshness_caution(fresh) == "" # and stays quiet when there is nothing to say
110
+
111
+
112
+ # --------------------------------------------------------------------------- #
113
+ # `--check`: upstream-signal drift, cheap and read-only
114
+ # --------------------------------------------------------------------------- #
115
+ def test_check_detects_drift_and_exits_nonzero(patched_cbio, monkeypatch):
116
+ _age_artifact("paad_tcga", 10, import_date="2026-01-01 00:00:00")
117
+ monkeypatch.setattr(
118
+ cbioportal_io, "study_metadata", lambda s: {"importDate": "2026-07-01 00:00:00"}
119
+ )
120
+ monkeypatch.setattr(curate.time, "sleep", lambda *_: None)
121
+
122
+ rows, code = curate.check(["paad_tcga"])
123
+
124
+ assert rows[0]["state"] == "drift"
125
+ assert code == 1
126
+
127
+
128
+ def test_check_reports_up_to_date_when_upstream_matches(patched_cbio, monkeypatch):
129
+ _age_artifact("paad_tcga", 10, import_date="2026-01-01 00:00:00")
130
+ monkeypatch.setattr(
131
+ cbioportal_io, "study_metadata", lambda s: {"importDate": "2026-01-01 00:00:00"}
132
+ )
133
+ rows, code = curate.check(["paad_tcga"])
134
+
135
+ assert rows[0]["state"] == "up_to_date"
136
+ assert code == 0
137
+
138
+
139
+ def test_check_makes_one_metadata_call_per_study_and_writes_nothing(patched_cbio, monkeypatch):
140
+ """The cadence is cheap by construction: a check is metadata, not a re-pull (C3).
141
+
142
+ It must also not mutate artifacts — re-curation stays a reviewed commit, and a checker that
143
+ repaired what it found would put the API back on an automatic path (C4).
144
+ """
145
+ _age_artifact("paad_tcga", 10)
146
+ before = json.dumps(curated_store.read("paad_tcga"), sort_keys=True)
147
+
148
+ calls = []
149
+ monkeypatch.setattr(
150
+ cbioportal_io,
151
+ "study_metadata",
152
+ lambda s: calls.append(s) or {"importDate": "2026-01-01 00:00:00"},
153
+ )
154
+ for name in ("fetch_mutations", "fetch_molecular_data", "clinical_data", "entrez_ids"):
155
+ monkeypatch.setattr(
156
+ cbioportal_io,
157
+ name,
158
+ lambda *a, **k: pytest.fail("--check must not re-fetch variant data"),
159
+ )
160
+ monkeypatch.setattr(curate.time, "sleep", lambda *_: None)
161
+
162
+ curate.check(["paad_tcga"])
163
+
164
+ assert calls == ["paad_tcga"]
165
+ assert json.dumps(curated_store.read("paad_tcga"), sort_keys=True) == before
166
+
167
+
168
+ def test_check_failure_is_not_reported_as_drift(patched_cbio, monkeypatch):
169
+ """An unreachable API means we do not know — it must not masquerade as a drift verdict."""
170
+
171
+ def _boom(_study):
172
+ raise RuntimeError("upstream down")
173
+
174
+ monkeypatch.setattr(cbioportal_io, "study_metadata", _boom)
175
+ rows, code = curate.check(["paad_tcga"])
176
+
177
+ assert rows[0]["state"] == "check_failed"
178
+ assert code == 0 # their outage is not our staleness
179
+
180
+
181
+ # --------------------------------------------------------------------------- #
182
+ # The scheduled job
183
+ # --------------------------------------------------------------------------- #
184
+ def test_weekly_job_is_report_only():
185
+ """The scheduled freshness job must invoke `--check` and nothing that writes.
186
+
187
+ The whole cadence rests on re-curation being a reviewed commit. A job that quietly gained a
188
+ `python -m src.curate <study>` step would re-establish the unattended API path ADR-0005 C4
189
+ removed — and it would do it in a file nobody reads once it is green.
190
+ """
191
+ from pathlib import Path
192
+
193
+ workflow = Path(__file__).resolve().parents[1] / ".github/workflows/curation-freshness.yml"
194
+ assert workflow.is_file(), "the weekly freshness job is missing"
195
+
196
+ runs = [
197
+ line.strip()
198
+ for line in workflow.read_text().splitlines()
199
+ if "src.curate" in line and not line.lstrip().startswith("#")
200
+ ]
201
+ assert runs, "the job does not invoke src.curate at all"
202
+ for line in runs:
203
+ assert "--check" in line, f"non-check curate invocation in the scheduled job: {line}"