Spaces:

anne-voigt
/

pdac-genomics-agent-dev

Sleeping

App Files Files Community

pdac-genomics-agent-dev / memory.md

avoigt1121

merge: label-join decided as route (c) — no cBioPortal PDAC subtype call exists

83b08b3 12 days ago

preview code

Raw

History Blame Contribute Delete

166 kB

A newer version of the Gradio SDK is available: 6.26.0

Upgrade

pdac-genomics-agent — memory

Current status + history. Architecture → CLAUDE.md. Backlog → TODO.md.

2026-08-14 — The subtype-label search is closed: no cohort exists, and the tab now says so

Re-opened the "find a cohort with variant data and an in-metadata subtype label" item, and it closes as a negative result with evidence rather than a curation. The 2026-07-28 sweep was re-run, not trusted: 539 cBioPortal studies (one added since), 19,165 clinical attributes, 680 of them on pancreatic studies. No study carries a Moffitt / Bailey / Collisson / PurIST molecular-subtype attribute.

Three near-misses, all correctly rejected — and the third is new to the record:

candidate	values	why it is not a subtype label
`paad_tcga_pan_can_atlas_2018.SUBTYPE`	`"PAAD"` ×152	a disease code, not a subtype
`paad_iatlas_prince_2022.IMMUNE_SUBTYPE`	C1–C6	Thorsson immune clusters, different taxonomy
`pdac_msk_2024.GENOMIC_GROUP` ("Genomic Subtype")	`KRAS-MUT` 2209 / `Other-MAPK-MUT` 76 / `MAPK-WT` 51	DNA-derived — it is mutation status. Joining it against KRAS would return a near-perfect association that is pure tautology wearing a p-value.

GENOMIC_GROUP is the one worth remembering: its display name is literally "Genomic Subtype", it sits on our largest cohort, and it would have produced the most persuasive-looking result in the repo. The closed vocabulary (ADR-0006 §3) rejects it on values, and it is not in SUBTYPE_ATTRIBUTES so it is never even probed — the guard held twice.

So the documented fallback was taken. The Variant × subtype tab now states plainly that the join is orchestrator-only today, names why each near-miss is refused, and says a routed result is the correct answer rather than a bug — the failure mode being a user who clicks "Test association" seven times, gets a route every time, and reads it as broken. The controls stay live: a cohort that does carry a label (or one named explicitly via subtype_attribute) still joins.

A false claim in ADR-0006 was corrected, not quietly patched. Its Context §2 cited paad_tcga → MOFFITT_SUBTYPE as a live local-join example, and §1's sample payload still uses it. That cohort has never carried it. A dated correction block now records this: the contract is unaffected — the local branch is specified, implemented and fixture-tested, and is correct the moment a labelled cohort is registered. What changed is only its status.

Two tests lock it (tests/test_variant_by_subtype.py): every curated artifact must route to the orchestrator (with a non-empty-list guard so it cannot pass vacuously), and the UI caution must carry the "no curated cohort does today" sentence. The first is the useful one — a future curation that does add a label fails it, forcing the UI copy to be revisited alongside the data instead of silently going stale. 247 tests pass (was 245).

Curation-time API only (ADR-0005 C4); nothing was curated, so no artifact changed.

2026-08-14 — the same question from the orchestrator side: route (a) was already built

Ran independently of, and reached the same negative result as, the entry above — read that one for the portal-wide sweep and the near-miss table; it is the fuller record. Confirmed here on a narrower probe (all attributes of the 7 curated studies + all 59 pancreas-matching studies), with one addition worth keeping: paad_cptac_2021's full 43-attribute list contains nothing subtype-like at all, so the "CPTAC carries Moffitt" intuition from the registry table does not transfer — that label lives in the expression manifest (cptac_pda_counts), the side a DNA agent structurally does not read. "Curate in a labelled cohort" was never a small unlisted- attribute fix; there is no attribute to list.

What this lane adds: the route with the real product value is already implemented and live, in the other repo. pdac-analysis-orchestrator has cross_modality_join.py (its ADR-0003), router._maybe_cross_modality_join, and three test files, and its agents.yaml registers this agent with cross_modality_join: {status: implemented, partner_agent: pdac_subtype_agent}. Its EXPRESSION_COUNTERPART maps paad_tcga → tcga_paad_tpm at 150/150 sample ids matched — after a barcode-normalisation fix, since cBioPortal's 15-char barcodes vs the h5ad's vial letter matched 0/150 exactly at first — and paad_cptac_2021 → cptac_pda at 131/131. Its statistics are a deliberate mirror of subtype_join.contingency_and_test, so one cohort gets one answer whichever route reaches it.

So the three-route question in TODO.md resolves to (c) on evidence, not preference: (b) is refuted upstream and (a) needs no scoping because it already ships. Nothing to build; the in-agent half stays correct, tested and dormant, and the only real defect was wording — now merged with the other lane's tab copy (both survived; neither was discarded).

Left for the orchestrator repo, not touched from here (file partitioning): its agents.yaml dataset_clarification still calls paad_cptac_2021 "the one cohort the variant × subtype join can run on" — stale, and contradicted by a comment four lines above it. Display-only text.

2026-08-14 — the 300,000-pixel raw-result accordion (`2f6f12d`, dev-verified)

The TODO's stated cause was wrong, and acting on it would not have fixed the page. The item blamed samples_profiled printing 2,336 ids three times. Measured: that is 7,008 lines of 21,187. per_sample + provenance are ~13,200 more — KRAS alone is altered in 2,188 of 2,336 samples. Eliding the named block would have left ~14,000 lines and a still-unusable page. Measure the payload before believing a size diagnosis; the block that is easiest to name is not necessarily the one that dominates.

Nothing was removed from the payload, and that was the constraint, not a courtesy. samples_profiled is the only way a machine caller can tell wild-type from never-sequenced (the per_sample_contract in deploy/orchestrator_registration.yaml); dropping it is exactly what made the orchestrator's join report every gene as "no variation in alteration status". So the fix is a DISPLAY split: _as_json (machine) is untouched and was verified over gradio_client to still return 3 × 2,336 ids as lists, while the UI gets _ui_json over the same object.

Four tiers, widest first. A summary table (gene × modality, each cell carrying the percentage AND the fraction — 93.7% (2188 / 2336), because the denominator is the thing people get wrong); the three charts; a download of the complete payload; then the abridged JSON. The table is what makes the abridgement acceptable — the numbers are on the page rather than inside a collapsed accordion.

The abridged block is deliberately a different SHAPE from what it replaces — a dict where a list was, stating the true count. A truncated list that still looks like a list is how a reader ends up believing a cohort has 8 profiled samples. Small payloads (denial, refusal, error) are under the threshold and pass through verbatim, so the ADR-0004 denial-visibility guarantee is untouched; the anonymous click-through on the dev Space confirms it live.

gr.Code(max_lines=...) DOES NOT WORK in gradio 6.18. Measured in the browser: the code box still rendered 23,323px tall with max_lines=25 set. The height cap is CSS on an elem_id, and that is what makes the page height constant rather than a function of the payload — abridgement alone only got ~300,000px down to ~25,681px. Final: 2,818px with the accordion open. If this regresses, check the CSS before checking the abridgement.

Empty summary cells NAME their absence ("not on DNA panel" / "not in cohort" / "not curated") and are never blank or zero — the same anti-padding rule as the charts. NRG1 on pdac_msk_2024 is the case that proves it: 0.3% (6 / 2336) in the SV column, refused in the other two.

The clearing frame blanks the two new slots as it does the charts. A stale download link is the worst version of that bug: the file leaves the page entirely and gets opened later with nothing to say which query produced it.

2026-08-14 — the modality radio was out of date with ADR-0008 (`9f3852c`, dev-verified)

The TODO item asked a judgment question — should the UI offer a third modality choice, or should the test suite own the unsupported_modality case alone? Both, and they are separable questions that were tangled together.

The refusal can never be reached from the radio, whatever its choices. A gr.Radio validates the submitted value against its OWN choices, so gradio returns the generic Value: 'fusion' … is not in the list of choices before variant_by_subtype runs. Widening the radio moves that wall; it does not open it. The only surface that can carry a free-form modality is the gr.api endpoint the orchestrator calls, so that is where the refusal is now exercised — permanently, not as a stopgap. "Make the refusal reachable from the UI" was the wrong frame.

But the radio was independently wrong. ["mutation", "cnv"] was a hand-written literal that outlived the SV modality (ADR-0008) for its whole existence: the tool accepted sv, deploy/orchestrator_registration.yaml published sv, and the only surface refusing it was the UI — offering a NARROWER contract than the machine endpoint. That is the same family as the modality fallback the refusal exists to prevent (a surface answering a different question than its label), one level up. Choices are now derived from a new variant_by_subtype.SUPPORTED_MODALITIES; the regression test asserts parity with the tool's closed set, not the refusal, so it fails on the old literal and on a future fourth modality. Verified failing against the two-choice radio first.

Checked the SV choice was not decorative before adding it. On ccle_broad_2019 (SV plus a named subtype attribute) the join runs and returns contingency counts; most genes fall below MIN_ALTERED_FOR_TEST and report testable: false. Sparse-but-labelled with the SV rarity caveat beats a capability the UI hides — the alternative is the UI deciding for the reader that an honest answer isn't worth showing.

A bug this change would otherwise have shipped. unavailable_modality and unsupported_modality had no branch in _subtype_caution_md and fell through to caveats, which a refusal does not carry — the box stayed visible=False and the reader saw an empty plot and nothing else. That is the ADR-0004 invisible-denial failure in a second place, harmless only while unreachable: 5 of the 7 curated cohorts publish no SV, so adding the choice made unavailable_modality an ordinary click result. Both routes now render; unavailable_modality also echoes modality so the answer can name its own subject.

Surveyed while deciding, worth keeping: no curated cohort auto-detects a subtype label. Six carry only CANCER_TYPE + ONCOTREE_CODE; ccle_broad_2019's SUBTYPE is histology, which looks_like_pdac_subtype rejects on purpose. So every registered cohort routes to the orchestrator unless the caller names subtype_attribute explicitly. Correct, but it means the label-join half and its modality argument are reachable only by the explicit path today → new TODO item. Also note the ordering: the label check precedes the modality check, so a bad modality on a label-less cohort reports orchestrator, not unsupported_modality.

Dev-verified live on anne-voigt/pdac-genomics-agent-dev at 9f3852c: /config serves [mutation, cnv, sv] with mutation as default; via gradio_client with an explicit x-orchestrator-token header, "fusion" and "mutaton" both return route: "unsupported_modality" with supported_modalities: [cnv, mutation, sv], "" and " " still resolve to mutation, and sv returns a real join. 255 tests pass.

PROMOTED TO PROD same day (f9cee52..f2d366c, Annie's call). Gates clean — 255 tests, pip-audit "no known vulnerabilities", gitleaks 97 commits no leaks; hook mirrored to github first; build watched to RUNNING (the wedged-restart trap did not fire). Verified on prod: /config serves [mutation, cnv, sv], "fusion" → unsupported_modality, blank → mutation, sv → a real join. The first push was rejected non-fast-forward — another lane promoted while the gates were running. Reconciled by merge, not rebase (those shas are published), then re-pushed. Worth remembering: run the gates, but re-fetch immediately before the push, because a clean gate run is not evidence the remote stood still.

Integration note: the push to hf-dev/main fast-forwarded cleanly from 35722c3, so the work is on the integration tip without a merge commit. The local dev branch is checked out in another lane's worktree and was left untouched — it needs only a pull. Not promoted to prod; that is Annie's call. No ADR claimed: this applies ADR-0008 to the UI surface rather than contesting it.

2026-08-13 — dev Space configured & verified: the gate is live on staging (no code change)

The dev Space anne-voigt/pdac-genomics-agent-dev went from "RUNNING but unconfigured" to a valid staging target for gate changes. Anne entered the settings; this lane prepared the exact values and verified afterward.

Settings now on dev — variables: ACCESS_CONTROL=enforce, LOG_SINK=hf, LOG_SINK_HF_DATASET=anne-voigt/pdac-genomics-results-dev; secrets: ADMIN_IDS, ALLOWED_IDS, LOG_SINK_HF_TOKEN. Two corrections to the TODO item as written:

It omitted LOG_SINK=hf — the dataset/token vars are inert while the sink defaults to local (logging_sink.get_log_sink).
"ADMIN_IDS/ALLOWED_IDS as variables, matching prod" was wrong about prod — checked via the HF API: prod holds them as secrets (only ACCESS_CONTROL is a visible variable). Dev now matches prod's actual posture.

Also found while checking prod: prod had no LOG_SINK* settings at all and its default dataset anne-voigt/pdac-genomics-results did not exist — prod audit records were going to ephemeral ./run_logs and dying on every restart. Spun off to a separate lane (which created the prod dataset and set the prod variables; its LOG_SINK_HF_TOKEN secret was still pending Anne at the time of writing — see the status fragment).

Verification (all live against the dev Space): machine API with no credential → machine_auth: no_token_header denial; bogus token → token_unresolved fail-closed denial; valid token → resolves and returns the 36-gene panel. Audit records with enforced: true land in pdac-genomics-results-dev under runs/, and none in prod's dataset. Signed-out UI access is moot while the Space is private (HF's own wall precedes ours), and the anonymous-denial records prove the gate's fail-closed path regardless. Note learned: a successful panel:api call writes no audit record by design — panel() returns after the gate without record_run; only denials are recorded on that endpoint.

Created the private dataset anne-voigt/pdac-genomics-results-dev. This item gated the "make the dev Space public?" decision — that is now unblocked.

2026-08-13 (latest) — dev Space PUBLIC; the staging lane is whole

Anne flipped visibility the same day. Post-flip verification, all green:

space_info (anonymous) → private: False; anne-voigt-pdac-genomics-agent-dev.hf.space returns HTTP 200 with no cookies and renders the full app — the 404 / cross-origin-iframe / __sign= password-form maze from 2026-08-05 is gone.
Signed-out click-through on "Query variant status" → {"status": "denied", "reason": "🔒 Please sign in…"}. The ADR-0004 UI gate is enforced and fail-closed on staging, matching prod's public+gated model.
Allow-listed access answers with grounded numbers (machine path, same day: KRAS 136/150 = 90.7% on paad_tcga); audit records land in pdac-genomics-results-dev only.

Consequence: the dev lane can now verify that the Space answers, not merely that it boots. Server-shaped changes (auth, secrets, artifact loading) no longer take their first real click-through in production. The 2026-08-05 caveat about local-run verification being the only option is retired.

2026-08-13 (later) — dev Space configured; gate verified pre-flip; only the public flip remains

Anne set the dev Space settings the same day. Verified from here while the Space is still private (machine path works through privacy; only browser click-through doesn't):

Variables read via HfApi.get_space_variables: ACCESS_CONTROL=enforce, LOG_SINK=hf, LOG_SINK_HF_DATASET=anne-voigt/pdac-genomics-results-dev. ADMIN_IDS/ALLOWED_IDS are secrets — and that MATCHES prod (another lane checked prod the same day); the old TODO wording "as variables, matching prod" misdescribed prod. CBIOPORTAL_CONTACT deliberately unset.
Gate, three ways: allow-listed token in x-orchestrator-token → real grounded answer (KRAS 136/150 = 90.7% on paad_tcga — the post-denominator-fix number); garbage token → token_unresolved; no header → no_token_header. Fail-closed confirmed on staging.
Sink separation: the test runs wrote 6 runs/*.json into pdac-genomics-results-dev; prod's pdac-genomics-results has none.
⚠️ Tooling trap worth remembering: gradio 6's Client(token=…) does not send x-hf-authorization — without an explicit headers={"x-orchestrator-token": …} even a valid admin token reads as no_token_header. The 2026-07-28 "gradio_client attaches it automatically" note no longer holds on the pinned client.

Remaining: Anne flips visibility → public (Settings UI), then the browser-level checks (*.hf.space loads, signed-out UI denied, signed-in allow-listed answers).

2026-08-13 — dev-Space visibility: recommendation written, flip awaits Anne

The "public vs prod-only click-through" decision (TODO, open since 2026-08-05) now carries a worked recommendation + runbook in TODO.md: make dev public with ACCESS_CONTROL=enforce, matching prod's model. The load-bearing findings: the licensing argument for privacy is stale — ccle_broad_2019 and pdac_msk_2024 have been served by the public prod Space since 2026-07-28, with the Broad continuity clause discharged in-repo and the ND question resolved 2026-08-05, so a public dev Space adds no new data exposure; secrets are unaffected by visibility; the only genuine precondition is sequencing — the gate is DARK on dev, so secrets/ variables must be configured (and gate-verified via gradio_client + token, which works while private) BEFORE the flip. Both the secret-setting and the visibility flip need the Space settings UI, so this is Annie's action; verification steps are in the runbook. No push made from this lane.

2026-08-06 — legibility: the header names SV, and the chart no longer stacks (`408908b`, NOT deployed)

Header. Title and the variant-status blurb both said "mutation and copy-number status". SV has been shipping since 0f39c3b and was named nowhere a reader looks first.

The stacked chart was wrong twice over, not merely cramped. A sample can be BOTH mutated and copy-number altered, so the segments are not disjoint and stacking double-counts; and the modalities have different denominators (ccle_broad_2019: 53 / 44 / 41 profiled; paad_tcga: 150 mutation vs 184 cnv), so the segments are not percentages of the same thing. A stacked bar asserts "these parts make up the whole". KRAS reached ~120% on an axis labelled "% altered".

Now one chart per modality, each naming its own denominator, each a single series. Gradio's native BarPlot has no grouping control and the Space has no other plotting library, so splitting is the only honest option inside the current dependency set — and it deletes the categorical colour scale whose stale domain caused both wrong-answer bugs of 2026-08-05.

Two further distortions surfaced only by looking at the rendered result:

Autoscale started the y-axis at the data minimum — a CCLE chart ran from 20%, so SMAD4 at ~21% looked like nothing beside KRAS at 90%. Baseline now pinned to zero.
Fixing the ceiling at 100 to match then made SV unreadable: NRG1 is 6/2336 = 0.26%, a sub-pixel sliver — the same "reported as nothing" failure in new clothes. Ceiling is fitted per chart, floored so one tiny bar is not stretched to full height, capped at 100.

A third bug caught BEFORE shipping, which is the point. The clearing frame first hid all three charts; the answer then re-showed them. That hide→show flip lost the race for the third chart about half the time — it stayed unmounted (browser: "null does not exist"), so a cohort with structural variants rendered as mutation+CNV only, reading as "no fusions here". The clearing frame now blanks each chart's VALUE and leaves visibility alone.

Deployed 3ad3d85 2026-08-07 and re-verified on the live prod Space: header + tab blurb name the fusion modality; a fresh-load FIRST query draws all three charts (53 / 44 / 41 profiled); paad_tcga draws exactly two and no SV; narrowing to NRG1 draws one SV chart on a single click and widening back draws three with all 33 genes and KRAS present. Gates clean beforehand (237 tests, pip-audit, gitleaks).

What caught it was re-running the same query from a fresh load instead of accepting one good render. The first render looked right; the failure only appeared on a first query after load, so a single happy screenshot would have shipped it. That is the third time in two days this exact class — a chart silently not showing — has appeared, and the first time it was caught before deploy. Verified across 4 fresh-load trials (including a click during hydration and the double-click that first exposed it), plus paad_tcga showing exactly 2 charts, plus both narrow→wide and wide→narrow on single clicks. 237 tests (was 235).

2026-08-05 (latest) — I shipped a regression to prod and caught it in the post-deploy click-through (`819da30`, NOT deployed)

23d37b6 went to prod and fixed the three real bugs. It also introduced a fourth, which the post-deploy click-through found: on a query that NARROWED the gene set, the chart body lagged one render behind — caution and legend updated, the bars kept the previous answer's data. One cohort's numbers under another cohort's warning, which is the same class of wrong answer the whole change set existed to remove.

colors_in_legend was the cause, and I had added it as the cosmetic half of the fix. Passing a value that CHANGES between renders is what forced the colour scale to rebuild — which is what fixed the narrow→wide direction. It was never cosmetic; it was load-bearing for a reason I had not understood. Measured, each against 40ab3dd as control:

build	narrow → wide	wide → narrow
`40ab3dd` (before)	marks dropped	correct
`23d37b6` (`colors_in_legend`)	correct	stale one render
`color_map` alone	marks dropped	correct

So the two configurations I had were the same bug pointing in opposite directions, and the direction I happened to test first was the one that looked fixed.

Also tried and rejected: holding the modality categories open with NaN placeholder rows so the domain never changes shape. NaN poisons the y extent — no bars and no axis render at all. (Zeros would have worked visually and were never an option: NRG1 is not 0% mutated, it is unassayed.)

The fix stops relying on the plot diffing correctly. _ui_variant_status is now a generator: it blanks the chart, then yields the answer. The worst intermediate state a reader can catch is an EMPTY chart, which claims nothing, rather than a stale one, which claims something false. Both directions verified repainting on a single click. test_ui_handler_gating reads the final frame via _final(); the ADR-0004 assertions are untouched and a denied path still yields exactly one frame.

Deployed 125b8a5 2026-08-06 (gates clean: 234 tests, pip-audit, gitleaks; hook mirrored to github first). Verified on the live prod Space in both directions, each on a single click — CCLE wide → MSK+NRG1 narrow (correct NRG1 bar immediately, not CCLE's) → full panel wide (KRAS ~94% across all 33 genes). The OAuth session drops when the Space rebuilds, so sign in again before verifying; an anonymous query renders nothing visible because the plot is None and the JSON accordion is collapsed.

Three process lessons, all of them mine:

I explained away the symptom the first time. I saw the stale chart locally, guessed "my click missed the button", and moved on. It had not missed. A surprising observation deserves one cheap check before a comfortable explanation.
A fix verified in one direction is not verified. Both bugs here are the same scale-rebuild fault; testing narrow→wide only was the whole gap.
Green tests proved nothing about either. Every version passed its own tests, because the tests asserted the payload and the hypothesis, not the rendered result across a SEQUENCE of renders.

2026-08-05 — first signed-in click-through since 2026-07-27; two display bugs found (`6c7b8cc`, NOT deployed)

Everything shipped that day had been verified only by reading the deployed tree over resolve/main and by the local suite. That proves what is served, not that it answers correctly, and the panel had since gone 19 → 36 genes and gained a third modality. So: signed in as anne-voigt and ran real queries. Every payload the Space produced was correct. Both bugs are display-layer, which is why 219 passing tests saw neither — and which also means the orchestrator's machine path, reading JSON rather than pixels, was never affected.

Passed as specified: NRG1 on pdac_msk_2024 returns assayed: false and a valid sv block at 6/2336 (the ADR-0008 modality-scoped gate, live); paad_tcga names sv in unavailable_modalities with no SV numbers anywhere; BYOD refuses a Fusion MAF with sv_not_supported and accepts a clean one as grounded=false; licence caution + citation.curation ride on every answer; the machine endpoints still refuse a cookie-only caller.

The chart could show an empty panel while the JSON said 93.7%. The colour scale's domain was derived from the current frame and never rebuilt when the next frame carried more categories. NRG1 alone → sv-only chart; the next full-panel query redrew with the legend still ["sv"] and two thirds of the marks dropped. The y-axis rescaled to 0–100 — proof the data had arrived and only the colour scale was stale. Same staleness in the other direction put an sv swatch on paad_tcga under a caution saying it has no SV data: one cause, both symptoms, and it only loses data narrow → wide, which is exactly the direction a scientist moves (check one gene, then open the panel). Fixed by pinning color_map + per-answer colors_in_legend.

The tempting fix for that is wrong and there is now a test saying so. Padding every gene to all three modalities would stabilise the category set — and report NRG1 as 0% mutated on a cohort that never sequenced it. That is the false negative assayed: false exists to prevent. Stabilise the SCALE, never the DATA.

caveats never reached the variant-status tab. Only _subtype_caution_md read it, so ccle_broad_2019's uncharacterized-SV warning — generated correctly, carried correctly — stopped one function short of the screen, on the one tab where SV data is actually queried. A warning that exists so "no in-frame fusions" cannot be read as biology is not doing its job from inside a collapsed JSON accordion.

226 tests pass (was 219). The new tests/test_plot_and_caveat_rendering.py asserts on the rendered surface — the plot's colour scale and the caution markdown — because that is the only place either bug existed.

Still open, reported not fixed (see TODO.md): item 5 of the check (unsupported-modality refusal) is unreachable from the UI because Modality is a radio, and Gradio's own choice validation intercepts first; no curated cohort carries a subtype label, so the label-join half cannot be demonstrated in the Space.

2026-08-05 (later) — the SV under-call: a plausible cause that was wrong, and the real one (`03e0388`, `e0d0ac0`, NOT deployed)

Follow-up to the click-through above. The six NRG1 events on pdac_msk_2024 really were being reported as bare rearrangement while the payload's own provenance called them in-frame fusions — but the cause was not what I first said it was, and the way that surfaced is the part worth keeping.

The wrong answer. The provenance text reads "an in-frame fusion", hyphenated, and classify_sv tested "in frame". Clean, sufficient, matched the evidence I had. Fixed it, added tests, they passed, committed (03e0388). Then re-curated — and the diff was empty. Zero cells moved. Nothing else would have caught it: the tests were green, because they tested the hypothesis rather than the cohort.

The real answer, read off the live API rather than inferred from the artifact:

field	value
`eventInfo`	`ATP1B1-NRG1 Fusion - Archer` ← a bare label
`annotation`	`POSITIVE FOR GENE FUSIONS…: The rearrangement is an in-frame fusion between genes ATP1B1 Exon2 (NM_001677) and NRG1 Exon2 (NM_004495).` ← the finding
`site2EffectOnFrame`	`'NA'` — on all 110 somatic panel rows

classify_sv read only eventInfo, so those rows fell to the bare-partner default correctly. The frame claim lived in a field nothing consulted. The hyphenation was real but downstream of the actual gap — it only matters once annotation is read at all, because that is where the hyphenated spelling lives. site2EffectOnFrame is the field that ought to carry this and is the same trap variantClass already set for CCLE: structured, present, and NA throughout.

Scope was larger than first reported: 9 cells over the whole KRAS-wild-type actionable tier — NRG1 ×6, NTRK1, NTRK3 (ETV6-NTRK3), RET (NCOA4-RET) — not 6 NRG1 alone. The RET cell went fusion_out_of_frame → fusion_in_frame, the only transition that reduces caution: that sample carries two RET events and the in-frame row now outranks the other under the existing specificity ordering, with provenance moving to match.

Why annotation is safe to read, checked across both SV cohorts before trusting it: no row says "negative", none says "out of frame" there, and CCLE's 322 annotations never mention a frame at all (they are partner/panel strings). So it can only fire where the source states a frame. eventInfo still wins where both make a claim — annotation is a fallback, not an override. --dry-run confirms CCLE still reports uncharacterized, so its caveat is untouched; that was checked deliberately, since the caveat rests entirely on bare labels never being promoted.

All three fixes verified by clicking through the merged build (d956d91), 2026-08-05:

ccle_broad_2019 now renders the caveat in full — "…this cohort can never return fusion_in_frame. Read an absence of in-frame fusions here as a fact about the annotation, NOT about the biology" — alongside the Broad licence line, not instead of it.
The chart regression is gone: NRG1-alone (single red sv bar, y-axis 0–0.30, sv-only legend) followed immediately by the full panel now draws KRAS ~94%, TP53 ~78%, CDKN2A ~40% across all 33 genes with the full mutation/cnv/sv legend. That is the exact sequence that used to draw empty.
sv renders red in both the narrow and wide charts — the colour no longer depends on how many modalities happen to be present.
NRG1 on pdac_msk_2024: assayed: false + its note (the ADR-0008 gate is intact), 6/2336 = 0.257%, and all six now fusion_in_frame rather than bare rearrangement.

⚠️ The dev Space could NOT be used for this, and this is structural, not a one-off. It is private, so anne-voigt-pdac-genomics-agent-dev.hf.space returns 404 — the session cookie lives on huggingface.co, not on the hf.space origin. Through the embed on the Space page the app renders but is a cross-origin iframe, and synthetic clicks do not reach it. Promoting the iframe's signed __sign= URL to top level lands on a username/password form with "Could not get API info. Connection errored out." So: while the dev Space is private, it can be built and booted but not clicked through. The verification above was done against the merged branch running locally on the pinned gradio 6.18.0 — legitimate here because both display bugs are purely client-side, but it is NOT a substitute for a real click-through of anything server-shaped. Make the dev Space public (or accept prod-only click-through) before relying on the dev lane for UI verification.

Two habits earned their keep and should be kept: re-curating with --check first (all seven up_to_date, so this was a re-derivation and not a data refresh hiding behind a code change), and reading the re-curation diff cell by cell instead of trusting the summary line — which was identical (106 SV cells (characterized)) before and after both attempts.

2026-07-29 — the registration descriptor caught up with the Space (docs-only)

deploy/orchestrator_registration.yaml exists so "the trigger keywords and tool list can't drift from what the agent actually exposes". It had drifted on six counts, all now fixed against the live Space at 894e710: genes is a comma-separated string (not list[str]; blank ⇒ the full panel), variant_by_subtype takes four args (genes, source, subtype_attribute, modality), /panel was missing entirely, samples_profiled was undocumented, the cohort list still said four, and the auth/visibility notes still said "PRIVATE until the gate is enforced" (the gate is ENFORCED, the Space PUBLIC since 0ffadd1) with no mention that a machine caller must use the custom x-orchestrator-token header.

Nothing was broken by any of it — the orchestrator's own agents.yaml was written from the live endpoints, not from this file. The risk was only that the next reader trusts it, which is exactly the risk the header claims the file removes.

Three things worth keeping:

The per_sample sparsity rule is now a named section of the descriptor (per_sample_contract), not a footnote: for a gene with assayed != false, a sample in samples_profiled[modality] but absent from per_sample is WT/neutral; a sample NOT in that list was never interrogated and is neither — exclude it from a join rather than counting it WT. The orchestrator shipped a join reporting all 19 genes as "no variation" precisely because it had to guess this.
The cohort set is no longer restated as a list of facts — the descriptor now says the set IS src/resources/curated/*.json and gives the seven only as an illustrative tail. This is the second time the restated list went stale, and prose is not the thing to trust twice.
Bare KRAS was DROPPED from trigger_keywords, with the reason recorded inline. This is the orchestrator's deliberate divergence, not an oversight to correct: bare KRAS collides with pdac-subtype-agent's KRAS-Science-2024 signature sets and DecoupleRpy pathway questions, and the hotspot alleles G12D/G12V/G12R carry the DNA intent. Verified live in prod — "Score the KRAS signatures on gse205154_sears" reaches the subtype agent. The comment says don't re-add it.

The drift was invisible to the tests, so the tests changed too. test_deploy_surface.py only did string containment on tool names, which catches a rename but not a wrong signature. Two new structural tests: the descriptor's args are compared to the live inspect.signature (names AND str types, per endpoint), and the cohort ids it mentions are compared to the curated directory. Both fail on the drift they exist to catch. 174 tests pass (was 169; the descriptor file also turns out not to have been valid YAML before — {genes: list[str]} is not a legal flow mapping, so nothing had ever parsed it).

2026-07-28 — `paad_cptac_2021` curated + DEPLOYED (`b79f05e`); and a payload gap it exposed

Curated alongside pancreas_cptac_gdc, not instead of it. Same CPTAC-3 patients, same paper (Cao et al. Cell 2021) — never pool them. Each covers what the other cannot:

cohort	n	CNV?	sample ids	why it exists
`pancreas_cptac_gdc`	183	yes	`C3L-00017-02` (aliquot-suffixed)	richer modality coverage — why it was picked first, and that still holds
`paad_cptac_2021`	140	no (mutation only)	`C3L-00017`	ids match the expression registry's `cptac_pda` 131/131; the suffixed GDC ids match it zero times

That id match is the point: it is what lets pdac-analysis-orchestrator run its cross-modality variant×subtype join (its ADR-0003), which aligns PurIST subtypes computed from cptac_pda against variant status from here. The earlier note calling paad_cptac_2021 "superseded by the GDC rebuild" was right about CNV and simply could not see the join constraint. Both cohorts are needed.

C2 discharged BEFORE fetching: access tier from cBioPortal's own publicStudy/readPermission (open), plus a COMMERCIAL_USE_REVIEW entry — permitted, CPTAC-3 open tier, confidence recorded as one step less direct than the GDC instrument since this is the authors' own deposition rather than a GDC rebuild. ATTRIBUTION.md carries the never-pool warning.

Deployed and verified live through the orchestrator's own x-orchestrator-token path: 140 samples, unavailable_modalities: ['cnv'], KRAS 96.4% / TP53 75.0% / CDKN2A 20.7% / SMAD4 17.1% (identical to local), study_terms surfacing open + permitted, variant_by_subtype correctly routing to the orchestrator. 169 tests pass.

Two process notes worth keeping. The push was rejected on a diverged remote (another lane had added the GitHub-mirror pre-push hook and docs) and was rebased, not forced — their two new tests then ran against the new artifact and passed. But the rebase silently dropped this entry's first version: both lanes had appended to memory.md at the same anchor, and the replay treated the hunk as already applied, keeping only the TODO.md half of the same commit. Nothing conflicted and nothing warned. If you rebase a docs commit in this family, re-read the file afterwards rather than trusting a clean exit.

The finding that matters more than the curation: `per_sample` is SPARSE

The first real run of the orchestrator's join exposed a payload contract gap in /query_variant_status — not a curation problem:

genes.<G>.mutation.per_sample contains only altered samples; WT is omitted. Verified on paad_cptac_2021 (KRAS 135 entries, all hotspot; SMAD4 24, none WT) and paad_tcga (KRAS 136, SMAD4 37) — the general shape, not new.

Internally harmless: variant_by_subtype feeds contingency_and_test a dense StatusMatrix row, so the local label-join is correct. But a machine caller cannot reconstruct WT — it receives altered samples only, and the payload carries n_samples/n_profiled counts with no profiled-sample id list. The orchestrator therefore scored a WT column of 0 for all 19 genes and reported "no variation in alteration status": a wrong reason wearing the clothes of a real finding. It fails safe (no fabricated p-value) but it misleads. Its alignment was perfect — 131 joined / 140 / 131.

FIXED (same day). /query_variant_status now returns samples_profiled — the denominator as an ID SET, per modality, not just as a count. samples_profiled was chosen over a dense per_sample for two reasons: one list per modality instead of the same list repeated across 19 genes (pdac_msk_2024 has 2336 samples), and it keeps per_sample meaning "what was found" rather than "the whole matrix".

Per MODALITY is the correct granularity here, and that had to be checked rather than assumed: within an assayed gene the denominator IS the modality's profiled set (StatusMatrix.denominator); genes a targeted panel never carried are refused separately via assayed: False/unassayed_genes, so they never reach the densification rule.

The documented caller rule, now stated in the payload docstring: for a gene with assayed != False, a sample in samples_profiled[modality] but absent from that gene's per_sample is WT/neutral — and a sample NOT in that list was never interrogated and is neither. Invariant asserted in tests: len(samples_profiled[m]) == n_profiled[m]. A modality the cohort lacks contributes no key at all rather than an empty list, because an empty list reads as "profiled nobody", which is a claim.

172 tests (+3, on the existing 4-sample partial-profiling fixture where S4 was never sequenced — precisely the sample a naive "absent ⇒ WT" would invent).

2026-07-23 — Repo created (scaffold)

New third specialist behind the PDAC Analysis Orchestrator, for DNA-level analysis (mutation/CNV status + the variant↔subtype question). Split out as its own agent per docs/adr/ADR-0001-genomics-specialist.md, which repeats pdac-subtype-agent's ADR-0001 reasoning (distinct domain → distinct agent) for the larger DNA-vs-RNA gap. Originating decision + resolved scope: pdac-subtype-agent ADR-0019 (Accepted 2026-07-23); Carl resolved scope (build now/ship later; mutation + CNV; variant×subtype join and lookup; cell-lines first, human v1).

Feasibility proven before scaffolding (pdac-subtype-agent docs/spikes/SPIKE-dna-panel-feasibility.md): cBioPortal's public REST API serves mutation + CNV + expression for PDAC cell lines (ccle_broad_2019) and tumors (paad_tcga) via per-gene JSON — no bulk downloads — and the sample-id join across all three modalities works. KRAS returned G12D/G12R/G12V; driver frequencies matched the literature.

Scaffold in place: README (HF Space front-matter, hf_oauth: true), CLAUDE.md, TODO.md (milestones), ADR-0001 + the number ledger (started day one), the cBioPortal client seeded from the spike (src/workflows/cbioportal_io.py), and the 19-gene v1 panel (src/resources/pdac_panel.tsv). Tools are contracts, not implemented. Local only — no HF/GitHub remote yet (awaiting an account/Space choice; working name pdac-genomics-agent).

Now in SHOWCASE_STATUS.md (row added 2026-07-24) — see the 2026-07-24 Space-created entry below.

Next: Milestone 1 — curate the cBioPortal status matrix + query_variant_status.

2026-07-24 — Milestone 1 done (status matrix + `query_variant_status`)

Core curation implemented and tested hermetically. Contract pinned in ADR-0002 (cBioPortal → gene×sample status-matrix curation contract).

variant_status.py — build_status_matrix(source, sample_ids=None) pulls mutations + discrete CNV via cbioportal_io, classifies each: mutation → WT/missense/truncating/hotspot (hotspot = HOTSPOTS explicit set + HOTSPOT_CODONS recurrent-codon rule → KRAS G12x/G13x/Q61x caught for any residue; truncating from mutationType; silent → WT), CNV -2..2 → deep_del/loss/neutral/gain/amp. Specific variant kept as provenance. WT resolved against the full sample-id list (mutation queries return only mutated samples).
Per-study modality gate — modalities set from real profile presence; no CNV profile ⇒ cnv=None, tool omits it + lists it in unavailable_modalities. Never a null "no alteration" column.
lineage.py — pancreatic subset of pan-cancer CCLE via CANCER_TYPE/ONCOTREE_CODE; refuses on empty. Registered PDAC cohorts skip it.
query_variant_status(genes, source) — per-sample status + cohort frequency + provenance + modalities_available/grounded. Applies the lineage filter for pan-cancer sources.
Added two client helpers: cbioportal_io.sample_ids() and .clinical_data().
Tests — tests/fixtures/cbioportal_paad_mini.json (real field shapes, anonymized samples) + test_variant_status.py / test_lineage.py; hermetic (conftest monkeypatches the client). Smoke test updated (M1 contracts now callable; variant_by_subtype still an M2 stub). 13 passed, no network.
Live spot-check reproduces the spike (not committed to CI): TCGA-PDAC CDKN2A deep-del 52, MYC amp 23 (exact); KRAS 136 / TP53 104 / SMAD4 37 (within a few of the spike's 140/105/38); paad_qcmg_uq_2016 reports cnv unavailable; CCLE narrows to 57 pancreatic lines (KRAS 84%).

Run tests: DecoupleRpy_Agent/.venv/bin/python -m pytest -q (this repo has no own venv). Still local only — no remote. Next: Milestone 2 — variant_by_subtype label-join + association test.

2026-07-24 — Milestone 2 done (`variant_by_subtype` label-join)

The stateless half of the cross-modality question. No new ADR — this implements what ADR-0001 already decided (general join is orchestration; the label-join is the half this agent can do alone).

subtype_join.py — find_subtype_labels(study, attr=None) probes known subtype clinical attributes (SUBTYPE_ATTRIBUTES: Moffitt/MRNA/Bailey/Collisson/PurIST/…) or an explicit one; contingency_and_test(per_sample_status, labels, altered) inner-joins on sample id and runs Fisher's exact (2×2) or chi-square (R×2). Returns testable=False + reason for a single subtype class or an all-altered/all-WT column (no spurious p-value).
variant_by_subtype(genes, source, subtype_attribute=None, modality="mutation") — for a label-bearing cohort returns {join_available: True, subtype_source, modality, n_labeled, genes: {G: {contingency, n, testable, test?/reason}}, caveats}. modality="cnv" supported. Honors the M1 coverage + lineage gates.
Routing contract: BYOD source, or no subtype attribute in metadata, returns a structured {join_available: False, route: "orchestrator", reason, note} — the orchestrator then gets subtype from pdac-subtype-agent + variant status from query_variant_status and joins on sample id. Orchestrator-side wiring is NOT built here (belongs in pdac-analysis-orchestrator).
Tests — test_variant_by_subtype.py (association unit: Fisher/chi-square/untestable; tool join over the labeled fixture; CNV modality; both orchestrator-routing paths). Fixture gained a MOFFITT_SUBTYPE clinical block; conftest patches clinical_data. Smoke test updated (variant_by_subtype now callable, no longer a stub). 20 passed, no network.
Live check: real paad_tcga carries no subtype clinical attribute → correctly routes to the orchestrator (confirms the label-join only fires when a label genuinely exists).

Still local only — no remote. Next: Milestone 3 — BYOD upload + deploy hardening.

2026-07-24 — Milestone 3 done (BYOD + deploy hardening, code)

Code-complete; the remaining M3 items are ops tasks that need an HF Space (see TODO.md "M3 deploy checklist"). Two ADRs published: ADR-0003 (BYOD contract), ADR-0004 (auth gate).

BYOD (src/workflows/byod.py, ADR-0003) — BYODSpec(genome_build, species, maf_text?, cnv_text?) → curate_byod_upload() → StatusMatrix grounded=False, never persisted. Build + species are REQUIRED declarations (untrusted, ADR-0006 mirror). Hard-gate refusals via BYODRefusal(reason=…): build_unsupported, build_mismatch (file build column contradicts declared), non_human_species (mouse), unparseable (missing cols / non-integer / out-of-range GISTIC / empty). Reuses M1 classify_mutation/classify_cnv/HOTSPOTS verbatim; off-panel genes dropped. Routed from build_status_matrix(BYODSpec) (lazy import — byod depends on variant_status).
Auth gate (access.py + tests/test_access.py, ADR-0004) — ported verbatim from the siblings: ALLOWED_IDS/ADMIN_IDS(+UPLOAD_ADMIN_IDS) allow-list, ACCESS_CONTROL dark-launch (OFF by default, fail-closed when ON), check_access/role_for/is_authorized/principal_trace_fields.
cBioPortal cache/backoff — cbioportal_io._req now caches idempotent GET metadata (process-local, clear_cache()/_CACHE_GET for tests) and backs off exponentially. ToS confirmation for automated use is still an open pre-prod ops task.
Orchestrator registration — deploy/orchestrator_registration.yaml (this repo's artifact; tools, DNA-specific trigger keywords, the cross-modality handoff contract). NOT wired into the orchestrator here — that repo merges it at deploy (ADR-0001: we don't touch siblings).
Security scan port — security/scan.sh (bandit/pip-audit/gitleaks/trivy → git-ignored timestamped reports) + security/.gitignore, mirroring the siblings. Not run in CI; run before the OHSU review. (bandit not in the shared venv.)
Tests — test_byod.py (curate + all five refusal gates + routing through build_status_matrix) and test_access.py (ported gate suite). 49 passed, no network.

Deploy surface still stubbed on purpose: gradio_ui.py/src/server.py are built at the deploy step (M3 checklist), where the access.py gate + audit sink get wired in. Still local only — no remote.

2026-07-24 — cBioPortal terms & deployability (pre-prod gate closed)

Closed the spike's open item ("confirm cBioPortal automated-use terms + add caching/backoff") → docs/cbioportal-terms-and-deployability.md. Verdict: usable for a deployed prod Space WITH CONDITIONS — no hard blocker.

No ToS ban on automated/programmatic access to the public API (www.cbioportal.org/api) — the website runs on the same API and maintainers publish official clients. Self-host NOT required for v1. v1 = public instance + curate-and-cache (runtime reads the cached status matrix; API is a curation-time dependency only), which neutralizes rate-limit/availability risk without ops cost.
Conditions (all engineering/provenance, fold into M1/M2): (C1) attribution — cite cBioPortal [Cerami 2012 / Gao 2013 / de Bruijn 2023] and each per-study source; (C2) carry per-study license flag (ODbL default; some studies restrict commercial use; TCGA/GDC + ICGC per-study terms; serve only open-access, refuse controlled); (C3) polite client — exp backoff+jitter on 429/5xx, self-throttle, descriptive User-Agent w/ contact; (C4) curate-and-cache so no live API call on the request path.
No documented public rate limit — absence ≠ permission; self-throttle (~2–5 req/s, serial).

2026-07-24 — HF Space created

Space anne-voigt/pdac-genomics-agent created via hf CLI (auth'd as anne-voigt), SDK gradio (matches README front-matter), PRIVATE + empty — nothing deployed (app code isn't ready and the M3 auth gate is unbuilt, so keep it private/ungated until launch). Working name is now confirmed as the real slug. SHOWCASE_STATUS.md Fleet row added (honest state: created/private/no-deploy). Repo is still LOCAL ONLY — origin is NOT yet pointed at the Space (per the sibling model, origin remote IS the Space; wire that up at first deploy, on main, after M1/M2 merge from claude/serene-diffie-b17166). Do NOT push an empty/broken tree to it.

2026-07-24 — cBioPortal public-vs-self-host ADR

Public-vs-self-host decision ratified as its own ADR → docs/adr/ADR-0005-cbioportal-public-vs-selfhost.md (Accepted 2026-07-24): public instance + curate-and-cache for v1, no self-host; revisit triggers = unavailable studies / sustained throttling / controlled-access need. ADR-number note: written on this lane as 0003 (it had seen the sibling lane's 0002 and skipped it) and renumbered to 0005 on 2026-07-25, because that lane had also published 0003 (BYOD) ~2 min earlier plus 0004 (auth gate). Different filenames → git would have merged two 0003s silently. Earlier claim keeps the number; see docs/adr/RESERVED.md.
License: data default ODbL (attribution + share-alike on redistributed derived database; serving derived answers is lower-friction). TODO Milestone 3 item marked done (research half); implementation of cache/backoff+attribution remains in M1/M2.

2026-07-25 — Founding docs seeded, 0003 collision fixed, both lanes merged to `main`

Three things, in order: the founding planning docs were seeded and reconciled, the ADR-number collision between the two lanes was resolved, and both branches were merged. main now holds all of M0–M3.

Seeded — the founding material had been drafted in pdac-subtype-agent/docs/genomics-agent-seed/ before this repo existed (its README said "move these to the new repo"). Now here:

docs/design/DESIGN-pdac-genomics-agent.md — the north-star design. Body kept verbatim; a new §0 records what held and what diverged. (It self-dates 2026-07-26 — a drafting artifact.)
ADR-0001 enriched, not duplicated. The seed's own draft ADR-0001 was a longer version of the one already published here; its missing content (founding commitments — new modality never a data_level, status matrix, hard gates, family maturity from commit one; alternatives 4–6; follow-ups) was merged into the existing file rather than landing a second ADR-0001.
RESERVED.md rebuilt with the family-wide EXTERNAL rows (0014 security scan → DecoupleRpy_Agent; 0015 registry semantics → biodata-registry), which were missing here, plus a new protocol rule 5: claims land on main, alone and immediately.

The design's "hard blocker" is not one. It made a biodata-registry modality extension (somatic_mutation/copy_number) a hard prerequisite for M1. M1–M3 shipped without it: the build reads cBioPortal directly and does not depend on biodata-registry at all (absent from requirements.txt, no imports). Recorded as an ADR-0001 amendment — not a blocker, but an open fork (direct-source vs registry-routed) needing its own ADR if reopened.

ADR-0003 collision — resolved. serene published 0003 (BYOD) at 14:04:34; upbeat claimed 0003 (cBioPortal public-vs-self-host) at 14:06:28, having skipped the sibling's 0002 but not its two-minute-old 0003. Different filenames → git would have merged both silently. The ledger failed to catch it because each lane kept its own copy of RESERVED.md; the one-shared-row design only conflicts when both lanes edit the same file in the same history. Fixed on the branch before merge (15b3ade): earlier claim keeps 0003, cBioPortal ADR renumbered 0003 → 0005 (file, header rationale, terms-doc ref, memory refs, ledger). When the two lanes then met in main, the ledger rows did conflict — the mechanism works, it just needs one shared copy.

Merged — claude/serene-diffie-b17166 (M1/M2/M3: status matrix, query_variant_status, variant_by_subtype, BYOD, access.py, security scan, orchestrator descriptor, ADR-0002/0003/0004) then claude/upbeat-hofstadter-353bb8 (cBioPortal terms doc, ADR-0005, Space-created note). Conflicts resolved by hand: ledger rows unioned to a clean 0001–0005; memory.md keeps both lanes' history chronologically; TODO.md took serene's completed M3 with the ToS checklist item flipped to done by upbeat's research. 49 tests pass, no network.

Carried into TODO.md as pre-deploy gates: the Rosie/Carl confirm that per-sample status is the real near-term need (specified pre-M1, never done, now pre-deploy), and cBioPortal conditions C1–C4 (attribution, per-study license flag, polite client, curate-and-cache) — research closed, only the backoff/cache half implemented.

Left alone deliberately: the seed folder still exists in pdac-subtype-agent; its README says to delete it after moving, but deleting files in a sibling repo is that repo's call.

Still local only — no remote. Next: the deploy step (gradio_ui.py + src/server.py with the gate and audit sink wired in), which is what unblocks pointing origin at the Space.

2026-07-25 — Deploy surface built (`gradio_ui.py` + `src/server.py`)

The last M3 checklist item that needed code. The Space can now actually run; what remains before a deploy is ops (secrets, enforcement) and the C4 caching gap, not app-building.

gradio_ui.py — three tabs (variant status / variant×subtype / BYOD upload) over the existing tools, gr.LoginButton() for OAuth, and three gr.api machine endpoints for the orchestrator. Handler shape ported from pdac-subtype-agent: gate first via access.check_access, audit either way, return (caution markdown, plot, JSON). Refusals are data, never exceptions — a BYOD hard-gate refusal (mouse, bad build, unparseable) surfaces as {status: "refused", reason} with an explanation, and any tool blow-up becomes {status: "error"} rather than a crashed Space.

src/server.py — one sub-FastMCP per tool module, mounted, mirroring both siblings.

Ported to support the handlers (ADR-0004 explicitly deferred the sink to this step): audit.py + logging_sink.py (LOG_SINK = local|hf) and licenses.py.

C1 (attribution) is now discharged — licenses.py stamps a citation block onto every grounded payload with the cBioPortal papers and the per-study source, which is what the terms review said belongs in the response contract. C2/C4 remain open; C4 is the real gap: ADR-0005 ratified curate-and-cache, but every tool call still hits cBioPortal live on the request path.

Three MCP/api surfaces, one set of names. Found and fixed: gradio's MCP bridge (launch(mcp_server=True)) names each tool after the Python function, gradio_client dispatches on api_name, and src/server.py registers its own names — so the same tool was appearing as _run_variant_status, query_variant_status, and query_variant_status_tool. The endpoint functions are now named exactly as the contract does (tools imported under _-aliases to free the names), and the FastMCP registrations pin name= explicitly. A test asserts the MCP names equal the descriptor's. Verified live: all three surfaces report query_variant_status / variant_by_subtype.

Live finding — gradio_client sends the caller's own HF token. A tokenless gradio_client call still arrived carrying x-hf-authorization: Bearer <the calling machine's HF token>, which the gate resolved to that machine's username. Two consequences, both now in a code comment: identity locally is ambient, and per the inherited ADR-0018 finding HF Space edge infra strips x-hf-*, so the same call that authenticates locally will be denied in the deployed Space unless the orchestrator sets the custom x-orchestrator-token. Fail-closed either way, but it will look like a bug if forgotten.

Verified, not assumed: app boots and serves HTTP 200; /panel answered end-to-end over gradio_client (19 genes); with ACCESS_CONTROL=enforce and a non-matching allow-list the same call was denied by name. 79 tests pass, no network (30 new: UI gating, machine-endpoint auth, audit-record shape, deploy-surface contract).

The audit test worth keeping honest is the negative one: sample ids, per-sample status, and provenance strings must never reach the sink. Gene names may (a fixed public panel).

Deps pinned to what this was built against (gradio 6.18.0, fastmcp 3.2.3, huggingface_hub 1.10.1) and sdk_version/python_version added to the README front-matter. run_logs/ gitignored.

Still local only — no remote. Next: the ops half — point origin at the Space, set ALLOWED_IDS/ADMIN_IDS, flip ACCESS_CONTROL=enforce, and close C4 (curate-and-cache) before prod traffic.

2026-07-25 — C4 closed: curate-and-cache (the API is no longer on the request path)

ADR-0005 ratified "public cBioPortal + curate-and-cache" back on 2026-07-24, but the code kept calling the API live on every request — the ADR and the implementation disagreed. They now agree.

The split. python -m src.curate <study> (new, src/curate.py) is the only thing that touches cBioPortal: it fetches, classifies through the same M1 rules, and writes src/resources/curated/<study>.json. The request path reads that artifact via src/workflows/curated_store.py and never fetches. An uncurated study raises NotCuratedError telling the operator to curate it. No fallback on purpose — a fallback makes the coupling intermittent rather than absent, which is harder to debug and shows up under load.

Artifacts carry clinical attributes, not just the matrix. Caching the matrix alone would have left two live calls on the path: the pancreatic-lineage filter and the subtype label-join both queried clinical-data per request. Both now read the artifact. Artifacts are sparse (only non-WT / non-neutral cells) — a dense wall of "WT" is noise in a diff.

Four cohorts curated and committed (real data, ~940KB total): paad_tcga (186 samples), ccle_broad_2019 (1739), paad_qcmg_uq_2016 (456, no CNV — the coverage gate confirmed against live data), paad_utsw_2015 (109). The offline path reproduces the spike exactly: TCGA KRAS 136 / TP53 104 / SMAD4 37, CCLE narrowing to 57 pancreatic lines with KRAS 84%. A live /query_variant_status now answers in 0.44s; an uncurated study refuses in 0.07s. The UI's cohort dropdown is read from the artifacts rather than hardcoded — "registered" and "curated" are the same set now.

Two bugs found by verification, not by unit tests.

A C4 leak. clinical_values returned None both for "study not curated" and for "curated but has no such attribute", and the second case fell through to a live call. Probing the eight known subtype attributes against a cohort carrying none of them meant eight live API calls per request — on real paad_tcga, which has no subtype attribute at all. Absence has to be an answer ({}), not a miss. Caught by sabotaging urllib against the real artifacts; the unit test missed it because the fixture study happens to carry MOFFITT_SUBTYPE and matched on the first probe. Regression test added.
CCLE's SUBTYPE is histology, not molecular subtype — "Adenocarcinoma", "Melanoma", "Small Cell Lung Cancer". SUBTYPE is in SUBTYPE_ATTRIBUTES, so auto-detection would have reported a confident "variant × subtype association" that was really variant × histology across a pan-cancer panel. Pre-existing, but curating CCLE is what made it reachable. Auto-detection now validates a candidate's values against a closed PDAC subtype vocabulary (basal/classical/squamous/ progenitor/immunogenic/ADEX/…) and skips attributes that don't speak it; an explicitly named attribute is still trusted. A plausible-looking wrong answer is worse than no answer.

Tests: 90 pass, no network (12 new). patched_cbio now patches the client and curates from it, so tests exercise the offline path production actually takes; no_curated_artifacts isolates the curation-time tests, since committed artifacts correctly take precedence over a patched client.

Remaining on the source conditions: C2 (per-study license flag) is untouched; C3 is partial (backoff + cache, still no jitter/self-throttle/User-Agent contact). New backlog item: nothing warns when an artifact goes stale — each carries curated_at, but re-curation cadence is undecided.

Still local only — no remote. Next: the ops half (point origin at the Space, allow-list, ACCESS_CONTROL=enforce).

2026-07-25 — C2 + C3 done: the cBioPortal pre-prod gate is GREEN

All four source conditions from docs/cbioportal-terms-and-deployability.md (ratified in ADR-0005) are now implemented rather than merely accepted. The checklist in that doc is ticked.

C2 — per-study license/access, grounded in the source. The key call: the access tier comes from cBioPortal's own publicStudy / readPermission fields (cbioportal_io.study_metadata), not from a list we maintain by hand — a hand-kept list is exactly the thing that goes stale silently and is then trusted. licenses.study_terms() builds the block; it is stored in every artifact and surfaced in every answer as citation.study_terms, because provenance that stops at the storage layer is provenance nobody reads.

Enforced in two places: curation refuses a non-open study before fetching any variant data (ControlledAccessError), and the serve path refuses an artifact declaring controlled access (ControlledAccessArtifactError). The second should be unreachable — which is precisely why it exists, since artifacts are files a human can hand-add or copy between deployments.

Commercial-use restrictions are not an API field, so they stay a hand-maintained overlay (licenses.COMMERCIAL_USE_RESTRICTED) whose default is "unreviewed", never "unrestricted". Silence about terms must not read as permission.

Schema bumped v1 → v2. A pre-C2 artifact has no terms block, and the serve gate only fires on a present non-open block — so a v1 artifact would have served with C2 silently dropped. Refusing it outright and re-curating was the only honest option. All four cohorts re-curated.

C3 — polite client. Descriptive User-Agent carrying the project URL, plus an optional CBIOPORTAL_CONTACT env var (env, not hardcoded, so no personal address lands in a public repo — a real address still needs setting in the Space before prod, now a TODO gate). ~4 req/s serial self-throttle, backoff with jitter honouring Retry-After, and retries limited to 429/5xx: a 404 is our own bug and retrying it just repeats the mistake at their expense. All of this is curation-time cost only (C4), so being conservative is effectively free.

Tests: 106 pass, no network (16 new). Includes a test that every artifact committed to this repo is open-access — the C2 posture asserted, not assumed.

Two follow-ups recorded in TODO.md: set CBIOPORTAL_CONTACT at deploy, and actually review commercial-use terms per curated study (today they all correctly read "unreviewed").

Still local only — no remote. Next: the ops half of the deploy checklist.

2026-07-25 — Both C-condition follow-ups closed; CCLE turns out to be commercially RESTRICTED

Follow-up 1 — the contact address — was a requirement I invented. Annie pushed back: why an email? Correct. The polite-bot convention identifies via +<URL> (as Googlebot/2.1; +http://www.google.com/bot.html does), and the Space URL we already carry is the contact channel — the Space page has a Community tab anyone at cBioPortal can post to. No sibling repo sets a User-Agent at all. CBIOPORTAL_CONTACT stays as an optional second channel; unset is now documented as a complete configuration, the curation nag is gone, and the pre-deploy gate is deleted rather than deferred. Added .env.example covering the auth gate, audit sink, and this.

Follow-up 2 — commercial-use review — overturned an assumption in our own code. licenses.COMMERCIAL_USE_REVIEW now records all four curated cohorts with instrument, source URL, confidence, and obligations:

study	verdict	basis
`paad_tcga`	permitted	NIH/NCI GDS open-access tier; TCGA moratoria lifted
`ccle_broad_2019`	restricted	Broad DepMap/CCLE Portal Terms
`paad_qcmg_uq_2016`	no_restriction_found	ICGC E.8.1 open tier
`paad_utsw_2015`	permitted	CC BY 4.0 (article + supplementary)

CCLE is not CC BY 4.0. That licence covers DepMap releases; ccle_broad_2019 is sourced from the Broad CCLE Portal, whose terms bar commercial use and also bar using the data to train or enhance ML/AI models beyond internal research. We had asserted "CC-BY-4.0" in both licenses.py and CLAUDE.md — wrong, now corrected in both. Our academic use stays clear either way; the flag exists so nobody downstream reads our silence as permission.

Two structural findings about cBioPortal licensing, worth not re-learning: it has no machine-readable licence field (meta_study.txt has no such key), and every datahub study ships an identical ODbL boilerplate LICENSE file — so that file is not a per-study assertion. The FAQ's promise that restrictions "will be explicitly mentioned in the study information" is not reliably kept (CCLE's description says nothing). Terms must always resolve to the upstream source, never to cBioPortal's redistribution, and where they conflict — they do for CCLE — upstream governs.

Vocabulary distinction that is load-bearing: no_restriction_found ≠ permitted. commercial_use_permitted() returns False for it, and for unreviewed. Absence of a prohibition is not a grant, and anyone asking programmatically is about to rely on the answer.

The restriction also surfaces in the UI, not only in citation.study_terms — a licence constraint nobody reads is one nobody honours.

Two items a human must confirm before anyone relies on this commercially (both in TODO.md): read https://depmap.org/portal/terms/ in a real browser (bot-protected, so the verbatim CCLE clause was obtained indirectly — substance high-confidence, wording not directly verified), and establish whether ICGC affirmatively permits open-tier commercial use rather than merely not prohibiting it.

Also rescued: the founding seed originals were untracked in pdac-subtype-agent (present in one working tree, in no commit, in a checkout several lanes share). Their content had been merged here rather than copied, so the originals existed in no history at all. Archived verbatim under docs/design/seed/ with a map to where each one's content now lives (bdac9d2).

Tests: 109 pass, no network. Still local only — no remote.

2026-07-25 — DEPLOYED: `origin` points at the Space, app is RUNNING (private)

First deploy. origin now is the HF Space (anne-voigt/pdac-genomics-agent), matching the sibling model — there is no separate GitHub repo, and git push origin main rebuilds the Space.

Merged HF's scaffold rather than force-pushing over it. The Space carried an auto-generated initial commit (a placeholder README + .gitattributes). Force-pushing would have discarded .gitattributes, which configures LFS patterns; I merged with --allow-unrelated-histories and resolved the README to ours. HF's default front-matter was wrong for us anyway — it pointed at app.py, gradio 6.20.0 and python 3.13, against our tested gradio_ui.py / 6.18.0 / 3.11 (and sdk_version must match the requirements.txt pin or the Space builds a different UI than the tests cover). Checked first: .gitattributes has no *.json LFS rule, so the curated artifacts stay plain git.

Shipped an attribution file for the artifacts before pushing. Committing the curated matrices makes them downloadable, which is exactly the case the terms review singled out: serving derived answers is a produced work, but "if the matrix is ever offered for download, ship it ODbL + attribution". src/resources/curated/ATTRIBUTION.md carries the platform citation, each study's original citation, and each study's commercial-use verdict (CCLE restricted).

Verified live, not just reported RUNNING: /panel → 19 genes; /query_variant_status on paad_tcga → 186 samples with KRAS 136 / TP53 104 / SMAD4 37, identical to local and to the original spike; CCLE → 57 pancreatic lines with commercial_use: restricted surfacing. Response 0.80s — curate-and-cache (C4) working in the Space, no API on the request path.

Auth state — deliberately incomplete. hf_oauth: true is live and ADMIN_IDS=anne-voigt is set as a Space secret. ACCESS_CONTROL is not flipped, so the gate is still dark-launched OFF; the Space is PRIVATE, which is what contains the exposure. Two decisions left for a human, both in TODO.md: who goes in ALLOWED_IDS (a membership question, not an ops step), and when to enforce.

Enforcement has a known consequence, worth deciding with eyes open: HF edge infra strips the x-hf-* header gradio_client sends automatically, so under ACCESS_CONTROL=enforce the machine API denies every caller until the orchestrator sends a custom x-orchestrator-token whose identity is allow-listed. The OAuth UI path is unaffected. Nothing consumes the machine API yet, so this costs nothing today — but it will look like a broken deploy if hit unaware.

The one combination that must never exist is public + gate-off. Private + gate-off is fine; flipping the Space public requires enforcing first.

Still outstanding before this is real to users: the Rosie/Carl confirm that per-sample variant status is the near-term need (the "should it ship at all" gate — never done), and merging deploy/orchestrator_registration.yaml into the orchestrator's agents.yaml.

2026-07-27 — `ACCESS_CONTROL=enforce` flipped; gate verified fail-closed

The gate is on. ACCESS_CONTROL=enforce is set as a Space variable rather than a secret — it carries nothing sensitive, and someone auditing "is the gate actually on?" should be able to read that off the settings page without needing access to secrets. ADMIN_IDS=anne-voigt stays a secret.

Verified live: the machine API (/panel) now returns {"status": "denied"} with the sign-in message. That is the correct, predicted behaviour and it confirms the documented asymmetry — HF edge infra strips the x-hf-* header gradio_client attaches automatically, so no identity reaches the handler and the gate fails closed. The orchestrator cannot call this Space until it sends a custom x-orchestrator-token with an allow-listed identity. Expected, not a regression.

Two things are NOT done, and the first one matters today:

ALLOWED_IDS is unset, so only anne-voigt can use the Space. Setting it needs the lab members' actual HF usernames. Guessing them fails in both directions — a wrong guess either locks out someone real or hands access to an unrelated account that happens to own that username. Asked; not invented.
The allowed path is unverified. The denial is confirmed; nobody has signed in through OAuth and run a query. That is the path that must work, and it cannot be tested without an interactive browser sign-in. Rollback if the owner is locked out: set ACCESS_CONTROL to empty.

The Space also slept between the deploy and this change (idle timeout on the free tier, not an error) — a restart brought it back in ~2 minutes. Worth knowing before someone reads SLEEPING as a failure.

2026-07-27 — Carl allow-listed; UI gate verified BOTH ways; machine auth still failing

ALLOWED_IDS=anne-voigt,cpelz741. Carl's HF id was taken from pdac-subtype-agent prod (SHOWCASE_STATUS.md records its live config) rather than guessed — a wrong username would either lock out a colleague or hand access to an unrelated account that owns that name.

The OAuth UI path is now VERIFIED, which was the last open deploy question. Signed in as anne-voigt in the deployed Space: a CCLE query rendered the frequency chart (KRAS ~84% across the 57 lineage-filtered pancreatic lines) and displayed the licence caution — "commercial use of this cohort is restricted under Broad DepMap/CCLE Portal Terms". Signed out, the same query returns {"status":"denied"}. Both halves of the gate confirmed end-to-end in the real UI.

Incidental: the Space does not render inside the huggingface.co page frame (blank), but works at the direct host https://anne-voigt-pdac-genomics-agent.hf.space. Use the direct host for testing. The cohort dropdown renders its value only on focus — a Gradio display quirk, not an empty default.

Broad terms read directly (see previous entry): confirmed, and the Continuity of Terms clause means rehosting requires reposting them in full — done in CCLE_BROAD_TERMS.md.

Open and now spun off: the machine API denies the orchestrator despite correct configuration. The orchestrator already sends x-orchestrator-token (its commit 05bda30, built for the subtype agent's ADR-0018 gate) and that token resolves to anne-voigt, which is allow-listed here. It is still denied. What is ruled out: the header is not being dropped — a local probe confirmed x-orchestrator-token arrives in gr.Request.headers alongside x-hf-authorization. So the failure is server-side identity resolution: _resolve_token_identity() calls huggingface_hub.whoami() at request time from inside the Space and returns None on any failure. It fails closed, which is right, but silently, which is why this took a live test to notice.

The design question worth answering before patching: should a gate depend on an outbound network call per request at all? That is a third-party dependency on the request path — the same class of coupling ADR-0005 C4 removed for cBioPortal. A shared-secret comparison would have neither the latency nor the failure mode.

2026-07-28 — ADR-0006: the variant×subtype join contract (written down at last)

The agent's differentiator was the only major decision without an ADR — it lived as prose in TODO.md, DESIGN-pdac-genomics-agent.md §4, and module docstrings, despite being the one part of the design that spans two Spaces. Now docs/adr/ADR-0006-variant-subtype-join-contract.md.

No behaviour changed. This documents what M2 already ships (plus the vocabulary fix already in code) and, crucially, states the orchestrator's obligations normatively so the other repo has a contract to build against rather than a docstring to interpret.

What it settles:

The split. Label-join is local (cohort carries a validated subtype clinical attribute → join on sample id + test here); the live-PurIST join is orchestration (ADR-0001/0011 — stateless siblings never call each other), returning {join_available: false, route: "orchestrator"}. altered vocabularies and the Fisher-2×2 / χ²-R×2 rule are pinned here as the single normative statement, because the orchestrator will implement the same test on its side.
The routing contract — the discriminated return shape (switch on join_available, never on the presence of genes), the orchestrator's three ordered steps, and an explicit prohibition: it may not synthesize a label and call back through subtype_attribute. Sample-id alignment across Spaces stays OPEN (the spike measured 145/146 within one cBioPortal cohort — not the cross-Space case that would actually need a map). Interim rule: exact string equality, report join coverage, no fuzzy/prefix matching, and a collapsed join is a refusal with the numbers shown. A silent low-coverage join is the dangerous failure.
Label validation is load-bearing. Attribute-id matching alone is unsafe on evidence: ccle_broad_2019's SUBTYPE holds histology ("Adenocarcinoma", "Melanoma"), which produced a confident variant×histology association dressed as variant×subtype. Auto-detect now validates VALUES against a closed PDAC subtype vocabulary (≥0.5 of non-blank values); an explicitly named attribute bypasses it (caller took responsibility) and is always visible in subtype_source.
Statistical honesty — kept as-is, deliberately. No multiple-testing correction across genes (descriptive screen over a user-chosen panel; the right denominator is session-level knowledge a stateless tool lacks — FDR is the orchestrator's call), mandatory caveat saying so, and testable=false + machine-readable reason instead of a bogus p-value. Contingency table and n are returned even when untestable — KRAS is altered in ~90%+ of PDAC, so the flagship gene is the one that degenerates most often. No hard minimum-n refusal; show n, caveat, refuse only the genuinely undefined.
Metadata label ≠ live PurIST call. Never interchangeable, never mixed in one contingency table, never blended to fill gaps. subtype_source is required on every joined result (clinical_attribute:<ATTR> vs purist:<version>) and may not be dropped downstream. "Do they agree?" is a concordance analysis, not a licence to pool.

Cross-references pdac-subtype-agent ADR-0020, whose alignment-by-sample-id primitive this reuses — variant×subtype is a cross-modality instance of the same cross-source alignment pattern, and inherits its align-never-pool + mandatory-caveat stance.

Ledger protocol followed to the letter: 0006 claimed with status CLAIMED, committed alone, pushed to main (a792086) before the ADR was written — the step that exists because of the real 0003 collision. Flipped to PUBLISHED after.

Known follow-up (not a blocker): two association-test implementations will exist (here and orchestrator-side) and must agree on §1's rules, or the same cohort answers differently by route.

2026-07-28 — Machine-API auth: the Space was never broken; the bug is caller-side

The 2026-07-27 diagnosis in the entry above is WRONG and is corrected here. It concluded the failure was "server-side identity resolution — _resolve_token_identity() calls whoami per request and returns None". Probing the deployed Space directly disproves that:

probe (against prod, `ACCESS_CONTROL=enforce`)	result
valid HF token in `x-orchestrator-token`	`status: ok` — full panel returned
bogus token in `x-orchestrator-token`	denied
header absent entirely	denied

So whoami resolves fine from inside the Space and the gate discriminates exactly as designed. Root cause is the caller: pdac-analysis-orchestrator's router._build_client_kwargs() attaches x-orchestrator-token only when its own HF_TOKEN is set — an unset secret on that Space sends no header at all. The fix is a secret on the orchestrator Space; no code change was needed here.

Why it was misdiagnosed, and the actual fix shipped. All three failure modes returned one message — "🔒 Please sign in with your HuggingFace account" — which reads as a this-Space auth bug and sent the investigation server-side, where nothing was wrong. A machine caller cannot sign in; telling it to is a category error. Machine denials now carry a machine_auth diagnosis:

no_token_header — caller misconfigured, message names the orchestrator's HF_TOKEN.
token_unresolved — a token arrived but did not resolve (bad/expired, or an upstream blip).
not_allowlisted — resolved fine, simply not on ALLOWED_IDS. The only one of the three that is a real authorization decision; still names who was refused.

4 regression tests pin the three states apart (tests/test_machine_api_auth.py). 113 green.

Method note worth keeping: the earlier conclusion came from reading the code and a local header probe; it took ~4 curl calls against the deployed Space to overturn it. When a gate is "failing", probe the deployed thing with a known-good credential before theorising about why the resolution path is broken — the cheap experiment discriminates caller-side from server-side in one step. The lookalike messages are what made theorising feel productive.

Still open (unchanged, not urgent): should an auth gate depend on an outbound whoami on the request path at all, versus a shared-secret comparison? Same coupling class ADR-0005 C4 removed for cBioPortal. It works today; this is a latency/failure-mode question, not a correctness one.

2026-07-28 — Re-curation cadence (ADR-0007): drift, visibility, and no refusal

Closed the gap ADR-0005 C4 created and left open. curated_at had been written into every artifact since commit one and read by nothing — no threshold, no warning, no policy. A cohort could drift from upstream for years and every answer would look equally confident.

Cadence is upstream-driven, not calendar-driven. python -m src.curate --check reads one /studies/<id> per curated study and compares upstream's importDate against the stamp now recorded at curation (source_import_date). Drift → re-curate; no drift → do nothing, however old the file is. The empirical confirmation is the point: all four cohorts were last imported upstream in January 2026 and curated in July — six months of a published cohort not moving. A "refresh monthly" policy would have rewritten four byte-identical files and spent the C3 politeness budget for zero new data. REVIEW_AFTER_DAYS=180 / STALE_AFTER_DAYS=365 are only a backstop for when nobody runs the check.

Age is visible to the reader, not just the maintainer. Every grounded answer's citation block now carries curation (curated_at, age_days, level, source_import_date, drift_comparable, plain-language note); the UI renders a line past the horizon, next to the existing licence caution. Same stance that put terms in the response contract — provenance that stops at the storage layer is provenance nobody honours.

Stale never refuses. A stale-but-valid cohort is old, not wrong: its calls were correct for the release they were curated from. Refusing would withhold a correct answer over a maintenance lapse and would dilute what a refusal from this agent means. Refusal stays for the incorrectness gates (build, species, absent modality, access tier). Pinned by test_stale_artifact_still_answers_and_says_so.

--check never writes, deliberately — re-curation stays a reviewed commit. A checker that repaired what it found would put the API back on an unattended path, i.e. undo C4 quietly.

Two implementation notes worth keeping:

No schema bump. source_import_date is optional; absence degrades to an honest drift_comparable: false. Bumping to v3 would have refused all four existing artifacts and forced exactly the blanket re-curation this cadence exists to avoid.
The four pre-existing artifacts had their source_import_date backfilled once in the same commit from a live metadata read (metadata only — no status call touched), so --check is useful today rather than "unknown until each happens to be re-curated".

121 tests pass (11 new, tests/test_recuration_cadence.py). --check verified live against the public instance: all four up_to_date. Not deployed — the branch claude/infallible-kowalevski-2d4648 is pushed to origin and rebased onto main, but origin/main IS the live Space, so promoting is a deliberate deploy decision.

Renumbered 0006 → 0007, and the near-miss is the lesson again. A concurrent lane had already claimed and published 0006 (the variant×subtype join contract) on main. This lane's worktree held its own copy of RESERVED.md and predated that claim, so the one-shared-row conflict could not fire — the third time that root cause has bitten this family. The claim here was committed alone and pushed, but to the feature branch rather than main (protocol rule 5), because origin/main IS the live Space and pushing it is a deploy: rule 5 and the deploy model are in genuine tension on this repo. What actually caught it was reading SHOWCASE_STATUS.md before updating it. Practical check until rule 5 has a deploy-safe form here: git fetch origin main && git show origin/main:docs/adr/RESERVED.md before claiming.

Weekly job added the same day (.github/workflows/curation-freshness.yml, Mondays 07:00 UTC + workflow_dispatch, report-only). It does not run, and the file says so in its own header rather than looking green. origin IS the HF Space and HuggingFace does not execute GitHub Actions — and unlike every sibling (DecoupleRpy_Agent, pdac-subtype-agent, pdac-analysis-orchestrator, each of which carries a second github remote), this repo has no GitHub mirror at all, so it has never had CI of any kind. Committed anyway on the DecoupleRpy_Agent/.github/workflows/security.yml precedent: the definition never forks, and adding the mirror later is then zero work. The one remaining ops step is creating Anne-Voigt/pdac-genomics-agent on GitHub and git push github main — that turns the check on and gives the repo CI. A local launchd agent was deliberately NOT installed: a job that runs on one laptop is invisible to everyone else, which is how the original gap happened. A test (test_weekly_job_is_report_only) asserts the job can never gain a writing src.curate step — that would re-establish the unattended API path C4 removed, in a file nobody reads once it is green. 122 tests.

Deployed 2026-07-28 (6f23703 → live; a third lane's 4b8cb52 landed on top minutes later and the Space is RUNNING at that sha, which contains ours). Rebased onto fdfbfb3 (the machine-auth diagnosis lane) before pushing — 126 tests green with both lanes' changes together, up from our 122. Verified in prod, not just RUNNING: paad_tcga → 186 samples, KRAS 136, TP53 104 — identical to local and to the original spike — and citation.curation serves level: current, age_days: 2, source_import_date: 2026-01-12, drift_comparable: true, confirming the backfill landed in the artifacts the Space actually reads.

Fixed the same day (see the next entry): the machine endpoints take genes as a comma-separated string (_split_genes), not a list, and a list used to return AttributeError: 'list' object has no attribute 'split'.

2026-07-28 — Cohort survey; +2 curated (6 total); per-GENE coverage gate added

Surveyed all 538 cBioPortal studies for pancreatic cohorts and curated the two that earned it. Curated set is now 6: paad_tcga, ccle_broad_2019, paad_qcmg_uq_2016, paad_utsw_2015, pancreas_cptac_gdc (new), pdac_msk_2024 (new). The UI cohort list reads from the artifacts, so both appeared with no code change.

The scarce thing is confirmed absent. No pancreatic study in cBioPortal carries a real Moffitt/Bailey/Collisson/PurIST subtype clinical attribute — checked by sweeping all 19,136 clinical attributes portal-wide, not just the candidates. The two near-misses are both correctly rejected by subtype_join.looks_like_pdac_subtype: paad_tcga_pan_can_atlas_2018's SUBTYPE is the literal string "PAAD" (a disease code, 152 patients), and paad_iatlas_prince_2022's IMMUNE_SUBTYPE is Thorsson C1–C6 immune clusters, not PDAC molecular subtypes. So variant_by_subtype's in-agent label-join still has no cohort to run on, and this is now a searched-and-settled fact rather than an untested assumption — the live-PurIST route through the orchestrator is the only path, exactly as ADR-0001 has it. Do not re-survey for this; re-check only if cBioPortal adds a study.

Curated, with why:

Study	n	Modalities	Note
`pdac_msk_2024`	2,336	mut + discrete CNV	Largest PDAC cohort available anywhere here — 12× TCGA. Textbook frequencies (below). Targeted MSK-IMPACT panel — see the gate. Commercial use RESTRICTED.
`pancreas_cptac_gdc`	183	mut + discrete CNV + expr	CPTAC-3 pancreas, GRCh38, exome-wide, commercial use permitted. The GDC rebuild was chosen over `paad_cptac_2021` (140, same cohort) precisely because that one has no CNV.

Driver-frequency sanity check passed (the check that a wildly-off number is a curation bug, not biology): pdac_msk_2024 KRAS 93.7% / TP53 76.1% / SMAD4 21.9% (+4.4% deep-del) / CDKN2A 23.4% (+15.4% deep-del); pancreas_cptac_gdc KRAS 83.1% / TP53 69.4% / SMAD4 19.1% / CDKN2A 21.3%. Both sit on the published ~90/60-70/20/20. (paad_tcga's KRAS 73% remains the known outlier — that cohort carries non-PDAC histologies.)

Rejected, and why — more cohorts is not better, each one is committed data someone maintains:

pancreas_msk_2024 (395) — 348 of its 395 samples are also in pdac_msk_2024. Curating both would double-count every frequency. Its 22 non-PDAC pancreatic histologies are not worth that.
paad_icgc (99) — same ICGC PACA-AU patients as paad_qcmg_uq_2016 (456), and mutation-only. Strictly dominated by the cohort already curated.
paad_tcga_gdc (186) / paad_tcga_pan_can_atlas_2018 (184) — re-releases of paad_tcga.
pancan_pcawg_2020 (285 PAAD) — an aggregation of ICGC PACA + TCGA; overlaps two curated cohorts.
ccle_broad_2025 (1,981) — supersedes ccle_broad_2019 on paper but ships no discrete CNV (LOG2-VALUE only), so it would be a downgrade, and it duplicates the lines we have.
paad_iatlas_prince_2022 (93) — mutation-only, and KRAS 73% in a pure-PAAD metastatic cohort is low enough to suggest under-powered calling. No subtype label (see above). Nothing gained.
pancreas_ctdna_msk_2025 (412) — cfDNA, not tumour tissue. Different detection sensitivity; its frequencies are not comparable to the tissue cohorts and pooling them would mislead.
paad_cptac_2021 (140) — superseded by the GDC rebuild, which has CNV.
paad_msk_2025 (29) — too small to add anything.
paac_* / panet_* / pact_jhu_2011 — acinar, neuroendocrine, cystic. Not PDAC; the driver panel does not describe them.

New: the per-GENE assay-coverage gate (assayed_panel_genes + genes_assayed). The existing per-study modality gate does nothing for a targeted cohort. MSK-IMPACT does not sequence ELAVL1, GATA6, ACVR1B — 3 of our 19 panel genes — so without this, pdac_msk_2024 would have reported GATA6 as 0% altered in 2,336 samples: not a low frequency but no measurement, wearing the most persuasive number in the repo. GATA6 is the one that would have hurt, since CLAUDE.md names it the clearest variant×subtype signal. Curation now resolves coverage from the GENE_PANEL attribute (intersection across panel versions, not union — a gene on only some samples' panel has no cohort-wide denominator), stores genes_assayed, and both tools return assayed: false + unassayed_genes instead of a frequency. genes_assayed: null means exome/genome-wide; all four pre-existing artifacts are, so no re-curation was needed and the schema was not bumped (unlike v1→v2 for terms, a missing key here is a true statement about them).

Licensing resolved upstream for both (licenses.COMMERCIAL_USE_REVIEW + STUDY_CITATIONS + ATTRIBUTION.md), each with its own reviewed: 2026-07-28 — study_terms now honours a per-entry review date instead of stamping everything with the module default.

pancreas_cptac_gdc → permitted, high confidence. NIH/NCI GDS open tier (GDC), the same instrument as paad_tcga; this study is literally a GDC rebuild. CPTAC's embargo is retired.
pdac_msk_2024 → restricted, and this one needs a human eye. The article (Varghese et al., Nat Med 2025;31:466–477) is CC BY-NC-ND 4.0 — non-commercial — while the authors' own data-availability statement deposits the cohort in cBioPortal as "publicly accessible and downloadable", where it carries the generic ODbL boilerplate. Both are recorded; the stricter reading governs, consistent with "resolve upstream". The open question is ND, not NC: the artifact we commit is a derived status matrix, arguably an adaptation, and we redistribute it. Fine for the non-profit research this agent does; anyone going further should resolve ND with the authors. Tracked in TODO.md.

Found in passing, then fixed the same day — the denominator was wrong, on BOTH modalities. Frequencies divided by the whole cohort, but a study assays only part of what it contains, and never-assayed samples read as WT/neutral and got counted as denominators. The headline:

paad_tcga reported KRAS at 73% (136/186). The truth is 91% (136/150) — 36 of its samples were never sequenced. paad_qcmg_uq_2016 likewise 75% → 90%.

This retires a "known quirk" that was actually a bug. The 73% had been explained away as "the TCGA cohort includes non-PDAC histologies". That explanation was wrong, and it was load-bearing — it made a wrong number look understood, which is why it survived so long. All four tumour cohorts now land KRAS at 90–94%, matching the published ~90%. Do not re-derive the histology story.

The fix (profiled block, schema v2→v3): curation records cBioPortal's own declared per-modality sample lists (<study>_sequenced, <study>_cna) via cbioportal_io.profiled_sample_ids; StatusMatrix.denominator(modality) intersects them with the (possibly lineage-filtered) cohort; every frequency divides by that and reports n_profiled alongside n_altered, so the denominator is stated rather than inferred. Unprofiled samples are also dropped from per_sample (a neutral there reads as "measured, normal") and from variant_by_subtype's contingency table (where they had inflated the WT column and biased the test toward "no association" — a denominator error that moves a p-value). The UI axis now says "% of samples profiled for that modality", not "% of cohort".

Bumped the schema on purpose. A v2 artifact carries no profiled block, which reads as "all samples assayed" — false for three of the six, and it produces a wrong number that looks right. That must fail loudly, so v2 artifacts are refused and all six were re-curated. (Contrast genes_assayed, added in the same pass without a bump: its absence was a true statement about every prior artifact.)

Corrected denominators now in the artifacts: paad_tcga 186 cohort / 150 sequenced / 184 CNV; paad_qcmg_uq_2016 456 / 383 (no CNV); pancreas_cptac_gdc 183 / 181 / 149; ccle_broad_2019 57 pancreatic lines / 53 / 44; paad_utsw_2015 and pdac_msk_2024 fully profiled.

Tests: 136 passed (109 before; +12 tests/test_panel_coverage.py, +15 tests/test_profiled_denominator.py).

The new literature test is the real guardrail. test_kras_lands_near_the_published_90_percent pins KRAS at ~90% in every tumour cohort. That test would have caught the denominator bug on day one; the sanity-check-against-published-frequencies step had been done by eye and not encoded.

Also fixed: the test suite was not actually hermetic. conftest.py's docstring claimed "NO live network", but patched_cbio calls src.curate.curate, which fetches study_metadata — and that function was never patched. Every fixture-using test made a real request to cbioportal.org per fixture study. It surfaced as a 12-error, 54-second run mid-session that passed on retry: a third party's latency reported as our bug. study_metadata is now patched from the fixture, and _req is stubbed to pytest.fail, so an unanticipated call fails loudly instead of leaving CI. Verified by running the suite with socket.connect blocked outright — 111 tests pass with no network at all (the three UI test modules still touch it, but that is Gradio's mocked-OAuth HF call at import in a local dev env, not our code and not present on a Space).

2026-07-28 — Machine endpoints: caller-side argument errors now say so

Third instance of one lesson in two days, and the pattern is now explicit: a message that doesn't name the responsible side sends the reader to the wrong repo. The auth lane fixed it for denials (machine_auth: no_token_header | token_unresolved | not_allowlisted); this does the same for arguments (machine_input: <field>_not_a_string), including the exact string to send instead (genes="KRAS,TP53") and an explicit "this is a caller-side argument error — nothing is wrong with this Space".

A real bug surfaced next to the message. registered_source(source) is called to build the audit descriptor before the auth gate — outside _safe_result — so a non-string source raised there and became an uncaught 500 rather than a result. It was the one input that escaped the catch-all. _safe_text now feeds the descriptor defensively; the caller gets the typed error after the gate.

Deliberately NOT coercing a list into a string. Gradio publishes these parameters as str in the endpoint schema (Client.view_api()); silently accepting a second undeclared shape would make the published contract and the real one disagree, and the next caller reads the schema and is misled again. One contract, with instructions when you miss it.

Ordering is gate-first: an unauthenticated caller gets `denied` and learns nothing about our inputs — pinned by `test_argument_errors_are_reported_only_after_the_auth_gate`. 132 tests (7 new, alongside the denial-diagnosis guards they mirror).

2026-07-28 — Hermetic is now enforced, and the whole suite is hermetic

Finishes the item above. That fix patched the one hole that had been found; this one removes the reason holes could survive — and closes the last remaining one.

The lesson being encoded: the "NO live network" claim lived in a docstring for weeks while being false. Nothing checked it. So the property is now enforced by a guard and the guard is itself under test. Do not "simplify" either away.

Socket guard (tests/conftest.py, module scope) — wraps socket.socket.connect / connect_ex / socket.create_connection for the whole session and raises on any non-loopback address, naming the host and pointing at conftest for the fix. Installed at import of conftest, not as an autouse fixture, deliberately: the leaks that mattered happened at collection time (a test module's import gradio_ui runs the app graph at module scope), and a fixture runs too late to catch those. Loopback and AF_UNIX are allowed — local sockets are not what the guard is about.
The three UI modules no longer touch the network. test_ui_handler_gating.py, test_deploy_surface.py, test_machine_api_auth.py all import gradio_ui, whose gr.Blocks graph is built at module scope. Off a Space, gradio's attach_oauth falls back to _add_mocked_oauth_routes, which calls HF whoami to mock a local login profile. That is gradio's debug convenience, not our code, and it does not happen on the deployed Space (there SPACE_ID is set and the real OAuth routes are used). conftest now stubs gradio.oauth._get_mocked_oauth_info.
- Why that seam and not the others considered: it is the only function on that path that touches the network, so stubbing it leaves every route gradio registers intact and the UI tests still exercise the real graph. Restructuring gradio_ui.py to build Blocks inside a function was rejected as far more invasive for a test-only problem — and its module-level demo is what HF Spaces launches, so it must keep working. gradio_ui.py is unchanged; the entire fix is test-only.
- Known coupling: _get_mocked_oauth_info is a gradio private. A gradio upgrade that renames it would break the stub — which is why test_hermetic.py imports all three UI modules explicitly, so that breakage surfaces as a named test failure rather than as a mystery connection at collection time.
tests/test_hermetic.py (new, 7 tests) — asserts the guard is installed, that it refuses a routable address, that the message names the host and points at the fix (a guard with an unhelpful message gets disabled by the next person who trips it), that loopback still works, and that the three UI modules import cleanly.
Bonus: CI no longer needs an HF login. _get_mocked_oauth_info raises ValueError when the machine has no HF token, so those three modules could not have been imported on a clean CI box at all. Verified with HF_TOKEN unset and HF_HOME pointed at an empty dir.

Verified — 143 passed with sockets blocked outright and no --ignore (136 before + 7 new), using the same method that exposed the original bug: stub socket.socket.connect / socket.create_connection to raise, then pytest.main(["-q"]). Previously 111 passed that way with the three UI modules ignored.

Mutation-tested, so the guard is not vacuous: removing the study_metadata patch and the _req stub reproduces the original bug, and the guard catches it with AssertionError: the test suite attempted a network connection to ('www.cbioportal.org', 443). Restored after.

Suite runtime 22s → 4s — that difference was the network, on a good day. On a bad one it was the 54-second, 12-error run that started this.

No ADR. The contested part (stub a gradio private vs. restructure gradio_ui) is test-only, reversible, and documented at both the seam and the test. If it should be one, next free is 0008 — main has 0006 (variant×subtype join) and 0007 (re-curation cadence) published — and per RESERVED.md the claim must land on main alone, first.

⚠️ This branch's RESERVED.md is stale: it stops at 0005 and states "next free 0006". Both 0006 and 0007 are burned. This is the third instance of the same failure the ledger's own postmortem describes — per-lane copies mean the one-shared-row conflict cannot fire between branches, so the ledger in your working tree is exactly the thing you cannot trust for this. Verify with git show main:docs/adr/RESERVED.md before claiming a number. (The rollup suggests origin/main, but origin here IS the HF Space; local main is the reliable read.) Found by reading SHOWCASE_STATUS.md before updating it — the same habit that caught collision #3.

Environment note: this repo now has a venv at ../../.venv (Python 3.14) created for this work — gradio[oauth]==6.18.0 (the oauth extra is required or the UI modules fail to import at all), plus the rest of requirements.txt and pytest. Previously the memory said to borrow DecoupleRpy_Agent/.venv.

2026-07-28 — Routing leg VERIFIED end-to-end; dispatch blocked by a private-Space 404

The one thing the orchestrator registration task could not check headlessly — does the router actually classify a DNA question and dispatch here — is now verified in a signed-in browser session against the orchestrator dev Space (5cbf1d9).

Asked: "Which samples in the TCGA pancreatic cohort carry a KRAS G12D mutation, and what is the KRAS alteration frequency?"

Classification: PASS. The orchestrator answered "Routing to PDAC Genomics Agent for computation." — it recognised a DNA question and picked this specialist over the subtype and DE agents. That is the registration working, including the deliberate choice to drop bare KRAS from the trigger keywords (the hotspot allele G12D carried the intent instead).

Dispatch: FAIL, and the error names its own cause. The call came back:

The specialist tool server could not be reached… 404 Client Error. Repository Not Found for url: https://huggingface.co/api/spaces/anne-voigt/pdac-genomics-agent. If you are trying to access a private or gated repo, make sure you are authenticated…

This Space is PRIVATE, so without credentials the orchestrator cannot even resolve the repo, let alone reach the gate. The failure is a repo-lookup 404, not an allow-list denial — consistent with the caller-side root cause already found: the orchestrator attaches its token only when HF_TOKEN is set, and that secret is unset on its Space.

Nothing to change here. The fix is one secret on the orchestrator's Space, and it is a credential operation for a human. Worth noting the order of failure for whoever does it: with HF_TOKEN unset the symptom is a 404 (invisible private repo); once set, the next thing exercised is the allow-list — and the identity it resolves to must be on ALLOWED_IDS here (anne-voigt,cpelz741 today).

2026-07-28 — Repaired `main`: conflict markers were committed and deployed

main carried literal <<<<<<< HEAD / ======= / >>>>>>> markers in memory.md, and they went out to the live Space. Introduced by b676112, which was a cherry-pick of the hermetic-suite commit (dbaeaa3) that hit an append-only collision in memory.md, was committed unresolved, and was pushed. Two further things about that commit are worth knowing, because both make it hard to find again:

Its message describes different work. It reads "verify: routing leg passes; dispatch blocked by private-Space 404", but its diff is the hermetic suite — tests/conftest.py (+109), tests/test_hermetic.py (+69), plus TODO/memory. Searching the log for the socket guard will not find it.
The code half landed fine. The socket guard and test_hermetic.py are correct and complete on main; only the prose merge was broken. So this was cosmetically invisible — tests pass with conflict markers sitting in memory.md, because nothing tests prose.

Repair: kept both colliding entries (machine-endpoint argument errors, and hermetic enforcement) in chronological order — an append-only log has no genuine conflict, only an ordering question.

Also corrected a statement main was making about itself. That cherry-pick carried over a TODO item reading "This branch's RESERVED.md is stale — it stops at 0005". True of the branch it was written on; false on main, whose ledger correctly shows 0006 + 0007 published and next free 0008. Rewritten to keep the lesson (check git show main:docs/adr/RESERVED.md, never your working copy — the per-lane-copy failure means the shared-row conflict cannot fire) without the false claim.

Deliberately NOT carried over from that branch's follow-up commit (52bb514): its narrative records a rebase onto main, but main received a merge (80b5e01) instead, and it flags a gap — pancreas_cptac_gdc / pdac_msk_2024 missing source_import_date — that does not exist here. All six artifacts on main carry it (verified), because the merge re-curated all six rather than trusting git's textual merge of the JSON. Importing that entry would have written a history that did not happen and a backlog item already closed.

The generalizable bit: prose is the only part of this repo nothing verifies. Code has tests, artifacts have --check, licences have the review table — memory.md and TODO.md have nothing, so a broken merge in them survives a fully green suite and ships. Worth a lint step (grep the tree for conflict markers) if it happens twice.

2026-07-28 — Routing re-tested with `HF_TOKEN` set: still 404. It is a VISIBILITY problem.

Re-ran the signed-in dev-orchestrator test after HF_TOKEN was set and the Space restarted (confirmed RUNNING, fresh container start). Routing passed again — "Routing to PDAC Genomics Agent for computation" — and dispatch failed with the same repo-lookup 404.

The token is set. router.py prints a loud startup warning when HF_TOKEN is unset; the container log for the 23:47:29 start shows no such warning. So the earlier "just set HF_TOKEN" theory is now discharged and the remaining cause is narrower: the orchestrator has a token, but it cannot see anne-voigt/pdac-genomics-agent. HF returns 404 (not 403) for a private repo the caller cannot view, which is exactly the observed error.

The decisive context — this Space is the only private specialist:

Space	private?
`pdac-subtype-agent-prod`	No
`Paper2Agent_decoupleRpy`	No
`pdac-genomics-agent`	YES

The two specialists the orchestrator already calls successfully are public, and they are protected by the ADR-0004 allow-list gate rather than by repo privacy. We are the outlier.

Two ways forward, and the choice is architectural, not a bug fix:

Grant the token repo-read scope on this Space (fine-grained tokens need explicit read on the namespace). Keeps the Space private; keeps us different from the siblings; every future caller hits the same wall.
Make this Space public and rely on the enforced gate — the family's actual convention. ACCESS_CONTROL=enforce + ALLOWED_IDS is exactly the protection pdac-subtype-agent-prod runs on, and ours is already enforced and verified in both directions. The redistribution question this would raise was already handled: ATTRIBUTION.md and the full-text CCLE_BROAD_TERMS.md ship beside the artifacts precisely because they are downloadable.

Not doing either unilaterally: option 2 publishes a repository, which is the owner's call.

2026-07-28 — GitHub mirror live (first CI ever); scope gate closed

Two long-standing items closed, both of which had been waiting on a human.

1. GitHub mirror — Anne-Voigt/pdac-genomics-agent, PRIVATE. main pushed; curation-freshness is active and was proven with a manual workflow_dispatch run rather than left to fire on Monday and be discovered broken. This repo had never had CI of any kind — unlike every sibling it had no github remote at all, because origin IS the HF Space and HuggingFace does not run GitHub Actions. That is why ADR-0007's weekly check shipped inert.

PRIVATE was a deliberate call, not a default, and it should not be flipped casually. The Space is private on purpose, and src/resources/curated/ redistributes two cohorts with real restrictions: pdac_msk_2024 under CC BY-NC-ND (the ND term over a derived status matrix is explicitly unresolved — see licenses.COMMERCIAL_USE_REVIEW) and ccle_broad_2019 under Broad's terms (commercial use restricted, plus a continuity clause binding onward users). A public mirror publishes both, to a second host, under terms nobody has cleared. Resolve ND before going public.

Remotes now mean two different things, and confusing them deploys something: origin → the live HF Space (push to main = DEPLOY). github → the mirror (push = CI, no deploy). Push both; they are not interchangeable.

2. The "should this exist at all" gate is closed. Per-sample variant status is confirmed as the near-term need. The design doc made this a pre-M1 confirm; M1–M3 were built without it on Carl's "build now, ship later", and it had quietly become a pre-deploy confirm that the deploy went ahead of. Recorded with its provenance rather than as a bare tick: this is Anne's call as the consultant, not a direct statement from Dr. Sears or Carl Pelz, and the original item asked for theirs. Enough to proceed; NOT the lab having reviewed the tool. A session with Carl is still worth having before anyone leans on this for real analysis.

Still blocked on a human, and it is now the only thing between the orchestrator and this agent: HF_TOKEN must be set in the orchestrator's prod Space secrets. Not doable from here — handling an API token is out of scope for an assistant, and it is the one action in this whole sequence that genuinely requires the person who owns the credential. This agent's gate is verified working from both sides; the orchestrator simply sends no header when that secret is unset.

2026-07-28 — Space made PUBLIC; the orchestrator boundary is fixed and proven

Flipped anne-voigt/pdac-genomics-agent to public, matching the two sibling specialists the orchestrator already calls (pdac-subtype-agent-prod, Paper2Agent_decoupleRpy). The protection model is now the family's: repo visibility is not the boundary — the ADR-0004 gate is.

Scanned before publishing, and it caught something. docs/cbioportal-terms-and-deployability.md carried a literal personal gmail address in an example User-Agent. The implementation had deliberately never hardcoded a contact (the +URL is the channel, with an optional CBIOPORTAL_CONTACT env var); the doc had not caught up. Removed before the flip — publishing would have made it world-readable and scrapeable. Also verified: no .env, no token-shaped strings, no tracked run_logs, audit records go to a separate dataset repo.

Verified after the flip:

check	result
anonymous `/panel` (no credential at all)	denied — `machine_auth: no_token_header`
tokenless client can resolve the Space	yes (this is exactly what 404'd while private)
orchestrator's pattern: resolvable repo + `x-orchestrator-token`	ok — real data returned

ALLOWED_IDS remains a secret, not a variable — on a public Space, variables are world-readable, so the sibling's ALLOWED_IDS (a variable) currently exposes its collaborator list. Keep ours a secret; worth fixing there.

A correctness signal worth noting: the live answer is now KRAS 136/150 = 91%, not the 136/186 = 73% seen earlier. The difference is the denominator fix that landed from another lane — frequencies are now over the profiled subset rather than the whole cohort. 91% is what the PDAC literature reports for KRAS; 73% was an artifact of counting unsequenced samples as wild-type.

Still unverified: the orchestrator's UI end-to-end. Its chat panel returned no rendered answer within ~20s across two attempts, and its Space restarted mid-test (fresh container starts at 23:47:29 and 23:53:05), which wipes conversation state. Routing itself was already proven twice before the flip ("Routing to PDAC Genomics Agent for computation"), and the dispatch leg is now proven at the API level — so the gap is orchestrator-side rendering/session, not this agent.

2026-07-28 — Pre-push hook: the mirror now follows the deploy

scripts/hooks/pre-push + make install-hooks (this repo's first Makefile; the pattern copies DecoupleRpy_Agent/scripts/hooks/pre-push). Pushing main to origin — i.e. deploying — now mirrors that same commit to github first, so CI can never run against less than what is live.

Why "first" and not "after": git has no post-push hook. So a failed deploy can leave the mirror briefly AHEAD. That is deliberately the error we accept: CI running a superset of production self-corrects on the next deploy, whereas CI running a subset — the stale-mirror case — reads as a pass. Picking which way to be wrong is the whole design decision here.

A mirror failure warns and never blocks. The mirror is CI; origin is production. Refusing to ship because a secondary remote was unreachable is disproportionate, so every path exits 0 and the warning names the one command that fixes it (a test asserts exit 1 appears nowhere in the hook).

Verified by driving the hook's stdin protocol directly rather than by pushing six times: non-deploy remote, non-main branch, branch delete (all-zero local sha — must never mirror a deletion), missing mirror remote, a real in-sync mirror push, and a deliberately broken remote to see the warn-don't-block path. All six correct.

Worktree gotcha worth keeping: the installer uses git rev-parse --git-path hooks, not a literal .git/hooks. Every lane in this repo works from a linked worktree where .git is a file, so the naive path installs nothing and fails silently. Checked empirically that git honours the resolved path from inside a worktree — and that it resolves to the SHARED .git/hooks, so one install covers every lane. Pinned by test_makefile_installs_the_hook_where_git_looks.

Still needs one manual step per clone: hooks are not versioned by git, so make install-hooks has to be run once. A lane that skips it is back to remembering git push github main.

Also corrected CLAUDE.md: the Space went public in 0ffadd1, so the two-remote table's "(private)" was already wrong. The mirror stays private.

169 tests.

2026-07-28 — The orchestrator → this agent path is LIVE and proven in production

The three-tier architecture stopped being a design and started answering questions. Verified in the prod orchestrator UI, signed in as anne-voigt:

"Which pancreatic cell lines carry a KRAS mutation?" → "Routing to PDAC Genomics Agent for computation" → 57 pancreatic lines (lineage filter), 48/53 mutation-profiled (90.6%) KRAS-hotspot (G12D dominant; MIAPACA2 = G12C; Q61H/Q61R), 13/44 CNV-profiled (29.5%) also amplified, CCLE commercial-use restriction + Ghandi 2019 / cBioPortal citations in the answer.

Two things worth noting in that output. The two denominators (53 vs 44) are the per-study modality-coverage gate working — it did not pretend 57 everywhere. And BxPC-3 is correctly absent from the mutant list: the canonical KRAS-wild-type control line, so the biology is right, not just the plumbing.

The blocker was never auth — and there were TWO blockers wearing one symptom. Four theories died on the way here: "transient whoami" (wrong), "Bearer None broken probe" (right about that probe, not about why prod failed), "HF_TOKEN unset on prod" (wrong — it was set all along; the prod orchestrator's startup block carries no unset-token warning), and by then the dev-side failure had already been correctly reframed by another lane as visibility, fixed by making this Space public (0ffadd1) — see that entry above.

What remained, and what this entry adds: agents.yaml as deployed on the orchestrator's PROD (f32afbd, 07-24) contained no pdac-genomics-agent at all. The registration had only ever reached its dev. A router cannot dispatch to an agent it does not know exists, so prod answered from the Claude API and recommended DepMap/COSMIC/cBioPortal — while this agent, built on cBioPortal, sat one dispatch away. Fixed by promoting dev → prod (11452f7, 6 commits, +1381).

So the two Spaces were failing for two unrelated reasons at once — dev couldn't see this Space, prod didn't know about it — which is exactly why single-cause theories kept half-fitting the evidence and dying.

Method note, again the same lesson as the machine-auth entry above: the theories were all built from code and logs; the answer came from driving the actual UI end-to-end. When a cross-Space path fails, exercise the real path before theorising about which layer broke — and check the deployed config, not the branch you happen to have checked out.

Bonus, relevant to ADR-0006: the orchestrator's half of the join now exists (cross_modality_join.py + its own ADR-0003), and it carries an agreement test that imports this repo's contingency_and_test and asserts identical output — which is exactly the "two implementations must agree or the same cohort answers differently by route" risk ADR-0006's Consequences flagged. That risk is now pinned by a test rather than by prose. ⚠️ Its success path is still fixture-only: no cohort can actually run the live-PurIST join yet (id-alignment failures on every pair tried so far), so treat the first real cohort as a first run, not a regression check.

2026-07-28 — Mouse/KPC spike: the refusal STANDS (no code changed)

Discovery spike, no implementation: docs/spikes/SPIKE-mouse-kpc-variant-path.md. Verdict FAILED as a feature. byod.py's non_human_species gate and CLAUDE.md's mouse hard gate are unchanged and should stay unchanged. RESERVED.md's "Mouse/species variant boundary" entry stays unclaimed — this spike is the evidence for not writing that ADR.

Four independent reasons, each sufficient:

No data. Probed the public cBioPortal: 538 studies, refGenome ∈ {hg19×466, hg38×72}, zero mouse builds; the two keyword hits are human (one is a PDX study — human tumour in a mouse, i.e. evidence against mouse coverage). The gene namespace rejects MGI symbols (Trp53/Kras/Cdkn2a/Smad4 → Gene not found), so curate_from_cbioportal would return an empty matrix. A registered mouse path = adopting a whole second source, not a new study id.
The flagship question is tautological. KPC is engineered (LSL-Kras^G12D; Trp53^R172H; Pdx1-Cre), so "KRAS status in my KPC mice" restates the breeding record — and our response shape (per-sample status + cohort frequency) is structurally indistinguishable from a genuine somatic finding. 100% KRAS-hotspot would be a fact about the colony, not the disease.
The hotspot rules silently corrupt. Human TP53 (393 aa) vs mouse Trp53 (390 aa) is a clean −3 offset: our HOTSPOTS["TP53"] 175/248/273/282 → mouse 172/245/270/279. So the canonical KPC allele Trp53 R172H classifies as missense, not hotspot, and a mouse variant at 175 would be promoted spuriously. KRAS is the trap — codons 12/13/61 ARE conserved (identical first 90 aa), which generalises to the wrong conclusion.
CNV has no cross-species mapping at all — GISTIC calls are assembly-specific, and half the panel's signal (CDKN2A deep-del, MYC amp) is CNV. CDKN2A also breaks orthology at the protein level (ARF: human p14 132 aa vs mouse p19 169 aa).

ADR-0017 is not precedent for this — reading it that way inverts its argument. It permitted the analyses with no fitted human-specific object (GSVA) and gated the one that had (PurIST). Our hotspot table is exactly a fitted human object (human residue positions), so it lands in the PurIST column — and differs only in being worse: PurIST on mouse is unvalidated, the TP53 rule is arithmetically wrong. ADR-0017's 1:1 gene-level ortholog table also cannot help: expression needs gene-level mapping, variants need residue-level equivalence per isoform.

The real need reframed: not per-sample mouse status but "how does my engineered model compare to the human cohorts?" — mostly an expression question already owned by pdac-subtype-agent. Spike proposes one small alternative for Carl's review, deliberately not built: compare_model_genotype_to_human_cohort, which takes the declared genotype (not a MAF), maps ~5 KPC alleles to human equivalents via a tiny hand-curated table, and reports the human cohort frequency from artifacts we already have. No mouse data, no new source, no ADR needed. Default outcome — build nothing — remains fully defensible.

2026-08-01 — Branch reconciliation: seven lanes collapsed into `main`

A week of parallel worktree lanes had left 7 local branches + 5 worktrees, several of them rebase copies of each other, and it was no longer obvious which carried work main lacked. Audited all of them; the answer was that main was already a superset of everything except one commit.

The only unmerged work: claude/infallible-payne-2134a8 (e364948) — the mouse/KPC spike. Cherry-picked onto main as 84001c2. Its docs/spikes/SPIKE-mouse-kpc-variant-path.md and the RESERVED.md annotation applied cleanly; TODO.md and memory.md conflicted because the branch forked back at 085b9f7 and both files have been rewritten since. Resolution kept main's newer text in both and grafted in only the genuinely new spike entries. Two blocks from the branch were deliberately dropped as superseded, not lost:

its scope-confirm entry said "confirmed with Rosie/Carl"; main's version records the same gate more precisely (it was Anne's call as consultant, explicitly not the lab having reviewed the tool). Over-reading that is the exact error main's wording guards against.
its "three operational blockers" list, plus the correction underneath saying they were already closed. main records the closed state directly (ALLOWED_IDS set, gate verified both ways, orchestrator reaching the machine API), so re-importing the stale-then-corrected pair would have put a contradiction back into the file. The durable lesson from it — _resolve_token_identity() doing an outbound whoami per request under a broad except — is already recorded on main.

claude/suspicious-cannon-b68c4b looked like unmerged work and was not. All three of its commits (75746cf cohorts/denominators, dbaeaa3 socket guard, 52bb514 memory) are content that reached main by another route — git cherry flagged them + only because the patch-ids differ after rebasing. Verified file-by-file rather than by subject line: tests/conftest.py is identical and main's test_hermetic.py is strictly larger (it has the conflict-marker test the branch lacks). Same story for origin/claude/infallible-kowalevski-2d4648 — its three "unique" gradio_ui.py lines are pre-denominator-fix code that main superseded.

The method worth reusing: git cherry / branch names / commit subjects all lie after a rebase. What actually settled it was diff <(git ls-tree -r main) <(git ls-tree -r <branch>) for unique files, then a per-file diff of the suspects. Only one branch of seven had a unique file.

Deleted: 7 branches, 5 worktrees, and the 2 redundant origin/claude/* remote branches. Every deleted tip is preserved as an annotated archive/<branch> tag, matching the convention the orchestrator and pdac-subtype-agent lanes used the same day — pushed to the github mirror (the private remote; deliberately not to origin, which is the public Space). Each tag's message carries that branch's audit verdict, so the reasoning survives with the object instead of only in this file:

tag	sha	verdict
`archive/claude/busy-herschel-2bc3c9`	`f91fafb`	identical to `main`
`archive/claude/dazzling-moser-5d97a7`	`6a17ae2`	ancestor of `main`
`archive/claude/gallant-ellis-5ca74e`	`64d7f04`	merged via `80b5e01`
`archive/claude/infallible-kowalevski-2d4648`	`09cba72`	ancestor of `main`
`archive/claude/infallible-payne-2134a8`	`e364948`	⚠️ unreachable — the one real branch; cherry-picked as `84001c2`
`archive/claude/suspicious-cannon-b68c4b`	`52bb514`	⚠️ unreachable — no unique content (the rebase-lookalike case)
`archive/pick/hermetic`	`a90b9fc`	ancestor of `main`

Only the two marked unreachable actually needed a tag — the other five are ancestors of main and were never at risk. Verified rather than assumed: after git reflog expire --expire-unreachable=now --all && git gc --prune=now, both e364948 and 52bb514 still resolve. Recorded SHAs in prose would not have survived that; this is the difference between the tag habit and the note habit, and it is why the tag habit is the right one.

The infallible-payne tag additionally records the two blocks of e364948 that were deliberately not carried onto main (the over-read scope-confirm; the stale-then-corrected blockers list), so a later reader who diffs the tag against main and sees content "missing" finds the reason attached to the object rather than concluding it was lost.

174 tests pass after the merge, including the conflict-marker guard — the test that exists precisely because a botched prose merge once shipped to the Space.

Deployed: f91fafb..2ca2107 to prod; build watched through RUNNING_BUILDING → RUNNING_APP_STARTING → RUNNING, direct host HTTP 200. The pre-push hook mirrored to github first, so CI tests the deployed tree. All three refs at 2ca2107, one branch.

⚠️ Still unverified: the gate was not re-exercised (needs the x-orchestrator-token credential) and no signed-in click-through was done. This deploy is docs/tests only over 894e710, so served behaviour should be unchanged — but RUNNING + HTTP 200 proves the app boots and serves, not that it answers correctly.

2026-08-03 — the two remaining small code items, closed

Both were long-tail "Open — payload contract" / mouse-spike follow-ups; neither changes behaviour for any deployed caller.

1. query_variant_status([]) now expands to the full panel (src/tools/query_variant_status.py). The machine endpoint's blank genes already did this via _split_genes in gradio_ui.py; calling the Python function directly with [] returned an empty genes map, which reads as "nothing altered" rather than "you asked about nothing" — the same class of confident-false-negative the coverage gates exist to prevent, just aimed at a direct caller instead of an orchestrator. The two entry points now agree. Test asserts [] and load_panel() return the identical genes map.

2. The BYOD non_human_species refusal now names its reason (src/workflows/byod.py). It said only "not supported", which reads as an arbitrary gate. It now states the actual failure mode: classification is keyed on human residue coordinates, so mouse Trp53 R172H — the ortholog of human R175H — would score as generic missense against our human hotspot table, and CNV has no cross-species mapping at all. Points at the mouse expression path in pdac-subtype-agent and at SPIKE-mouse-kpc-variant-path.md. String change only; the refusal itself is unchanged and still STANDS per that spike.

176 tests pass (was 174) — two new, one per item.

2026-08-05 — six backlog decisions taken, and the panel widened 19 → 29

A backlog-triage session with Anne. Five open items closed by decision rather than by code, one reopened with a real shape, and the one remaining code item — the panel widening — built.

The decisions

ND on pdac_msk_2024 — RESOLVED, and it unblocks the mirror. The CC BY-NC-ND licence attaches to the article and its supplementary material; the authors deposited this cohort separately as publicly accessible and downloadable data, so the deposition is the instrument governing the data and a derived status matrix is not an ND-restricted adaptation. NC is untouched — commercial use still barred, verdict stays restricted. Recorded as a reasoned consultant position, explicitly NOT an author grant. This removes one of the two blockers on making the GitHub mirror public; ccle_broad_2019's Broad terms are the other and have not moved.

ICGC open tier — no_restriction_found → permitted. commercial_use_permitted() now returns True for paad_qcmg_uq_2016. E.4 survives the upgrade and still binds commercially (no IP claims over primary somatic data; downstream licensing must not obstruct further research) — a test pins that clause to the entry, because a grant is not the absence of obligations. ⚠️ The granting clause itself was established off-repo and is uncited in the entry; that gap is recorded in the entry's own confidence field, not only in TODO.md, since a caller can now rely on the verdict.

whoami stays. The open item asked whether the machine gate should drop its outbound whoami for a shared secret. It should not, and the reasoning inverts the item: a shared secret is one credential shared by all callers, so every machine call would audit as the same undifferentiated principal. principal_trace_fields records {principal, role} per run precisely so that field names a person. The coupling buys per-user attribution — unlike ADR-0005 C4's, which bought nothing the artifact didn't already give us. Successor item is narrower and real: the resolution cache's TTL and what happens on an HF outage mid-session (fail closed vs. serve on a warm cache).

Mouse compare_model_genotype_to_human_cohort — DECLINED. Not deferred, not blocked on Carl. The refusal in byod.py is the whole answer; the species-boundary ADR stays unclaimed.

Stub-vs-restructure ADR — CLOSED, not contested. An ADR for an uncontested three-line test-only decision is ceremony. The gradio-upgrade watch item carries the residual risk.

biodata-registry — REOPENED, with the actual obstacle named. Anne wants the registry's structure reused, and the objection was never philosophical: the registry does not model DNA. VALID_MODALITIES is expression-only, and its value-add (data_level → path A/B/P, get_integration_plan) has no meaning for a status matrix. What transfers is the pattern, which this repo already reimplemented locally as curated/*.json + COMMERCIAL_USE_REVIEW. The proposal is to extend the registry with a variant modality and move cohort semantics there — a cross-repo change (new manifest kind, version bump, re-pin everywhere) needing ADR-0008 and its own milestone.

The panel: 19 → 29 genes, all seven artifacts re-curated

Added HR-beyond-BRCA (RAD51C, RAD51D, BRIP1, BARD1, CHEK2, FANCC — PARP-inhibitor eligibility) and MMR (MLH1, MSH2, MSH6, PMS2 — MSI-high / pembrolizumab). Both tiers are answerable from mutation+CNV alone.

The fusion-driven actionable genes were deliberately left off — NRG1, NTRK1/2/3, ALK, ROS1, RET. In PDAC these are structural-variant events, so a mutation+CNV status matrix would report ~0% for exactly the alterations that matter, on the most clinically actionable question this agent could be asked. They stay off the panel until a structural-variant modality exists (now a backlog item with its own coverage-gate requirement).

New-gene frequencies are literature-plausible (0–2%; MSI-H PDAC is ~1%), and KRAS is unchanged at 90.7% TCGA / 93.7% MSK, so the profiled-denominator work did not regress. MSK-IMPACT assays all ten new genes, so those MSK numbers are real rather than gated false negatives; its three unassayed genes are still ELAVL1/GATA6/ACVR1B.

The bug the widening exposed

Widening the panel is not config-only — every curated artifact bakes in the panel_genes it was curated against, and query_variant_status filtered requested genes down to what the artifact carried. A panel gene absent from a not-yet-re-curated artifact therefore vanished from the payload with no note, and a caller reads a short genes map as "we looked and found nothing". Same confident false negative the assayed: False gate exists to prevent, reached by another door, and it would have fired on any future widening too.

Such a gene now returns {curated: False, note: ...} and is named in uncurated_genes. Kept distinct from assayed: False on purpose: that one means the cohort's assay never interrogated the gene (a fact about the data, unfixable here), this one means we have not curated it yet (a fact about our snapshot, fixed by re-running python -m src.curate <study>). Collapsing them would tell a user their cohort cannot answer a question a re-curation answers fine.

Two tests: the gap is stated rather than dropped, and every committed artifact carries the current panel — the second fails loudly if a future widening skips the re-curation pass.

Test-suite notes

179 pass (was 176). Three existing tests needed changing, and two of them were pinning the wrong thing: test_panel_endpoint_returns_the_v1_panel hardcoded n_genes == 19 and now asserts the endpoint agrees with load_panel(), and test_no_restriction_found_is_not_reported_as_permitted pinned paad_qcmg_uq_2016 and broke when that study's verdict legitimately changed — it now tests the function's semantics across all verdicts, which is the rule it was always guarding. The scaffold smoke test's len(genes) == 29 is pinned deliberately, so a panel edit that skips re-curation cannot drift in quietly.

Not deployed at the time this entry was written — code, artifacts, licences and docs only. Superseded the same day: this shipped as 7ae3398 — see the next entry.

2026-08-05 (later) — DEPLOYED the panel widening, then built the structural-variant modality

Two deploys today. The first shipped the morning's 19 → 29 widening; the second is the SV work.

Deploy 1 — the panel widening went live (`7ae3398`)

35c0da0..7ae3398 to origin, hook mirrored to github first, all three refs aligned. Build watched RUNNING_BUILDING → RUNNING_APP_STARTING → RUNNING at 7ae3398, root HTTP 200. Security scan clean (the three bandit findings — B104 bind-all on the MCP server, B311 jitter, B310 urlopen — are pre-existing and in files this change did not touch).

Verification note worth keeping: the machine endpoint correctly returned status: denied to an unauthenticated gradio_client call, which is the enforced gate working, not a deploy failure. Since a signed-in click-through needs a browser, the deployed TREE was verified instead by fetching files from the Space over resolve/main: panel 29 genes, all seven artifacts panel_genes=29 / schema v3 / curated 2026-08-05, and uncurated_genes present in the deployed tool. A signed-in click-through is still owed — tree verification proves what is served, not that it answers correctly.

Deploy 2 — the structural-variant modality (ADR-0008)

Third modality alongside mutation/CNV. Statuses none / fusion_in_frame / fusion_out_of_frame / intragenic / rearrangement, somatic only, schema v3→v4, panel 29 → 36 with the fusion tier finally added. ADR-0008 was claimed on main alone and pushed before the ADR was written, per the ledger protocol.

Probing cBioPortal before designing was the whole game. Three things the probe found, none of which were guessable:

Only 2 of 7 cohorts publish SV at all — pdac_msk_2024 and ccle_broad_2019. The other five name sv in unavailable_modalities and emit no sv block. The per-study gate now fires far more often than it ever did for CNV.

The two SV cohorts differ in KIND, not just volume. MSK states frame ("Protein Fusion: in frame {ETV6:NTRK3}"); CCLE has variantClass: NA on all 153 panel-gene rows and labels events only as free text ("EML4-ALK fusion"). So classification reads eventInfo, never variantClass — the latter describes the genomic event, not its protein consequence, and a classifier keyed on it would have promoted every translocation to a "fusion". And a per-study sv_annotation flag (characterized / uncharacterized) collapses every event in an uncharacterized cohort to rearrangement, including the odd row that mentions a frame, so a frequency is never a mixture of "genuinely in-frame" and "we could not tell". Those answers carry a caveat saying an absence of in-frame fusions there is a fact about the annotation, not the biology. Without it, someone comparing MSK to CCLE would read "0 in-frame fusions in cell lines" as a finding.

The per-gene gate was wrong for SV — a real bug, surfaced by building the modality. genes_assayed derives from the DNA gene panel (IMPACT341/468/505…), so it answers "was this sequenced for mutations and copy number?" and nothing else. Fusion calling is a separate assay with a different gene set: MSK reports 6 somatic NRG1 rearrangements while NRG1 is absent from the DNA-panel intersection. The old gate suppressed all six. That is the mirror image of the failure the gate was built for — refusing an answer we hold, rather than inventing one we don't — and both are dishonest. The gate is now modality-scoped: off the DNA panel with observed SV events ⇒ assayed: false and a valid sv block plus a note forbidding the reader from taking the SV frequency as a mutation frequency; off the panel with no events ⇒ still fully refused. Evidence overrides the gate toward reporting, never toward a zero.

Also: sv_provenance is a separate dict from provenance. Same (gene, sample) key, and TP53/CDKN2A each carry a mutation and a fusion in the same sample, so sharing one would have let a rearrangement overwrite a KRAS G12D.

ADR-number near-miss, checked and cleared (the fifth in this family). audit.py, logging_sink.py and tests/test_audit.py all said "ADR-0008 parity" before this repo had an 0008 — that is the SIBLINGS' numbering (pdac-subtype-agent and DecoupleRpy_Agent each numbered their own audit-logging ADR 0008). ADR numbers are per-repo; only genuine cross-repo standards burn a number family-wide, and RESERVED.md lists those explicitly (0014, 0015). So 0008 was free here. The three references were reworded to name the sibling repo, since "ADR-0008 parity" now points at the wrong decision. Recorded in the ledger with the lesson: never cite a bare ADR-#### for a sibling's decision.

Frequencies: NRG1 0.26%, NTRK3 0.21%, ROS1 0.21%, NTRK1 0.17%, RET 0.09%, ALK 0.04% in MSK. KRAS SV is a real measured 0% (on the DNA panel, so genuinely interrogated) against its 93.7% mutation rate — a good illustration that the modalities are independent.

204 tests pass (was 179): 25 new in tests/test_structural_variants.py, most of them about what the modality declines to say. Four existing tests needed shape updates (modalities dict gains sv, MSK's unassayed set gains NRG1, the denominator map gains sv, panel count 29 → 36); the UI's _frequency_frame needed no change — its existing "frequency" in block guard already skips the new mixed assayed/note entries and picks up sv automatically.

Still open on this modality: variant_by_subtype does not accept modality="sv" (per-gene counts are 1–6 in 2,336, so nearly every cell would be testable: false — decide if that is worth exposing), and BYOD has no SV path (an uploaded fusion table cannot answer the annotation-depth question, so it should refuse explicitly rather than ignore an SV column).

2026-08-05 (third pass) — the two SV follow-ups, which turned out to be two more live bugs

Closing the two items left open by the SV modality. Both were framed as "missing features"; both were actually silent wrong answers already in production. Same defect class as everything else today: a fallback where there should have been a refusal.

`variant_by_subtype` answered a different question than it was asked

The modality branch was if modality == "cnv": … else: mutation. So modality="sv" produced a MUTATION association — and so did "banana", and so did a typo like "mutaton" — returning join_available: true with real p-values. The payload did say "modality": "mutation", so a scrupulous caller could have caught it, but the orchestrator asked about fusions and received a confident answer about point mutations.

The modality set is now closed (mutation / cnv / sv), and an unrecognized value returns route: "unsupported_modality" with supported_modalities. Blank and whitespace-only still mean "unspecified" and default to mutation — the machine endpoint sends blank for an untouched field, so refusing that would break the most ordinary call there is. (A test caught the whitespace case: (" " or "mutation") is truthy, so defaulting before stripping left "" and refused. Strip first, then default.)

Fixing the validation is what made not wiring SV a safe choice rather than a silent one — worth recording, because "leave it unwired for now" looked conservative and was the opposite.

The sparse-table floor, and why it applies to everything

MIN_ALTERED_FOR_TEST = 5. Below it a gene reports testable: false with its contingency counts instead of a p-value. Fisher's exact returns a number from a 2×2 containing a 2, and an odds ratio of ∞ from a single-sample cell reads as a strong finding — arithmetically correct, scientifically empty. SV made this unmissable (1–6 events per gene in 2,336 samples), but the floor is applied to every modality: a mutation association resting on 3 altered samples is exactly as empty, and a rule that fired for one modality and not another would be a distinction the reader cannot see. Nothing is hidden — the counts still ship, only the inference is withheld.

This changed one existing test, and the change is the lesson in miniature: test_variant_by_subtype_joins_existing_label asserted that fisher_exact ran on a five-sample fixture. That assertion was the bug written down as an expectation.

An uploaded MAF reported fusions as missense mutations

classify_mutation has no fusion branch, so Variant_Classification: Fusion is neither silent nor truncating and fell through to missense. Latent for months and harmless while the panel held no fusion genes — it became serious this morning, when NRG1/NTRK1-3/ALK/ROS1/RET joined, because an upload would then report precisely the actionable fusion genes as point mutations.

Now a BYODRefusal("sv_not_supported") naming why: a fusion status is only meaningful alongside an annotation-depth claim (in frame or not) that an uploaded row cannot make (ADR-0008), and the alternative is the missense answer above. It points at the registered cohorts that can answer.

The registered path was checked and is clean — cBioPortal keeps fusions in the SV profile, and no curated artifact carries a fusion-like string in its mutation provenance. Blast radius was BYOD only. Also added "sv": False to the BYOD modalities dict: the behaviour was already "no SV", but unavailable_modalities is built from that dict, so a BYOD answer never named structural variants as unavailable. Silence about a modality is what this whole family of gates exists to prevent.

219 tests pass (was 204).

Pattern worth naming, since it has now happened four times in one day: every one of these bugs was a default that fired where a refusal belonged — absent gene dropped from the payload, DNA-panel gate applied to fusions, unknown modality falling back to mutation, fusion row falling through to missense. None of them raised. All of them produced a well-formatted, citation-stamped answer to a question nobody asked. When reviewing this codebase, grep for else: on a branch that picks a modality, a gene set, or a status — that is where they live.

2026-08-05 — dev lane established (local half); mirror hook fixed

32b56aa on the new dev branch. This was the only showcase agent repo without a dev Space — main IS the public prod Space, so every deploy shipped straight to production with no staging.

What landed (all local, nothing on HuggingFace):

hf-dev remote → anne-voigt/pdac-genomics-agent-dev. The Space does not exist yet — that is Annie's call. Until it does, hf-dev resolves to nothing and git fetch --all warns on it.
dev cut from main, tracking hf-dev/main; main tracking origin/main. Both branches previously tracked nothing, which is why unpushed work here was invisible to git status and the generated fleet table showed (no upstream) instead of an ahead-count.
scripts/hooks/pre-push bug, found by adding the remote and fixed in the same commit. The hook selected deploy pushes with origin*|*huggingface.co* and then acted on anything headed for refs/heads/main. git push hf-dev dev:main satisfies BOTH — so the first dev-Space push would have mirrored unreleased dev code onto the GitHub mirror's main, where the ADR-0007 freshness job reads it as production. Exactly the wrong-CI failure the hook exists to prevent, inverted. Now bails on hf-dev (and any *-dev URL) before the deploy test. All four paths exercised by piping synthetic stdin at the hook: hf-dev by name, hf-dev by URL, github, origin.
CLAUDE.md: three-remote table, branch-lifecycle section, and a correction — the memory-maintenance list still told agents to hand-edit SHOWCASE_STATUS.md, which is generated from status/ fragments now.

To activate: create the Space (private, gradio 6.18.0, py3.11, hardware matching prod), then copy ACCESS_CONTROL / ADMIN_IDS / ALLOWED_IDS / LOG_SINK_HF_DATASET (a separate dataset, so staging runs don't pollute anne-voigt/pdac-genomics-results) / LOG_SINK_HF_TOKEN / CBIOPORTAL_CONTACT. Repo is 588 KiB packed with no LFS payload, so the push is cheap.

Update, same day — the dev Space is LIVE. Created anne-voigt/pdac-genomics-agent-dev (private, gradio, cpu-basic matching prod's free tier), pushed dev:main, watched BUILDING → APP_STARTING → RUNNING at 9b5fe82. dev now tracks hf-dev/main for real. build-status.sh's REPOS list needed the dev-ref flipped from - to hf-dev/main or the fleet row kept printing — for a Space that exists.

Two things the wiring taught, both worth keeping:

make install-hooks COPIES. .git/hooks/pre-push was still the stale pre-fix copy after the fix was committed. An edited hook is inert until reinstalled — always re-run the target.
The hook fix is verified against a real push, not just synthetic stdin: git push hf-dev dev:main emitted no mirror line and left github/main at 40ab3dd.

⚠️ The Space has no secrets set yet — gate dark, sink a no-op. Not a valid place to accept a gate change until that is done (TODO).

2026-08-12 — ADR-0009: the whoami cache's TTL and outage behaviour, decided

Closed the M3 narrowed-successor TODO. _WHOAMI_CACHE in gradio_ui.py was a bare dict — resolutions trusted until restart, and an HF outage served warm tokens forever while denying cold ones, all undecided-by-default. Now (ADR-0009): entries are (identity, monotonic-stamp) with positive TTL 15 min, negative TTL 60 s, and stale-if-error grace 24 h — on a transient whoami failure only, a previously-positive entry is served stale so an HF blip cannot take the orchestrator down. The deciding frame: whoami is attribution, not authorization — ALLOWED_IDS is local, fail-closed, and re-checked per call, so the worst the cache can do is let a revoked HF token keep attributing calls for the window. A definitive HfHubHTTPError is never rescued by staleness; a cold token stays denied in an outage; transient failures are never cached.

Mechanics worth remembering:

Ledger protocol followed: 0009 claimed on main alone (c4f9eb5), pushed to github only (origin push deploys prod and this session was told not to). The claim also fixed the stale "Next free number: 0008" line — 0008 was already published; next free is 0010.
HfHubHTTPError cannot be constructed bare — response= is a required kwarg now; the test builds it around a Mock. Sockets are blocked in tests, so the mock seam is huggingface_hub.HfApi plus a monkeypatched time.monotonic.
8 new tests in tests/test_whoami_cache_policy.py; full suite 245 green. NOT staged to the dev Space (it still has no secrets — gate dark there anyway) and NOT pushed to origin.

2026-08-13 — stray secrets removed from the prod Space (config only, no code)

Deleted `decoupler_results_dev_token` and `decoupler_results_token` from the prod Space's secrets at Annie's request — both were added there by mistake on 2026-08-13 while fixing the DecoupleRpy_Agent dev Space's log-sink token (that saga lives in that repo's memory.md); nothing in this repo ever read them. Deletion restarted prod (~90 s rebuild, back RUNNING). Post-restart health check: tokenless `/panel` → `denied / no_token_header` — serving, gate fail-closed, diagnosis correct. Remaining secrets: `ADMIN_IDS`, `ALLOWED_IDS`, `HF_TOKEN`.

2026-08-13 — prod audit sink wired to a durable HF dataset (settings work, no code)

The prod Space anne-voigt/pdac-genomics-agent had no LOG_SINK* settings (verified via HF API 2026-08-13), so logging_sink.py fell back to the local sink writing ./run_logs — ephemeral, lost on every restart of a free-tier Space with no persistent storage. The default dataset the HF sink points at did not exist either. Done this session:

Created the private dataset anne-voigt/pdac-genomics-results (the code's default repo id).
Set on the prod Space via API, as variables (auditable without secret access, matching the ACCESS_CONTROL choice): LOG_SINK=hf, LOG_SINK_HF_DATASET=anne-voigt/pdac-genomics-results. The variable change restarted the Space; watched it back through RUNNING_APP_STARTING.
Safe in the interim: HFDatasetLogSink no-ops without a token (checked the code before flipping LOG_SINK), so audit writes cannot error while the secret is missing — but records are DROPPED, not written locally, until the token lands. Strictly worse than local for retention until Anne acts, strictly better after.

Still needs Anne (Space settings UI, secret value): set LOG_SINK_HF_TOKEN (the research_agent_token value) on prod, then run one signed-in query and confirm a record file lands in the dataset. Note the sink also falls back to a research_agent_token env var if that name already exists as a secret on the Space.

Discrepancy noticed while in the settings API: prod's variables are only ACCESS_CONTROL

the two new LOG_SINK* ones — ADMIN_IDS/ALLOWED_IDS are evidently secrets on prod, while the dev-Space TODO says to set them "as variables, matching prod". The docs disagree with the Space; reconcile when configuring the dev Space (either flip prod's to variables or fix the TODO wording).

2026-08-14 — "dead button" diagnosed; prod token secret landed on the WRONG Space; secrets decision

Anne reported the prod UI's "Query variant status" button does nothing. Reproduced and root-caused by driving the public Space in a browser and speaking the gradio queue protocol directly:

The backend is healthy. The click DOES fire (queue/join → process_completed, success: true in ~1 ms) — but a signed-out session gets the ADR-0004 denial, and that payload updates ONLY the collapsed "Raw result (JSON)" accordion while hiding the caution box. So a signed-out (or session-expired) click is visually indistinguishable from a dead button — no toast, no chart, no visible change. The two prod restarts on 2026-08-13/14 dropped OAuth sessions, which is almost certainly why it "stopped working". Fix for the user: sign in again. UX finding worth a lane of its own: a denial should be visible without opening the accordion.
The intended prod LOG_SINK_HF_TOKEN was set on the DEV Space instead. Prod's /secrets API shows only ADMIN_IDS/ALLOWED_IDS/HF_TOKEN; dev gained LOG_SINK_HF_TOKEN 2026-08-13. Confirmed live: an authorized machine query (x-orchestrator-token, allow-listed caller) answered correctly (CCLE, 57 lines, n_profiled 53/44/41) and left NO record in the dataset and NO [logging_sink] persist_trace failed line in the run logs — the exact signature of the no-token no-op. Prod restarted 2026-08-14 to rule out stale env; still no-op. Remaining action is Anne's alone: add LOG_SINK_HF_TOKEN on PROD Space settings, then one authorized query and check runs/ in the dataset.
Decision (Anne): ADMIN_IDS/ALLOWED_IDS stay SECRETS. The dev-Space TODO's "as variables, matching prod" wording was the error; docs corrected, Spaces untouched.
The dev Space is now fully configured (gate + sink, separate pdac-genomics-results-dev dataset, private, exists) — the "not a valid place to accept a gate change" caveat is lifted.

Diagnostic trick worth keeping: with gradio 6 the UI click's fn is reachable without the UI — POST /gradio_api/queue/join with {fn_index, trigger_id, session_hash} + an EventSource on queue/data shows exactly what the server yields per frame; /gradio_api/call/<api_name> + x-orchestrator-token exercises the authorized machine path. Space secrets/variables are enumerable (keys only) at /api/spaces/<id>/secrets and /variables — no more guessing what a Space carries.

2026-08-14 — invisible-denial fix landed on dev (the "dead button" UX lane)

The lane the same-day diagnosis called for: an ADR-0004 denial on any of the three UI tabs now renders in the caution markdown box ("⛔ Access denied. …", visible), instead of only filling the collapsed "Raw result (JSON)" accordion while hiding the caution — which had made a signed-out/session-expired click visually indistinguishable from a dead button. New helper _denial_caution_md in gradio_ui.py; the gate itself is untouched (still first, still fail-closed). tests/test_ui_handler_gating.py now asserts denial visibility (_assert_denial_visible: caution visible: True + reason text) in both denial tests, so the regression can't come back silently. 237/237 tests pass. Merged to dev only; prod promotion is Annie's call.

2026-08-14 — invisible-denial fix PROMOTED TO PROD (`668841c..3853e93`)

Annie's call, executed. Gates first: 245 tests pass, gitleaks clean over 89 commits; pip-audit flags one advisory — cryptography 49.0.0 PYSEC-2026-3552 (fix 50.0.0) — which is a local-venv transitive, not pinned in requirements.txt, so it did not block a UI-only promotion (worth revisiting if it shows up in the Space build). Promoted with git push origin dev:main; the pre-push hook mirrored 3853e93 to github first ("mirror in sync"). Build watched RUNNING_APP_STARTING → RUNNING, root HTTP 200. Verified live via the queue protocol (the 2026-08-14 diagnostic trick): a signed-out click on "Query variant status" now yields caution {visible: true, "⛔ Access denied. 🔒 Please sign in…"} alongside the denied JSON — the dead-button symptom is gone in production. Local main could not be fast-forwarded (checked out in the peaceful-blackburn worktree — another lane); origin/main is the truth.

2026-08-14 — `cdaniel2008` added to `ALLOWED_IDS` (prod + dev, config only)

Fleet-wide widening (see pdac-subtype-agent memory.md): ALLOWED_IDS secret re-set on pdac-genomics-agent and -dev to the prior roster (Annie + Carl) plus cdaniel2008, via HfApi.add_space_secret. ADMIN_IDS/ACCESS_CONTROL unchanged. Spaces restarted on the write.

2026-08-14 (later) — dev `ALLOWED_IDS` narrowed to `anne-voigt`: lab members prod-only

Policy (fleet-wide, see pdac-subtype-agent memory.md): Carl + Colin removed from the dev Space's ALLOWED_IDS; prod unchanged (anne-voigt, cpelz741, cdaniel2008). The earlier "dev mirrors prod" posture note is superseded for the roster (posture — secret vs variable — still matches prod).

2026-08-14 — PROMOTED TO PROD (`2e971df`): invisible-denial fix + ADR-0009 + bookkeeping

Annie's call. Merged dev into main (--no-ff) in the worktree lane; her concurrent dev:main push had already landed 3853e93 on origin/main and the merge commit descends from it, so both pushes composed cleanly — nothing clobbered. Gates before push: 245 tests green on the merged tree (repo .venv); pre-push hook mirrored 2e971df to github first ("mirror in sync"). Build watched to RUNNING at 2e971df, root 200. Verified live signed-out: the click now renders "⛔ Access denied … please sign in" in the caution box on the variant-status tab — the 2026-08-14 dead-button report can no longer reproduce. ADR-0009's claim + cache policy are now on origin/main, closing the "reaches origin at next promotion" note.

Two restarts today dropped OAuth sessions — anyone who "was signed in" must sign in again.

⚠️ dev NOT realigned to main yet — the main checkout has dev checked out with another active session's uncommitted memory.md; fast-forwarding under it was not safe. After that lane finishes: git switch dev && git merge --ff-only main && git push hf-dev dev:main. ⚠️ Prod LOG_SINK_HF_TOKEN still missing (see 2026-08-14 entry above) — deploy doesn't change that.

2026-08-14 — PROD AUDIT SINK LIVE ✅ (the last mile, and the trap in it)

Anne set LOG_SINK_HF_TOKEN on prod. First verification FAILED with the exact signature of a missing secret — correct answers, no record, no persist_trace failed line — because the secret-add restart wedged: prod sat RUNNING_APP_STARTING 20+ minutes while the OLD container (no secret in env) kept serving. Ruled out a code/token problem by querying the DEV Space, which wrote runs/20260814_170438_132674.json immediately. One manual restart later prod cut over in ~60 s and the authorized query wrote runs/20260814_170536_091195.json — full record: tool: variant_status:api, principal: anne-voigt, role: admin, enforced: true.

Prod audit records are now durable. Worth keeping: a wedged Space restart serves the pre-change environment indefinitely while reporting APP_STARTING — after ANY secret/variable change, confirm the stage reaches RUNNING, and treat "no record + no error line" as "the serving container never saw the config", not "the sink is broken".

Also answered for Anne: a Space secret's NAME (LOG_SINK_HF_TOKEN) never appears in the HF access-token list — that list shows token names (research_agent_token); the secret is just an env key whose value is the token string.