data/taxonomy/decisions.csv

decision_id,date,question,decision,rationale,decided_in
dec_001,2026-07-10,"test_extraction_provenance.py needs the source PDFs, which are gitignored. #27 requires a fresh clone to be green. How do both hold?",Skip the provenance tests when data/taxonomy/sources/ is absent; never fail.,"Skipped-not-failed keeps the fresh-clone gate honest while the test still fails loudly for anyone who has run the fetcher. The repo has no CI workflow today, so this is a local-developer concern. If CI lands and can hold a licence to the sources, it should run these unskipped.",#29
dec_002,2026-07-10,"A PDF has no sentences, only positioned glyphs, so a copied snippet spans line breaks that get_text() reports as newlines. What does 'the snippet still appears on that page' mean?","Compare after collapsing every run of whitespace to one space. Do not normalize ligatures, dashes, or quotation marks.","Whitespace is a layout artifact and carries no meaning. A dash does: '0.2-2 mm' and '0.2–2 mm' differ, and a value set that silently rewrites a dash is the quiet edit this table exists to prevent. Both the puller and the test share one normalize().",#29
dec_003,2026-07-10,"FIGO 2023's page-4 footnote states the isolated-tumor-cell bound as '>=0.2 mm', which is the error the paywalled corrigendum fixes. The same paper states it correctly on page 7. Which page does the registry cite?",Cite page 7. Never anchor an ITC or micrometastasis extraction to the page-4 footnote.,"A page-anchored extraction pointed at the footnote would be verifiable, checksummed, and wrong: the evidence would be real and the value false. CAP page 14 independently gives the same bound as 'no greater than 0.2 mm', so the corrected reading has two open sources and the erroneous one has none.",#29
dec_004,2026-07-10,"induce_surface_forms.py emits per-dimension lexicons, anchors and windows. Which of them are committed?",Commit {dimension}_anchors.csv and {dimension}_lexicon.csv. Gitignore {dimension}_windows.parquet.,"The anchors ARE the definition of every count, so they must be reviewable in a diff. The lexicon is the finding. The windows are regenerable corpus text, large, and add nothing a reader can check. Parquet also stays out of git by default, where a .csv would be un-ignored by the data/taxonomy/**/*.csv negation and committed by accident.",#30
dec_005,2026-07-10,Which anchors define the LVSI dimension?,"Four: the lymphovascular family (tolerating the dropped 'o' and slash separators), angiolymphatic, the LVSI token, and bare 'vascular invasion'.","This set covers 467 of 546 reports, reproducing the PRD's figure exactly from anchors written against the corpus rather than copied from the PRD. Dropping bare 'vascular invasion' costs 44 reports (467 to 423). Adding bare 'lymphatic invasion' gains 15 (467 to 482) but that phrase also appears in lymph-node dissection descriptions. 467/546 is treated as a fact about the corpus; every other count here is a fact about these anchors.",#30
dec_006,2026-07-10,"The PRD's section 5.1 and 5.2 counts (101/232/81/118 geometry, 681 LVSI windows, 13 negation windows) do not reproduce. What do the regression tests assert?","Assert the numbers the committed anchors produce, and say in the test name and docstring that they characterize the anchor set. Assert only 467/546 and 'zero reports state a focus count' as facts about the corpus.","The ticket already warned that geometry counts are anchor-definitions, not corpus facts: a loose anchor set gives 374/461/364/205 against the PRD's 101/232/81/118. Measured again here with a third, unit-requiring set: 139/109/65/182. The same turns out to be true of the LVSI window count (635 here, 681 in the PRD) and the negation count (126 disagreeing windows here, 13 in the PRD), which the ticket presented as robust alongside 467/546. Only the report-level LVSI coverage and the zero focus counts survived every anchor set tried. Committing the anchors makes a moved number a visible diff.",#30
dec_007,2026-07-10,TCGA-EC-A24G's single page is a completed checklist printed sideways. Should ingestion detect and correct the rotation before rendering?,No. Render the page exactly as the PDF draws it. Record the rotation in visual_only_cases.csv and hand the problem to #32 (Channel B).,"Ingestion's job is to be faithful to the source, and orientation detection is a heuristic that would silently rewrite a page. But a page embedded sideways will produce sideways patch vectors, so #32 cannot ignore this: the one case the visual layer exists to rescue is also the one case whose page is not upright. Recorded on #32 rather than fixed here, because mechanism and preprocessing policy are separate commits.",#31
dec_008,2026-07-10,T5 calls the three restored cases an images-and-embeddings-only evaluation set. Are they?,"No. Only TCGA-EC-A24G is visual-only. TCGA-AJ-A3BH (747 words) and TCGA-BK-A139 (412 words) carry clean text layers in their GDC PDFs and are missing from the corpus, not from the record.","Scoring a visual extractor on a case whose PDF text you could simply read is not a test of the visual thesis. After T5 the corpus contains exactly one case where text collapsed, which sharpens rather than resolves PRD section 5.5: n=1 is a demonstration, not an evaluation set. The other two belong in a PDF-text ingestion path.",#31
dec_009,2026-07-10,"Embedding the three restored cases needs the ColPali weights, whose base repo is a ~5.4GB download that is not cached here.","Render the page images, wire `precompute_embeddings.py --case ... --skip-queries`, and stop. Do not pull 5.4GB unattended.","#42 exists because pytest silently downloaded this checkpoint over what the local address says is a phone hotspot. Doing the same thing deliberately but unasked is the same mistake with better intentions. The images are on disk and one command produces the vectors; the test asserts pending vectors are a subset of the restored three, so it holds now and after that command runs.",#31
dec_010,2026-07-10,"docs/taxonomy.md section 6 lists five reasons a field carries no plain value, under five names this project invented. A standard vocabulary for absent values already exists. Adopt it?","Adopt FHIR dataAbsentReason for the two states that are genuinely absences: not_applicable becomes `not-applicable`, not_stated becomes `asked-unknown`. Keep indeterminate, constrained and determined_by_join, which are not absences and which no null vocabulary can express.","Two of the five states are absences and both have exact codes in an existing CC0 standard, so inventing names for them was gratuitous. The other three are not absences. `indeterminate` is an assertion the pathologist made, and CAP prints `Cannot be determined` as a checklist choice, which #29 committed as a concept with evidence; coding it `unknown` would discard the assertion. `constrained` is a narrowed set, and coding it as absent would discard the set, contradicting invariant 1. `determined_by_join` is a statement about provenance, and the value exists. HL7 v3 NullFlavor was considered and is 1:1 with dataAbsentReason for the two codes needed; dataAbsentReason was chosen as the current vocabulary. Both are CC0.",#46
dec_011,2026-07-10,myometrial_invasion_category was specified as a single threshold at 50 percent. Is that the whole rule?,"No. The category has three values, cut at 0 percent and at 50 percent: none, less than half, half or more.","FIGO 2023 separates stage IA1 (confined to the endometrium or a polyp) from stage IA2 (less than half myometrial invasion), so a 2009 stage IA cannot be resolved without knowing whether any invasion is present. Clinical risk assessment makes the same three-way cut for the same reason, rendering the continuous depth as 0 percent, 1 to 50 percent, and more than 50 percent (pathologist correspondence, 2026-07-10). A report that states no depth has not asserted zero depth; the corpus states invasion is absent in 38 of 546 reports.",#48
dec_012,2026-07-10,Open decision 2: may the 43 reports whose text says 'extensive' near a lymphovascular anchor be mapped to figo2023.lvsi.substantial?,"No. Leave them `constrained`. The mapping is recorded as `noMatch`, not as `relatedMatch` with reduced certainty.","FIGO 2023 defines substantial by a count of five or more involved vessels, and that count-based definition follows a 2015 study whose adoption has been slow. A pathologist writing 'extensive' in a pre-2013 report was not applying the five-vessel rule, so the adjective and the staging variable are different variables that share a word. No report in this corpus states a vessel or focus count, verified corpus-wide rather than only inside the anchored windows. Deriving `substantial` from the adjective would upstage cases from FIGO 2023 stage I to stage II on evidence the report does not contain. Recovering the variable requires re-reading the slides, at a retrieval cost of about 50 dollars per case before pathologist time (pathologist correspondence, 2026-07-10).",#48
dec_013,2026-07-10,"A separate research brief (docs/taxonomy_agent_prd.md, first draft) proposed a flat documentation_state enum (reported, not reported, not assessed, not performed, pending, indeterminate) for every field's canonical spec. dataAbsentReason already covers not_applicable and not_stated (dec_010). Does the new enum replace or extend that choice?","Neither. Keep dataAbsentReason as the single axis for why a value is absent, and add not-performed to it if a real case ever needs the distinction; do not adopt documentation_state. Workflow state (registered, preliminary, final, amended, corrected, cancelled, entered-in-error, unknown) is a second, separate FHIR axis (Observation.status), not a value-absence reason, and this corpus has no reachable instance of it: every TCGA report is a finalized sign-out, so a pending state has nothing to attach to.","Fetched the actual FHIR code systems rather than trust memory of them. dataAbsentReason has 15 codes, not the 2 dec_010 adopted; code 14 is not-performed, defined as ""the observation procedure was not performed"", which is exactly the new enum's not performed state and already sits in the code system this project committed to. Observation.status is a documented, separate axis from dataAbsentReason: FHIR never merges why-a-value-is-missing with what-workflow-state-the-observation-is-in into one field. documentation_state's pending and not performed values silently merge those two axes, which is the anti-pattern the standard governing this exact domain (clinical Observation resources) deliberately avoids. Adopting it would reopen dec_010's question with a worse answer. Recorded in docs/log.md's 2026-07-10 entry (""The PRD's documentation_state re-derives two axes FHIR already keeps apart"").",#57
dec_014,2026-07-10,"CAP bundles serosal, adnexal, vaginal, parametrial, and pelvic-wall involvement into one multi-select field (""Other Tissue/Organ Involvement""); ICCR gives each site its own field, with laterality on adnexa and an explicit pelvic-versus-abdominal split on peritoneal biopsies. Which field shape should this registry's new anatomical-involvement dimensions follow?","ICCR's shape: one dimension per anatomic site (serosa, adnexa, parametria, vagina, omentum, peritoneal biopsies split pelvic/abdominal), not CAP's bundled multi-select.","No clinical fact is in dispute: CAP and ICCR draw the same stage boundary (taxonomy_conventions.md section 4.2), and CAP's bundling is a data-entry-form convenience, not a claim that the sites are indistinct. But FIGO 2023 itself already splits stage by site: IIIA1 (adnexa) versus IIIA2 (serosa), IIIB1 (vagina) versus IIIB2 (parametria), and IIIB2 versus IVB turns on pelvic-versus-abdominal peritoneum (taxonomy_conventions.md section 4.1). A bundled field would have to be re-parsed to recover exactly the distinction a separate-field shape stores natively, which is invariant 1's rule (store the most primitive observation, derive the category) applied to anatomy instead of a continuous quantity. #58's stated scope undercounts at four dimensions, bundling vagina and parametria as one; corrected to five to keep IIIB1 and IIIB2 separately derivable.",#73
dec_015,2026-07-10,"T9's crosswalk.csv predicate column was specified as the five SKOS values only (exactMatch, broadMatch, narrowMatch, relatedMatch, noMatch). docs/taxonomy_conventions.md's own conventions section independently defined a sixth, conflict, for when two standards state a rule for the same question and disagree (used in section 6.3 for the multiple-classifier priority-order disagreement, #72). Does the registry's predicate column adopt the sixth value?",Yes. predicate is six values: the five SKOS values plus conflict.,"noMatch means a concept named in one standard has no counterpart in the other; conflict means both standards have a rule and the rules disagree. Coercing a real disagreement into noMatch (implying neither standard addresses the question) or relatedMatch (implying compatible overlap) would misstate what the sources actually say. Neither #35 nor conflict's first use case (#72) is implemented yet, so this closes a design gap before the first crosswalk.csv row is written rather than migrating rows after the fact.",#85
dec_016,2026-07-11,"Open decision 1: LVSI 'suspicious' and 'indeterminate/equivocal' are pathologist hedges that no reference value set names. How do they survive reconciliation without being coerced to 'present'?","Carry each as an endopath/1 extension concept (endopath.lvsi.suspicious, endopath.lvsi.indeterminate), crosswalked noMatch to CAP and FIGO. gap_report.csv buckets rows lvsi_sf_suspicious and lvsi_sf_indeterminate as in_data_not_in_reference and names the extension in mapped_concept_id.","Evidence: data/taxonomy/induced/lvsi_lexicon.csv records suspicious and indeterminate/equivocal as real surface forms, and gap_report.csv carries them (rows lvsi_sf_suspicious, lvsi_sf_indeterminate). Coercing either to cap_uterus.lvsi.present would turn a hedge into an assertion, a silent unrecoverable edit. 'suspicious' has no home in any reference value set. 'indeterminate/equivocal' is a hedge about the finding and is distinct from CAP's cap_uterus.lvsi.cannot_be_determined, which asserts the finding could not be assessed. An endopath/1 extension with a noMatch crosswalk keeps the pathologist's word in the export and lets a later reviewer or slide re-read resolve it. Reversible if a reference standard adopts the term.",#34
dec_017,2026-07-11,"Open decision 3: is CAP's nested pelvic/para-aortic x sentinel x macro/micro/ITC regional-node tree, and FIGO 2023's IIIC*i/ii deposit split, in scope for this corpus?","Out of scope. Record the AJCC ITC concept and the FIGO 2023 IIIC*i/ii codes as in_reference_not_in_data in gap_report.csv, and do not model the full node tree.","Evidence: data/taxonomy/induced/nodes_lexicon.csv finds isolated tumor cells in 0 reports and micrometastasis in 2 (0.4%). gap_report.csv row nod_ref_itc (ajcc_8.regional_lymph_node_status.pN0_i_plus) and the GDC-derived figo2023 IIIC*i/ii rows record the gap. The micro-vs-macro-vs-ITC deposit distinction FIGO 2023 IIIC*i/ii turns on is unmeasurable at this coverage, so taxonomyprd.md section 9 marks IIIC*i/ii unmeasurable. Modelling the full CAP node tree would add structure no case in the corpus can populate. Reversible if a corpus that states deposit sizes is ingested.",#34
dec_018,2026-07-11,"The T8 residual audit samples the raw corpus (data/raw/TCGA_Reports.csv), which is gitignored. How is convergence.csv populated and demonstrated in a fresh clone?","Ship convergence.csv as a header-only ledger in the repo. reconcile_vocabulary.py --residual-audit appends one row per dimension when the corpus is present, and a slow test asserts the final iteration is below 5% for every dimension. Never fail when the corpus is absent.","Mirrors dec_001: a corpus-dependent check skips rather than fails, so the fresh-clone gate stays honest while the audit still runs for anyone who has the corpus. The committed anchors are #30's converged final set (dec_005, dec_006), so the audit measures the final iteration's residual against them. #30 did not preserve intermediate anchor sets, so earlier iterations are not mechanically reconstructable, and manufacturing them would invent the history #34's monotonicity correction warns against. A non-monotonic step, were one produced, carries its own decisions.csv row naming the pocket of forms it exposed.",#34
dec_019,2026-07-11,"Open decision 3: UCEC_CN_HIGH stands in for p53abn as a surrogate (about 95 percent, CAP Note E), routing 162 cases into IICmp53abn. May the surrogate upstage a case unattended, or must a reviewer confirm it first?","Require reviewer confirmation. The copy_number_high to p53abn crosswalk row carries predicate broadMatch and certainty uncertain, and derive_stage applies the p53abn molecular modifier only when promise_class is asserted from the report or a reviewer has confirmed the join. A raw tcga_class of copy_number_high alone yields the anatomic stage, and the surrogate p53abn stage is offered as a determined_by_join candidate a person accepts, never an automatic upstage.","The surrogate carries a stated 5 percent error (CAP Note E: copy-number-high tumours are TP53-mutated in about 95 percent of cases), and IICmp53abn is a stage change with treatment consequences, so a 1-in-20 misclassification that upstages silently is exactly the confident error the human-in-the-loop design exists to prevent. The crosswalk row records the surrogate and its uncertainty so the candidate stage and its provenance are auditable; the modifier is confined to stages I and II (docs section 3.2), so this touches only the 162 CN-high cases. Reversible if a corpus with direct p53 immunohistochemistry is ingested, at which point the join is no longer a surrogate.",#35