[NOTICKET] docs: correct 13f74e1 root cause — Go migrations are non-convergent

Verified against the Go repo at dd37b38 (pulled today). 13f74e1's commit message
and doc deltas claimed `reports.user_id` was absent from migrations 0001-0004.
That was wrong.

`0001_create_core_schema.sql` has declared `reports.user_id TEXT NOT NULL` since
the schema was written. The actual defect is that `0002` (L95) and `0004` (L61)
retrofit the column onto pre-existing DBs as a NULLABLE
`ALTER TABLE reports ADD COLUMN IF NOT EXISTS user_id TEXT`. Since
`CREATE TABLE IF NOT EXISTS` no-ops on an existing table, the same migration set
yields two schemas: the old dedorch DB got nullable and silently swallowed
Python's column-less INSERT for months; Neon was provisioned fresh and enforces
NOT NULL. `analyses_messages.user_id` has the identical split (0002 L92 / 0004
L57) but is Go-write-only, so it is inert.

The 13f74e1 code fix stands and is now positively confirmed by the canonical DDL.

Also from a full ORM-vs-live drift scan against Neon:
- All five tables Python writes are clean (reports, report_inputs,
message_traceability, message_charts, analyses).
- `report_inputs` now ships in Go's 0001, column-for-column identical to
`ReportInputRow` -> closes #22.
- `message_traceability` + `message_charts` are in NO migration; they exist on
Neon only because they were hand-created, so a fresh instance loses them and
fails silently (never-throw) -> new #32.
- Remaining drift (analyses_messages message_id/status/note, documents
chunks_count, message_sources) is read-only or unwired -> inert.

REPO_STATUS §8/§9/§12/§13 + header; DEV_PLAN #22 ✅, #30 corrected, #31/#32 new,
§5 critical path re-pointed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DEV_PLAN.md

@@ -215,7 +215,7 @@ Status legend: ⬜ not started · 🔄 in progress · ✅ done · ⛔ blocked ·
 | 19 | Decide report author display-name source (`users.fullname` vs Go-passed name) | Sofhia | ✅ | Done 2026-06-25. `AnalysisReport.user_name`; `generator` renders `user_name or user_id`; `api/v1/report.py::_resolve_user_name` reads `users.fullname` never-throw (fallback `user_id`). Decided: resolve in Python (unblocked); swap to Go-passed name later if preferred |
 | 20 | **Help handoff:** update `handlers/help.py` + `help.md` — drop the `problem_validated` tier + `define_problem_statement` action (the skill it points at is gone as of #1) | Sofhia | ✅ | Done 2026-06-25. `help.py`: actions = `ask_analysis_question` (always) + `generate_report` (if ready); renders objective/business_questions (getattr-tolerant). `help.md` v1→v2: 3 tiers, no `/problem_statement`, `/generate report`→`/report`. Local test_help updated → 11 pass |
 | 21 | Rename `analysis_records` → **`report_inputs`** (table, ORM `ReportInputRow`, store `*ReportInputStore`) | Rifqi | ✅ | Done 2026-06-26. `sed` rename across 9 files; Pydantic `AnalysisRecord` kept; columns stay String (pure rename — uuid+FK is the #22 Harry schema). Name `report_inputs` (purpose; avoids Langfuse/`analyses_messages` clash). Write scope = one row per slow-path run. Suite **284 passed** |
-| 22 | Finalize `report_inputs` schema → hand to Harry for the dedorch migration | Rifqi → Harry | 🔄 | **DDL ready** (uuid `id`/`analysis_id` + FK→`analyses(id)`; `user_id`/`plan_id` text; `data` jsonb = serialized `AnalysisRecord`, shape documented). dedorch has empty `analysis_records` → rename. Resolves #16. **Action: send Harry the DDL + `data` shape** |
+| 22 | Finalize `report_inputs` schema → hand to Harry for the dedorch migration | Rifqi → Harry | ✅ | **DDL ready** (uuid `id`/`analysis_id` + FK→`analyses(id)`; `user_id`/`plan_id` text; `data` jsonb = serialized `AnalysisRecord`, shape documented). dedorch has empty `analysis_records` → rename. Resolves #16. ~~**Action: send Harry the DDL + `data` shape**~~ **RESOLVED 2026-07-22:** Harry's `0001_create_core_schema.sql` now carries `report_inputs` with exactly this shape — FK to `analyses(id)` included. Python's `ReportInputRow` diffed against the live Neon table: **column-for-column identical, no drift.** The two remaining un-migrated Python tables (`message_traceability`, `message_charts`) split out as → #32 |
 | 23 | Report markdown formatting: tables, **bold**, *italic*, horizontal separators | Sofhia | ✅ | Done 2026-06-25. Added `---` separators between header + each section in `_render_markdown`. Tables (EDA) / bold (method labels) / italic (meta + citations) already emitted. Relaxed `report_summary.md` to allow inline `**bold**`/`*italic*` for emphasis (kept no-headings/no-bullets so it doesn't duplicate the section structure / Key Findings). Compile + ruff clean |
 | 24 | Clarify report input contract: records table (+ `last_report` for edit mode?) | Rifqi/Sofhia ↔ Harry | ⬜ new | Edit-mode input left open at the checkpoint |
 | 25 | Migrate Python chat path to Go `analyses_messages` (+ `analyses`) | Rifqi ↔ Harry | ✅ | Done 2026-07-02. Read path already on `analyses_messages` (commit `0066161`). This change makes Python **read-only**: removed the `save_messages` calls from `/api/v2/chat/stream` so **Go is the sole writer** — fixes the double-write both Go+Python were producing. `load_history` still reads `analyses_messages`. v1 `/chat/stream` is unwired so left untouched |
@@ -223,13 +223,14 @@ Status legend: ⬜ not started · 🔄 in progress · ✅ done · ⛔ blocked ·
 | 27 | **Images (DEFERRED):** image table (id, analysis_id, msg/report ref, order) + originals in a bucket | — | ⏸️ | Maintenance-heavy; parked |
 | 28 | **UI research** (FE): new-analysis form, knowledge menu (user vs analysis level), report artifacts + version selector | Team | ⬜ new | No dedicated UI person; interview + old analysis UI removed |
 | 29 | **LLM env quad rename `__4o` → `__54m`** + set the four `azureai__*__54m` secrets on the HF Space | Rifqi | 🔄 | Code done 2026-07-14 (pr/17): settings quad + 9 call sites + 2 eval runners renamed; deployment is `gpt-5.4-mini`. Suite **381 passed, 2 failed (both pre-existing), 7 skipped** — same as baseline; ruff clean on touched files; `import main` 0. **Hard rename — no `__4o` fallback**, so HF fails loudly on the first LLM call until the four `__54m` secrets are set. Root cause it fixes: HF silently ran GPT-4o while local ran 5.4-mini → same question, same catalog, planner hallucinated catalog ids on HF only. **Action: set HF secrets, redeploy, re-run the failing question.** Local `.env` still carries the now-dead `__4o` keys — safe to delete |
-| 30 | **Neon `reports.user_id` NOT NULL** — reconcile the ORM + `ReportStore` write | Rifqi | ✅ | Done 2026-07-22 (pr/18). The new Neon dedorch instance has `reports.user_id text NOT NULL` (verified via `information_schema`; **not** present in Go migrations `0001–0004`), a column `AnalysisReportRow` lacked → `ReportStore.save` INSERTed without it → **every `POST /api/v1/tools/report` 500'd** with `NotNullViolationError`. Fix: `models.py` adds `user_id = Column(Text)` (**nullable** on purpose — Python matches the loosest deployment shape, §7D getattr-tolerance) + `store.py` writes `report.user_id`, which the endpoint already required (`user_id: str = Query(...)`) and threaded via `generator.generate`. Live-tested by Rifqi → **201**. `import main` 0; ruff on touched files unchanged (3 pre-existing E501 in `models.py`, same count on HEAD). **Follow-up → #31** |
-| 31 | **Schema-change notification from Go/Harry** — Python must learn about new columns on tables it *writes* | Rifqi → Harry | ⬜ new | Raised by #30: the `reports.user_id` addition reached the Neon DB with no Python-side heads-up, and read-path getattr-tolerance hides such drift until an INSERT fails. Python writes only `reports`, `report_inputs`, `message_traceability`, `message_charts`, `analyses` (via `state_store.ensure`) — a column added to any of those breaks Python silently until runtime. **Ask:** Harry announces DDL touching those five, or the migration set becomes the single verifiable source (`0001–0004` did not contain this column). **Action: Rifqi raises with Harry** |
+| 30 | **Neon `reports.user_id` NOT NULL** — reconcile the ORM + `ReportStore` write | Rifqi | ✅ | Done 2026-07-22 (pr/18, commit `13f74e1`). `ReportStore.save` never wrote `user_id`; on Neon that column is `text NOT NULL` → **every `POST /api/v1/tools/report` 500'd** with `NotNullViolationError`. Fix: `models.py` adds `user_id = Column(Text)` (**nullable** on purpose — Python matches the loosest deployment shape, §7D getattr-tolerance) + `store.py` writes `report.user_id`, which the endpoint already required (`user_id: str = Query(...)`) and threaded via `generator.generate`. Live-tested by Rifqi → **201**. Suite 381 passed / 2 pre-existing failures / 7 skipped; `import main` 0; ruff unchanged (3 pre-existing E501 in `models.py`, same count on HEAD). **Correction 2026-07-22 (after pulling Go `dd37b38`):** the commit message and the first version of this row said the column was *not* in Go migrations `0001–0004`. **That was wrong** — `0001_create_core_schema.sql` has declared `reports.user_id TEXT NOT NULL` since the schema was written. Real root cause → #31 |
+| 31 | **Go migration set is not convergent** — fresh vs migrated dedorch DBs get different NOT NULL constraints | Rifqi → Harry | ⬜ new | Root cause of #30, verified in the Go source 2026-07-22. `0001_create_core_schema.sql` creates `reports.user_id TEXT NOT NULL` and `analyses_messages.user_id TEXT NOT NULL`; `0002_cleanup_legacy_schema.sql` (L95, L92) and `0004_replace_chat_with_analysis_scope.sql` (L61, L57) retrofit the same two columns onto pre-existing DBs as **nullable** `ALTER TABLE … ADD COLUMN IF NOT EXISTS user_id TEXT`. Because `CREATE TABLE IF NOT EXISTS` no-ops on an existing table, **the same migration set yields two different schemas** — the old dedorch DB got nullable (hiding Python's missing write for months), Neon got NOT NULL. **Ask Harry:** make the retrofit converge (backfill + `ALTER COLUMN … SET NOT NULL`, or add the constraint in a new migration) so every instance matches `0001`. Until then, `information_schema` on the target instance — not the migration files — is the only reliable schema source (REPO_STATUS §13). **Action: Rifqi raises with Harry** |
+| 32 | **`message_traceability` + `message_charts` are in no Go migration** — hand the DDL to Harry | Rifqi → Harry | ⬜ new | Found by the 2026-07-22 drift scan. Both tables exist on Neon **only because they were created by hand** (2026-07-06 / 2026-07-13); neither appears in `0001–0006`. Any newly provisioned dedorch instance will be missing them → traceability flush and chart persist fail (both never-throw, so they degrade **silently** — no 500 like #30 to make it visible). DDL for `message_charts` is in `SPINE_V2_PLAN.md` §4.4. This is the surviving half of #22. **Action: Rifqi sends Harry both DDL blocks** |
 
 ## 5. Critical path & sequencing
 
-- **Critical path:** #22 (send Harry the `report_inputs` schema). HF deploy (#13) for the playground. (#4 ✅, #21 ✅; Harry's #3 no longer blocks us — Python is getattr-tolerant.)
-- **Parallelizable now:** #22 (handoff). (#4 ✅, #11 ✅ done.)
+- **Critical path:** ~~#22 (send Harry the `report_inputs` schema)~~ **✅ resolved 2026-07-22** — now **#32** (`message_traceability` + `message_charts` DDL to Harry) and **#31** (non-convergent migration set). HF deploy (#13) for the playground. (#4 ✅, #21 ✅; Harry's #3 no longer blocks us — Python is getattr-tolerant.)
+- **Parallelizable now:** #31 + #32 (both are Harry handoffs). (#4 ✅, #11 ✅, #22 ✅ done.)
 - **Harry-blocked / coordinated:** #3 (now 🔄, blocks #4), #7 (Go endpoint), #18 (Go state ownership), #24 (contract). **#25 = chat-path migration to `analyses_messages` — a cutover blocker.**
 - **Demo gate (playground, #13):** deploy the remove-`problem_statement` work to HF — slow path (#15 ✅)
   and the report path are verified locally, and #16 is resolved (#22 hands Harry the schema). **Keep it

REPO_STATUS.md

@@ -2,7 +2,7 @@
 
 **Audience:** teammates onboarding onto the Python repo (`Agentic-Service-Data-Eyond-Catalog`).
 **Scope:** what the code does **right now** (branch `pr/4`, ticket KM-652). Describes current state only — no roadmap or to-dos.
-**Snapshot date:** 2026-06-25. **Data-layer reconcile 2026-07-01:** §8/§12 updated — dedorch cutover done, `data_catalog` model reconciled. **Query-path fix 2026-07-02:** §8/§13 — dedorch catalogs ship no FKs → Python infers them (`fk_inference.py`); shared-Fernet-key gotcha documented. **Agent-quality fixes 2026-07-08 (pr/13):** from the scoped live-test review — the planner gains an explicit **infeasible** outcome (`TaskList.infeasible_reason` → deterministic EN/ID data-gap reply via `refusals.data_gap_message`; no more force-mapping absent measures like `pa` AS "revenue"), the IR validator rejects bare selects under `group_by` (self-corrects via the planner retry), `analyze_trend` handles integer year/month columns (was collapsing every row into one 1970-01 bucket), planner few-shots add top-N (Example G) + infeasible (Example H), numeric catalog `sample_values` are base64-decoded at read (`catalog/sample_decode.py` — stopgap for Go's byte-marshaling; primary fix is Go-side), traceability no longer emits null source rows for failed retrievals, and `check_data` hides `-1` row counts. **Report v2 + analyze_merge planner support 2026-07-09 (pr/13):** Sofia's `analyze_merge` tool (8abf635, KM-703) is now planner-supported (`_validate_data_source` guards `data_right`, two-retrieve→merge few-shot Example I, planner.md "Two measures per entity" rule); the report gains per-business-question answers (`bq_answers` — drafted by the SAME single LLM call, index-based record refs, deterministic fallback unchanged), "Attempted, Unresolved" + "Excluded Analyses" sections (failed runs are no longer silently dropped), evidence tables copied from `results_snapshot` (table-kind outputs, ≤3/record ≤10 rows ≤8 cols, `check_*` skipped), normalized caveat dedupe with caps (12/10), and single-language output via `detect_reply_language`; the report surface adds `GET /tools/report/{analysis_id}/records` (curation list), `GET …/readiness` (FE delta guard), and `exclude_record_ids` on POST — see API_CONTRACT_BE_PYTHON.md. **Report compaction 2026-07-09 (pr/13):** the rendered markdown drops the "Notes & Limitations", "Attempted, Unresolved", and "How This Was Analyzed" sections (team decision — compact report; render blocks commented out in `report/generator.py`, not deleted). The JSON body keeps `caveats`/`open_questions`/`unresolved`/`method_steps` and the curation/records endpoints are unchanged. **Traceability `data_used` layer 2026-07-13 (pr/15):** `GET /api/v1/traceability` gains a resolved, user-facing `data_used[]` block (one per `retrieve_data` call) — real source/table/column names, joins, plain-language filters, and result columns split into read-from-data vs `computed` (with `formula`, e.g. `total_revenue` = `SUM(line_total)`, so an alias is never shown as a real column). Ids are kept but **machine-only (FE must not render)**. Also adds `tool_calls[].summary`, and `sources[]` now carry `source_name` + every table touched. Deterministic catalog resolution (new `src/traceability/resolve.py`), no LLM, never-throw; catalog threaded to the scratchpad at the slow-path composition root. Additive/non-breaking; contract + `TRACEABILITY_FE_HANDOFF.md` updated. **Spine v2 W2+W1 2026-07-13 (pr/16):** §6/§7/§8/§9/§12 — `render_chart` tool lands (first of the `render_*` family: deterministic Plotly-JSON `dataeyond.chart.v1` envelope, hand-built, **no plotly dependency**; planner-selected only on an explicit chart ask, EN/ID) + Python-owned `message_charts` store + `GET /api/v1/charts` (FE fetches on `done`, same pattern as traceability; empty list = valid 200); planner gains a named **recipe table** + viz few-shots (Example J tail, Example K viz-infeasible) + validator Check 10 (`render_chart.data` must reference a table-producing task); and the slow path gains the **S1a quality checkpoint** (`slow_path/checkpoint.py`, 0 LLM, never-throw, between runner and assembler: CK1 all-failed → deterministic honest-failure answer with **no** assembler call, CK2 empty retrieve + downstream, CK3 10k-cap truncation, CK4 single trend bucket, CK5 all-null column, CK6 chart-spec sanity; flags render as an "Execution assessment" block in the assembler input and every flag logs `repair_candidate` — the S1b evidence base). Design + handoff doc: `SPINE_V2_PLAN.md`. **LLM model + env rename 2026-07-14 (pr/17):** §2/§3/§9/§13 — the generation LLM is now **Azure GPT-5.4-mini** (deployment `gpt-5.4-mini`), not GPT-4o, and the settings quad is renamed `azureai__*__4o` → **`azureai__*__54m`** across all 9 LLM call sites. **Hard rename, no `__4o` fallback** — an environment that still sets only `__4o` resolves to empty strings and fails on the first LLM call, by design: the silent-wrong-model drift is exactly how HF stayed on GPT-4o while local ran 5.4-mini (identical question, identical catalog, divergent planner output — HF hallucinated catalog ids, local planned correctly). Deploying requires all four `__54m` vars set in the HF Space secrets. **`reports.user_id` fix 2026-07-22 (pr/18):** §8/§9/§13 — the new **Neon** dedorch instance carries `reports.user_id text NOT NULL`, a column Python's `AnalysisReportRow` did not have, so `ReportStore.save` emitted an INSERT without it and **every `POST /api/v1/tools/report` 500'd** (`NotNullViolationError`; report persistence deliberately re-raises rather than degrading). The ORM gains `user_id` (declared **nullable** so Python tolerates both the pre- and post-migration shape) and the store writes `report.user_id`, which the endpoint already required as a query param and threaded through the generator. Live-verified against `information_schema` + a green 201. Generalized as a §13 gotcha: Go-owned column additions are invisible on read paths and only surface on Python's few write paths. **Cross-repo update 2026-06-29:** §2/§8/§11/§12 re-verified against
+**Snapshot date:** 2026-06-25. **Data-layer reconcile 2026-07-01:** §8/§12 updated — dedorch cutover done, `data_catalog` model reconciled. **Query-path fix 2026-07-02:** §8/§13 — dedorch catalogs ship no FKs → Python infers them (`fk_inference.py`); shared-Fernet-key gotcha documented. **Agent-quality fixes 2026-07-08 (pr/13):** from the scoped live-test review — the planner gains an explicit **infeasible** outcome (`TaskList.infeasible_reason` → deterministic EN/ID data-gap reply via `refusals.data_gap_message`; no more force-mapping absent measures like `pa` AS "revenue"), the IR validator rejects bare selects under `group_by` (self-corrects via the planner retry), `analyze_trend` handles integer year/month columns (was collapsing every row into one 1970-01 bucket), planner few-shots add top-N (Example G) + infeasible (Example H), numeric catalog `sample_values` are base64-decoded at read (`catalog/sample_decode.py` — stopgap for Go's byte-marshaling; primary fix is Go-side), traceability no longer emits null source rows for failed retrievals, and `check_data` hides `-1` row counts. **Report v2 + analyze_merge planner support 2026-07-09 (pr/13):** Sofia's `analyze_merge` tool (8abf635, KM-703) is now planner-supported (`_validate_data_source` guards `data_right`, two-retrieve→merge few-shot Example I, planner.md "Two measures per entity" rule); the report gains per-business-question answers (`bq_answers` — drafted by the SAME single LLM call, index-based record refs, deterministic fallback unchanged), "Attempted, Unresolved" + "Excluded Analyses" sections (failed runs are no longer silently dropped), evidence tables copied from `results_snapshot` (table-kind outputs, ≤3/record ≤10 rows ≤8 cols, `check_*` skipped), normalized caveat dedupe with caps (12/10), and single-language output via `detect_reply_language`; the report surface adds `GET /tools/report/{analysis_id}/records` (curation list), `GET …/readiness` (FE delta guard), and `exclude_record_ids` on POST — see API_CONTRACT_BE_PYTHON.md. **Report compaction 2026-07-09 (pr/13):** the rendered markdown drops the "Notes & Limitations", "Attempted, Unresolved", and "How This Was Analyzed" sections (team decision — compact report; render blocks commented out in `report/generator.py`, not deleted). The JSON body keeps `caveats`/`open_questions`/`unresolved`/`method_steps` and the curation/records endpoints are unchanged. **Traceability `data_used` layer 2026-07-13 (pr/15):** `GET /api/v1/traceability` gains a resolved, user-facing `data_used[]` block (one per `retrieve_data` call) — real source/table/column names, joins, plain-language filters, and result columns split into read-from-data vs `computed` (with `formula`, e.g. `total_revenue` = `SUM(line_total)`, so an alias is never shown as a real column). Ids are kept but **machine-only (FE must not render)**. Also adds `tool_calls[].summary`, and `sources[]` now carry `source_name` + every table touched. Deterministic catalog resolution (new `src/traceability/resolve.py`), no LLM, never-throw; catalog threaded to the scratchpad at the slow-path composition root. Additive/non-breaking; contract + `TRACEABILITY_FE_HANDOFF.md` updated. **Spine v2 W2+W1 2026-07-13 (pr/16):** §6/§7/§8/§9/§12 — `render_chart` tool lands (first of the `render_*` family: deterministic Plotly-JSON `dataeyond.chart.v1` envelope, hand-built, **no plotly dependency**; planner-selected only on an explicit chart ask, EN/ID) + Python-owned `message_charts` store + `GET /api/v1/charts` (FE fetches on `done`, same pattern as traceability; empty list = valid 200); planner gains a named **recipe table** + viz few-shots (Example J tail, Example K viz-infeasible) + validator Check 10 (`render_chart.data` must reference a table-producing task); and the slow path gains the **S1a quality checkpoint** (`slow_path/checkpoint.py`, 0 LLM, never-throw, between runner and assembler: CK1 all-failed → deterministic honest-failure answer with **no** assembler call, CK2 empty retrieve + downstream, CK3 10k-cap truncation, CK4 single trend bucket, CK5 all-null column, CK6 chart-spec sanity; flags render as an "Execution assessment" block in the assembler input and every flag logs `repair_candidate` — the S1b evidence base). Design + handoff doc: `SPINE_V2_PLAN.md`. **LLM model + env rename 2026-07-14 (pr/17):** §2/§3/§9/§13 — the generation LLM is now **Azure GPT-5.4-mini** (deployment `gpt-5.4-mini`), not GPT-4o, and the settings quad is renamed `azureai__*__4o` → **`azureai__*__54m`** across all 9 LLM call sites. **Hard rename, no `__4o` fallback** — an environment that still sets only `__4o` resolves to empty strings and fails on the first LLM call, by design: the silent-wrong-model drift is exactly how HF stayed on GPT-4o while local ran 5.4-mini (identical question, identical catalog, divergent planner output — HF hallucinated catalog ids, local planned correctly). Deploying requires all four `__54m` vars set in the HF Space secrets. **`reports.user_id` fix 2026-07-22 (pr/18):** §8/§9/§12/§13 — `ReportStore.save` never wrote `reports.user_id`, and on the new **Neon** dedorch instance that column is `text NOT NULL`, so **every `POST /api/v1/tools/report` 500'd** (`NotNullViolationError`; report persistence deliberately re-raises rather than degrading). The ORM gains `user_id` (declared **nullable** so Python tolerates every deployment shape) and the store writes `report.user_id`, which the endpoint already required as a query param and threaded through the generator. Live-verified against `information_schema` + a green 201. **Root cause (verified against the Go source after pulling `dd37b38`):** the column is *not* new — `0001_create_core_schema.sql` has declared it `NOT NULL` all along, but `0002`/`0004` retrofit it onto pre-existing DBs as a **nullable** `ADD COLUMN IF NOT EXISTS`. The old DB therefore accepted the column-less INSERT and the bug stayed latent; a fresh DB enforces it. Generalized as a §13 gotcha — **Go's migration set is not convergent, so fresh and migrated dedorch DBs differ in NOT NULL constraints** and the migrations alone can't tell you which shape you have. A full ORM-vs-live drift scan the same day found all five Python-written tables clean; the remaining drift is on read-only or unwired paths. **Cross-repo update 2026-06-29:** §2/§8/§11/§12 re-verified against
 the **Go source** (`Orchestrator-Agent-Service`), not its docs. The Go service has moved well past its
 own (uncommitted, stale) design docs: it now hosts the **dedorch SQL migrations** in-repo and a full
 **`/api/v1/analyses` + `/api/v1/skills`** REST surface. Go does **not** call Python yet — those skills
@@ -361,10 +361,16 @@ Python is consumer-only). State **re-verified against the Go source 2026-06-29**
   **dropped** `problem_statement`/`problem_validated`/`owner_id`. Migration `0004` renames the legacy
   `rooms`/`chat_messages`/`interview_*` tables to `zdeprecated_*`.
 - **`report_inputs`** (the slow-path structured output, formerly `analysis_records`) stays
-  **Python-owned**; its finalized schema goes to Harry so the dedorch migration creates it post-cutover.
-  Same pattern for **`message_traceability`** (created manually 2026-07-06) and **`message_charts`**
-  (created manually 2026-07-13, DDL in `SPINE_V2_PLAN.md` §4.4; live e2e verified same day —
-  Harry's migration handoff for both is still the open item).
+  **Python-owned** in the sense that Python is its only writer — but **✅ 2026-07-22: Harry's
+  migration now creates it.** `0001_create_core_schema.sql` carries the table (uuid `id`, `analysis_id`
+  uuid nullable **with the FK to `analyses(id)`**, `user_id`/`plan_id` text NOT NULL, `data` jsonb
+  NOT NULL, `created_at` timestamptz), exactly the DDL handed over in #22 — which closes #22.
+  Python's `ReportInputRow` was diffed against the live Neon table the same day: **column-for-column
+  identical**, no drift.
+  **`message_traceability`** (created manually 2026-07-06) and **`message_charts`** (created manually
+  2026-07-13, DDL in `SPINE_V2_PLAN.md` §4.4) are **still in no Go migration** — they exist on Neon
+  because they were created by hand, so a future fresh dedorch DB will not have them. Harry's
+  migration handoff for those two remains the open item (DEV_PLAN #32).
 - **Connection-string cutover DONE (2026-07-01).** Python's `postgres_connstring` now points at
   **dedorch** and reads the Go-migrated tables directly. Every ORM model Python reads (`analyses`,
   `analyses_messages`, `data_catalog`) has been reconciled to its dedorch shape.
@@ -372,8 +378,11 @@ Python is consumer-only). State **re-verified against the Go source 2026-06-29**
   DDL (`ALTER TABLE rooms …`, index creation) fails on Go-owned tables
   (`InsufficientPrivilegeError: must be owner of table rooms`). Skipping is safe — Go migration `0001`
   already provides the `vector` extension + the langchain FTS index. Set `SKIP_INIT_DB=false` (.env or
-  env) only for a local Python-owned DB. `report_inputs` is not in any Go migration yet (#22) — create
-  it in dedorch before enabling the slow path, else report/slow-path writes fail (chat path unaffected).
+  env) only for a local Python-owned DB. ~~`report_inputs` is not in any Go migration yet (#22) — create
+  it in dedorch before enabling the slow path, else report/slow-path writes fail (chat path unaffected).~~
+  **RESOLVED 2026-07-22:** `report_inputs` ships in Go's `0001` (above), so a fresh dedorch DB has it.
+  The same is **not** true of `message_traceability`/`message_charts` — those still need the manual DDL
+  on any newly provisioned instance (DEV_PLAN #32).
 
 **⚠️ Integration gap (verified — the big one).** Go's `/api/v1/analyses` and `/api/v1/skills`
 (`help` / `report`) are **placeholders that return dummy data** — the `SendMessage` / `GenerateReport`
@@ -415,18 +424,32 @@ records-based report; floor: ≥1 `analyze_*` success). Wiring Go → Python is
   degrades it into an honest "data not available" answer, so it masquerades as a data problem.
   Tell-apart: `BlobNotFound` + `location_ref` starting `object_storage://` ⇒ env gap; set
   `storage_provider=supabase_s3` + the five `supabase_s3_*` values (match Go's data plane).
-- **Go-owned columns drift silently until an INSERT hits them (gotcha, found 2026-07-22).** Python
-  reads Go-owned tables getattr-tolerantly, so a *new* column on a table Python only reads is
-  invisible. But `reports` is one of the few tables Python **writes** — when the new Neon instance
-  brought a `reports.user_id text NOT NULL` the ORM did not have, the INSERT omitted the column and
-  every `POST /api/v1/tools/report` 500'd with `Report persistence failed: … NotNullViolationError`
-  (report persistence is **not** a never-throw seam — `api/v1/report.py` re-raises as a 500, by
-  design, so a lost report is loud). Tell-apart: `NotNullViolationError` naming a column absent from
-  `src/db/postgres/models.py`. Fix pattern: add the column to the ORM as **nullable** (Python
-  matches the loosest deployment shape) and always write it — never make Python's model NOT NULL to
-  match one DB. Verify with
-  `select column_name, is_nullable from information_schema.columns where table_name='<t>'` against
-  the live DB, not against the Go migrations (this column was not in `0001–0004`).
+- **Fresh vs migrated dedorch DBs have different NOT NULL constraints (gotcha, found 2026-07-22 —
+  verified against the Go source, corrects an earlier note in this file that said the column was
+  missing from the migrations).** Go's migration set is not convergent: `0001_create_core_schema.sql`
+  declares `reports.user_id TEXT NOT NULL` **and** `analyses_messages.user_id TEXT NOT NULL` in its
+  `CREATE TABLE`s, while `0002`/`0004` retrofit the same columns onto pre-existing databases as
+  `ALTER TABLE … ADD COLUMN IF NOT EXISTS user_id TEXT` — **nullable**. So a DB where the table
+  already existed ends up nullable (the `CREATE TABLE IF NOT EXISTS` no-ops), and a DB created fresh
+  from `0001` ends up NOT NULL. **Same migrations, two schemas.**
+  This is what the 2026-07-22 report outage actually was: `ReportStore.save` had *never* written
+  `user_id`, and the old dedorch DB's nullable column silently accepted it. The Neon instance was
+  provisioned fresh, so `NOT NULL` bit immediately — every `POST /api/v1/tools/report` 500'd with
+  `Report persistence failed: … NotNullViolationError` (report persistence is **not** a never-throw
+  seam — `api/v1/report.py` re-raises as a 500 by design, so a lost report is loud).
+  Tell-apart: `NotNullViolationError` naming a column absent from `src/db/postgres/models.py`.
+  Fix pattern: add the column to the ORM as **nullable** (Python then matches the loosest deployment
+  shape) and always write it — never mirror one DB's NOT NULL into Python's model.
+  **Reading the Go migrations is not sufficient** — the constraint depends on the DB's history, so
+  verify the instance you actually run against:
+  `select column_name, is_nullable from information_schema.columns where table_name='<t>'`.
+  Full ORM-vs-live drift scan run 2026-07-22 against Neon: all five tables Python **writes**
+  (`reports`, `report_inputs`, `message_traceability`, `message_charts`, `analyses`) are clean.
+  Remaining drift is on read-only or dead paths and is inert — `analyses_messages` has live
+  `message_id`/`status`/`note` the ORM lacks (Go-only writer, §2.3; added by Go `0006` +
+  `981b42b`/`5da7283`), `documents` has a live `chunks_count`, and `message_sources` diverges wholly
+  but is referenced only by the **unwired** `api/v1/room.py`. `rooms`/`chat_messages` are absent
+  (renamed `zdeprecated_*` by `0004`).
 - **Never-throw seams** are pervasive (tool invoker, query service, executors, state/catalog reads,
   record persistence, report summary). Failures degrade into soft output rather than raising — good
   for UX, but they can mask real breakage (e.g. a missing analysis-scope catalog silently falling