[NOTICKET] revert: drop F-3 user_id scoping on /charts + /traceability (lead decision)

Reverts the F-3 half of 0cb7d53. Rifqi restated the standing decision: traceability
does not take user_id, and by the same logic neither does charts. The message_id-only
lookup for /charts (lead decision 2026-07-13) and (analysis_id, message_id) for
/traceability stand as they were.

Reverted in full so nothing half-applied is left behind:
- both endpoints back to their pre-F-3 signatures, no optional query param, no
scoped=false logging
- PostgresChartStore.list_for_message / turn_exists back to message_id-only; the
ChartStore Protocol and NullChartStore likewise
- the contract's two FE-facing "becoming required" notes removed — no FE change is
being requested, and none should be

Deliberately KEPT (not part of F-3):
- degraded_seam="chart_persist" + repr(exc) on the chart-persist seam (#42)
- PostgresTraceabilityStore.get's optional user_id parameter, which predates this
work (shipped in fd4865b). Its docstring now records that the endpoint will NOT
pass it, so a future reader does not "finish" the wiring. Kept as dead capability
for a Go-forwarded identity (#43).

Accepted consequence, recorded rather than argued: both endpoints stay unauthenticated
over real customer data — charts[].spec.plotly.data is actual table values, and the
traceability payload carries 5-row previews, the executed SQL, and the owner's user_id.
The service-secret gate is now the ONLY control protecting them, which makes setting
dataeyond__service__secret (#37) load-bearing rather than merely advisable.

Docs: DEV_PLAN #40 struck through and marked RESOLVED/DECLINED with the reasoning;
CODE_REVIEW gains a "Declined by the lead" section so F-3 is not re-raised by a future
review pass.

Verification: ruff clean on all four touched files; `import main` OK; suite 424 passed
/ 0 failed / 7 skipped (unchanged — nothing depended on the reverted parameters).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

API_CONTRACT_BE_PYTHON.md

@@ -487,21 +487,11 @@ Query params:
 | --- | --- | --- |
 | `analysis_id` | Yes | Analysis identifier. |
 | `message_id` | Yes | Assistant answer identifier returned by the stream. |
-| `user_id` | **No — becoming Yes** | Owner of the turn. See the tenant-scoping note below. |
-
-> **Tenant scoping — added 2026-07-23, FE action requested.** `user_id` is **optional
-> today**: send it and the lookup is scoped to the turn's owner; omit it and the
-> response is byte-identical to before, so nothing breaks by not changing yet.
-> **It will become required.** Please start sending it whenever the FE has it —
-> the server logs every unscoped call, and that count is what tells us when the flip
-> is safe. Until then this endpoint is an unauthenticated capability URL over real
-> customer data: the payload carries 5-row previews of every retrieved result, the
-> executed SQL, and the owner's `user_id`.
 
 Example:
 
 ```text
-GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1&user_id=usr_7
+GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1
 ```
 
 `intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or Azure content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
@@ -704,25 +694,11 @@ Query params:
 | Query | Required | Description |
 | --- | --- | --- |
 | `message_id` | Yes | Assistant answer identifier returned by the stream's `done` event. |
-| `user_id` | **No — becoming Yes** | Owner of the turn. See the tenant-scoping note below. |
-
-> **Tenant scoping — added 2026-07-23, FE action requested.** `user_id` is **optional
-> today**: send it and both the chart lookup and the `empty`/`not_found` check are
-> scoped to the turn's owner; omit it and the response is byte-identical to before, so
-> nothing breaks by not changing yet. **It will become required.** Please start sending
-> it whenever the FE has it — the server logs every unscoped call, and that count is
-> what tells us when the flip is safe.
->
-> Why this matters more here than elsewhere: `charts[].spec.plotly.data` contains the
-> **actual values from the customer's tables**, and the only thing currently protecting
-> it is that `message_id` is a UUID4. A leaked id — an error report, a shared screenshot
-> of a network tab, a support ticket — exposes that data indefinitely, with no expiry
-> and no ownership check.
 
 Example:
 
 ```text
-GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2&user_id=usr_7
+GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2
 ```
 
 The `dataeyond.chart.v1` envelope (the shape of `charts[].spec`, verbatim from `render_chart`):
@@ -799,7 +775,7 @@ Field rules:
 - `spec` is the full `dataeyond.chart.v1` envelope, unmodified — it is the source of truth, not a projection; render straight from it.
 - `chart_type` / `title` are copied out of `spec` for convenience (list rendering without parsing `spec`); `title` may be `null`.
 - A turn can produce more than one chart (multiple `render_chart` calls in the same plan); `charts` is ordered by creation time.
-- The response payload carries no `user_id` / `analysis_id`. The *lookup* accepts an optional `user_id` (see the tenant-scoping note above); when supplied, another tenant's turn reads as `not_found` rather than `empty`, so the tri-state cannot be used to probe whether a `message_id` exists.
+- The payload carries no `user_id` / `analysis_id` — charts are keyed by `message_id` alone.
 
 > **DDL note (Harry / dedorch migration):** the original manual index is `(analysis_id, message_id)`, which does not serve a `message_id`-only lookup. Additive index for the migration (also safe to run manually now):
 > ```sql

CODE_REVIEW_2026-07-23.md

@@ -32,7 +32,6 @@ Legend: ✅ shipped · 🔎 shipped, needs a deployment action · ⬜ not starte
 
 | Ref | Finding | Sev | Note |
 |---|---|---|---|
-| **F-3** | `/charts` + `/traceability` unscoped capability URLs | High | Store-side scoping done; endpoints need the parameter — **FE contract change** |
 | **F-13** | Unbounded Parquet read (OOM escapes every seam) | High | Only finding that can kill the process |
 | **F-12** | Unbounded planner catalog render | High | Measured 120k tokens @ 200×30; ship a *safety net*, not a tight cap |
 | **F-9** | PII reaches traceability + report evidence | High | Decided: mask stored artifacts, assembler keeps values |
@@ -47,6 +46,12 @@ Legend: ✅ shipped · 🔎 shipped, needs a deployment action · ⬜ not starte
 | **F-8** | Planner/assembler prompts lack injection resistance | High | Gated on F-29 |
 | **F-11**, **F-14**, **F-16**, **F-25**, **F-6**, **F-23**, **F-27**, **F-28** | Assorted | Med/Low | Batch opportunistically |
 
+### Declined by the lead
+
+| Ref | Finding | Decision |
+|---|---|---|
+| **F-3** | `/charts` + `/traceability` unscoped capability URLs | **DECLINED 2026-07-23 (Rifqi).** The `message_id`-only lookup for `/charts` and `(analysis_id, message_id)` for `/traceability` are standing lead decisions; F-3 asked to reopen them and the answer is no. An optional-`user_id` implementation was built and fully reverted the same day. **Accepted consequence:** both endpoints stay unauthenticated over real customer data, so the **service-secret gate (#37) is the sole control** — that makes setting `dataeyond__service__secret` load-bearing rather than merely advisable. Not a finding to re-raise. |
+
 ### Downgraded on evidence
 
 | Ref | Was | Now | Why |

DEV_PLAN.md

@@ -241,7 +241,7 @@ plus the live report bug on analysis `966224d4…`. Same status legend as §0.
 | 37 | **F-2 service-secret gate** — `X-Dataeyond-Service-Secret`, router-level dependency | Rifqi | 🔎 | Code shipped 2026-07-23, **inert until `dataeyond__service__secret` is set**. Go makes no outbound call to Python (verified in the Go source), so the "Go fronts Python" premise in the code comments is not wired and the surface is currently open. **Action: Rifqi sets the secret on the HF Space + the FE/Go caller, then re-verify** |
 | 38 | **F-1 tenant scoping** — `user_id` predicate on the six analysis-keyed reads | Rifqi | ✅ | `CatalogStore.get_by_analysis` filtered on `analysis_id` alone where Go filters on both; the catalog payload carries the owner's `user_id`, so `DbExecutor`'s ownership check compared the victim's id against itself and passed → cross-tenant **query execution against a customer DB**. Defence-in-depth only until #37 is armed (`user_id` is caller-supplied, and `GET /traceability` leaks it) |
 | 39 | **Stale tests resolved** — the two long-standing suite failures | Rifqi | ✅ | Both encoded the pre-2026-07-13 user-scope fallback that `reader.py` deliberately removed. Not product bugs. Suite is now **394 passed / 0 failed / 7 skipped** — fully green for the first time |
-| 40 | **F-3** — scope `GET /charts` + `GET /traceability` by `user_id` | Rifqi ↔ FE | 🔄 | **Python side shipped 2026-07-23; awaiting FE.** Correction to this row's original claim that "store-side scoping is in place": that was true only of `PostgresTraceabilityStore.get`. `PostgresChartStore.list_for_message`/`turn_exists` had **no `user_id` parameter at all** — added now, including on `turn_exists` so another tenant's turn reads `not_found` rather than `empty` (the tri-state must not become an existence probe). Both endpoints take `user_id` as an **optional** query param: supplied ⇒ scoped, omitted ⇒ byte-identical to before, so it needed no FE change to land. Every unscoped call logs `scoped=false` — the same log-only-then-enforce rollout used for the catalog predicate in #38. **Action: FE starts sending `user_id`; when the unscoped log goes quiet, flip both to required** (one line per endpoint). Contract updated with an FE-facing note |
+| 40 | ~~**F-3** — scope `GET /charts` + `GET /traceability` by `user_id`~~ | Rifqi | ⛔ | **RESOLVED 2026-07-23 — DECLINED by Rifqi. Do not re-open without his sign-off.** Both endpoints keep their existing lookup keys: `/traceability` by `(analysis_id, message_id)`, `/charts` by `message_id` alone (the 2026-07-13 lead decision). F-3 proposed adding a `user_id` parameter to both; an optional-param version was implemented on 2026-07-23 and then **reverted in full** (endpoints, charts-store predicates, and the contract notes) once the decision was restated — no FE change is required and none should be requested. **Accepted consequence:** both endpoints remain unauthenticated capability URLs over real customer data (`charts[].spec.plotly.data` is actual table values; the traceability payload carries 5-row previews, the executed SQL, and the owner's `user_id`). **The service-secret gate (#37) is therefore the only control protecting them** — which raises #37 from important to load-bearing. The stores' optional `user_id` parameters are kept as dead capability for a future Go-forwarded identity (#43); `PostgresChartStore` was returned to its message_id-only form. |
 | 41 | **F-12 / F-13** — bound the planner catalog render and the tabular blob read | Rifqi | ✅ | Shipped 2026-07-23. **F-13:** neither storage backend could report an object size, so `object_size()` was added to both (S3 `head_object`→`ContentLength`, Azure `get_blob_properties().size`, each returning None rather than raising) and `TabularExecutor` now refuses >500 MB **before** downloading, with a post-download byte check as the fallback when the probe is unavailable. **F-12:** `render()` gained TWO ceilings — `_MAX_TABLES=150` and `_MAX_CATALOG_CHARS=250_000` — because a table-count cap alone leaves wide tables unbounded (20 tables × 300 cols is as fatal as 400 × 30). Measured after: 400×30 went from ~241k tokens to ~63k; 100×30 (~45k tokens) still renders **in full**, so no realistic catalog is touched. Truncation emits an explicit "N more tables not shown" line so the planner knows it saw a subset, and logs — that log is the signal to retune. 12 new tests |
 | 42 | **F-20 observability** — `degraded_seam=<name>` on every never-throw / silent-drop path | Rifqi | 🔄 | The 2026-07-23 report bug was invisible by construction: the record was dropped with zero logging. **Partially shipped 2026-07-23** — the 10 seams where silent degradation is user-visible now emit a stable `degraded_seam` field (+ `repr(e)` instead of `str(e)`, so an empty-`str()` Fernet error is no longer a blank log): `input_guard_fail_open`, `analysis_catalog_read`, `report_floor_record_read`, `traceability_persist`, `traceability_flush`, `chart_persist` (×2), `report_input_persist` (×2), `analysis_state_ensure`. **Remaining:** the other ~76 `except Exception` sites, most of which are in unwired routers (`db_client`, `data_catalog`, `users`) or non-live paths — deliberately not swept, since a blanket edit across unwired code is exactly the drive-by §7A forbids. Control flow unchanged throughout (§5.4) |
 | 43 | **Go identity contract** — what does Go forward, and when? | Rifqi ↔ Harry | ⬜ new | Needed to replace #37's shared secret with real per-user authorization. Until then the store predicates in #38 are defensive only |

src/api/v1/charts.py

@@ -14,17 +14,7 @@ Every response is HTTP 200 with an explicit `status` marker (lead ask 2026-07-13
                   id, or an error turn — those never write a row). Distinguished
                   from `empty` via the turn's traceability row.
 
-**Tenant scoping (F-3, 2026-07-23).** `user_id` is an OPTIONAL query parameter in this
-rollout: supplied, it scopes both reads to the turn's owner; omitted, the lookup stays
-message_id-only and the response is identical to before, so no FE change is needed to
-keep working. This is deliberately the same log-only-then-enforce pattern used for the
-catalog tenant predicate — every unscoped call logs `scoped=false`, which is the signal
-for when it is safe to make the parameter required.
-
-Until it IS required this endpoint remains a capability URL: `spec.plotly.data` carries
-the customer's real values, so anyone holding a leaked message_id can read them. Making
-`user_id` required is an FE contract change (DEV_PLAN #40); the service-secret gate
-(#37) is the control that matters in the meantime.
+No auth — Go fronts Python.
 """
 
 from typing import Literal
@@ -56,26 +46,10 @@ async def get_charts(
     message_id: str = Query(
         ..., description="Assistant turn id, taken from the `done` SSE event"
     ),
-    user_id: str | None = Query(
-        None,
-        description=(
-            "Owner of the turn. Optional during the F-3 rollout — when supplied the "
-            "lookup is scoped to this user; when omitted the response is unchanged. "
-            "Send it as soon as the FE can; it becomes required (DEV_PLAN #40)."
-        ),
-    ),
 ) -> ChartsResponse:
     """Fetch every chart for one turn. Always 200 — the `status` field carries the
     outcome, so the FE can call this unconditionally on every `done`."""
-    if user_id is None:
-        # The rollout signal: when this stops appearing, the parameter can be
-        # made required without breaking a live caller.
-        logger.warning(
-            "charts read unscoped — no user_id supplied",
-            message_id=message_id,
-            scoped=False,
-        )
-    charts = await _store.list_for_message(message_id, user_id)
+    charts = await _store.list_for_message(message_id)
     if charts:
         return ChartsResponse(
             status="success",
@@ -83,7 +57,7 @@ async def get_charts(
             count=len(charts),
             charts=charts,
         )
-    if await _store.turn_exists(message_id, user_id):
+    if await _store.turn_exists(message_id):
         return ChartsResponse(
             status="empty",
             message="This message completed without producing charts.",

src/api/v1/traceability.py

@@ -7,17 +7,7 @@ chat pipeline right before the `done` SSE event; the FE fires this GET on `done`
 
 Renamed from the contracted `/api/v1/observability` (team decision 2026-07-06) so it is
 never confused with the Langfuse *observability* stack (engineering-only, PII-masked).
-
-**Tenant scoping (F-3, 2026-07-23).** `user_id` is an OPTIONAL query parameter in this
-rollout: supplied, it scopes the read to the turn's owner; omitted, behaviour is exactly
-as before, so no FE change is needed to keep working. Every unscoped call logs
-`scoped=false` — that count is the evidence for when the parameter can safely be made
-required (an FE contract change, DEV_PLAN #40).
-
-Note what an unscoped read exposes: the payload carries 5-row previews of every
-`retrieve_data` result, the executed SQL, AND the owner's `user_id` — which is the id an
-attacker would need to satisfy every other tenant predicate. The service-secret gate
-(#37) is the control that matters until scoping is required.
+No auth — Go fronts Python.
 """
 
 from fastapi import APIRouter, HTTPException, Query
@@ -40,28 +30,11 @@ async def get_traceability(
     message_id: str = Query(
         ..., description="Assistant turn id, taken from the `done` SSE event"
     ),
-    user_id: str | None = Query(
-        None,
-        description=(
-            "Owner of the turn. Optional during the F-3 rollout — when supplied the "
-            "lookup is scoped to this user; when omitted the response is unchanged. "
-            "Send it as soon as the FE can; it becomes required (DEV_PLAN #40)."
-        ),
-    ),
 ) -> TraceabilityPayload:
     """Fetch one turn's provenance record. 404 while the turn is still running or if
     the id is unknown (the FE never gets a `message_id` for error turns, so 404 is
     the correct answer there)."""
-    if user_id is None:
-        # The rollout signal: when this stops appearing, the parameter can be
-        # made required without breaking a live caller.
-        logger.warning(
-            "traceability read unscoped — no user_id supplied",
-            analysis_id=analysis_id,
-            message_id=message_id,
-            scoped=False,
-        )
-    payload = await _store.get(analysis_id, message_id, user_id)
+    payload = await _store.get(analysis_id, message_id)
     if payload is None:
         raise HTTPException(
             status_code=404,

src/charts/store.py

@@ -64,13 +64,9 @@ class ChartStore(Protocol):
         envelope: dict,
     ) -> None: ...
 
-    async def list_for_message(
-        self, message_id: str, user_id: str | None = None
-    ) -> list[ChartRecord]: ...
+    async def list_for_message(self, message_id: str) -> list[ChartRecord]: ...
 
-    async def turn_exists(
-        self, message_id: str, user_id: str | None = None
-    ) -> bool: ...
+    async def turn_exists(self, message_id: str) -> bool: ...
 
 
 class NullChartStore:
@@ -91,12 +87,10 @@ class NullChartStore:
             chart_type=envelope.get("chart_type", "unknown"),
         )
 
-    async def list_for_message(
-        self, message_id: str, user_id: str | None = None
-    ) -> list[ChartRecord]:
+    async def list_for_message(self, message_id: str) -> list[ChartRecord]:
         return []
 
-    async def turn_exists(self, message_id: str, user_id: str | None = None) -> bool:
+    async def turn_exists(self, message_id: str) -> bool:
         return False
 
 
@@ -140,27 +134,15 @@ class PostgresChartStore:
                 error=repr(exc),
             )
 
-    async def list_for_message(
-        self, message_id: str, user_id: str | None = None
-    ) -> list[ChartRecord]:
+    async def list_for_message(self, message_id: str) -> list[ChartRecord]:
         # message_id-only lookup (lead decision 2026-07-13). NOTE for the Harry
         # migration: the manual DDL's composite index (analysis_id, message_id)
         # does not serve this predicate — an additive index on (message_id) is
         # part of the handoff.
-        #
-        # `user_id` scopes the read to the turn's owner (F-3, 2026-07-23). Optional
-        # because `GET /api/v1/charts` does not require the parameter yet — making it
-        # required is an FE contract change. Until the FE sends it, an unscoped read
-        # means the chart data (which is REAL customer values, in `spec.plotly.data`)
-        # is reachable by anyone holding the message_id. See the endpoint's
-        # `scoped=false` log line.
         async with AsyncSessionLocal() as session:
-            where = [MessageChartRow.message_id == message_id]
-            if user_id is not None:
-                where.append(MessageChartRow.user_id == user_id)
             result = await session.execute(
                 select(MessageChartRow)
-                .where(*where)
+                .where(MessageChartRow.message_id == message_id)
                 .order_by(MessageChartRow.created_at)
             )
             rows = result.scalars().all()
@@ -175,22 +157,17 @@ class PostgresChartStore:
             for row in rows
         ]
 
-    async def turn_exists(self, message_id: str, user_id: str | None = None) -> bool:
+    async def turn_exists(self, message_id: str) -> bool:
         """True iff the turn flushed its traceability row (written before `done`).
 
         Lets the endpoint tri-state a zero-chart GET: `empty` (completed turn, no
         charts — the common case) vs `not_found` (unknown/mistyped id, or an error
         turn, which never writes traceability). PK lookup — cheap.
-
-        Scoped by `user_id` when the caller supplies one (F-3, 2026-07-23), so another
-        tenant's turn reads as `not_found` rather than `empty` — the tri-state must not
-        become a probe that confirms a message_id exists.
         """
         async with AsyncSessionLocal() as session:
-            where = [MessageTraceabilityRow.message_id == message_id]
-            if user_id is not None:
-                where.append(MessageTraceabilityRow.user_id == user_id)
             result = await session.execute(
-                select(MessageTraceabilityRow.message_id).where(*where)
+                select(MessageTraceabilityRow.message_id).where(
+                    MessageTraceabilityRow.message_id == message_id
+                )
             )
             return result.scalar_one_or_none() is not None

src/traceability/store.py

@@ -100,12 +100,23 @@ class PostgresTraceabilityStore:
     ) -> TraceabilityPayload | None:
         """One turn's payload, or None on a miss.
 
-        `user_id` scopes the read to the turn's owner (2026-07-23). Optional because
-        `GET /api/v1/traceability` does not yet accept the parameter — adding it is an
-        FE contract change. Until then this row is reachable by `(analysis_id,
-        message_id)` alone, and the payload itself carries `user_id`, so the endpoint
-        also hands an attacker the id needed to pass every other scoped read. That is
-        why the service-secret gate, not this predicate, is the control that matters.
+        `user_id` scopes the read to the turn's owner when a caller supplies one.
+
+        **It stays unused by the endpoint — lead decision, reaffirmed 2026-07-23.**
+        `GET /api/v1/traceability` looks up by `(analysis_id, message_id)` alone, and
+        `GET /api/v1/charts` by `message_id` alone (the 2026-07-13 decision). Review
+        finding F-3 proposed adding the parameter to both; that was **declined**, so
+        do not "finish" this by wiring it into the endpoint — reopening it needs the
+        same sign-off path as any other locked decision.
+
+        The consequence is deliberate and accepted: this row is reachable by
+        `(analysis_id, message_id)` alone, and the payload itself carries `user_id`.
+        **The service-secret gate is the control that protects these endpoints**, not
+        this predicate — which is why `dataeyond__service__secret` being set matters
+        more than it would otherwise (DEV_PLAN #37).
+
+        The parameter is kept because it costs nothing and a future Go-forwarded
+        identity (#43) would use it.
         """
         async with AsyncSessionLocal() as session:
             where = [