[NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams
Browse filesDEV_PLAN #40, #41, #42.
#41 F-13 — TabularExecutor buffered an entire Parquet blob and read_parquet'd it with
no size check. Filtering and the 10k row cap both happen after the frame exists, so
they bound the RESULT, never the working set. An OOM here is the one failure in this
service that escapes every never-throw seam: the process dies, taking every
concurrent request with it. Neither storage backend could report an object size, so
`object_size()` is added to both (S3 head_object -> ContentLength, Azure
get_blob_properties().size), each returning None rather than raising. The executor now
refuses >500 MB BEFORE downloading, with a post-download byte check as the fallback
when the probe is unavailable — that one cannot prevent the download, but it does stop
read_parquet from multiplying the bytes into a frame several times larger.
#41 F-12 — CatalogSummary.render() had no truncation at all; 400 tables x 30 cols
measured ~241k tokens per planner call, x3 retries. Two ceilings now, whichever binds
first: _MAX_TABLES=150 and _MAX_CATALOG_CHARS=250_000. A table-count cap alone is not
enough — 20 tables x 300 columns is as fatal as 400 x 30, and wide fact tables are
common. Measured after: 400x30 drops from ~241k to ~63k tokens, while 100x30 (~45k)
still renders IN FULL, so no realistic catalog is affected. These are safety nets, not
tight caps: there is no relevance ordering here, so a low cap could drop the very table
the question is about. Truncation emits an explicit "N more tables not shown" line so
the planner knows it saw a subset rather than confidently claiming a table is absent,
and logs — that log is what tells us when a real customer approaches the ceiling.
#40 F-3 — correcting this row's own claim that "store-side scoping is in place": that
was true only of PostgresTraceabilityStore.get. PostgresChartStore.list_for_message
and turn_exists had NO user_id parameter at all. Both now take one, including
turn_exists so another tenant's turn reads not_found rather than empty — the tri-state
must not become a probe confirming a message_id exists. Both endpoints accept user_id
as an OPTIONAL query param: supplied => scoped, omitted => byte-identical to before.
That is what lets the Python half land without waiting on the FE. Every unscoped call
logs scoped=false; when that goes quiet the flip to required is one line per endpoint.
Same log-only-then-enforce rollout used for the catalog predicate in #38.
#42 F-20 — the 2026-07-23 report bug was invisible by construction. Ten live seams
where silent degradation is user-visible now carry a stable `degraded_seam` field, so
a dashboard can count them: input_guard_fail_open, analysis_catalog_read,
report_floor_record_read, traceability_persist, traceability_flush, chart_persist (x2),
report_input_persist (x2), analysis_state_ensure. Each also moves str(e) -> repr(e), so
an empty-str() Fernet InvalidToken is no longer a blank log line. Control flow is
unchanged throughout (CLAUDE.md 5.4). The remaining ~76 except-sites are mostly in
unwired routers and were deliberately not swept — a blanket edit across unwired code is
the drive-by 7A forbids.
Docs: contract gains FE-facing tenant-scoping notes on both endpoints (optional now,
becoming required, with why it matters most for /charts — spec.plotly.data is real
customer values). DEV_PLAN #40 -> in progress, #41 -> done, #42 -> in progress, plus
new rows #44 (the F-17/F-18/F-25 batch) and #45 (F-9 PII in persisted artifacts, which
was decided 2026-07-23 but never tracked).
Verification: 12 new tests for the two ceilings. Suite 424 passed / 0 failed / 7
skipped. Ruff on touched paths identical to HEAD (3 pre-existing). `import main` OK.
Readiness eval 17/17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- API_CONTRACT_BE_PYTHON.md +27 -3
- DEV_PLAN.md +5 -3
- eval/readiness/results/readiness_result_2026-07-23_150632.json +263 -0
- eval/readiness/results/readiness_result_2026-07-23_150859.json +250 -0
- eval/readiness/results/readiness_result_2026-07-23_150948.json +274 -0
- src/agents/chat_handler.py +21 -4
- src/agents/guard.py +8 -1
- src/agents/planner/inputs.py +68 -1
- src/agents/report/readiness.py +4 -1
- src/agents/slow_path/store.py +2 -1
- src/api/v1/charts.py +29 -3
- src/api/v1/traceability.py +29 -2
- src/catalog/reader.py +4 -1
- src/charts/store.py +35 -11
- src/query/executor/tabular.py +83 -0
- src/storage/az_blob/az_blob.py +19 -0
- src/storage/object_storage/supabase_s3.py +23 -0
- src/traceability/store.py +2 -1
|
@@ -487,11 +487,21 @@ Query params:
|
|
| 487 |
| --- | --- | --- |
|
| 488 |
| `analysis_id` | Yes | Analysis identifier. |
|
| 489 |
| `message_id` | Yes | Assistant answer identifier returned by the stream. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 490 |
|
| 491 |
Example:
|
| 492 |
|
| 493 |
```text
|
| 494 |
-
GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1
|
| 495 |
```
|
| 496 |
|
| 497 |
`intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or Azure content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
|
|
@@ -694,11 +704,25 @@ Query params:
|
|
| 694 |
| Query | Required | Description |
|
| 695 |
| --- | --- | --- |
|
| 696 |
| `message_id` | Yes | Assistant answer identifier returned by the stream's `done` event. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 697 |
|
| 698 |
Example:
|
| 699 |
|
| 700 |
```text
|
| 701 |
-
GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2
|
| 702 |
```
|
| 703 |
|
| 704 |
The `dataeyond.chart.v1` envelope (the shape of `charts[].spec`, verbatim from `render_chart`):
|
|
@@ -775,7 +799,7 @@ Field rules:
|
|
| 775 |
- `spec` is the full `dataeyond.chart.v1` envelope, unmodified — it is the source of truth, not a projection; render straight from it.
|
| 776 |
- `chart_type` / `title` are copied out of `spec` for convenience (list rendering without parsing `spec`); `title` may be `null`.
|
| 777 |
- A turn can produce more than one chart (multiple `render_chart` calls in the same plan); `charts` is ordered by creation time.
|
| 778 |
-
- The payload carries no `user_id` / `analysis_id`
|
| 779 |
|
| 780 |
> **DDL note (Harry / dedorch migration):** the original manual index is `(analysis_id, message_id)`, which does not serve a `message_id`-only lookup. Additive index for the migration (also safe to run manually now):
|
| 781 |
> ```sql
|
|
|
|
| 487 |
| --- | --- | --- |
|
| 488 |
| `analysis_id` | Yes | Analysis identifier. |
|
| 489 |
| `message_id` | Yes | Assistant answer identifier returned by the stream. |
|
| 490 |
+
| `user_id` | **No — becoming Yes** | Owner of the turn. See the tenant-scoping note below. |
|
| 491 |
+
|
| 492 |
+
> **Tenant scoping — added 2026-07-23, FE action requested.** `user_id` is **optional
|
| 493 |
+
> today**: send it and the lookup is scoped to the turn's owner; omit it and the
|
| 494 |
+
> response is byte-identical to before, so nothing breaks by not changing yet.
|
| 495 |
+
> **It will become required.** Please start sending it whenever the FE has it —
|
| 496 |
+
> the server logs every unscoped call, and that count is what tells us when the flip
|
| 497 |
+
> is safe. Until then this endpoint is an unauthenticated capability URL over real
|
| 498 |
+
> customer data: the payload carries 5-row previews of every retrieved result, the
|
| 499 |
+
> executed SQL, and the owner's `user_id`.
|
| 500 |
|
| 501 |
Example:
|
| 502 |
|
| 503 |
```text
|
| 504 |
+
GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1&user_id=usr_7
|
| 505 |
```
|
| 506 |
|
| 507 |
`intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or Azure content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
|
|
|
|
| 704 |
| Query | Required | Description |
|
| 705 |
| --- | --- | --- |
|
| 706 |
| `message_id` | Yes | Assistant answer identifier returned by the stream's `done` event. |
|
| 707 |
+
| `user_id` | **No — becoming Yes** | Owner of the turn. See the tenant-scoping note below. |
|
| 708 |
+
|
| 709 |
+
> **Tenant scoping — added 2026-07-23, FE action requested.** `user_id` is **optional
|
| 710 |
+
> today**: send it and both the chart lookup and the `empty`/`not_found` check are
|
| 711 |
+
> scoped to the turn's owner; omit it and the response is byte-identical to before, so
|
| 712 |
+
> nothing breaks by not changing yet. **It will become required.** Please start sending
|
| 713 |
+
> it whenever the FE has it — the server logs every unscoped call, and that count is
|
| 714 |
+
> what tells us when the flip is safe.
|
| 715 |
+
>
|
| 716 |
+
> Why this matters more here than elsewhere: `charts[].spec.plotly.data` contains the
|
| 717 |
+
> **actual values from the customer's tables**, and the only thing currently protecting
|
| 718 |
+
> it is that `message_id` is a UUID4. A leaked id — an error report, a shared screenshot
|
| 719 |
+
> of a network tab, a support ticket — exposes that data indefinitely, with no expiry
|
| 720 |
+
> and no ownership check.
|
| 721 |
|
| 722 |
Example:
|
| 723 |
|
| 724 |
```text
|
| 725 |
+
GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2&user_id=usr_7
|
| 726 |
```
|
| 727 |
|
| 728 |
The `dataeyond.chart.v1` envelope (the shape of `charts[].spec`, verbatim from `render_chart`):
|
|
|
|
| 799 |
- `spec` is the full `dataeyond.chart.v1` envelope, unmodified — it is the source of truth, not a projection; render straight from it.
|
| 800 |
- `chart_type` / `title` are copied out of `spec` for convenience (list rendering without parsing `spec`); `title` may be `null`.
|
| 801 |
- A turn can produce more than one chart (multiple `render_chart` calls in the same plan); `charts` is ordered by creation time.
|
| 802 |
+
- The response payload carries no `user_id` / `analysis_id`. The *lookup* accepts an optional `user_id` (see the tenant-scoping note above); when supplied, another tenant's turn reads as `not_found` rather than `empty`, so the tri-state cannot be used to probe whether a `message_id` exists.
|
| 803 |
|
| 804 |
> **DDL note (Harry / dedorch migration):** the original manual index is `(analysis_id, message_id)`, which does not serve a `message_id`-only lookup. Additive index for the migration (also safe to run manually now):
|
| 805 |
> ```sql
|
|
@@ -241,10 +241,12 @@ plus the live report bug on analysis `966224d4…`. Same status legend as §0.
|
|
| 241 |
| 37 | **F-2 service-secret gate** — `X-Dataeyond-Service-Secret`, router-level dependency | Rifqi | 🔎 | Code shipped 2026-07-23, **inert until `dataeyond__service__secret` is set**. Go makes no outbound call to Python (verified in the Go source), so the "Go fronts Python" premise in the code comments is not wired and the surface is currently open. **Action: Rifqi sets the secret on the HF Space + the FE/Go caller, then re-verify** |
|
| 242 |
| 38 | **F-1 tenant scoping** — `user_id` predicate on the six analysis-keyed reads | Rifqi | ✅ | `CatalogStore.get_by_analysis` filtered on `analysis_id` alone where Go filters on both; the catalog payload carries the owner's `user_id`, so `DbExecutor`'s ownership check compared the victim's id against itself and passed → cross-tenant **query execution against a customer DB**. Defence-in-depth only until #37 is armed (`user_id` is caller-supplied, and `GET /traceability` leaks it) |
|
| 243 |
| 39 | **Stale tests resolved** — the two long-standing suite failures | Rifqi | ✅ | Both encoded the pre-2026-07-13 user-scope fallback that `reader.py` deliberately removed. Not product bugs. Suite is now **394 passed / 0 failed / 7 skipped** — fully green for the first time |
|
| 244 |
-
| 40 | **F-3** — scope `GET /charts` + `GET /traceability` by `user_id` | Rifqi ↔ FE |
|
| 245 |
-
| 41 | **F-12 / F-13** — bound the planner catalog render and the tabular blob read | Rifqi |
|
| 246 |
-
| 42 | **F-20 observability** — `degraded_seam=<name>` on every never-throw / silent-drop path | Rifqi |
|
| 247 |
| 43 | **Go identity contract** — what does Go forward, and when? | Rifqi ↔ Harry | ⬜ new | Needed to replace #37's shared secret with real per-user authorization. Until then the store predicates in #38 are defensive only |
|
|
|
|
|
|
|
| 248 |
|
| 249 |
**Not re-raised:** F-4 (non-Postgres read-only/timeout gap) was **downgraded to latent** —
|
| 250 |
Go's `database_clients.Service.Create` enforces `isSupportedActive`, and only `postgres` is
|
|
|
|
| 241 |
| 37 | **F-2 service-secret gate** — `X-Dataeyond-Service-Secret`, router-level dependency | Rifqi | 🔎 | Code shipped 2026-07-23, **inert until `dataeyond__service__secret` is set**. Go makes no outbound call to Python (verified in the Go source), so the "Go fronts Python" premise in the code comments is not wired and the surface is currently open. **Action: Rifqi sets the secret on the HF Space + the FE/Go caller, then re-verify** |
|
| 242 |
| 38 | **F-1 tenant scoping** — `user_id` predicate on the six analysis-keyed reads | Rifqi | ✅ | `CatalogStore.get_by_analysis` filtered on `analysis_id` alone where Go filters on both; the catalog payload carries the owner's `user_id`, so `DbExecutor`'s ownership check compared the victim's id against itself and passed → cross-tenant **query execution against a customer DB**. Defence-in-depth only until #37 is armed (`user_id` is caller-supplied, and `GET /traceability` leaks it) |
|
| 243 |
| 39 | **Stale tests resolved** — the two long-standing suite failures | Rifqi | ✅ | Both encoded the pre-2026-07-13 user-scope fallback that `reader.py` deliberately removed. Not product bugs. Suite is now **394 passed / 0 failed / 7 skipped** — fully green for the first time |
|
| 244 |
+
| 40 | **F-3** — scope `GET /charts` + `GET /traceability` by `user_id` | Rifqi ↔ FE | 🔄 | **Python side shipped 2026-07-23; awaiting FE.** Correction to this row's original claim that "store-side scoping is in place": that was true only of `PostgresTraceabilityStore.get`. `PostgresChartStore.list_for_message`/`turn_exists` had **no `user_id` parameter at all** — added now, including on `turn_exists` so another tenant's turn reads `not_found` rather than `empty` (the tri-state must not become an existence probe). Both endpoints take `user_id` as an **optional** query param: supplied ⇒ scoped, omitted ⇒ byte-identical to before, so it needed no FE change to land. Every unscoped call logs `scoped=false` — the same log-only-then-enforce rollout used for the catalog predicate in #38. **Action: FE starts sending `user_id`; when the unscoped log goes quiet, flip both to required** (one line per endpoint). Contract updated with an FE-facing note |
|
| 245 |
+
| 41 | **F-12 / F-13** — bound the planner catalog render and the tabular blob read | Rifqi | ✅ | Shipped 2026-07-23. **F-13:** neither storage backend could report an object size, so `object_size()` was added to both (S3 `head_object`→`ContentLength`, Azure `get_blob_properties().size`, each returning None rather than raising) and `TabularExecutor` now refuses >500 MB **before** downloading, with a post-download byte check as the fallback when the probe is unavailable. **F-12:** `render()` gained TWO ceilings — `_MAX_TABLES=150` and `_MAX_CATALOG_CHARS=250_000` — because a table-count cap alone leaves wide tables unbounded (20 tables × 300 cols is as fatal as 400 × 30). Measured after: 400×30 went from ~241k tokens to ~63k; 100×30 (~45k tokens) still renders **in full**, so no realistic catalog is touched. Truncation emits an explicit "N more tables not shown" line so the planner knows it saw a subset, and logs — that log is the signal to retune. 12 new tests |
|
| 246 |
+
| 42 | **F-20 observability** — `degraded_seam=<name>` on every never-throw / silent-drop path | Rifqi | 🔄 | The 2026-07-23 report bug was invisible by construction: the record was dropped with zero logging. **Partially shipped 2026-07-23** — the 10 seams where silent degradation is user-visible now emit a stable `degraded_seam` field (+ `repr(e)` instead of `str(e)`, so an empty-`str()` Fernet error is no longer a blank log): `input_guard_fail_open`, `analysis_catalog_read`, `report_floor_record_read`, `traceability_persist`, `traceability_flush`, `chart_persist` (×2), `report_input_persist` (×2), `analysis_state_ensure`. **Remaining:** the other ~76 `except Exception` sites, most of which are in unwired routers (`db_client`, `data_catalog`, `users`) or non-live paths — deliberately not swept, since a blanket edit across unwired code is exactly the drive-by §7A forbids. Control flow unchanged throughout (§5.4) |
|
| 247 |
| 43 | **Go identity contract** — what does Go forward, and when? | Rifqi ↔ Harry | ⬜ new | Needed to replace #37's shared secret with real per-user authorization. Until then the store predicates in #38 are defensive only |
|
| 248 |
+
| 44 | **F-17 / F-18 / F-25 compiler-parity batch** | Rifqi | ✅ new | Shipped 2026-07-23. Three execution-verified review findings that had **no task row** — the tracker jumped from #39 to #40 and lost them. **F-17 (High):** the bare-select check was gated on `if ir.group_by`, so a mixed select with `group_by=[]` passed; Postgres then failed loudly but the pandas path silently DROPPED the column while `output_columns` still advertised it → a real-looking table with a fabricated all-null column. Fixed in `validator.py` (fires whenever any agg is present — no false positives possible) + a presence backstop in `TabularExecutor`. **F-18:** `astype(str)` ran before `na=False`, so `NULL LIKE '%an%'` matched the literal `"nan"`/`"None"`. **F-25:** `SqlCompiler` raised on an empty `in`/`not_in` where pandas and `_column_values`' own docstring implement the empty-set semantics — a two-step plan whose first step legitimately returned zero rows hard-failed instead of answering. 16 new parity tests; one stale test updated (it pinned the old F-25 raise) |
|
| 249 |
+
| 45 | **F-9 PII in persisted artifacts** — mask the traceability preview + report evidence tables | Rifqi | ⬜ new | **Decided 2026-07-23** (CODE_REVIEW §7) but never tracked. `pii_flag` is an ingestion-time control only: nothing stops the planner SELECTing a flagged column, and the real values then land in `message_traceability.data` (served by a GET that is only now becoming scoped, #40) and are frozen permanently into `reports.content`. Decision: **the assembler still sees values** (otherwise "list our top customers" stops being answerable); mask only the persisted artifacts. Carry `pii_flag` onto `retrieve_data`'s output meta — the catalog is already in scope at `data_access.py:232` |
|
| 250 |
|
| 251 |
**Not re-raised:** F-4 (non-Postgres read-only/timeout gap) was **downgraded to latent** —
|
| 252 |
Go's `database_clients.Service.Create` enforces `isSupportedActive`, and only `postgres` is
|
|
@@ -0,0 +1,263 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"run": {
|
| 3 |
+
"timestamp": "2026-07-23T15:06:32",
|
| 4 |
+
"dataset": "readiness_dataset.json",
|
| 5 |
+
"target": "src/agents/report/readiness.is_report_ready",
|
| 6 |
+
"total": 15,
|
| 7 |
+
"passed": 4,
|
| 8 |
+
"accuracy": 0.267,
|
| 9 |
+
"runtime_avg_ms": 0.11
|
| 10 |
+
},
|
| 11 |
+
"alignment_gap": {
|
| 12 |
+
"count": 0,
|
| 13 |
+
"ids": []
|
| 14 |
+
},
|
| 15 |
+
"by_group": {
|
| 16 |
+
"floor": {
|
| 17 |
+
"n": 6,
|
| 18 |
+
"passed": 3,
|
| 19 |
+
"accuracy": 0.5
|
| 20 |
+
},
|
| 21 |
+
"delta": {
|
| 22 |
+
"n": 5,
|
| 23 |
+
"passed": 0,
|
| 24 |
+
"accuracy": 0.0
|
| 25 |
+
},
|
| 26 |
+
"edge": {
|
| 27 |
+
"n": 1,
|
| 28 |
+
"passed": 1,
|
| 29 |
+
"accuracy": 1.0
|
| 30 |
+
},
|
| 31 |
+
"alignment": {
|
| 32 |
+
"n": 3,
|
| 33 |
+
"passed": 0,
|
| 34 |
+
"accuracy": 0.0
|
| 35 |
+
}
|
| 36 |
+
},
|
| 37 |
+
"cases": [
|
| 38 |
+
{
|
| 39 |
+
"id": "floor_01",
|
| 40 |
+
"group": "floor",
|
| 41 |
+
"expected_ready": false,
|
| 42 |
+
"got_ready": false,
|
| 43 |
+
"expected_missing": [
|
| 44 |
+
"at least one completed analysis"
|
| 45 |
+
],
|
| 46 |
+
"got_missing": [
|
| 47 |
+
"at least one completed analysis"
|
| 48 |
+
],
|
| 49 |
+
"correct": true,
|
| 50 |
+
"aligned": false,
|
| 51 |
+
"gap": false,
|
| 52 |
+
"latency_ms": 0.3
|
| 53 |
+
},
|
| 54 |
+
{
|
| 55 |
+
"id": "floor_02",
|
| 56 |
+
"group": "floor",
|
| 57 |
+
"expected_ready": false,
|
| 58 |
+
"got_ready": false,
|
| 59 |
+
"expected_missing": [
|
| 60 |
+
"at least one completed analysis"
|
| 61 |
+
],
|
| 62 |
+
"got_missing": [
|
| 63 |
+
"at least one completed analysis"
|
| 64 |
+
],
|
| 65 |
+
"correct": true,
|
| 66 |
+
"aligned": false,
|
| 67 |
+
"gap": false,
|
| 68 |
+
"latency_ms": 0.1
|
| 69 |
+
},
|
| 70 |
+
{
|
| 71 |
+
"id": "floor_03",
|
| 72 |
+
"group": "floor",
|
| 73 |
+
"expected_ready": false,
|
| 74 |
+
"got_ready": false,
|
| 75 |
+
"expected_missing": [
|
| 76 |
+
"at least one completed analysis"
|
| 77 |
+
],
|
| 78 |
+
"got_missing": [
|
| 79 |
+
"at least one completed analysis"
|
| 80 |
+
],
|
| 81 |
+
"correct": true,
|
| 82 |
+
"aligned": false,
|
| 83 |
+
"gap": false,
|
| 84 |
+
"latency_ms": 0.1
|
| 85 |
+
},
|
| 86 |
+
{
|
| 87 |
+
"id": "floor_04",
|
| 88 |
+
"group": "floor",
|
| 89 |
+
"expected_ready": true,
|
| 90 |
+
"got_ready": false,
|
| 91 |
+
"expected_missing": [],
|
| 92 |
+
"got_missing": [
|
| 93 |
+
"at least one completed analysis"
|
| 94 |
+
],
|
| 95 |
+
"correct": false,
|
| 96 |
+
"aligned": true,
|
| 97 |
+
"gap": false,
|
| 98 |
+
"latency_ms": 0.1
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"id": "floor_05",
|
| 102 |
+
"group": "floor",
|
| 103 |
+
"expected_ready": true,
|
| 104 |
+
"got_ready": false,
|
| 105 |
+
"expected_missing": [],
|
| 106 |
+
"got_missing": [
|
| 107 |
+
"at least one completed analysis"
|
| 108 |
+
],
|
| 109 |
+
"correct": false,
|
| 110 |
+
"aligned": true,
|
| 111 |
+
"gap": false,
|
| 112 |
+
"latency_ms": 0.1
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"id": "floor_06",
|
| 116 |
+
"group": "floor",
|
| 117 |
+
"expected_ready": true,
|
| 118 |
+
"got_ready": false,
|
| 119 |
+
"expected_missing": [],
|
| 120 |
+
"got_missing": [
|
| 121 |
+
"at least one completed analysis"
|
| 122 |
+
],
|
| 123 |
+
"correct": false,
|
| 124 |
+
"aligned": true,
|
| 125 |
+
"gap": false,
|
| 126 |
+
"latency_ms": 0.1
|
| 127 |
+
},
|
| 128 |
+
{
|
| 129 |
+
"id": "delta_01",
|
| 130 |
+
"group": "delta",
|
| 131 |
+
"expected_ready": false,
|
| 132 |
+
"got_ready": false,
|
| 133 |
+
"expected_missing": [
|
| 134 |
+
"a new analysis since the last report"
|
| 135 |
+
],
|
| 136 |
+
"got_missing": [
|
| 137 |
+
"at least one completed analysis"
|
| 138 |
+
],
|
| 139 |
+
"correct": false,
|
| 140 |
+
"aligned": true,
|
| 141 |
+
"gap": false,
|
| 142 |
+
"latency_ms": 0.1
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"id": "delta_02",
|
| 146 |
+
"group": "delta",
|
| 147 |
+
"expected_ready": true,
|
| 148 |
+
"got_ready": false,
|
| 149 |
+
"expected_missing": [],
|
| 150 |
+
"got_missing": [
|
| 151 |
+
"at least one completed analysis"
|
| 152 |
+
],
|
| 153 |
+
"correct": false,
|
| 154 |
+
"aligned": true,
|
| 155 |
+
"gap": false,
|
| 156 |
+
"latency_ms": 0.1
|
| 157 |
+
},
|
| 158 |
+
{
|
| 159 |
+
"id": "delta_03",
|
| 160 |
+
"group": "delta",
|
| 161 |
+
"expected_ready": true,
|
| 162 |
+
"got_ready": false,
|
| 163 |
+
"expected_missing": [],
|
| 164 |
+
"got_missing": [
|
| 165 |
+
"at least one completed analysis"
|
| 166 |
+
],
|
| 167 |
+
"correct": false,
|
| 168 |
+
"aligned": true,
|
| 169 |
+
"gap": false,
|
| 170 |
+
"latency_ms": 0.1
|
| 171 |
+
},
|
| 172 |
+
{
|
| 173 |
+
"id": "delta_04",
|
| 174 |
+
"group": "delta",
|
| 175 |
+
"expected_ready": false,
|
| 176 |
+
"got_ready": false,
|
| 177 |
+
"expected_missing": [
|
| 178 |
+
"a new analysis since the last report"
|
| 179 |
+
],
|
| 180 |
+
"got_missing": [
|
| 181 |
+
"at least one completed analysis"
|
| 182 |
+
],
|
| 183 |
+
"correct": false,
|
| 184 |
+
"aligned": true,
|
| 185 |
+
"gap": false,
|
| 186 |
+
"latency_ms": 0.1
|
| 187 |
+
},
|
| 188 |
+
{
|
| 189 |
+
"id": "delta_05",
|
| 190 |
+
"group": "delta",
|
| 191 |
+
"expected_ready": false,
|
| 192 |
+
"got_ready": false,
|
| 193 |
+
"expected_missing": [
|
| 194 |
+
"a new analysis since the last report"
|
| 195 |
+
],
|
| 196 |
+
"got_missing": [
|
| 197 |
+
"at least one completed analysis"
|
| 198 |
+
],
|
| 199 |
+
"correct": false,
|
| 200 |
+
"aligned": true,
|
| 201 |
+
"gap": false,
|
| 202 |
+
"latency_ms": 0.1
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"id": "edge_01",
|
| 206 |
+
"group": "edge",
|
| 207 |
+
"expected_ready": false,
|
| 208 |
+
"got_ready": false,
|
| 209 |
+
"expected_missing": [
|
| 210 |
+
"at least one completed analysis"
|
| 211 |
+
],
|
| 212 |
+
"got_missing": [
|
| 213 |
+
"at least one completed analysis"
|
| 214 |
+
],
|
| 215 |
+
"correct": true,
|
| 216 |
+
"aligned": false,
|
| 217 |
+
"gap": false,
|
| 218 |
+
"latency_ms": 0.1
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"id": "align_01",
|
| 222 |
+
"group": "alignment",
|
| 223 |
+
"expected_ready": true,
|
| 224 |
+
"got_ready": false,
|
| 225 |
+
"expected_missing": [],
|
| 226 |
+
"got_missing": [
|
| 227 |
+
"at least one completed analysis"
|
| 228 |
+
],
|
| 229 |
+
"correct": false,
|
| 230 |
+
"aligned": false,
|
| 231 |
+
"gap": false,
|
| 232 |
+
"latency_ms": 0.1
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"id": "align_02",
|
| 236 |
+
"group": "alignment",
|
| 237 |
+
"expected_ready": true,
|
| 238 |
+
"got_ready": false,
|
| 239 |
+
"expected_missing": [],
|
| 240 |
+
"got_missing": [
|
| 241 |
+
"at least one completed analysis"
|
| 242 |
+
],
|
| 243 |
+
"correct": false,
|
| 244 |
+
"aligned": false,
|
| 245 |
+
"gap": false,
|
| 246 |
+
"latency_ms": 0.1
|
| 247 |
+
},
|
| 248 |
+
{
|
| 249 |
+
"id": "align_03",
|
| 250 |
+
"group": "alignment",
|
| 251 |
+
"expected_ready": true,
|
| 252 |
+
"got_ready": false,
|
| 253 |
+
"expected_missing": [],
|
| 254 |
+
"got_missing": [
|
| 255 |
+
"at least one completed analysis"
|
| 256 |
+
],
|
| 257 |
+
"correct": false,
|
| 258 |
+
"aligned": true,
|
| 259 |
+
"gap": false,
|
| 260 |
+
"latency_ms": 0.1
|
| 261 |
+
}
|
| 262 |
+
]
|
| 263 |
+
}
|
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"run": {
|
| 3 |
+
"timestamp": "2026-07-23T15:08:59",
|
| 4 |
+
"dataset": "readiness_dataset.json",
|
| 5 |
+
"target": "src/agents/report/readiness.is_report_ready",
|
| 6 |
+
"total": 15,
|
| 7 |
+
"passed": 15,
|
| 8 |
+
"accuracy": 1.0,
|
| 9 |
+
"runtime_avg_ms": 0.0
|
| 10 |
+
},
|
| 11 |
+
"alignment_gap": {
|
| 12 |
+
"count": 2,
|
| 13 |
+
"ids": [
|
| 14 |
+
"align_01",
|
| 15 |
+
"align_02"
|
| 16 |
+
]
|
| 17 |
+
},
|
| 18 |
+
"by_group": {
|
| 19 |
+
"floor": {
|
| 20 |
+
"n": 6,
|
| 21 |
+
"passed": 6,
|
| 22 |
+
"accuracy": 1.0
|
| 23 |
+
},
|
| 24 |
+
"delta": {
|
| 25 |
+
"n": 5,
|
| 26 |
+
"passed": 5,
|
| 27 |
+
"accuracy": 1.0
|
| 28 |
+
},
|
| 29 |
+
"edge": {
|
| 30 |
+
"n": 1,
|
| 31 |
+
"passed": 1,
|
| 32 |
+
"accuracy": 1.0
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"n": 3,
|
| 36 |
+
"passed": 3,
|
| 37 |
+
"accuracy": 1.0
|
| 38 |
+
}
|
| 39 |
+
},
|
| 40 |
+
"cases": [
|
| 41 |
+
{
|
| 42 |
+
"id": "floor_01",
|
| 43 |
+
"group": "floor",
|
| 44 |
+
"expected_ready": false,
|
| 45 |
+
"got_ready": false,
|
| 46 |
+
"expected_missing": [
|
| 47 |
+
"at least one completed analysis"
|
| 48 |
+
],
|
| 49 |
+
"got_missing": [
|
| 50 |
+
"at least one completed analysis"
|
| 51 |
+
],
|
| 52 |
+
"correct": true,
|
| 53 |
+
"aligned": false,
|
| 54 |
+
"gap": false,
|
| 55 |
+
"latency_ms": 0.0
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"id": "floor_02",
|
| 59 |
+
"group": "floor",
|
| 60 |
+
"expected_ready": false,
|
| 61 |
+
"got_ready": false,
|
| 62 |
+
"expected_missing": [
|
| 63 |
+
"at least one completed analysis"
|
| 64 |
+
],
|
| 65 |
+
"got_missing": [
|
| 66 |
+
"at least one completed analysis"
|
| 67 |
+
],
|
| 68 |
+
"correct": true,
|
| 69 |
+
"aligned": false,
|
| 70 |
+
"gap": false,
|
| 71 |
+
"latency_ms": 0.0
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"id": "floor_03",
|
| 75 |
+
"group": "floor",
|
| 76 |
+
"expected_ready": false,
|
| 77 |
+
"got_ready": false,
|
| 78 |
+
"expected_missing": [
|
| 79 |
+
"at least one completed analysis"
|
| 80 |
+
],
|
| 81 |
+
"got_missing": [
|
| 82 |
+
"at least one completed analysis"
|
| 83 |
+
],
|
| 84 |
+
"correct": true,
|
| 85 |
+
"aligned": false,
|
| 86 |
+
"gap": false,
|
| 87 |
+
"latency_ms": 0.0
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"id": "floor_04",
|
| 91 |
+
"group": "floor",
|
| 92 |
+
"expected_ready": true,
|
| 93 |
+
"got_ready": true,
|
| 94 |
+
"expected_missing": [],
|
| 95 |
+
"got_missing": [],
|
| 96 |
+
"correct": true,
|
| 97 |
+
"aligned": true,
|
| 98 |
+
"gap": false,
|
| 99 |
+
"latency_ms": 0.0
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"id": "floor_05",
|
| 103 |
+
"group": "floor",
|
| 104 |
+
"expected_ready": true,
|
| 105 |
+
"got_ready": true,
|
| 106 |
+
"expected_missing": [],
|
| 107 |
+
"got_missing": [],
|
| 108 |
+
"correct": true,
|
| 109 |
+
"aligned": true,
|
| 110 |
+
"gap": false,
|
| 111 |
+
"latency_ms": 0.0
|
| 112 |
+
},
|
| 113 |
+
{
|
| 114 |
+
"id": "floor_06",
|
| 115 |
+
"group": "floor",
|
| 116 |
+
"expected_ready": true,
|
| 117 |
+
"got_ready": true,
|
| 118 |
+
"expected_missing": [],
|
| 119 |
+
"got_missing": [],
|
| 120 |
+
"correct": true,
|
| 121 |
+
"aligned": true,
|
| 122 |
+
"gap": false,
|
| 123 |
+
"latency_ms": 0.0
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"id": "delta_01",
|
| 127 |
+
"group": "delta",
|
| 128 |
+
"expected_ready": false,
|
| 129 |
+
"got_ready": false,
|
| 130 |
+
"expected_missing": [
|
| 131 |
+
"a new analysis since the last report"
|
| 132 |
+
],
|
| 133 |
+
"got_missing": [
|
| 134 |
+
"a new analysis since the last report"
|
| 135 |
+
],
|
| 136 |
+
"correct": true,
|
| 137 |
+
"aligned": true,
|
| 138 |
+
"gap": false,
|
| 139 |
+
"latency_ms": 0.0
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"id": "delta_02",
|
| 143 |
+
"group": "delta",
|
| 144 |
+
"expected_ready": true,
|
| 145 |
+
"got_ready": true,
|
| 146 |
+
"expected_missing": [],
|
| 147 |
+
"got_missing": [],
|
| 148 |
+
"correct": true,
|
| 149 |
+
"aligned": true,
|
| 150 |
+
"gap": false,
|
| 151 |
+
"latency_ms": 0.0
|
| 152 |
+
},
|
| 153 |
+
{
|
| 154 |
+
"id": "delta_03",
|
| 155 |
+
"group": "delta",
|
| 156 |
+
"expected_ready": true,
|
| 157 |
+
"got_ready": true,
|
| 158 |
+
"expected_missing": [],
|
| 159 |
+
"got_missing": [],
|
| 160 |
+
"correct": true,
|
| 161 |
+
"aligned": true,
|
| 162 |
+
"gap": false,
|
| 163 |
+
"latency_ms": 0.0
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"id": "delta_04",
|
| 167 |
+
"group": "delta",
|
| 168 |
+
"expected_ready": false,
|
| 169 |
+
"got_ready": false,
|
| 170 |
+
"expected_missing": [
|
| 171 |
+
"a new analysis since the last report"
|
| 172 |
+
],
|
| 173 |
+
"got_missing": [
|
| 174 |
+
"a new analysis since the last report"
|
| 175 |
+
],
|
| 176 |
+
"correct": true,
|
| 177 |
+
"aligned": true,
|
| 178 |
+
"gap": false,
|
| 179 |
+
"latency_ms": 0.0
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"id": "delta_05",
|
| 183 |
+
"group": "delta",
|
| 184 |
+
"expected_ready": false,
|
| 185 |
+
"got_ready": false,
|
| 186 |
+
"expected_missing": [
|
| 187 |
+
"a new analysis since the last report"
|
| 188 |
+
],
|
| 189 |
+
"got_missing": [
|
| 190 |
+
"a new analysis since the last report"
|
| 191 |
+
],
|
| 192 |
+
"correct": true,
|
| 193 |
+
"aligned": true,
|
| 194 |
+
"gap": false,
|
| 195 |
+
"latency_ms": 0.0
|
| 196 |
+
},
|
| 197 |
+
{
|
| 198 |
+
"id": "edge_01",
|
| 199 |
+
"group": "edge",
|
| 200 |
+
"expected_ready": false,
|
| 201 |
+
"got_ready": false,
|
| 202 |
+
"expected_missing": [
|
| 203 |
+
"at least one completed analysis"
|
| 204 |
+
],
|
| 205 |
+
"got_missing": [
|
| 206 |
+
"at least one completed analysis"
|
| 207 |
+
],
|
| 208 |
+
"correct": true,
|
| 209 |
+
"aligned": false,
|
| 210 |
+
"gap": false,
|
| 211 |
+
"latency_ms": 0.0
|
| 212 |
+
},
|
| 213 |
+
{
|
| 214 |
+
"id": "align_01",
|
| 215 |
+
"group": "alignment",
|
| 216 |
+
"expected_ready": true,
|
| 217 |
+
"got_ready": true,
|
| 218 |
+
"expected_missing": [],
|
| 219 |
+
"got_missing": [],
|
| 220 |
+
"correct": true,
|
| 221 |
+
"aligned": false,
|
| 222 |
+
"gap": true,
|
| 223 |
+
"latency_ms": 0.0
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"id": "align_02",
|
| 227 |
+
"group": "alignment",
|
| 228 |
+
"expected_ready": true,
|
| 229 |
+
"got_ready": true,
|
| 230 |
+
"expected_missing": [],
|
| 231 |
+
"got_missing": [],
|
| 232 |
+
"correct": true,
|
| 233 |
+
"aligned": false,
|
| 234 |
+
"gap": true,
|
| 235 |
+
"latency_ms": 0.0
|
| 236 |
+
},
|
| 237 |
+
{
|
| 238 |
+
"id": "align_03",
|
| 239 |
+
"group": "alignment",
|
| 240 |
+
"expected_ready": true,
|
| 241 |
+
"got_ready": true,
|
| 242 |
+
"expected_missing": [],
|
| 243 |
+
"got_missing": [],
|
| 244 |
+
"correct": true,
|
| 245 |
+
"aligned": true,
|
| 246 |
+
"gap": false,
|
| 247 |
+
"latency_ms": 0.0
|
| 248 |
+
}
|
| 249 |
+
]
|
| 250 |
+
}
|
|
@@ -0,0 +1,274 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"run": {
|
| 3 |
+
"timestamp": "2026-07-23T15:09:48",
|
| 4 |
+
"dataset": "readiness_dataset.json",
|
| 5 |
+
"target": "src/agents/report/readiness.is_report_ready",
|
| 6 |
+
"total": 17,
|
| 7 |
+
"passed": 17,
|
| 8 |
+
"accuracy": 1.0,
|
| 9 |
+
"runtime_avg_ms": 0.0
|
| 10 |
+
},
|
| 11 |
+
"alignment_gap": {
|
| 12 |
+
"count": 2,
|
| 13 |
+
"ids": [
|
| 14 |
+
"align_01",
|
| 15 |
+
"align_02"
|
| 16 |
+
]
|
| 17 |
+
},
|
| 18 |
+
"by_group": {
|
| 19 |
+
"floor": {
|
| 20 |
+
"n": 8,
|
| 21 |
+
"passed": 8,
|
| 22 |
+
"accuracy": 1.0
|
| 23 |
+
},
|
| 24 |
+
"delta": {
|
| 25 |
+
"n": 5,
|
| 26 |
+
"passed": 5,
|
| 27 |
+
"accuracy": 1.0
|
| 28 |
+
},
|
| 29 |
+
"edge": {
|
| 30 |
+
"n": 1,
|
| 31 |
+
"passed": 1,
|
| 32 |
+
"accuracy": 1.0
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"n": 3,
|
| 36 |
+
"passed": 3,
|
| 37 |
+
"accuracy": 1.0
|
| 38 |
+
}
|
| 39 |
+
},
|
| 40 |
+
"cases": [
|
| 41 |
+
{
|
| 42 |
+
"id": "floor_01",
|
| 43 |
+
"group": "floor",
|
| 44 |
+
"expected_ready": false,
|
| 45 |
+
"got_ready": false,
|
| 46 |
+
"expected_missing": [
|
| 47 |
+
"at least one completed analysis"
|
| 48 |
+
],
|
| 49 |
+
"got_missing": [
|
| 50 |
+
"at least one completed analysis"
|
| 51 |
+
],
|
| 52 |
+
"correct": true,
|
| 53 |
+
"aligned": false,
|
| 54 |
+
"gap": false,
|
| 55 |
+
"latency_ms": 0.0
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"id": "floor_02",
|
| 59 |
+
"group": "floor",
|
| 60 |
+
"expected_ready": false,
|
| 61 |
+
"got_ready": false,
|
| 62 |
+
"expected_missing": [
|
| 63 |
+
"at least one completed analysis"
|
| 64 |
+
],
|
| 65 |
+
"got_missing": [
|
| 66 |
+
"at least one completed analysis"
|
| 67 |
+
],
|
| 68 |
+
"correct": true,
|
| 69 |
+
"aligned": false,
|
| 70 |
+
"gap": false,
|
| 71 |
+
"latency_ms": 0.0
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"id": "floor_03",
|
| 75 |
+
"group": "floor",
|
| 76 |
+
"expected_ready": false,
|
| 77 |
+
"got_ready": false,
|
| 78 |
+
"expected_missing": [
|
| 79 |
+
"at least one completed analysis"
|
| 80 |
+
],
|
| 81 |
+
"got_missing": [
|
| 82 |
+
"at least one completed analysis"
|
| 83 |
+
],
|
| 84 |
+
"correct": true,
|
| 85 |
+
"aligned": false,
|
| 86 |
+
"gap": false,
|
| 87 |
+
"latency_ms": 0.0
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"id": "floor_04",
|
| 91 |
+
"group": "floor",
|
| 92 |
+
"expected_ready": true,
|
| 93 |
+
"got_ready": true,
|
| 94 |
+
"expected_missing": [],
|
| 95 |
+
"got_missing": [],
|
| 96 |
+
"correct": true,
|
| 97 |
+
"aligned": true,
|
| 98 |
+
"gap": false,
|
| 99 |
+
"latency_ms": 0.0
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"id": "floor_05",
|
| 103 |
+
"group": "floor",
|
| 104 |
+
"expected_ready": true,
|
| 105 |
+
"got_ready": true,
|
| 106 |
+
"expected_missing": [],
|
| 107 |
+
"got_missing": [],
|
| 108 |
+
"correct": true,
|
| 109 |
+
"aligned": true,
|
| 110 |
+
"gap": false,
|
| 111 |
+
"latency_ms": 0.0
|
| 112 |
+
},
|
| 113 |
+
{
|
| 114 |
+
"id": "floor_06",
|
| 115 |
+
"group": "floor",
|
| 116 |
+
"expected_ready": true,
|
| 117 |
+
"got_ready": true,
|
| 118 |
+
"expected_missing": [],
|
| 119 |
+
"got_missing": [],
|
| 120 |
+
"correct": true,
|
| 121 |
+
"aligned": true,
|
| 122 |
+
"gap": false,
|
| 123 |
+
"latency_ms": 0.0
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"id": "floor_07",
|
| 127 |
+
"group": "floor",
|
| 128 |
+
"expected_ready": true,
|
| 129 |
+
"got_ready": true,
|
| 130 |
+
"expected_missing": [],
|
| 131 |
+
"got_missing": [],
|
| 132 |
+
"correct": true,
|
| 133 |
+
"aligned": true,
|
| 134 |
+
"gap": false,
|
| 135 |
+
"latency_ms": 0.0
|
| 136 |
+
},
|
| 137 |
+
{
|
| 138 |
+
"id": "floor_08",
|
| 139 |
+
"group": "floor",
|
| 140 |
+
"expected_ready": true,
|
| 141 |
+
"got_ready": true,
|
| 142 |
+
"expected_missing": [],
|
| 143 |
+
"got_missing": [],
|
| 144 |
+
"correct": true,
|
| 145 |
+
"aligned": true,
|
| 146 |
+
"gap": false,
|
| 147 |
+
"latency_ms": 0.0
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"id": "delta_01",
|
| 151 |
+
"group": "delta",
|
| 152 |
+
"expected_ready": false,
|
| 153 |
+
"got_ready": false,
|
| 154 |
+
"expected_missing": [
|
| 155 |
+
"a new analysis since the last report"
|
| 156 |
+
],
|
| 157 |
+
"got_missing": [
|
| 158 |
+
"a new analysis since the last report"
|
| 159 |
+
],
|
| 160 |
+
"correct": true,
|
| 161 |
+
"aligned": true,
|
| 162 |
+
"gap": false,
|
| 163 |
+
"latency_ms": 0.0
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"id": "delta_02",
|
| 167 |
+
"group": "delta",
|
| 168 |
+
"expected_ready": true,
|
| 169 |
+
"got_ready": true,
|
| 170 |
+
"expected_missing": [],
|
| 171 |
+
"got_missing": [],
|
| 172 |
+
"correct": true,
|
| 173 |
+
"aligned": true,
|
| 174 |
+
"gap": false,
|
| 175 |
+
"latency_ms": 0.0
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"id": "delta_03",
|
| 179 |
+
"group": "delta",
|
| 180 |
+
"expected_ready": true,
|
| 181 |
+
"got_ready": true,
|
| 182 |
+
"expected_missing": [],
|
| 183 |
+
"got_missing": [],
|
| 184 |
+
"correct": true,
|
| 185 |
+
"aligned": true,
|
| 186 |
+
"gap": false,
|
| 187 |
+
"latency_ms": 0.0
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"id": "delta_04",
|
| 191 |
+
"group": "delta",
|
| 192 |
+
"expected_ready": false,
|
| 193 |
+
"got_ready": false,
|
| 194 |
+
"expected_missing": [
|
| 195 |
+
"a new analysis since the last report"
|
| 196 |
+
],
|
| 197 |
+
"got_missing": [
|
| 198 |
+
"a new analysis since the last report"
|
| 199 |
+
],
|
| 200 |
+
"correct": true,
|
| 201 |
+
"aligned": true,
|
| 202 |
+
"gap": false,
|
| 203 |
+
"latency_ms": 0.0
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"id": "delta_05",
|
| 207 |
+
"group": "delta",
|
| 208 |
+
"expected_ready": false,
|
| 209 |
+
"got_ready": false,
|
| 210 |
+
"expected_missing": [
|
| 211 |
+
"a new analysis since the last report"
|
| 212 |
+
],
|
| 213 |
+
"got_missing": [
|
| 214 |
+
"a new analysis since the last report"
|
| 215 |
+
],
|
| 216 |
+
"correct": true,
|
| 217 |
+
"aligned": true,
|
| 218 |
+
"gap": false,
|
| 219 |
+
"latency_ms": 0.0
|
| 220 |
+
},
|
| 221 |
+
{
|
| 222 |
+
"id": "edge_01",
|
| 223 |
+
"group": "edge",
|
| 224 |
+
"expected_ready": false,
|
| 225 |
+
"got_ready": false,
|
| 226 |
+
"expected_missing": [
|
| 227 |
+
"at least one completed analysis"
|
| 228 |
+
],
|
| 229 |
+
"got_missing": [
|
| 230 |
+
"at least one completed analysis"
|
| 231 |
+
],
|
| 232 |
+
"correct": true,
|
| 233 |
+
"aligned": false,
|
| 234 |
+
"gap": false,
|
| 235 |
+
"latency_ms": 0.0
|
| 236 |
+
},
|
| 237 |
+
{
|
| 238 |
+
"id": "align_01",
|
| 239 |
+
"group": "alignment",
|
| 240 |
+
"expected_ready": true,
|
| 241 |
+
"got_ready": true,
|
| 242 |
+
"expected_missing": [],
|
| 243 |
+
"got_missing": [],
|
| 244 |
+
"correct": true,
|
| 245 |
+
"aligned": false,
|
| 246 |
+
"gap": true,
|
| 247 |
+
"latency_ms": 0.0
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"id": "align_02",
|
| 251 |
+
"group": "alignment",
|
| 252 |
+
"expected_ready": true,
|
| 253 |
+
"got_ready": true,
|
| 254 |
+
"expected_missing": [],
|
| 255 |
+
"got_missing": [],
|
| 256 |
+
"correct": true,
|
| 257 |
+
"aligned": false,
|
| 258 |
+
"gap": true,
|
| 259 |
+
"latency_ms": 0.0
|
| 260 |
+
},
|
| 261 |
+
{
|
| 262 |
+
"id": "align_03",
|
| 263 |
+
"group": "alignment",
|
| 264 |
+
"expected_ready": true,
|
| 265 |
+
"got_ready": true,
|
| 266 |
+
"expected_missing": [],
|
| 267 |
+
"got_missing": [],
|
| 268 |
+
"correct": true,
|
| 269 |
+
"aligned": true,
|
| 270 |
+
"gap": false,
|
| 271 |
+
"latency_ms": 0.0
|
| 272 |
+
}
|
| 273 |
+
]
|
| 274 |
+
}
|
|
@@ -361,7 +361,10 @@ class ChatHandler:
|
|
| 361 |
analysis_state = await self._get_state_store().ensure(analysis_id, user_id)
|
| 362 |
except Exception as e:
|
| 363 |
logger.warning(
|
| 364 |
-
"analysis state ensure failed",
|
|
|
|
|
|
|
|
|
|
| 365 |
)
|
| 366 |
|
| 367 |
# ---- 1b. Gate (REMOVED 2026-06-24) ---------------------------
|
|
@@ -671,7 +674,11 @@ class ChatHandler:
|
|
| 671 |
payload = pad.build(analysis_id or "", user_id, pad.message_id)
|
| 672 |
await self._get_traceability_store().save(payload)
|
| 673 |
except Exception as e: # noqa: BLE001 — never break the answer on a trace slip
|
| 674 |
-
logger.warning(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 675 |
|
| 676 |
async def _run_slow_path(
|
| 677 |
self,
|
|
@@ -774,7 +781,12 @@ class ChatHandler:
|
|
| 774 |
# tool_calls were already recorded by the wrapped invoker.
|
| 775 |
pad.set_planning_from_record(record)
|
| 776 |
except Exception as e: # persistence must never break the user's answer
|
| 777 |
-
logger.error(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 778 |
# SPINE_V2_PLAN §4.4: chart rows are written before `done`; the FE fetches
|
| 779 |
# GET /api/v1/charts unconditionally on every `done` (no polling race).
|
| 780 |
try:
|
|
@@ -790,7 +802,12 @@ class ChatHandler:
|
|
| 790 |
envelope=output.value,
|
| 791 |
)
|
| 792 |
except Exception as e: # chart persist must never break the user's answer
|
| 793 |
-
logger.error(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 794 |
tracer.end() # output omitted (chat_answer may contain PII on Cloud)
|
| 795 |
if pad is not None:
|
| 796 |
await self._flush_trace(pad, analysis_id, user_id)
|
|
|
|
| 361 |
analysis_state = await self._get_state_store().ensure(analysis_id, user_id)
|
| 362 |
except Exception as e:
|
| 363 |
logger.warning(
|
| 364 |
+
"analysis state ensure failed",
|
| 365 |
+
degraded_seam="analysis_state_ensure",
|
| 366 |
+
analysis_id=analysis_id,
|
| 367 |
+
error=repr(e),
|
| 368 |
)
|
| 369 |
|
| 370 |
# ---- 1b. Gate (REMOVED 2026-06-24) ---------------------------
|
|
|
|
| 674 |
payload = pad.build(analysis_id or "", user_id, pad.message_id)
|
| 675 |
await self._get_traceability_store().save(payload)
|
| 676 |
except Exception as e: # noqa: BLE001 — never break the answer on a trace slip
|
| 677 |
+
logger.warning(
|
| 678 |
+
"traceability flush failed",
|
| 679 |
+
degraded_seam="traceability_flush",
|
| 680 |
+
error=repr(e),
|
| 681 |
+
)
|
| 682 |
|
| 683 |
async def _run_slow_path(
|
| 684 |
self,
|
|
|
|
| 781 |
# tool_calls were already recorded by the wrapped invoker.
|
| 782 |
pad.set_planning_from_record(record)
|
| 783 |
except Exception as e: # persistence must never break the user's answer
|
| 784 |
+
logger.error(
|
| 785 |
+
"analysis_record persist failed",
|
| 786 |
+
degraded_seam="report_input_persist",
|
| 787 |
+
user_id=user_id,
|
| 788 |
+
error=repr(e),
|
| 789 |
+
)
|
| 790 |
# SPINE_V2_PLAN §4.4: chart rows are written before `done`; the FE fetches
|
| 791 |
# GET /api/v1/charts unconditionally on every `done` (no polling race).
|
| 792 |
try:
|
|
|
|
| 802 |
envelope=output.value,
|
| 803 |
)
|
| 804 |
except Exception as e: # chart persist must never break the user's answer
|
| 805 |
+
logger.error(
|
| 806 |
+
"chart persist failed",
|
| 807 |
+
degraded_seam="chart_persist",
|
| 808 |
+
user_id=user_id,
|
| 809 |
+
error=repr(e),
|
| 810 |
+
)
|
| 811 |
tracer.end() # output omitted (chat_answer may contain PII on Cloud)
|
| 812 |
if pad is not None:
|
| 813 |
await self._flush_trace(pad, analysis_id, user_id)
|
|
@@ -146,7 +146,14 @@ class InputGuard:
|
|
| 146 |
)
|
| 147 |
# A genuine guard outage (auth, timeout, network): fail open so a guard
|
| 148 |
# failure never blocks legitimate chat.
|
| 149 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 150 |
return GuardVerdict(allow=True, category="safe", reason="guard_error")
|
| 151 |
|
| 152 |
allow = decision.category == "safe"
|
|
|
|
| 146 |
)
|
| 147 |
# A genuine guard outage (auth, timeout, network): fail open so a guard
|
| 148 |
# failure never blocks legitimate chat.
|
| 149 |
+
# The guard being DOWN and the guard PASSING everything are
|
| 150 |
+
# indistinguishable downstream — this marker is what makes "the primary
|
| 151 |
+
# jailbreak defense is currently off" a countable event (F-20).
|
| 152 |
+
logger.warning(
|
| 153 |
+
"input guard errored — allowing",
|
| 154 |
+
degraded_seam="input_guard_fail_open",
|
| 155 |
+
error=repr(e),
|
| 156 |
+
)
|
| 157 |
return GuardVerdict(allow=True, category="safe", reason="guard_error")
|
| 158 |
|
| 159 |
allow = decision.category == "safe"
|
|
@@ -20,6 +20,25 @@ from typing import Any
|
|
| 20 |
from pydantic import BaseModel, Field
|
| 21 |
|
| 22 |
from ...catalog.models import Catalog, DataType
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
|
| 25 |
class ColumnSummary(BaseModel):
|
|
@@ -127,10 +146,32 @@ class CatalogSummary(BaseModel):
|
|
| 127 |
return cls(structured_sources=structured, unstructured_sources=unstructured)
|
| 128 |
|
| 129 |
def render(self) -> str:
|
| 130 |
-
"""Render the summary as compact text for the planner prompt.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 131 |
if not self.structured_sources and not self.unstructured_sources:
|
| 132 |
return "(catalog is empty — the user has not registered any data yet)"
|
| 133 |
|
|
|
|
|
|
|
|
|
|
| 134 |
lines: list[str] = []
|
| 135 |
for source in self.structured_sources:
|
| 136 |
lines.append(f"Source: {source.name} ({source.source_type}) — id={source.source_id}")
|
|
@@ -141,6 +182,14 @@ class CatalogSummary(BaseModel):
|
|
| 141 |
c.column_id: c.name for t in source.tables for c in t.columns
|
| 142 |
}
|
| 143 |
for table in source.tables:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 144 |
rc = f" ({table.row_count:,} rows)" if table.row_count is not None else ""
|
| 145 |
lines.append(f" Table: {table.name}{rc} — id={table.table_id}")
|
| 146 |
for col in table.columns:
|
|
@@ -160,8 +209,26 @@ class CatalogSummary(BaseModel):
|
|
| 160 |
f"left_column_id={fk.column_id}, "
|
| 161 |
f"right_column_id={fk.target_column_id})"
|
| 162 |
)
|
|
|
|
| 163 |
lines.append("")
|
| 164 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
if self.unstructured_sources:
|
| 166 |
lines.append("Unstructured sources (for retrieve_knowledge):")
|
| 167 |
for src in self.unstructured_sources:
|
|
|
|
| 20 |
from pydantic import BaseModel, Field
|
| 21 |
|
| 22 |
from ...catalog.models import Catalog, DataType
|
| 23 |
+
from ...middlewares.logging import get_logger
|
| 24 |
+
|
| 25 |
+
logger = get_logger("planner_inputs")
|
| 26 |
+
|
| 27 |
+
# Ceilings on what `CatalogSummary.render` emits into the planner prompt (F-12,
|
| 28 |
+
# 2026-07-23). See that method's docstring for why these are safety nets set high
|
| 29 |
+
# rather than tight caps. Whichever binds first wins:
|
| 30 |
+
# _MAX_TABLES guards many-small-tables catalogs
|
| 31 |
+
# _MAX_CATALOG_CHARS guards few-very-wide-tables catalogs (a 200-column fact table
|
| 32 |
+
# blows the budget long before the table count does)
|
| 33 |
+
# ~250k chars is roughly 62k tokens. Sizing rationale, measured this session:
|
| 34 |
+
# 100 tables x 30 cols ~121k chars -> renders IN FULL (a plausible real warehouse)
|
| 35 |
+
# 150 tables x 30 cols ~182k chars -> renders IN FULL
|
| 36 |
+
# 400 tables x 30 cols ~482k chars -> truncated to ~250k (was ~241k TOKENS/call)
|
| 37 |
+
# The point is to make the catastrophic case survivable without touching anyone real.
|
| 38 |
+
# These are provisional: nobody knows the largest actual customer catalog yet (review
|
| 39 |
+
# open question #4), so the truncation log is what tells us when to revisit them.
|
| 40 |
+
_MAX_TABLES = 150
|
| 41 |
+
_MAX_CATALOG_CHARS = 250_000
|
| 42 |
|
| 43 |
|
| 44 |
class ColumnSummary(BaseModel):
|
|
|
|
| 146 |
return cls(structured_sources=structured, unstructured_sources=unstructured)
|
| 147 |
|
| 148 |
def render(self) -> str:
|
| 149 |
+
"""Render the summary as compact text for the planner prompt.
|
| 150 |
+
|
| 151 |
+
**Bounded since 2026-07-23 (F-12).** This used to emit one line per column
|
| 152 |
+
across every table of every source with no truncation anywhere, and
|
| 153 |
+
`PlannerService.plan` rebuilds the whole prompt on each of its 3 retries.
|
| 154 |
+
Measured: 200 tables x 30 cols renders ~481k chars (~120k tokens) per call,
|
| 155 |
+
~361k tokens across the retries — past the context window, so the Azure call
|
| 156 |
+
fails and the never-throw path degrades it to "Analysis failed". A 400-table
|
| 157 |
+
warehouse is simply unusable, and the symptom looks like a data problem.
|
| 158 |
+
|
| 159 |
+
`_MAX_TABLES` is a SAFETY NET, not a tight cap. A low cap would be actively
|
| 160 |
+
harmful: there is no relevance ordering here, so dropping tables can drop the
|
| 161 |
+
very table the question is about. It is set far above any catalog seen so far,
|
| 162 |
+
every truncation is logged (that log is the signal a real customer is
|
| 163 |
+
approaching it), and the planner is TOLD it saw a subset so it can ask the
|
| 164 |
+
user to name a table rather than silently assume it saw everything.
|
| 165 |
+
|
| 166 |
+
The durable fix is a relevance-ranked, question-keyed subset — which must not
|
| 167 |
+
be attempted without an eval proving the planner still picks the right table.
|
| 168 |
+
"""
|
| 169 |
if not self.structured_sources and not self.unstructured_sources:
|
| 170 |
return "(catalog is empty — the user has not registered any data yet)"
|
| 171 |
|
| 172 |
+
rendered_tables = 0
|
| 173 |
+
omitted_tables = 0
|
| 174 |
+
budget_used = 0
|
| 175 |
lines: list[str] = []
|
| 176 |
for source in self.structured_sources:
|
| 177 |
lines.append(f"Source: {source.name} ({source.source_type}) — id={source.source_id}")
|
|
|
|
| 182 |
c.column_id: c.name for t in source.tables for c in t.columns
|
| 183 |
}
|
| 184 |
for table in source.tables:
|
| 185 |
+
# Two ceilings, whichever binds first. Table count alone is not
|
| 186 |
+
# enough: 150 tables of 200 columns is just as unbounded as 400
|
| 187 |
+
# tables of 30, and wide fact tables are common in warehouses.
|
| 188 |
+
if rendered_tables >= _MAX_TABLES or budget_used >= _MAX_CATALOG_CHARS:
|
| 189 |
+
omitted_tables += 1
|
| 190 |
+
continue
|
| 191 |
+
rendered_tables += 1
|
| 192 |
+
table_start = len(lines)
|
| 193 |
rc = f" ({table.row_count:,} rows)" if table.row_count is not None else ""
|
| 194 |
lines.append(f" Table: {table.name}{rc} — id={table.table_id}")
|
| 195 |
for col in table.columns:
|
|
|
|
| 209 |
f"left_column_id={fk.column_id}, "
|
| 210 |
f"right_column_id={fk.target_column_id})"
|
| 211 |
)
|
| 212 |
+
budget_used += sum(len(ln) + 1 for ln in lines[table_start:])
|
| 213 |
lines.append("")
|
| 214 |
|
| 215 |
+
if omitted_tables:
|
| 216 |
+
# The planner must know it saw a subset — otherwise "that table isn't in
|
| 217 |
+
# the catalog" becomes a confident, wrong answer.
|
| 218 |
+
lines.append(
|
| 219 |
+
f"... and {omitted_tables} more table(s) not shown (catalog too large "
|
| 220 |
+
f"to render in full). If the question names a table you cannot see "
|
| 221 |
+
f"here, say so and ask the user to name it explicitly."
|
| 222 |
+
)
|
| 223 |
+
logger.warning(
|
| 224 |
+
"catalog render truncated",
|
| 225 |
+
rendered_tables=rendered_tables,
|
| 226 |
+
omitted_tables=omitted_tables,
|
| 227 |
+
chars_used=budget_used,
|
| 228 |
+
max_tables=_MAX_TABLES,
|
| 229 |
+
max_chars=_MAX_CATALOG_CHARS,
|
| 230 |
+
)
|
| 231 |
+
|
| 232 |
if self.unstructured_sources:
|
| 233 |
lines.append("Unstructured sources (for retrieve_knowledge):")
|
| 234 |
for src in self.unstructured_sources:
|
|
@@ -222,8 +222,11 @@ async def report_floor(
|
|
| 222 |
except Exception as exc: # noqa: BLE001 — never-throw; fail closed to not-ready
|
| 223 |
logger.warning(
|
| 224 |
"report_floor: record store read failed — not ready",
|
|
|
|
|
|
|
|
|
|
| 225 |
analysis_id=analysis_id,
|
| 226 |
-
error=
|
| 227 |
)
|
| 228 |
return [_MISSING_ANALYSIS], []
|
| 229 |
|
|
|
|
| 222 |
except Exception as exc: # noqa: BLE001 — never-throw; fail closed to not-ready
|
| 223 |
logger.warning(
|
| 224 |
"report_floor: record store read failed — not ready",
|
| 225 |
+
# A 409 from a read failure looks identical to a 409 from a genuinely
|
| 226 |
+
# empty analysis. This marker separates them (F-20).
|
| 227 |
+
degraded_seam="report_floor_record_read",
|
| 228 |
analysis_id=analysis_id,
|
| 229 |
+
error=repr(exc),
|
| 230 |
)
|
| 231 |
return [_MISSING_ANALYSIS], []
|
| 232 |
|
|
@@ -99,8 +99,9 @@ class PostgresReportInputStore:
|
|
| 99 |
except Exception as exc: # never break the user's answer (§8.3)
|
| 100 |
logger.error(
|
| 101 |
"analysis_record persist failed",
|
|
|
|
| 102 |
record_id=record.record_id,
|
| 103 |
-
error=
|
| 104 |
)
|
| 105 |
|
| 106 |
async def list_for_analysis(
|
|
|
|
| 99 |
except Exception as exc: # never break the user's answer (§8.3)
|
| 100 |
logger.error(
|
| 101 |
"analysis_record persist failed",
|
| 102 |
+
degraded_seam="report_input_persist",
|
| 103 |
record_id=record.record_id,
|
| 104 |
+
error=repr(exc),
|
| 105 |
)
|
| 106 |
|
| 107 |
async def list_for_analysis(
|
|
@@ -14,7 +14,17 @@ Every response is HTTP 200 with an explicit `status` marker (lead ask 2026-07-13
|
|
| 14 |
id, or an error turn — those never write a row). Distinguished
|
| 15 |
from `empty` via the turn's traceability row.
|
| 16 |
|
| 17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
"""
|
| 19 |
|
| 20 |
from typing import Literal
|
|
@@ -46,10 +56,26 @@ async def get_charts(
|
|
| 46 |
message_id: str = Query(
|
| 47 |
..., description="Assistant turn id, taken from the `done` SSE event"
|
| 48 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
) -> ChartsResponse:
|
| 50 |
"""Fetch every chart for one turn. Always 200 — the `status` field carries the
|
| 51 |
outcome, so the FE can call this unconditionally on every `done`."""
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
if charts:
|
| 54 |
return ChartsResponse(
|
| 55 |
status="success",
|
|
@@ -57,7 +83,7 @@ async def get_charts(
|
|
| 57 |
count=len(charts),
|
| 58 |
charts=charts,
|
| 59 |
)
|
| 60 |
-
if await _store.turn_exists(message_id):
|
| 61 |
return ChartsResponse(
|
| 62 |
status="empty",
|
| 63 |
message="This message completed without producing charts.",
|
|
|
|
| 14 |
id, or an error turn — those never write a row). Distinguished
|
| 15 |
from `empty` via the turn's traceability row.
|
| 16 |
|
| 17 |
+
**Tenant scoping (F-3, 2026-07-23).** `user_id` is an OPTIONAL query parameter in this
|
| 18 |
+
rollout: supplied, it scopes both reads to the turn's owner; omitted, the lookup stays
|
| 19 |
+
message_id-only and the response is identical to before, so no FE change is needed to
|
| 20 |
+
keep working. This is deliberately the same log-only-then-enforce pattern used for the
|
| 21 |
+
catalog tenant predicate — every unscoped call logs `scoped=false`, which is the signal
|
| 22 |
+
for when it is safe to make the parameter required.
|
| 23 |
+
|
| 24 |
+
Until it IS required this endpoint remains a capability URL: `spec.plotly.data` carries
|
| 25 |
+
the customer's real values, so anyone holding a leaked message_id can read them. Making
|
| 26 |
+
`user_id` required is an FE contract change (DEV_PLAN #40); the service-secret gate
|
| 27 |
+
(#37) is the control that matters in the meantime.
|
| 28 |
"""
|
| 29 |
|
| 30 |
from typing import Literal
|
|
|
|
| 56 |
message_id: str = Query(
|
| 57 |
..., description="Assistant turn id, taken from the `done` SSE event"
|
| 58 |
),
|
| 59 |
+
user_id: str | None = Query(
|
| 60 |
+
None,
|
| 61 |
+
description=(
|
| 62 |
+
"Owner of the turn. Optional during the F-3 rollout — when supplied the "
|
| 63 |
+
"lookup is scoped to this user; when omitted the response is unchanged. "
|
| 64 |
+
"Send it as soon as the FE can; it becomes required (DEV_PLAN #40)."
|
| 65 |
+
),
|
| 66 |
+
),
|
| 67 |
) -> ChartsResponse:
|
| 68 |
"""Fetch every chart for one turn. Always 200 — the `status` field carries the
|
| 69 |
outcome, so the FE can call this unconditionally on every `done`."""
|
| 70 |
+
if user_id is None:
|
| 71 |
+
# The rollout signal: when this stops appearing, the parameter can be
|
| 72 |
+
# made required without breaking a live caller.
|
| 73 |
+
logger.warning(
|
| 74 |
+
"charts read unscoped — no user_id supplied",
|
| 75 |
+
message_id=message_id,
|
| 76 |
+
scoped=False,
|
| 77 |
+
)
|
| 78 |
+
charts = await _store.list_for_message(message_id, user_id)
|
| 79 |
if charts:
|
| 80 |
return ChartsResponse(
|
| 81 |
status="success",
|
|
|
|
| 83 |
count=len(charts),
|
| 84 |
charts=charts,
|
| 85 |
)
|
| 86 |
+
if await _store.turn_exists(message_id, user_id):
|
| 87 |
return ChartsResponse(
|
| 88 |
status="empty",
|
| 89 |
message="This message completed without producing charts.",
|
|
@@ -7,7 +7,17 @@ chat pipeline right before the `done` SSE event; the FE fires this GET on `done`
|
|
| 7 |
|
| 8 |
Renamed from the contracted `/api/v1/observability` (team decision 2026-07-06) so it is
|
| 9 |
never confused with the Langfuse *observability* stack (engineering-only, PII-masked).
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
"""
|
| 12 |
|
| 13 |
from fastapi import APIRouter, HTTPException, Query
|
|
@@ -30,11 +40,28 @@ async def get_traceability(
|
|
| 30 |
message_id: str = Query(
|
| 31 |
..., description="Assistant turn id, taken from the `done` SSE event"
|
| 32 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
) -> TraceabilityPayload:
|
| 34 |
"""Fetch one turn's provenance record. 404 while the turn is still running or if
|
| 35 |
the id is unknown (the FE never gets a `message_id` for error turns, so 404 is
|
| 36 |
the correct answer there)."""
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
if payload is None:
|
| 39 |
raise HTTPException(
|
| 40 |
status_code=404,
|
|
|
|
| 7 |
|
| 8 |
Renamed from the contracted `/api/v1/observability` (team decision 2026-07-06) so it is
|
| 9 |
never confused with the Langfuse *observability* stack (engineering-only, PII-masked).
|
| 10 |
+
|
| 11 |
+
**Tenant scoping (F-3, 2026-07-23).** `user_id` is an OPTIONAL query parameter in this
|
| 12 |
+
rollout: supplied, it scopes the read to the turn's owner; omitted, behaviour is exactly
|
| 13 |
+
as before, so no FE change is needed to keep working. Every unscoped call logs
|
| 14 |
+
`scoped=false` — that count is the evidence for when the parameter can safely be made
|
| 15 |
+
required (an FE contract change, DEV_PLAN #40).
|
| 16 |
+
|
| 17 |
+
Note what an unscoped read exposes: the payload carries 5-row previews of every
|
| 18 |
+
`retrieve_data` result, the executed SQL, AND the owner's `user_id` — which is the id an
|
| 19 |
+
attacker would need to satisfy every other tenant predicate. The service-secret gate
|
| 20 |
+
(#37) is the control that matters until scoping is required.
|
| 21 |
"""
|
| 22 |
|
| 23 |
from fastapi import APIRouter, HTTPException, Query
|
|
|
|
| 40 |
message_id: str = Query(
|
| 41 |
..., description="Assistant turn id, taken from the `done` SSE event"
|
| 42 |
),
|
| 43 |
+
user_id: str | None = Query(
|
| 44 |
+
None,
|
| 45 |
+
description=(
|
| 46 |
+
"Owner of the turn. Optional during the F-3 rollout — when supplied the "
|
| 47 |
+
"lookup is scoped to this user; when omitted the response is unchanged. "
|
| 48 |
+
"Send it as soon as the FE can; it becomes required (DEV_PLAN #40)."
|
| 49 |
+
),
|
| 50 |
+
),
|
| 51 |
) -> TraceabilityPayload:
|
| 52 |
"""Fetch one turn's provenance record. 404 while the turn is still running or if
|
| 53 |
the id is unknown (the FE never gets a `message_id` for error turns, so 404 is
|
| 54 |
the correct answer there)."""
|
| 55 |
+
if user_id is None:
|
| 56 |
+
# The rollout signal: when this stops appearing, the parameter can be
|
| 57 |
+
# made required without breaking a live caller.
|
| 58 |
+
logger.warning(
|
| 59 |
+
"traceability read unscoped — no user_id supplied",
|
| 60 |
+
analysis_id=analysis_id,
|
| 61 |
+
message_id=message_id,
|
| 62 |
+
scoped=False,
|
| 63 |
+
)
|
| 64 |
+
payload = await _store.get(analysis_id, message_id, user_id)
|
| 65 |
if payload is None:
|
| 66 |
raise HTTPException(
|
| 67 |
status_code=404,
|
|
@@ -126,8 +126,11 @@ class AnalysisScopedCatalogReader(CatalogReader):
|
|
| 126 |
except Exception as e: # noqa: BLE001 — never block check on the analysis read
|
| 127 |
logger.warning(
|
| 128 |
"analysis catalog read failed — returning empty",
|
|
|
|
|
|
|
|
|
|
| 129 |
analysis_id=self._analysis_id,
|
| 130 |
-
error=
|
| 131 |
)
|
| 132 |
catalog = None
|
| 133 |
|
|
|
|
| 126 |
except Exception as e: # noqa: BLE001 — never block check on the analysis read
|
| 127 |
logger.warning(
|
| 128 |
"analysis catalog read failed — returning empty",
|
| 129 |
+
# Distinguishes a FAILED read from an analysis that genuinely has no
|
| 130 |
+
# sources bound — the user sees "no data bound" either way (F-20).
|
| 131 |
+
degraded_seam="analysis_catalog_read",
|
| 132 |
analysis_id=self._analysis_id,
|
| 133 |
+
error=repr(e),
|
| 134 |
)
|
| 135 |
catalog = None
|
| 136 |
|
|
@@ -64,9 +64,13 @@ class ChartStore(Protocol):
|
|
| 64 |
envelope: dict,
|
| 65 |
) -> None: ...
|
| 66 |
|
| 67 |
-
async def list_for_message(
|
|
|
|
|
|
|
| 68 |
|
| 69 |
-
async def turn_exists(
|
|
|
|
|
|
|
| 70 |
|
| 71 |
|
| 72 |
class NullChartStore:
|
|
@@ -87,10 +91,12 @@ class NullChartStore:
|
|
| 87 |
chart_type=envelope.get("chart_type", "unknown"),
|
| 88 |
)
|
| 89 |
|
| 90 |
-
async def list_for_message(
|
|
|
|
|
|
|
| 91 |
return []
|
| 92 |
|
| 93 |
-
async def turn_exists(self, message_id: str) -> bool:
|
| 94 |
return False
|
| 95 |
|
| 96 |
|
|
@@ -129,19 +135,32 @@ class PostgresChartStore:
|
|
| 129 |
except Exception as exc: # never break the user's answer
|
| 130 |
logger.error(
|
| 131 |
"chart persist failed",
|
|
|
|
| 132 |
message_id=message_id,
|
| 133 |
-
error=
|
| 134 |
)
|
| 135 |
|
| 136 |
-
async def list_for_message(
|
|
|
|
|
|
|
| 137 |
# message_id-only lookup (lead decision 2026-07-13). NOTE for the Harry
|
| 138 |
# migration: the manual DDL's composite index (analysis_id, message_id)
|
| 139 |
# does not serve this predicate — an additive index on (message_id) is
|
| 140 |
# part of the handoff.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
async with AsyncSessionLocal() as session:
|
|
|
|
|
|
|
|
|
|
| 142 |
result = await session.execute(
|
| 143 |
select(MessageChartRow)
|
| 144 |
-
.where(
|
| 145 |
.order_by(MessageChartRow.created_at)
|
| 146 |
)
|
| 147 |
rows = result.scalars().all()
|
|
@@ -156,17 +175,22 @@ class PostgresChartStore:
|
|
| 156 |
for row in rows
|
| 157 |
]
|
| 158 |
|
| 159 |
-
async def turn_exists(self, message_id: str) -> bool:
|
| 160 |
"""True iff the turn flushed its traceability row (written before `done`).
|
| 161 |
|
| 162 |
Lets the endpoint tri-state a zero-chart GET: `empty` (completed turn, no
|
| 163 |
charts — the common case) vs `not_found` (unknown/mistyped id, or an error
|
| 164 |
turn, which never writes traceability). PK lookup — cheap.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
"""
|
| 166 |
async with AsyncSessionLocal() as session:
|
|
|
|
|
|
|
|
|
|
| 167 |
result = await session.execute(
|
| 168 |
-
select(MessageTraceabilityRow.message_id).where(
|
| 169 |
-
MessageTraceabilityRow.message_id == message_id
|
| 170 |
-
)
|
| 171 |
)
|
| 172 |
return result.scalar_one_or_none() is not None
|
|
|
|
| 64 |
envelope: dict,
|
| 65 |
) -> None: ...
|
| 66 |
|
| 67 |
+
async def list_for_message(
|
| 68 |
+
self, message_id: str, user_id: str | None = None
|
| 69 |
+
) -> list[ChartRecord]: ...
|
| 70 |
|
| 71 |
+
async def turn_exists(
|
| 72 |
+
self, message_id: str, user_id: str | None = None
|
| 73 |
+
) -> bool: ...
|
| 74 |
|
| 75 |
|
| 76 |
class NullChartStore:
|
|
|
|
| 91 |
chart_type=envelope.get("chart_type", "unknown"),
|
| 92 |
)
|
| 93 |
|
| 94 |
+
async def list_for_message(
|
| 95 |
+
self, message_id: str, user_id: str | None = None
|
| 96 |
+
) -> list[ChartRecord]:
|
| 97 |
return []
|
| 98 |
|
| 99 |
+
async def turn_exists(self, message_id: str, user_id: str | None = None) -> bool:
|
| 100 |
return False
|
| 101 |
|
| 102 |
|
|
|
|
| 135 |
except Exception as exc: # never break the user's answer
|
| 136 |
logger.error(
|
| 137 |
"chart persist failed",
|
| 138 |
+
degraded_seam="chart_persist",
|
| 139 |
message_id=message_id,
|
| 140 |
+
error=repr(exc),
|
| 141 |
)
|
| 142 |
|
| 143 |
+
async def list_for_message(
|
| 144 |
+
self, message_id: str, user_id: str | None = None
|
| 145 |
+
) -> list[ChartRecord]:
|
| 146 |
# message_id-only lookup (lead decision 2026-07-13). NOTE for the Harry
|
| 147 |
# migration: the manual DDL's composite index (analysis_id, message_id)
|
| 148 |
# does not serve this predicate — an additive index on (message_id) is
|
| 149 |
# part of the handoff.
|
| 150 |
+
#
|
| 151 |
+
# `user_id` scopes the read to the turn's owner (F-3, 2026-07-23). Optional
|
| 152 |
+
# because `GET /api/v1/charts` does not require the parameter yet — making it
|
| 153 |
+
# required is an FE contract change. Until the FE sends it, an unscoped read
|
| 154 |
+
# means the chart data (which is REAL customer values, in `spec.plotly.data`)
|
| 155 |
+
# is reachable by anyone holding the message_id. See the endpoint's
|
| 156 |
+
# `scoped=false` log line.
|
| 157 |
async with AsyncSessionLocal() as session:
|
| 158 |
+
where = [MessageChartRow.message_id == message_id]
|
| 159 |
+
if user_id is not None:
|
| 160 |
+
where.append(MessageChartRow.user_id == user_id)
|
| 161 |
result = await session.execute(
|
| 162 |
select(MessageChartRow)
|
| 163 |
+
.where(*where)
|
| 164 |
.order_by(MessageChartRow.created_at)
|
| 165 |
)
|
| 166 |
rows = result.scalars().all()
|
|
|
|
| 175 |
for row in rows
|
| 176 |
]
|
| 177 |
|
| 178 |
+
async def turn_exists(self, message_id: str, user_id: str | None = None) -> bool:
|
| 179 |
"""True iff the turn flushed its traceability row (written before `done`).
|
| 180 |
|
| 181 |
Lets the endpoint tri-state a zero-chart GET: `empty` (completed turn, no
|
| 182 |
charts — the common case) vs `not_found` (unknown/mistyped id, or an error
|
| 183 |
turn, which never writes traceability). PK lookup — cheap.
|
| 184 |
+
|
| 185 |
+
Scoped by `user_id` when the caller supplies one (F-3, 2026-07-23), so another
|
| 186 |
+
tenant's turn reads as `not_found` rather than `empty` — the tri-state must not
|
| 187 |
+
become a probe that confirms a message_id exists.
|
| 188 |
"""
|
| 189 |
async with AsyncSessionLocal() as session:
|
| 190 |
+
where = [MessageTraceabilityRow.message_id == message_id]
|
| 191 |
+
if user_id is not None:
|
| 192 |
+
where.append(MessageTraceabilityRow.user_id == user_id)
|
| 193 |
result = await session.execute(
|
| 194 |
+
select(MessageTraceabilityRow.message_id).where(*where)
|
|
|
|
|
|
|
| 195 |
)
|
| 196 |
return result.scalar_one_or_none() is not None
|
|
@@ -35,6 +35,21 @@ _OBJECT_STORAGE_PREFIX = "object_storage://"
|
|
| 35 |
_LOCATION_REF_PREFIXES = (_AZ_BLOB_PREFIX, _OBJECT_STORAGE_PREFIX)
|
| 36 |
_ROW_HARD_CAP = 10_000
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
class TabularExecutor(BaseExecutor):
|
| 40 |
"""Executes compiled pandas chain on a Parquet blob.
|
|
@@ -47,10 +62,31 @@ class TabularExecutor(BaseExecutor):
|
|
| 47 |
self,
|
| 48 |
catalog: Catalog,
|
| 49 |
fetch_blob: Callable[[str], Coroutine[Any, Any, bytes]] | None = None,
|
|
|
|
| 50 |
) -> None:
|
| 51 |
self._catalog = catalog
|
| 52 |
self._compiler = PandasCompiler(catalog)
|
| 53 |
self._fetch_blob = fetch_blob or self._default_fetch_blob
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
@staticmethod
|
| 56 |
async def _default_fetch_blob(blob_name: str) -> bytes:
|
|
@@ -86,13 +122,60 @@ class TabularExecutor(BaseExecutor):
|
|
| 86 |
rendered_query = _render_query(ir, {c.column_id: c for c in table.columns})
|
| 87 |
logger.info("pandas query", query=rendered_query)
|
| 88 |
blob_name = _resolve_blob_name(source, table)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
blob_bytes = await self._fetch_blob(blob_name)
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
result_df = await asyncio.to_thread(_load_and_apply, blob_bytes, compiled)
|
| 92 |
|
| 93 |
truncated = len(result_df) > _ROW_HARD_CAP
|
| 94 |
capped = result_df.head(_ROW_HARD_CAP)
|
| 95 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
columns = compiled.output_columns
|
| 97 |
rows = capped.to_dict(orient="records")
|
| 98 |
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
|
|
|
| 35 |
_LOCATION_REF_PREFIXES = (_AZ_BLOB_PREFIX, _OBJECT_STORAGE_PREFIX)
|
| 36 |
_ROW_HARD_CAP = 10_000
|
| 37 |
|
| 38 |
+
# Largest Parquet blob we will pull into memory (F-13, 2026-07-23).
|
| 39 |
+
#
|
| 40 |
+
# The whole object is buffered and then `pd.read_parquet`-ed, so peak RSS is the file
|
| 41 |
+
# plus the decompressed frame — often 3-5x for string-heavy data. Filtering and
|
| 42 |
+
# `_ROW_HARD_CAP` both happen strictly AFTER the frame exists, so they bound the
|
| 43 |
+
# RESULT, never the working set. An OOM here is not catchable: it kills the process
|
| 44 |
+
# and every concurrent request with it, which is the one failure mode in this service
|
| 45 |
+
# that escapes every never-throw seam.
|
| 46 |
+
#
|
| 47 |
+
# Deliberately a SAFETY NET, not a tight cap: 500 MB of Parquet is far beyond any
|
| 48 |
+
# upload seen so far, so no real user should ever meet it. Every rejection logs, and
|
| 49 |
+
# that log is the signal to revisit the number (or to implement the size-tiered
|
| 50 |
+
# strategy this module's docstring already sketches: pyarrow pushdown, polars lazy).
|
| 51 |
+
_MAX_BLOB_BYTES = 500 * 1024 * 1024
|
| 52 |
+
|
| 53 |
|
| 54 |
class TabularExecutor(BaseExecutor):
|
| 55 |
"""Executes compiled pandas chain on a Parquet blob.
|
|
|
|
| 62 |
self,
|
| 63 |
catalog: Catalog,
|
| 64 |
fetch_blob: Callable[[str], Coroutine[Any, Any, bytes]] | None = None,
|
| 65 |
+
blob_size: Callable[[str], Coroutine[Any, Any, int | None]] | None = None,
|
| 66 |
) -> None:
|
| 67 |
self._catalog = catalog
|
| 68 |
self._compiler = PandasCompiler(catalog)
|
| 69 |
self._fetch_blob = fetch_blob or self._default_fetch_blob
|
| 70 |
+
# Injected alongside `fetch_blob` so a test that fakes the download can also
|
| 71 |
+
# fake the size probe. When only `fetch_blob` is injected the probe is skipped
|
| 72 |
+
# and the post-download check still applies.
|
| 73 |
+
self._blob_size = blob_size or (
|
| 74 |
+
self._default_blob_size if fetch_blob is None else None
|
| 75 |
+
)
|
| 76 |
+
|
| 77 |
+
@staticmethod
|
| 78 |
+
async def _default_blob_size(blob_name: str) -> int | None:
|
| 79 |
+
from ...config.settings import settings
|
| 80 |
+
|
| 81 |
+
provider = (settings.storage_provider or "").strip().lower()
|
| 82 |
+
if provider == "supabase_s3":
|
| 83 |
+
from ...storage.object_storage import object_storage
|
| 84 |
+
|
| 85 |
+
return await object_storage.object_size(blob_name)
|
| 86 |
+
|
| 87 |
+
from ...storage.az_blob.az_blob import blob_storage
|
| 88 |
+
|
| 89 |
+
return await blob_storage.object_size(blob_name)
|
| 90 |
|
| 91 |
@staticmethod
|
| 92 |
async def _default_fetch_blob(blob_name: str) -> bytes:
|
|
|
|
| 122 |
rendered_query = _render_query(ir, {c.column_id: c for c in table.columns})
|
| 123 |
logger.info("pandas query", query=rendered_query)
|
| 124 |
blob_name = _resolve_blob_name(source, table)
|
| 125 |
+
|
| 126 |
+
# Refuse an oversized blob BEFORE buffering it — see _MAX_BLOB_BYTES.
|
| 127 |
+
# A None size means the backend couldn't tell us (HEAD failed, or a test
|
| 128 |
+
# injected only `fetch_blob`); we proceed and rely on the post-download
|
| 129 |
+
# check below, which still beats no bound at all. (F-13)
|
| 130 |
+
if self._blob_size is not None:
|
| 131 |
+
size = await self._blob_size(blob_name)
|
| 132 |
+
if size is not None and size > _MAX_BLOB_BYTES:
|
| 133 |
+
raise ValueError(
|
| 134 |
+
f"file is too large to analyse ({size / 1024 / 1024:.0f} MB; "
|
| 135 |
+
f"limit {_MAX_BLOB_BYTES / 1024 / 1024:.0f} MB) — "
|
| 136 |
+
"filter it down or split it before uploading"
|
| 137 |
+
)
|
| 138 |
+
|
| 139 |
blob_bytes = await self._fetch_blob(blob_name)
|
| 140 |
|
| 141 |
+
# Backstop for the case where the size probe was unavailable. The bytes
|
| 142 |
+
# are already resident here, so this cannot prevent the download — but it
|
| 143 |
+
# does stop `pd.read_parquet` from multiplying them into a frame several
|
| 144 |
+
# times larger, which is where the OOM actually happens. (F-13)
|
| 145 |
+
if len(blob_bytes) > _MAX_BLOB_BYTES:
|
| 146 |
+
raise ValueError(
|
| 147 |
+
f"file is too large to analyse ({len(blob_bytes) / 1024 / 1024:.0f} MB; "
|
| 148 |
+
f"limit {_MAX_BLOB_BYTES / 1024 / 1024:.0f} MB) — "
|
| 149 |
+
"filter it down or split it before uploading"
|
| 150 |
+
)
|
| 151 |
+
|
| 152 |
result_df = await asyncio.to_thread(_load_and_apply, blob_bytes, compiled)
|
| 153 |
|
| 154 |
truncated = len(result_df) > _ROW_HARD_CAP
|
| 155 |
capped = result_df.head(_ROW_HARD_CAP)
|
| 156 |
|
| 157 |
+
# `output_columns` is derived from the SELECT list, not from the frame the
|
| 158 |
+
# compiler actually produced, and the rows below are mapped BY NAME
|
| 159 |
+
# (`data_access._retrieve_data` does `row.get(c)`). A declared name that is
|
| 160 |
+
# absent from the frame silently becomes a column of None — a real-looking
|
| 161 |
+
# table with a fabricated column, which is worse than an error. F-17's
|
| 162 |
+
# validator fix closes the one known way in (mixed select, no group_by);
|
| 163 |
+
# this is the backstop that turns any future divergence into an honest
|
| 164 |
+
# failure instead of a wrong answer.
|
| 165 |
+
#
|
| 166 |
+
# Presence, not order: the grouped path builds its frame via
|
| 167 |
+
# `reset_index()`, which emits the group columns first regardless of where
|
| 168 |
+
# they sat in the select list, so a positional comparison would reject
|
| 169 |
+
# perfectly good queries. Name-mapping makes order irrelevant. (F-17)
|
| 170 |
+
produced = set(capped.columns)
|
| 171 |
+
fabricated = [c for c in compiled.output_columns if c not in produced]
|
| 172 |
+
if fabricated:
|
| 173 |
+
raise ValueError(
|
| 174 |
+
f"pandas compiler did not produce declared column(s) {fabricated!r} "
|
| 175 |
+
f"(frame has {list(capped.columns)!r}) — refusing to return a "
|
| 176 |
+
"result whose columns would be fabricated as null"
|
| 177 |
+
)
|
| 178 |
+
|
| 179 |
columns = compiled.output_columns
|
| 180 |
rows = capped.to_dict(orient="records")
|
| 181 |
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
|
@@ -57,6 +57,25 @@ class AzureBlobStorage:
|
|
| 57 |
logger.error(f"Failed to download blob {blob_name}", error=str(e))
|
| 58 |
raise
|
| 59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
async def upload_bytes(self, content: bytes, blob_name: str) -> str:
|
| 61 |
"""Upload bytes to Azure Blob Storage using a specific blob name.
|
| 62 |
|
|
|
|
| 57 |
logger.error(f"Failed to download blob {blob_name}", error=str(e))
|
| 58 |
raise
|
| 59 |
|
| 60 |
+
async def object_size(self, blob_name: str) -> int | None:
|
| 61 |
+
"""Blob size in bytes without downloading it, or None if unavailable.
|
| 62 |
+
|
| 63 |
+
Lets `TabularExecutor` refuse an oversized Parquet BEFORE buffering it
|
| 64 |
+
(F-13). Returns None rather than raising when the size can't be determined —
|
| 65 |
+
the caller falls back to its post-download check. Interface-compatible with
|
| 66 |
+
SupabaseS3Storage.object_size.
|
| 67 |
+
"""
|
| 68 |
+
try:
|
| 69 |
+
async with self._get_blob_client(blob_name) as blob_client:
|
| 70 |
+
props = await blob_client.get_blob_properties()
|
| 71 |
+
size = getattr(props, "size", None)
|
| 72 |
+
return int(size) if size is not None else None
|
| 73 |
+
except Exception as e:
|
| 74 |
+
logger.warning(
|
| 75 |
+
f"Could not determine size of blob {blob_name}", error=repr(e)
|
| 76 |
+
)
|
| 77 |
+
return None
|
| 78 |
+
|
| 79 |
async def upload_bytes(self, content: bytes, blob_name: str) -> str:
|
| 80 |
"""Upload bytes to Azure Blob Storage using a specific blob name.
|
| 81 |
|
|
@@ -69,6 +69,29 @@ class SupabaseS3Storage:
|
|
| 69 |
resp = client.get_object(Bucket=self._bucket, Key=object_name)
|
| 70 |
return resp["Body"].read()
|
| 71 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
async def download_file(self, object_name: str) -> bytes:
|
| 73 |
"""Download an object's bytes. Interface-compatible with AzureBlobStorage.download_file."""
|
| 74 |
try:
|
|
|
|
| 69 |
resp = client.get_object(Bucket=self._bucket, Key=object_name)
|
| 70 |
return resp["Body"].read()
|
| 71 |
|
| 72 |
+
def _size_sync(self, object_name: str) -> int | None:
|
| 73 |
+
client = self._get_client()
|
| 74 |
+
resp = client.head_object(Bucket=self._bucket, Key=object_name)
|
| 75 |
+
size = resp.get("ContentLength")
|
| 76 |
+
return int(size) if size is not None else None
|
| 77 |
+
|
| 78 |
+
async def object_size(self, object_name: str) -> int | None:
|
| 79 |
+
"""Object size in bytes without downloading it, or None if unavailable.
|
| 80 |
+
|
| 81 |
+
A HEAD is what lets `TabularExecutor` refuse an oversized Parquet BEFORE
|
| 82 |
+
buffering it (F-13). Returns None rather than raising when the size can't be
|
| 83 |
+
determined — the caller then falls back to its post-download check, which is
|
| 84 |
+
still better than no bound at all. Interface-compatible with
|
| 85 |
+
AzureBlobStorage.object_size.
|
| 86 |
+
"""
|
| 87 |
+
try:
|
| 88 |
+
return await asyncio.to_thread(self._size_sync, object_name)
|
| 89 |
+
except Exception as e:
|
| 90 |
+
logger.warning(
|
| 91 |
+
f"Could not determine size of object {object_name}", error=repr(e)
|
| 92 |
+
)
|
| 93 |
+
return None
|
| 94 |
+
|
| 95 |
async def download_file(self, object_name: str) -> bytes:
|
| 96 |
"""Download an object's bytes. Interface-compatible with AzureBlobStorage.download_file."""
|
| 97 |
try:
|
|
@@ -90,8 +90,9 @@ class PostgresTraceabilityStore:
|
|
| 90 |
except Exception as exc: # never break the user's answer
|
| 91 |
logger.error(
|
| 92 |
"traceability persist failed",
|
|
|
|
| 93 |
message_id=payload.message_id,
|
| 94 |
-
error=
|
| 95 |
)
|
| 96 |
|
| 97 |
async def get(
|
|
|
|
| 90 |
except Exception as exc: # never break the user's answer
|
| 91 |
logger.error(
|
| 92 |
"traceability persist failed",
|
| 93 |
+
degraded_seam="traceability_persist",
|
| 94 |
message_id=payload.message_id,
|
| 95 |
+
error=repr(exc),
|
| 96 |
)
|
| 97 |
|
| 98 |
async def get(
|