[NOTICKET] fix: review cheap batch — F-19 SSE order, F-22/F-24 analysis_id, F-26, F-16, F-4
Browse filesSix low-risk review findings, batched. No behaviour anyone depends on is removed.
F-19 — SSE order/presence did not match the contract. `check` and router-`help`
emitted NO `sources` event at all, and `structured_flow` emitted `status` BEFORE
`sources`, inverting the documented order on exactly the turns that take longest. A
frontend that initializes per-turn state on `sources` (documented always-present)
therefore never initialized on check/help turns and initialized late on slow ones —
every one of those a plausible FE bug that would have been blamed on the frontend.
Note `stream_help` (the /tools/help endpoint) already emitted it, so the two help
paths disagreed with each other. Purely additive; `sources` stays `[]`.
F-22 — `analysis_id` now 422s unless it parses as a UUID, on BOTH live endpoints
(chat and help kept deliberately identical — two endpoints taking the same field
should not disagree on what is valid). It was never usable: `analyses.id` is
`uuid NOT NULL`, so a non-UUID silently matched nothing, `state_store.ensure` failed
its INSERT, the caller logged a warning and continued with `analysis_state = None`,
and help/readiness/report-write-back all quietly no-op'd while the user still saw a
normal-looking answer.
F-24 — traceability and chart writes are now SKIPPED, and logged, when `analysis_id`
is falsy. Go migration 0007 declares `analysis_id UUID NOT NULL` on both tables (plus
an FK to `analyses(id)` on `message_charts`), and Python passed `analysis_id or ""` —
an empty string cannot cast to uuid, the insert failed, and both writes are
never-throw, so the row vanished with no user-visible symptom and GET /charts then
answered `not_found` for a turn that really did produce a chart. Both live endpoints
require `analysis_id`, so this cannot fire in production; it is the honest fallback.
F-26 — `QueryResult.error` uses `str(e) or repr(e)`. A Fernet InvalidToken has an
empty `str()`, so it reached the assembler prompt, the traceability record and the
report caveats as an EMPTY string while the log (already repr) stayed diagnosable.
Falling back only when str() is empty means no existing error text changes.
F-16 — retrieval cache key gains `settings.redis_prefix`, matching every other cache
key in the service. Two environments sharing one Redis — which the single shared .env
makes plausible — were cross-serving retrieval results. Not cross-tenant (`user_id`
is in the key), but cross-environment. One-time cache-miss storm, bounded by the 1h
TTL.
F-4 — non-Postgres sources are now refused at the executor. ZERO blast radius today:
Go's `database_clients.Service.Create` gates on `isSupportedActive` and only
`postgres` is active, so no such source can be registered. This is a tripwire. The
legacy branch it replaces had no read-only session and no statement_timeout, i.e.
only four of the five defense layers CLAUDE.md §2.5 states unconditionally — and
since the compiler is built with dialect="postgres" regardless of db_type, those
queries would fail on a parse error first, which the never-throw path degrades into
"data not available". The danger was someone fixing THAT without noticing the pooling
branch. Branch commented out per the house convention, with the orphaned import
commented alongside it.
Tests: two event-sequence assertions updated for F-19 (they pinned the missing
`sources`), and six traceability tests now pass an `analysis_id` — they called
`handle()` without one, a shape neither live endpoint can produce.
Verification: suite 434 passed / 0 failed / 7 skipped (was 424; +10 from F-8).
Ruff on touched paths identical to HEAD baseline (S324/I001/E501, all pre-existing).
`import main` OK. Readiness eval 17/17.
Docs: contract gains the corrected SSE guarantee + the 422 rule; DEV_PLAN rows 46-48
added (48 = the floor_08 decision, open for Rifqi).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- API_CONTRACT_BE_PYTHON.md +2 -0
- DEV_PLAN.md +3 -0
- eval/readiness/results/readiness_result_2026-07-24_084154.json +274 -0
- src/agents/chat_handler.py +37 -6
- src/api/v1/help.py +19 -1
- src/api/v2/chat.py +26 -1
- src/query/executor/db.py +42 -8
- src/query/executor/tabular.py +5 -2
- src/retrieval/router.py +8 -1
|
@@ -147,6 +147,8 @@ Behavior notes:
|
|
| 147 |
- The router may classify messages into intents such as `chat`, `help`, `check`, `unstructured_flow`, or `structured_flow`.
|
| 148 |
- `sources` in the stream is **always `[]`** (KM-691) — read the real `sources[]` from `GET /api/v1/traceability` after `done`.
|
| 149 |
- `status` events are optional and should be safe for the frontend to ignore.
|
|
|
|
|
|
|
| 150 |
|
| 151 |
## Tools
|
| 152 |
|
|
|
|
| 147 |
- The router may classify messages into intents such as `chat`, `help`, `check`, `unstructured_flow`, or `structured_flow`.
|
| 148 |
- `sources` in the stream is **always `[]`** (KM-691) — read the real `sources[]` from `GET /api/v1/traceability` after `done`.
|
| 149 |
- `status` events are optional and should be safe for the frontend to ignore.
|
| 150 |
+
- **Fixed 2026-07-23 — `sources` is now genuinely always first, on every path.** It previously did not match this document: the `check` and router-`help` turns emitted **no `sources` event at all**, and the `structured_flow` (slow) path emitted `status` *before* `sources`, inverting the documented order on exactly the turns that take longest. A frontend that initializes per-turn state on `sources` therefore never initialized on check/help turns and initialized late on slow turns. The change is **purely additive** — no event was removed or renamed, and `sources` is still `[]` — so no frontend change is required; behaviour simply now matches what this contract always said.
|
| 151 |
+
- **`analysis_id` must be a UUID (2026-07-23).** `POST /api/v2/chat/stream` and `POST /api/v1/tools/help` now return **`422`** for a non-UUID `analysis_id` instead of accepting it. It was never actually usable: `analyses.id` is `uuid NOT NULL`, so a non-UUID id silently matched no analysis and the turn ran with no state — help, readiness, the report write-back, and the traceability/chart rows all quietly no-op'd while the user still saw a normal answer. The frontend passes Go-issued analysis ids, which are UUIDs, so this should never fire in practice.
|
| 152 |
|
| 153 |
## Tools
|
| 154 |
|
|
@@ -247,6 +247,9 @@ plus the live report bug on analysis `966224d4…`. Same status legend as §0.
|
|
| 247 |
| 43 | **Go identity contract** — what does Go forward, and when? | Rifqi ↔ Harry | ⬜ new | Needed to replace #37's shared secret with real per-user authorization. Until then the store predicates in #38 are defensive only |
|
| 248 |
| 44 | **F-17 / F-18 / F-25 compiler-parity batch** | Rifqi | ✅ new | Shipped 2026-07-23. Three execution-verified review findings that had **no task row** — the tracker jumped from #39 to #40 and lost them. **F-17 (High):** the bare-select check was gated on `if ir.group_by`, so a mixed select with `group_by=[]` passed; Postgres then failed loudly but the pandas path silently DROPPED the column while `output_columns` still advertised it → a real-looking table with a fabricated all-null column. Fixed in `validator.py` (fires whenever any agg is present — no false positives possible) + a presence backstop in `TabularExecutor`. **F-18:** `astype(str)` ran before `na=False`, so `NULL LIKE '%an%'` matched the literal `"nan"`/`"None"`. **F-25:** `SqlCompiler` raised on an empty `in`/`not_in` where pandas and `_column_values`' own docstring implement the empty-set semantics — a two-step plan whose first step legitimately returned zero rows hard-failed instead of answering. 16 new parity tests; one stale test updated (it pinned the old F-25 raise) |
|
| 249 |
| 45 | **F-9 PII in persisted artifacts** — mask the traceability preview + report evidence tables | Rifqi | ⬜ new | **Decided 2026-07-23** (CODE_REVIEW §7) but never tracked. `pii_flag` is an ingestion-time control only: nothing stops the planner SELECTing a flagged column, and the real values then land in `message_traceability.data` (served by a GET that is only now becoming scoped, #40) and are frozen permanently into `reports.content`. Decision: **the assembler still sees values** (otherwise "list our top customers" stops being answerable); mask only the persisted artifacts. Carry `pii_flag` onto `retrieve_data`'s output meta — the catalog is already in scope at `data_access.py:232` |
|
|
|
|
|
|
|
|
|
|
| 250 |
|
| 251 |
**Reading `eval/readiness/results/` (note for future sessions).** Four files are dated
|
| 252 |
2026-07-23. `…_150632.json` scores **4/15 (26.7%)** — that is **not** a product
|
|
|
|
| 247 |
| 43 | **Go identity contract** — what does Go forward, and when? | Rifqi ↔ Harry | ⬜ new | Needed to replace #37's shared secret with real per-user authorization. Until then the store predicates in #38 are defensive only |
|
| 248 |
| 44 | **F-17 / F-18 / F-25 compiler-parity batch** | Rifqi | ✅ new | Shipped 2026-07-23. Three execution-verified review findings that had **no task row** — the tracker jumped from #39 to #40 and lost them. **F-17 (High):** the bare-select check was gated on `if ir.group_by`, so a mixed select with `group_by=[]` passed; Postgres then failed loudly but the pandas path silently DROPPED the column while `output_columns` still advertised it → a real-looking table with a fabricated all-null column. Fixed in `validator.py` (fires whenever any agg is present — no false positives possible) + a presence backstop in `TabularExecutor`. **F-18:** `astype(str)` ran before `na=False`, so `NULL LIKE '%an%'` matched the literal `"nan"`/`"None"`. **F-25:** `SqlCompiler` raised on an empty `in`/`not_in` where pandas and `_column_values`' own docstring implement the empty-set semantics — a two-step plan whose first step legitimately returned zero rows hard-failed instead of answering. 16 new parity tests; one stale test updated (it pinned the old F-25 raise) |
|
| 249 |
| 45 | **F-9 PII in persisted artifacts** — mask the traceability preview + report evidence tables | Rifqi | ⬜ new | **Decided 2026-07-23** (CODE_REVIEW §7) but never tracked. `pii_flag` is an ingestion-time control only: nothing stops the planner SELECTing a flagged column, and the real values then land in `message_traceability.data` (served by a GET that is only now becoming scoped, #40) and are frozen permanently into `reports.content`. Decision: **the assembler still sees values** (otherwise "list our top customers" stops being answerable); mask only the persisted artifacts. Carry `pii_flag` onto `retrieve_data`'s output meta — the catalog is already in scope at `data_access.py:232` |
|
| 250 |
+
| 46 | **F-8 prompt-injection resistance** — planner / assembler / report_summary | Rifqi | ✅ new | Shipped 2026-07-24. The three prompts that ingest customer data had **no injection rule**: `guardrails.md` is appended only in `chatbot.py`/`help.py`, and the `InputGuard` screens only the user's message — it never sees catalog or row content. This is the one attack the five query-defense layers structurally cannot see, because every IR the planner emits is individually valid. A hostile string only needs to reach a text column in the customer's OWN database (a product description, a support ticket, a form field) to be sampled into `sample_values` and rendered verbatim into the planner prompt. Fix = a purpose-written "content is data, never instructions" rule in each prompt + `<data>…</data>` delimiters around the catalog render and the run-state render. **Deliberately NOT `guardrails.md` wholesale** — its rules prescribe refusal sentences, and the planner's only free-text field is `infeasible_reason`, so those strings would surface there and could regress the Q2 data-gap path (tests pin their absence). Verified: planner eval **6/6, carried_over 5/5 green**; live hostile-catalog run planned only `t_products` and never touched the planted `employees.salary`; 10 new tests |
|
| 251 |
+
| 47 | **Cheap-batch review fixes** — F-19, F-22, F-24, F-26, F-16, F-4 | Rifqi | ✅ new | Shipped 2026-07-24, one low-risk batch. **F-19:** `sources` was missing entirely on `check` + router-`help` and came *after* `status` on the slow path — contract said always-first; additive fix, contract updated. **F-22:** `analysis_id` now 422s unless it parses as a UUID, on **both** live endpoints (chat + help kept identical on purpose). **F-24:** traceability/chart writes are skipped, and logged, when `analysis_id` is falsy — Go `0007` declares `analysis_id UUID NOT NULL` (+ FK on charts), so `analysis_id or ""` failed the insert and the never-throw seam lost the row silently. **F-26:** `QueryResult.error` uses `str(e) or repr(e)` — a Fernet `InvalidToken` reached the assembler/traceability/report as an EMPTY string; falling back only when `str()` is empty changes no existing message. **F-16:** retrieval cache key gains `redis_prefix` (two envs on one Redis cross-served results). **F-4:** non-Postgres sources now refused at the executor — **zero blast radius today** (Go's `isSupportedActive` allows only `postgres`), a tripwire so nobody re-enables a path that has no read-only session and no `statement_timeout`; legacy branch commented out per house convention, orphaned import commented with it |
|
| 252 |
+
| 48 | **floor_08 — floor/body disagreement** | Rifqi | ⬜ new | **Needs a decision, found while adding #34 coverage.** When a plan HAS an `analyze_*` step that FAILS but its upstream `retrieve_data` returned rows, the floor passes (`_produced_rows`) while the body rejects it (`has_reportable_result` keys on the failed analysis step). Verified by execution — it is the only shape where the two disagree. Because the "Attempted, Unresolved" section is commented out (`generator.py:527`), such a run leaves **no trace anywhere**: if it is the session's only run, the report generates but comes out empty and the business question renders "Unanswered" — exactly the #33 bug through a different door. Suggested fix: apply the floor's second arm only when the plan has NO analysis step, mirroring the body predicate; `floor_08` then flips to `expected_ready: false`. **Not changed — Rifqi owns the floor** |
|
| 253 |
|
| 254 |
**Reading `eval/readiness/results/` (note for future sessions).** Four files are dated
|
| 255 |
2026-07-23. `…_150632.json` scores **4/15 (26.7%)** — that is **not** a product
|
|
@@ -0,0 +1,274 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"run": {
|
| 3 |
+
"timestamp": "2026-07-24T08:41:54",
|
| 4 |
+
"dataset": "readiness_dataset.json",
|
| 5 |
+
"target": "src/agents/report/readiness.is_report_ready",
|
| 6 |
+
"total": 17,
|
| 7 |
+
"passed": 17,
|
| 8 |
+
"accuracy": 1.0,
|
| 9 |
+
"runtime_avg_ms": 0.0
|
| 10 |
+
},
|
| 11 |
+
"alignment_gap": {
|
| 12 |
+
"count": 2,
|
| 13 |
+
"ids": [
|
| 14 |
+
"align_01",
|
| 15 |
+
"align_02"
|
| 16 |
+
]
|
| 17 |
+
},
|
| 18 |
+
"by_group": {
|
| 19 |
+
"floor": {
|
| 20 |
+
"n": 8,
|
| 21 |
+
"passed": 8,
|
| 22 |
+
"accuracy": 1.0
|
| 23 |
+
},
|
| 24 |
+
"delta": {
|
| 25 |
+
"n": 5,
|
| 26 |
+
"passed": 5,
|
| 27 |
+
"accuracy": 1.0
|
| 28 |
+
},
|
| 29 |
+
"edge": {
|
| 30 |
+
"n": 1,
|
| 31 |
+
"passed": 1,
|
| 32 |
+
"accuracy": 1.0
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"n": 3,
|
| 36 |
+
"passed": 3,
|
| 37 |
+
"accuracy": 1.0
|
| 38 |
+
}
|
| 39 |
+
},
|
| 40 |
+
"cases": [
|
| 41 |
+
{
|
| 42 |
+
"id": "floor_01",
|
| 43 |
+
"group": "floor",
|
| 44 |
+
"expected_ready": false,
|
| 45 |
+
"got_ready": false,
|
| 46 |
+
"expected_missing": [
|
| 47 |
+
"at least one completed analysis"
|
| 48 |
+
],
|
| 49 |
+
"got_missing": [
|
| 50 |
+
"at least one completed analysis"
|
| 51 |
+
],
|
| 52 |
+
"correct": true,
|
| 53 |
+
"aligned": false,
|
| 54 |
+
"gap": false,
|
| 55 |
+
"latency_ms": 0.0
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"id": "floor_02",
|
| 59 |
+
"group": "floor",
|
| 60 |
+
"expected_ready": false,
|
| 61 |
+
"got_ready": false,
|
| 62 |
+
"expected_missing": [
|
| 63 |
+
"at least one completed analysis"
|
| 64 |
+
],
|
| 65 |
+
"got_missing": [
|
| 66 |
+
"at least one completed analysis"
|
| 67 |
+
],
|
| 68 |
+
"correct": true,
|
| 69 |
+
"aligned": false,
|
| 70 |
+
"gap": false,
|
| 71 |
+
"latency_ms": 0.0
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"id": "floor_03",
|
| 75 |
+
"group": "floor",
|
| 76 |
+
"expected_ready": false,
|
| 77 |
+
"got_ready": false,
|
| 78 |
+
"expected_missing": [
|
| 79 |
+
"at least one completed analysis"
|
| 80 |
+
],
|
| 81 |
+
"got_missing": [
|
| 82 |
+
"at least one completed analysis"
|
| 83 |
+
],
|
| 84 |
+
"correct": true,
|
| 85 |
+
"aligned": false,
|
| 86 |
+
"gap": false,
|
| 87 |
+
"latency_ms": 0.0
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"id": "floor_04",
|
| 91 |
+
"group": "floor",
|
| 92 |
+
"expected_ready": true,
|
| 93 |
+
"got_ready": true,
|
| 94 |
+
"expected_missing": [],
|
| 95 |
+
"got_missing": [],
|
| 96 |
+
"correct": true,
|
| 97 |
+
"aligned": true,
|
| 98 |
+
"gap": false,
|
| 99 |
+
"latency_ms": 0.0
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"id": "floor_05",
|
| 103 |
+
"group": "floor",
|
| 104 |
+
"expected_ready": true,
|
| 105 |
+
"got_ready": true,
|
| 106 |
+
"expected_missing": [],
|
| 107 |
+
"got_missing": [],
|
| 108 |
+
"correct": true,
|
| 109 |
+
"aligned": true,
|
| 110 |
+
"gap": false,
|
| 111 |
+
"latency_ms": 0.0
|
| 112 |
+
},
|
| 113 |
+
{
|
| 114 |
+
"id": "floor_06",
|
| 115 |
+
"group": "floor",
|
| 116 |
+
"expected_ready": true,
|
| 117 |
+
"got_ready": true,
|
| 118 |
+
"expected_missing": [],
|
| 119 |
+
"got_missing": [],
|
| 120 |
+
"correct": true,
|
| 121 |
+
"aligned": true,
|
| 122 |
+
"gap": false,
|
| 123 |
+
"latency_ms": 0.0
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"id": "floor_07",
|
| 127 |
+
"group": "floor",
|
| 128 |
+
"expected_ready": true,
|
| 129 |
+
"got_ready": true,
|
| 130 |
+
"expected_missing": [],
|
| 131 |
+
"got_missing": [],
|
| 132 |
+
"correct": true,
|
| 133 |
+
"aligned": true,
|
| 134 |
+
"gap": false,
|
| 135 |
+
"latency_ms": 0.0
|
| 136 |
+
},
|
| 137 |
+
{
|
| 138 |
+
"id": "floor_08",
|
| 139 |
+
"group": "floor",
|
| 140 |
+
"expected_ready": true,
|
| 141 |
+
"got_ready": true,
|
| 142 |
+
"expected_missing": [],
|
| 143 |
+
"got_missing": [],
|
| 144 |
+
"correct": true,
|
| 145 |
+
"aligned": true,
|
| 146 |
+
"gap": false,
|
| 147 |
+
"latency_ms": 0.0
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"id": "delta_01",
|
| 151 |
+
"group": "delta",
|
| 152 |
+
"expected_ready": false,
|
| 153 |
+
"got_ready": false,
|
| 154 |
+
"expected_missing": [
|
| 155 |
+
"a new analysis since the last report"
|
| 156 |
+
],
|
| 157 |
+
"got_missing": [
|
| 158 |
+
"a new analysis since the last report"
|
| 159 |
+
],
|
| 160 |
+
"correct": true,
|
| 161 |
+
"aligned": true,
|
| 162 |
+
"gap": false,
|
| 163 |
+
"latency_ms": 0.0
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"id": "delta_02",
|
| 167 |
+
"group": "delta",
|
| 168 |
+
"expected_ready": true,
|
| 169 |
+
"got_ready": true,
|
| 170 |
+
"expected_missing": [],
|
| 171 |
+
"got_missing": [],
|
| 172 |
+
"correct": true,
|
| 173 |
+
"aligned": true,
|
| 174 |
+
"gap": false,
|
| 175 |
+
"latency_ms": 0.0
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"id": "delta_03",
|
| 179 |
+
"group": "delta",
|
| 180 |
+
"expected_ready": true,
|
| 181 |
+
"got_ready": true,
|
| 182 |
+
"expected_missing": [],
|
| 183 |
+
"got_missing": [],
|
| 184 |
+
"correct": true,
|
| 185 |
+
"aligned": true,
|
| 186 |
+
"gap": false,
|
| 187 |
+
"latency_ms": 0.0
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"id": "delta_04",
|
| 191 |
+
"group": "delta",
|
| 192 |
+
"expected_ready": false,
|
| 193 |
+
"got_ready": false,
|
| 194 |
+
"expected_missing": [
|
| 195 |
+
"a new analysis since the last report"
|
| 196 |
+
],
|
| 197 |
+
"got_missing": [
|
| 198 |
+
"a new analysis since the last report"
|
| 199 |
+
],
|
| 200 |
+
"correct": true,
|
| 201 |
+
"aligned": true,
|
| 202 |
+
"gap": false,
|
| 203 |
+
"latency_ms": 0.0
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"id": "delta_05",
|
| 207 |
+
"group": "delta",
|
| 208 |
+
"expected_ready": false,
|
| 209 |
+
"got_ready": false,
|
| 210 |
+
"expected_missing": [
|
| 211 |
+
"a new analysis since the last report"
|
| 212 |
+
],
|
| 213 |
+
"got_missing": [
|
| 214 |
+
"a new analysis since the last report"
|
| 215 |
+
],
|
| 216 |
+
"correct": true,
|
| 217 |
+
"aligned": true,
|
| 218 |
+
"gap": false,
|
| 219 |
+
"latency_ms": 0.0
|
| 220 |
+
},
|
| 221 |
+
{
|
| 222 |
+
"id": "edge_01",
|
| 223 |
+
"group": "edge",
|
| 224 |
+
"expected_ready": false,
|
| 225 |
+
"got_ready": false,
|
| 226 |
+
"expected_missing": [
|
| 227 |
+
"at least one completed analysis"
|
| 228 |
+
],
|
| 229 |
+
"got_missing": [
|
| 230 |
+
"at least one completed analysis"
|
| 231 |
+
],
|
| 232 |
+
"correct": true,
|
| 233 |
+
"aligned": false,
|
| 234 |
+
"gap": false,
|
| 235 |
+
"latency_ms": 0.0
|
| 236 |
+
},
|
| 237 |
+
{
|
| 238 |
+
"id": "align_01",
|
| 239 |
+
"group": "alignment",
|
| 240 |
+
"expected_ready": true,
|
| 241 |
+
"got_ready": true,
|
| 242 |
+
"expected_missing": [],
|
| 243 |
+
"got_missing": [],
|
| 244 |
+
"correct": true,
|
| 245 |
+
"aligned": false,
|
| 246 |
+
"gap": true,
|
| 247 |
+
"latency_ms": 0.0
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"id": "align_02",
|
| 251 |
+
"group": "alignment",
|
| 252 |
+
"expected_ready": true,
|
| 253 |
+
"got_ready": true,
|
| 254 |
+
"expected_missing": [],
|
| 255 |
+
"got_missing": [],
|
| 256 |
+
"correct": true,
|
| 257 |
+
"aligned": false,
|
| 258 |
+
"gap": true,
|
| 259 |
+
"latency_ms": 0.0
|
| 260 |
+
},
|
| 261 |
+
{
|
| 262 |
+
"id": "align_03",
|
| 263 |
+
"group": "alignment",
|
| 264 |
+
"expected_ready": true,
|
| 265 |
+
"got_ready": true,
|
| 266 |
+
"expected_missing": [],
|
| 267 |
+
"got_missing": [],
|
| 268 |
+
"correct": true,
|
| 269 |
+
"aligned": true,
|
| 270 |
+
"gap": false,
|
| 271 |
+
"latency_ms": 0.0
|
| 272 |
+
}
|
| 273 |
+
]
|
| 274 |
+
}
|
|
@@ -470,6 +470,11 @@ class ChatHandler:
|
|
| 470 |
yield {"event": "error", "data": f"Document retrieval failed: {e}"}
|
| 471 |
return
|
| 472 |
elif intent == "check":
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 473 |
try:
|
| 474 |
# Scope check to the analysis catalog: it holds only this room's
|
| 475 |
# bound sources and their real names (a DB shows as "xl test", not
|
|
@@ -516,6 +521,10 @@ class ChatHandler:
|
|
| 516 |
# yield {"event": "done", "data": ""}
|
| 517 |
# return
|
| 518 |
elif intent == "help":
|
|
|
|
|
|
|
|
|
|
|
|
|
| 519 |
try:
|
| 520 |
state = analysis_state or await self._load_analysis_state(analysis_id)
|
| 521 |
except Exception as e:
|
|
@@ -670,8 +679,21 @@ class ChatHandler:
|
|
| 670 |
"""
|
| 671 |
if pad.message_id is None:
|
| 672 |
return
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 673 |
try:
|
| 674 |
-
payload = pad.build(analysis_id
|
| 675 |
await self._get_traceability_store().save(payload)
|
| 676 |
except Exception as e: # noqa: BLE001 — never break the answer on a trace slip
|
| 677 |
logger.warning(
|
|
@@ -726,6 +748,13 @@ class ChatHandler:
|
|
| 726 |
if ac:
|
| 727 |
run_kw["assembler_callbacks"] = ac
|
| 728 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 729 |
# R4: bridge the coordinator's per-stage progress callback to SSE `status`
|
| 730 |
# events so the stream isn't silent for ~12s (and proxies don't drop the
|
| 731 |
# idle connection). Status events only appear if the coordinator calls back.
|
|
@@ -763,9 +792,6 @@ class ChatHandler:
|
|
| 763 |
yield {"event": "error", "data": f"Analysis failed: {e}"}
|
| 764 |
return
|
| 765 |
|
| 766 |
-
# Sources live in traceability now (KM-691), derived from the run's
|
| 767 |
-
# retrieve_data calls; the stream stays text-only.
|
| 768 |
-
yield {"event": "sources", "data": json.dumps([])}
|
| 769 |
yield {"event": "chunk", "data": result.chat_answer}
|
| 770 |
try:
|
| 771 |
# Stamp identity from the request scope: owner + the shared session id
|
|
@@ -790,13 +816,18 @@ class ChatHandler:
|
|
| 790 |
# SPINE_V2_PLAN §4.4: chart rows are written before `done`; the FE fetches
|
| 791 |
# GET /api/v1/charts unconditionally on every `done` (no polling race).
|
| 792 |
try:
|
| 793 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 794 |
for task_result in result.analysis_record.results_snapshot.values():
|
| 795 |
for output in task_result.outputs:
|
| 796 |
if output.kind == "chart" and isinstance(output.value, dict):
|
| 797 |
await self._get_chart_store().save(
|
| 798 |
message_id=pad.message_id,
|
| 799 |
-
analysis_id=analysis_id
|
| 800 |
user_id=user_id,
|
| 801 |
record_id=result.analysis_record.record_id,
|
| 802 |
envelope=output.value,
|
|
|
|
| 470 |
yield {"event": "error", "data": f"Document retrieval failed: {e}"}
|
| 471 |
return
|
| 472 |
elif intent == "check":
|
| 473 |
+
# The contract documents `sources` as always present and always first
|
| 474 |
+
# (it stays `[]` — real sources live in GET /traceability). This branch
|
| 475 |
+
# used to skip it, so an FE that initializes per-turn state on `sources`
|
| 476 |
+
# silently never initialized on a check turn. Purely additive. (F-19)
|
| 477 |
+
yield {"event": "sources", "data": json.dumps([])}
|
| 478 |
try:
|
| 479 |
# Scope check to the analysis catalog: it holds only this room's
|
| 480 |
# bound sources and their real names (a DB shows as "xl test", not
|
|
|
|
| 521 |
# yield {"event": "done", "data": ""}
|
| 522 |
# return
|
| 523 |
elif intent == "help":
|
| 524 |
+
# Same as the check branch: `sources` is contractually always-first.
|
| 525 |
+
# Note `stream_help` (the dedicated /tools/help endpoint) already emits
|
| 526 |
+
# it, so the two help paths disagreed with each other until now. (F-19)
|
| 527 |
+
yield {"event": "sources", "data": json.dumps([])}
|
| 528 |
try:
|
| 529 |
state = analysis_state or await self._load_analysis_state(analysis_id)
|
| 530 |
except Exception as e:
|
|
|
|
| 679 |
"""
|
| 680 |
if pad.message_id is None:
|
| 681 |
return
|
| 682 |
+
if not analysis_id:
|
| 683 |
+
# `message_traceability.analysis_id` is `UUID NOT NULL` (Go migration
|
| 684 |
+
# 0007). Passing `analysis_id or ""` sent an empty string, which cannot
|
| 685 |
+
# cast to uuid — the INSERT failed, the never-throw seam swallowed it, and
|
| 686 |
+
# the row vanished with no user-visible symptom. Skipping the write for a
|
| 687 |
+
# turn that genuinely has no analysis is the honest outcome, and it says
|
| 688 |
+
# so in the log instead of failing invisibly. (F-24)
|
| 689 |
+
logger.info(
|
| 690 |
+
"traceability flush skipped — no analysis_id",
|
| 691 |
+
degraded_seam="traceability_flush_no_analysis",
|
| 692 |
+
message_id=pad.message_id,
|
| 693 |
+
)
|
| 694 |
+
return
|
| 695 |
try:
|
| 696 |
+
payload = pad.build(analysis_id, user_id, pad.message_id)
|
| 697 |
await self._get_traceability_store().save(payload)
|
| 698 |
except Exception as e: # noqa: BLE001 — never break the answer on a trace slip
|
| 699 |
logger.warning(
|
|
|
|
| 748 |
if ac:
|
| 749 |
run_kw["assembler_callbacks"] = ac
|
| 750 |
|
| 751 |
+
# Contract order is `sources` -> `status`* -> `chunk`* -> `done`. This used to
|
| 752 |
+
# be emitted after the status loop, so the slow path (the ONLY path that emits
|
| 753 |
+
# `status`) inverted the documented order and the FE saw `status` first on
|
| 754 |
+
# exactly the turns that take longest. Sources live in traceability now
|
| 755 |
+
# (KM-691); the event stays `[]` and the stream stays text-only. (F-19)
|
| 756 |
+
yield {"event": "sources", "data": json.dumps([])}
|
| 757 |
+
|
| 758 |
# R4: bridge the coordinator's per-stage progress callback to SSE `status`
|
| 759 |
# events so the stream isn't silent for ~12s (and proxies don't drop the
|
| 760 |
# idle connection). Status events only appear if the coordinator calls back.
|
|
|
|
| 792 |
yield {"event": "error", "data": f"Analysis failed: {e}"}
|
| 793 |
return
|
| 794 |
|
|
|
|
|
|
|
|
|
|
| 795 |
yield {"event": "chunk", "data": result.chat_answer}
|
| 796 |
try:
|
| 797 |
# Stamp identity from the request scope: owner + the shared session id
|
|
|
|
| 816 |
# SPINE_V2_PLAN §4.4: chart rows are written before `done`; the FE fetches
|
| 817 |
# GET /api/v1/charts unconditionally on every `done` (no polling race).
|
| 818 |
try:
|
| 819 |
+
# `message_charts.analysis_id` is `uuid NOT NULL REFERENCES analyses(id)`
|
| 820 |
+
# (Go migration 0007), so an empty string both fails the cast and violates
|
| 821 |
+
# the FK. The write is never-throw, so the chart row disappeared silently
|
| 822 |
+
# and GET /api/v1/charts then answered `not_found` for a turn that really
|
| 823 |
+
# did produce a chart. Skip rather than lose it invisibly. (F-24)
|
| 824 |
+
if pad is not None and pad.message_id and analysis_id:
|
| 825 |
for task_result in result.analysis_record.results_snapshot.values():
|
| 826 |
for output in task_result.outputs:
|
| 827 |
if output.kind == "chart" and isinstance(output.value, dict):
|
| 828 |
await self._get_chart_store().save(
|
| 829 |
message_id=pad.message_id,
|
| 830 |
+
analysis_id=analysis_id,
|
| 831 |
user_id=user_id,
|
| 832 |
record_id=result.analysis_record.record_id,
|
| 833 |
envelope=output.value,
|
|
@@ -20,7 +20,7 @@ import json
|
|
| 20 |
import uuid
|
| 21 |
|
| 22 |
from fastapi import APIRouter, Depends, HTTPException
|
| 23 |
-
from pydantic import BaseModel
|
| 24 |
from sqlalchemy.ext.asyncio import AsyncSession
|
| 25 |
from sse_starlette.sse import EventSourceResponse
|
| 26 |
|
|
@@ -40,6 +40,24 @@ class HelpRequest(BaseModel):
|
|
| 40 |
user_id: str
|
| 41 |
analysis_id: str
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
@router.post("/help")
|
| 45 |
@log_execution(logger)
|
|
|
|
| 20 |
import uuid
|
| 21 |
|
| 22 |
from fastapi import APIRouter, Depends, HTTPException
|
| 23 |
+
from pydantic import BaseModel, field_validator
|
| 24 |
from sqlalchemy.ext.asyncio import AsyncSession
|
| 25 |
from sse_starlette.sse import EventSourceResponse
|
| 26 |
|
|
|
|
| 40 |
user_id: str
|
| 41 |
analysis_id: str
|
| 42 |
|
| 43 |
+
@field_validator("analysis_id")
|
| 44 |
+
@classmethod
|
| 45 |
+
def _analysis_id_is_uuid(cls, v: str) -> str:
|
| 46 |
+
"""Same boundary rule as `POST /api/v2/chat/stream`. (F-22)
|
| 47 |
+
|
| 48 |
+
Kept identical on purpose: a non-UUID id can never match a row in `analyses`
|
| 49 |
+
(`id uuid NOT NULL`), so the state read silently returns nothing and Help
|
| 50 |
+
answers from an empty state instead of saying the id was wrong. Two live
|
| 51 |
+
endpoints taking the same field should not disagree on what is valid.
|
| 52 |
+
"""
|
| 53 |
+
try:
|
| 54 |
+
uuid.UUID(v)
|
| 55 |
+
except (ValueError, AttributeError, TypeError):
|
| 56 |
+
raise ValueError(
|
| 57 |
+
"analysis_id must be a UUID (the id of an existing analysis)"
|
| 58 |
+
) from None
|
| 59 |
+
return v
|
| 60 |
+
|
| 61 |
|
| 62 |
@router.post("/help")
|
| 63 |
@log_execution(logger)
|
|
@@ -24,7 +24,7 @@ import uuid
|
|
| 24 |
from typing import Any
|
| 25 |
|
| 26 |
from fastapi import APIRouter, Depends, HTTPException, Request
|
| 27 |
-
from pydantic import BaseModel
|
| 28 |
from sqlalchemy.ext.asyncio import AsyncSession
|
| 29 |
from sse_starlette.sse import EventSourceResponse
|
| 30 |
|
|
@@ -82,6 +82,31 @@ class ChatRequest(BaseModel):
|
|
| 82 |
analysis_id: str
|
| 83 |
message: str
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
@router.post("/chat/stream")
|
| 87 |
# Rate limit per client IP. `slowapi` needs a Starlette `Request` param named
|
|
|
|
| 24 |
from typing import Any
|
| 25 |
|
| 26 |
from fastapi import APIRouter, Depends, HTTPException, Request
|
| 27 |
+
from pydantic import BaseModel, field_validator
|
| 28 |
from sqlalchemy.ext.asyncio import AsyncSession
|
| 29 |
from sse_starlette.sse import EventSourceResponse
|
| 30 |
|
|
|
|
| 82 |
analysis_id: str
|
| 83 |
message: str
|
| 84 |
|
| 85 |
+
@field_validator("analysis_id")
|
| 86 |
+
@classmethod
|
| 87 |
+
def _analysis_id_is_uuid(cls, v: str) -> str:
|
| 88 |
+
"""422 on a non-UUID `analysis_id` instead of degrading silently. (F-22)
|
| 89 |
+
|
| 90 |
+
`analyses.id` is `uuid NOT NULL`, so a non-UUID value can never match a real
|
| 91 |
+
analysis. Without this the turn still ran: `state_store.ensure` failed its
|
| 92 |
+
INSERT, the caller logged a warning and continued with `analysis_state = None`,
|
| 93 |
+
and help, readiness and the report write-back all silently no-op'd for that
|
| 94 |
+
turn. The user saw a normal-looking answer whose analysis state was never
|
| 95 |
+
persisted. A clear client error is the honest outcome.
|
| 96 |
+
|
| 97 |
+
Also closes the F-24 half of the same problem: `message_traceability` and
|
| 98 |
+
`message_charts` declare `analysis_id UUID NOT NULL` (Go migration 0007, with
|
| 99 |
+
an FK to `analyses(id)` on the charts table), and both writes are never-throw
|
| 100 |
+
— so a non-UUID id silently lost the provenance and chart rows too.
|
| 101 |
+
"""
|
| 102 |
+
try:
|
| 103 |
+
uuid.UUID(v)
|
| 104 |
+
except (ValueError, AttributeError, TypeError):
|
| 105 |
+
raise ValueError(
|
| 106 |
+
"analysis_id must be a UUID (the id of an existing analysis)"
|
| 107 |
+
) from None
|
| 108 |
+
return v
|
| 109 |
+
|
| 110 |
|
| 111 |
@router.post("/chat/stream")
|
| 112 |
# Rate limit per client IP. `slowapi` needs a Starlette `Request` param named
|
|
@@ -32,12 +32,16 @@ from ...database_client.database_client_service import database_client_service
|
|
| 32 |
from ...database_client.engine import user_engine_cache
|
| 33 |
from ...db.postgres.connection import AsyncSessionLocal
|
| 34 |
from ...middlewares.logging import get_logger
|
| 35 |
-
from ...pipeline.db_pipeline import db_pipeline_service
|
| 36 |
from ...utils.db_credential_encryption import decrypt_credentials_dict
|
| 37 |
from ..compiler.sql import CompiledSql, SqlCompiler
|
| 38 |
from ..ir.models import QueryIR
|
| 39 |
from .base import BaseExecutor, QueryResult
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
logger = get_logger("db_executor")
|
| 42 |
|
| 43 |
_QUERY_TIMEOUT_SECONDS = 30
|
|
@@ -131,7 +135,13 @@ class DbExecutor(BaseExecutor):
|
|
| 131 |
source_id=ir.source_id,
|
| 132 |
backend="sql",
|
| 133 |
elapsed_ms=elapsed_ms,
|
| 134 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
table_id=ir.table_id,
|
| 136 |
table_name=table_name,
|
| 137 |
source_name=source_name,
|
|
@@ -207,12 +217,36 @@ class DbExecutor(BaseExecutor):
|
|
| 207 |
result = conn.execute(text(compiled.sql), compiled.params)
|
| 208 |
return list(result.keys()), [dict(row) for row in result.mappings()]
|
| 209 |
|
| 210 |
-
#
|
| 211 |
-
#
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
|
| 217 |
# ------------------------------------------------------------------
|
| 218 |
# Speculative pre-connect (DB3)
|
|
|
|
| 32 |
from ...database_client.engine import user_engine_cache
|
| 33 |
from ...db.postgres.connection import AsyncSessionLocal
|
| 34 |
from ...middlewares.logging import get_logger
|
|
|
|
| 35 |
from ...utils.db_credential_encryption import decrypt_credentials_dict
|
| 36 |
from ..compiler.sql import CompiledSql, SqlCompiler
|
| 37 |
from ..ir.models import QueryIR
|
| 38 |
from .base import BaseExecutor, QueryResult
|
| 39 |
|
| 40 |
+
# Orphaned by the F-4 tripwire (2026-07-23): the only consumer was the legacy
|
| 41 |
+
# non-postgres branch in `_run_sync`, now commented out there. Restore this import
|
| 42 |
+
# alongside that branch (kept out of the block above so import sorting stays clean).
|
| 43 |
+
# from ...pipeline.db_pipeline import db_pipeline_service
|
| 44 |
+
|
| 45 |
logger = get_logger("db_executor")
|
| 46 |
|
| 47 |
_QUERY_TIMEOUT_SECONDS = 30
|
|
|
|
| 135 |
source_id=ir.source_id,
|
| 136 |
backend="sql",
|
| 137 |
elapsed_ms=elapsed_ms,
|
| 138 |
+
# `str(e) or repr(e)`, not bare repr: the log above already uses repr,
|
| 139 |
+
# but this payload reaches the assembler prompt, the traceability
|
| 140 |
+
# record and the report caveats — where a Fernet InvalidToken arrived
|
| 141 |
+
# as an EMPTY string, so the user-facing artifact said nothing while
|
| 142 |
+
# the log was diagnosable. Falling back only when str() is empty means
|
| 143 |
+
# no existing error text changes. (F-26)
|
| 144 |
+
error=str(e) or repr(e),
|
| 145 |
table_id=ir.table_id,
|
| 146 |
table_name=table_name,
|
| 147 |
source_name=source_name,
|
|
|
|
| 217 |
result = conn.execute(text(compiled.sql), compiled.params)
|
| 218 |
return list(result.keys()), [dict(row) for row in result.mappings()]
|
| 219 |
|
| 220 |
+
# TRIPWIRE (F-4, 2026-07-23). Below this line was the legacy per-call path for
|
| 221 |
+
# non-postgres db_types, whose own comment conceded "these never set
|
| 222 |
+
# read-only/timeout before, so behavior is unchanged". That means such a source
|
| 223 |
+
# would get only four of the five documented defense layers: IR validation, the
|
| 224 |
+
# compiler whitelist, the sqlglot guard and LIMIT — but NO read-only session and
|
| 225 |
+
# NO statement_timeout. `CLAUDE.md` §2.5 states all five unconditionally; in
|
| 226 |
+
# truth they were conditional on db_type, and nothing said so where a reader
|
| 227 |
+
# would look.
|
| 228 |
+
#
|
| 229 |
+
# Zero blast radius today: Go's `database_clients.Service.Create` gates on
|
| 230 |
+
# `isSupportedActive`, and only `postgres` is `active` — mysql/sqlserver/
|
| 231 |
+
# bigquery/snowflake are all "Coming soon", so no such source can be registered.
|
| 232 |
+
# This refuses loudly the day someone flips that flag, instead of silently
|
| 233 |
+
# executing against a customer's database with two guardrails missing.
|
| 234 |
+
#
|
| 235 |
+
# Note the compiler is built with dialect="postgres" regardless of db_type and
|
| 236 |
+
# the sqlglot guard parses with read="postgres", so these queries would fail on
|
| 237 |
+
# a parse error anyway — which the never-throw path would degrade into "data not
|
| 238 |
+
# available", masquerading as a data problem. The danger is someone fixing THAT
|
| 239 |
+
# without noticing the pooling branch. Re-enabling this path requires
|
| 240 |
+
# dialect-correct compilation AND session hardening, not just a dialect string.
|
| 241 |
+
raise ValueError(
|
| 242 |
+
f"source type {db_type!r} is not supported for analysis yet — only "
|
| 243 |
+
"PostgreSQL sources can be queried safely (read-only session and query "
|
| 244 |
+
"timeout are not yet implemented for other database types)"
|
| 245 |
+
)
|
| 246 |
+
# with db_pipeline_service.engine_scope(db_type, creds) as eng:
|
| 247 |
+
# with eng.connect() as conn:
|
| 248 |
+
# result = conn.execute(text(compiled.sql), compiled.params)
|
| 249 |
+
# return list(result.keys()), [dict(row) for row in result.mappings()]
|
| 250 |
|
| 251 |
# ------------------------------------------------------------------
|
| 252 |
# Speculative pre-connect (DB3)
|
|
@@ -204,15 +204,18 @@ class TabularExecutor(BaseExecutor):
|
|
| 204 |
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
| 205 |
logger.error(
|
| 206 |
"tabular executor failed",
|
|
|
|
| 207 |
source_id=ir.source_id,
|
| 208 |
-
error=
|
| 209 |
elapsed_ms=elapsed_ms,
|
| 210 |
)
|
| 211 |
return QueryResult(
|
| 212 |
source_id=ir.source_id,
|
| 213 |
backend="tabular",
|
| 214 |
elapsed_ms=elapsed_ms,
|
| 215 |
-
|
|
|
|
|
|
|
| 216 |
table_id=ir.table_id,
|
| 217 |
table_name=table_name,
|
| 218 |
source_name=source_name,
|
|
|
|
| 204 |
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
| 205 |
logger.error(
|
| 206 |
"tabular executor failed",
|
| 207 |
+
degraded_seam="tabular_execute",
|
| 208 |
source_id=ir.source_id,
|
| 209 |
+
error=repr(e),
|
| 210 |
elapsed_ms=elapsed_ms,
|
| 211 |
)
|
| 212 |
return QueryResult(
|
| 213 |
source_id=ir.source_id,
|
| 214 |
backend="tabular",
|
| 215 |
elapsed_ms=elapsed_ms,
|
| 216 |
+
# See db.py: fall back to repr only when str() is empty, so no
|
| 217 |
+
# existing user-facing error text changes. (F-26)
|
| 218 |
+
error=str(e) or repr(e),
|
| 219 |
table_id=ir.table_id,
|
| 220 |
table_name=table_name,
|
| 221 |
source_name=source_name,
|
|
@@ -13,6 +13,7 @@ import hashlib
|
|
| 13 |
import json
|
| 14 |
from dataclasses import asdict
|
| 15 |
|
|
|
|
| 16 |
from src.db.redis.connection import get_redis
|
| 17 |
from src.middlewares.logging import get_logger
|
| 18 |
from src.retrieval.base import RetrievalResult
|
|
@@ -21,7 +22,13 @@ from src.retrieval.document import DocumentRetriever
|
|
| 21 |
logger = get_logger("retrieval_router")
|
| 22 |
|
| 23 |
_CACHE_TTL = 3600
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
|
| 27 |
class RetrievalRouter:
|
|
|
|
| 13 |
import json
|
| 14 |
from dataclasses import asdict
|
| 15 |
|
| 16 |
+
from src.config.settings import settings
|
| 17 |
from src.db.redis.connection import get_redis
|
| 18 |
from src.middlewares.logging import get_logger
|
| 19 |
from src.retrieval.base import RetrievalResult
|
|
|
|
| 22 |
logger = get_logger("retrieval_router")
|
| 23 |
|
| 24 |
_CACHE_TTL = 3600
|
| 25 |
+
# Namespaced with `settings.redis_prefix`, like every other cache key in the service
|
| 26 |
+
# (cf. `api/v1/chat.py::build_cache_key`). Without it, two environments pointed at the
|
| 27 |
+
# same Redis — which the single shared `.env` makes entirely plausible — cross-serve
|
| 28 |
+
# each other's retrieval results. Not cross-TENANT (`user_id` is in the key), but
|
| 29 |
+
# cross-environment, which is confusing in exactly the way stale-cache bugs are. The
|
| 30 |
+
# one-time cost is a cache-miss storm bounded by the 1h TTL. (F-16)
|
| 31 |
+
_CACHE_KEY_PREFIX = f"{settings.redis_prefix}retrieval"
|
| 32 |
|
| 33 |
|
| 34 |
class RetrievalRouter:
|