Rifqi Hafizuddin commited on
Commit Β·
d953ce6
1
Parent(s): 6a3b169
[NOTICKET] update table names to dedorch db
Browse files- DEV_PLAN.md +11 -11
- src/db/postgres/models.py +25 -8
DEV_PLAN.md
CHANGED
|
@@ -29,15 +29,15 @@ describes the *in-flight plan* that changes it. New decisions from the 2026-06-2
|
|
| 29 |
Confirms the 2026-06-24 direction and adds these concrete changes (folded into Β§4 as tasks 21β28):
|
| 30 |
|
| 31 |
1. **Rename `analysis_records` β `agent_observability`** β no "record" in the name (avoids clashing
|
| 32 |
-
with Go's `
|
| 33 |
**hand the schema to Harry** so his dedorch migration creates it on env-move β this is how #16 is
|
| 34 |
resolved (the table exists post-`SKIP_INIT_DB`). Open: does the write scope broaden to *every agent
|
| 35 |
call* (true observability) or stay *one row per slow-path analysis*?
|
| 36 |
-
2. **`
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
3. **`
|
| 40 |
-
|
| 41 |
4. **Reports: Go owns ALL writes.** Report stays a **skill** (no router intent): FE β Go β Python;
|
| 42 |
Python only returns content. Input = the records table (now `agent_observability`); edit-mode may
|
| 43 |
also need the last report. (β #7/#18/#24)
|
|
@@ -112,8 +112,8 @@ Status legend: β¬ not started Β· π in progress Β· β
done Β· β blocked Β·
|
|
| 112 |
|---|---|---|---|---|
|
| 113 |
| 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | β
| Done 2026-06-25 (one commit w/ #2). Unwired in `orchestration.py`, `intent_router.md`, `chat_handler.py`, `tools.py`; `problem_statement.py` kept intact |
|
| 114 |
| 2 | Drop `problem_validated`: gate neutered; `is_report_ready`/`report_floor` β **β₯1 completed analysis** only, no-LLM | Rifqi | β
| Done 2026-06-25. `gate.py` no-op, gate call site commented in `chat_handler.py`, `report_floor` drops the goal check. Tests updated (`test_gate`/`test_chat_handler`/`test_readiness`). Suite: **284 passed, 7 skipped**; ruff clean |
|
| 115 |
-
| 3 | dedorch `
|
| 116 |
-
| 4 | Update Python `
|
| 117 |
| 5 | Report generator β `objective`+`business_questions`, "generated by {user}", formal outline | Sofhia | β
| Goal-shape (KM-674) + author name (#19) + outline (KM-680): Objective β Business Questions β Executive Summary β Key Findings β EDA β Notes & Limitations β How This Was Analyzed |
|
| 118 |
| 6 | Report skill input contract: `analysis_id` + `user_id` (no `chat_history`) | Sofhia/Rifqi | β
| No-op: `POST /report` already takes only analysis_id + user_id (records-based). Documented in API_ENDPOINTS.md Β§5. *(Edit-mode input revisited in #24.)* |
|
| 119 |
| 7 | `report_id` state update via request to Go, not direct DB | Sofhia + Harry | β¬ | Needs Go endpoint. **Checkpoint:** Go owns ALL `reports` writes; Python stops any direct insert/update and only returns content; report stays a **skill** (no intent). See #18 |
|
|
@@ -134,7 +134,7 @@ Status legend: β¬ not started Β· π in progress Β· β
done Β· β blocked Β·
|
|
| 134 |
| 22 | Finalize `agent_observability` schema β hand to Harry for the dedorch migration/startup | Rifqi β Harry | β¬ new | Resolves #16 (table created post-cutover). Python still creates it locally until then |
|
| 135 |
| 23 | Report markdown formatting: tables, **bold**, *italic*, horizontal separators | Sofhia | β
| Done 2026-06-25. Added `---` separators between header + each section in `_render_markdown`. Tables (EDA) / bold (method labels) / italic (meta + citations) already emitted. Relaxed `report_summary.md` to allow inline `**bold**`/`*italic*` for emphasis (kept no-headings/no-bullets so it doesn't duplicate the section structure / Key Findings). Compile + ruff clean |
|
| 136 |
| 24 | Clarify report input contract: records table (+ `last_report` for edit mode?) | Rifqi/Sofhia β Harry | β¬ new | Edit-mode input left open at the checkpoint |
|
| 137 |
-
| 25 |
|
| 138 |
| 26 | **Charts (DEFERRED):** store Plotly JSON in a future `chart` table (not matplotlib PNG) | β | βΈοΈ | After the markdown path is done end-to-end |
|
| 139 |
| 27 | **Images (DEFERRED):** image table (id, analysis_id, msg/report ref, order) + originals in a bucket | β | βΈοΈ | Maintenance-heavy; parked |
|
| 140 |
| 28 | **UI research** (FE): new-analysis form, knowledge menu (user vs analysis level), report artifacts + version selector | Team | β¬ new | No dedicated UI person; interview + old analysis UI removed |
|
|
@@ -142,8 +142,8 @@ Status legend: β¬ not started Β· π in progress Β· β
done Β· β blocked Β·
|
|
| 142 |
## 5. Critical path & sequencing
|
| 143 |
|
| 144 |
- **Critical path:** #3 β #4 (de-risked by getattr tolerance). #21 β #22 (rename + hand schema to Harry). HF deploy (#13) for the playground.
|
| 145 |
-
- **Parallelizable now
|
| 146 |
-
- **Harry-blocked:** #3 (blocks #4), #7 (Go endpoint), #18 (Go state ownership), #24
|
| 147 |
- **Demo gate (playground, #13):** deploy the remove-`problem_statement` work to HF β slow path (#15 β
)
|
| 148 |
and the report path are verified locally, and #16 is resolved (#22 hands Harry the schema). **Keep it
|
| 149 |
playground-able.**
|
|
|
|
| 29 |
Confirms the 2026-06-24 direction and adds these concrete changes (folded into Β§4 as tasks 21β28):
|
| 30 |
|
| 31 |
1. **Rename `analysis_records` β `agent_observability`** β no "record" in the name (avoids clashing
|
| 32 |
+
with Go's `analyses_messages`). It **stays Python-owned** for now; once the columns/types are fixed,
|
| 33 |
**hand the schema to Harry** so his dedorch migration creates it on env-move β this is how #16 is
|
| 34 |
resolved (the table exists post-`SKIP_INIT_DB`). Open: does the write scope broaden to *every agent
|
| 35 |
call* (true observability) or stay *one row per slow-path analysis*?
|
| 36 |
+
2. **`analyses` table (Go) β `status`, `data_bind` + `data_bind_version`, `report_collection`** (id+version).
|
| 37 |
+
**Verified 2026-06-25: these + `user_id` are ALREADY present in dedorch `analyses`.** Plus Harry drops
|
| 38 |
+
the duplicate/wrong singular `analysis` table. (β #3)
|
| 39 |
+
3. **`analyses_messages` (Go) = the analysis chat room** (user Q + agent A) β replaces the now-**deprecated**
|
| 40 |
+
`chat_messages`/`rooms`; Python's chat read/write must migrate here before cutover. (β #25)
|
| 41 |
4. **Reports: Go owns ALL writes.** Report stays a **skill** (no router intent): FE β Go β Python;
|
| 42 |
Python only returns content. Input = the records table (now `agent_observability`); edit-mode may
|
| 43 |
also need the last report. (β #7/#18/#24)
|
|
|
|
| 112 |
|---|---|---|---|---|
|
| 113 |
| 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | β
| Done 2026-06-25 (one commit w/ #2). Unwired in `orchestration.py`, `intent_router.md`, `chat_handler.py`, `tools.py`; `problem_statement.py` kept intact |
|
| 114 |
| 2 | Drop `problem_validated`: gate neutered; `is_report_ready`/`report_floor` β **β₯1 completed analysis** only, no-LLM | Rifqi | β
| Done 2026-06-25. `gate.py` no-op, gate call site commented in `chat_handler.py`, `report_floor` drops the goal check. Tests updated (`test_gate`/`test_chat_handler`/`test_readiness`). Suite: **284 passed, 7 skipped**; ruff clean |
|
| 115 |
+
| 3 | dedorch `analyses` migration: drop `problem_statement`/`problem_validated`, add `objective` + `business_questions` | Harry | π | **Verified dedorch 2026-06-25:** `analyses` (plural) ALREADY has `user_id` + `status` + `data_bind` + `data_bind_version` + `report_collection` β those parts done. **Remaining:** drop `problem_statement`/`problem_validated` + add `objective`/`business_questions`. Singular `analysis` = deprecated duplicate to drop |
|
| 116 |
+
| 4 | Update Python `analyses` model + `state_store` + `analysis.py` to match dedorch; `owner_id`β`user_id` | Rifqi/Sofhia | π | **Tablename done** (`models.py` `analysis`β`analyses`, uncommitted). **Remaining:** `owner_id`β`user_id` + add `status`/`data_bind`/`data_bind_version`/`report_collection` across `models.py`/`gate.py`/`state_store.py`/`analysis.py` (getattr-tolerant) + a test run before HF |
|
| 117 |
| 5 | Report generator β `objective`+`business_questions`, "generated by {user}", formal outline | Sofhia | β
| Goal-shape (KM-674) + author name (#19) + outline (KM-680): Objective β Business Questions β Executive Summary β Key Findings β EDA β Notes & Limitations β How This Was Analyzed |
|
| 118 |
| 6 | Report skill input contract: `analysis_id` + `user_id` (no `chat_history`) | Sofhia/Rifqi | β
| No-op: `POST /report` already takes only analysis_id + user_id (records-based). Documented in API_ENDPOINTS.md Β§5. *(Edit-mode input revisited in #24.)* |
|
| 119 |
| 7 | `report_id` state update via request to Go, not direct DB | Sofhia + Harry | β¬ | Needs Go endpoint. **Checkpoint:** Go owns ALL `reports` writes; Python stops any direct insert/update and only returns content; report stays a **skill** (no intent). See #18 |
|
|
|
|
| 134 |
| 22 | Finalize `agent_observability` schema β hand to Harry for the dedorch migration/startup | Rifqi β Harry | β¬ new | Resolves #16 (table created post-cutover). Python still creates it locally until then |
|
| 135 |
| 23 | Report markdown formatting: tables, **bold**, *italic*, horizontal separators | Sofhia | β
| Done 2026-06-25. Added `---` separators between header + each section in `_render_markdown`. Tables (EDA) / bold (method labels) / italic (meta + citations) already emitted. Relaxed `report_summary.md` to allow inline `**bold**`/`*italic*` for emphasis (kept no-headings/no-bullets so it doesn't duplicate the section structure / Key Findings). Compile + ruff clean |
|
| 136 |
| 24 | Clarify report input contract: records table (+ `last_report` for edit mode?) | Rifqi/Sofhia β Harry | β¬ new | Edit-mode input left open at the checkpoint |
|
| 137 |
+
| 25 | Migrate Python chat path to Go `analyses_messages` (+ `analyses`) | Rifqi β Harry | β¬ | **Bigger than "confirm" (verified 2026-06-25):** dedorch `rooms` + `chat_messages` are **deprecated** (`zdeprecated_*`). Python's `Room`/`ChatMessage` models + `chat.py` `load_history`/`save_messages` target them β **break post-cutover**. Move history read/write to `analyses_messages` before the conn-string cutover |
|
| 138 |
| 26 | **Charts (DEFERRED):** store Plotly JSON in a future `chart` table (not matplotlib PNG) | β | βΈοΈ | After the markdown path is done end-to-end |
|
| 139 |
| 27 | **Images (DEFERRED):** image table (id, analysis_id, msg/report ref, order) + originals in a bucket | β | βΈοΈ | Maintenance-heavy; parked |
|
| 140 |
| 28 | **UI research** (FE): new-analysis form, knowledge menu (user vs analysis level), report artifacts + version selector | Team | β¬ new | No dedicated UI person; interview + old analysis UI removed |
|
|
|
|
| 142 |
## 5. Critical path & sequencing
|
| 143 |
|
| 144 |
- **Critical path:** #3 β #4 (de-risked by getattr tolerance). #21 β #22 (rename + hand schema to Harry). HF deploy (#13) for the playground.
|
| 145 |
+
- **Parallelizable now:** #11, #21. #4 in progress (tablename done; columns + readers next).
|
| 146 |
+
- **Harry-blocked / coordinated:** #3 (now π, blocks #4), #7 (Go endpoint), #18 (Go state ownership), #24 (contract). **#25 = chat-path migration to `analyses_messages` β a cutover blocker.**
|
| 147 |
- **Demo gate (playground, #13):** deploy the remove-`problem_statement` work to HF β slow path (#15 β
)
|
| 148 |
and the report path are verified locally, and #16 is resolved (#22 hands Harry the schema). **Keep it
|
| 149 |
playground-able.**
|
src/db/postgres/models.py
CHANGED
|
@@ -138,8 +138,15 @@ class AnalysisRecordRow(Base):
|
|
| 138 |
|
| 139 |
`analysis_id` is nullable until the Analysis State (owned upstream) is wired
|
| 140 |
into the slow path; records still persist (and carry `user_id`) before then.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
"""
|
| 142 |
-
__tablename__ = "analysis_records"
|
| 143 |
|
| 144 |
id = Column(String, primary_key=True) # AnalysisRecord.record_id
|
| 145 |
analysis_id = Column(String, index=True) # FK to the analysis session (nullable for now)
|
|
@@ -169,16 +176,26 @@ class AnalysisReportRow(Base):
|
|
| 169 |
|
| 170 |
|
| 171 |
class AnalysisStateRow(Base):
|
| 172 |
-
"""Per-analysis session state β the dedorch `
|
| 173 |
|
| 174 |
One session = one analysis = one conversation; `id` is the shared session id
|
| 175 |
-
(canonical UUID).
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
`
|
| 179 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
"""
|
| 181 |
-
__tablename__ = "
|
| 182 |
|
| 183 |
id = Column(UUID(as_uuid=False), primary_key=True) # shared session id (uuid)
|
| 184 |
analysis_title = Column(String, nullable=False, default="New analysis")
|
|
|
|
| 138 |
|
| 139 |
`analysis_id` is nullable until the Analysis State (owned upstream) is wired
|
| 140 |
into the slow path; records still persist (and carry `user_id`) before then.
|
| 141 |
+
|
| 142 |
+
OWNERSHIP / HANDOFF (2026-06-25 checkpoint): **Python-owned for now**, but to be
|
| 143 |
+
**renamed `agent_observability`** (task #21 β Harry doesn't want "record" in the name;
|
| 144 |
+
it clashes with Go's `analyses_messages`) with the finalized schema handed to Harry so
|
| 145 |
+
the dedorch migration creates it (task #22). Slated to **migrate to Go ownership later** β
|
| 146 |
+
keep this docstring + DEV_PLAN #21/#22 as the handoff record. NOTE: `analysis_records`
|
| 147 |
+
already exists (empty) in dedorch with this exact shape.
|
| 148 |
"""
|
| 149 |
+
__tablename__ = "analysis_records" # β rename to `agent_observability` (#21)
|
| 150 |
|
| 151 |
id = Column(String, primary_key=True) # AnalysisRecord.record_id
|
| 152 |
analysis_id = Column(String, index=True) # FK to the analysis session (nullable for now)
|
|
|
|
| 176 |
|
| 177 |
|
| 178 |
class AnalysisStateRow(Base):
|
| 179 |
+
"""Per-analysis session state β the dedorch **`analyses`** table (plural; Go-owned).
|
| 180 |
|
| 181 |
One session = one analysis = one conversation; `id` is the shared session id
|
| 182 |
+
(canonical UUID). Verified against the dedorch DB 2026-06-25.
|
| 183 |
+
|
| 184 |
+
dedorch `analyses` ACTUAL columns: `id` (uuid), `analysis_title`, `user_id` (text),
|
| 185 |
+
`report_id` (uuid), `created_at`, `updated_at`, `problem_statement`,
|
| 186 |
+
`problem_validated`, `status` (text 'active'|'inactive' β soft-delete),
|
| 187 |
+
`data_bind` (jsonb), `data_bind_version` (int), `report_collection` (jsonb).
|
| 188 |
+
|
| 189 |
+
β οΈ This model is NOT yet column-reconciled to that shape (task #4): it still declares
|
| 190 |
+
`owner_id` (dedorch uses `user_id`) and omits `status`/`data_bind`/`data_bind_version`/
|
| 191 |
+
`report_collection`. dedorch still carries `problem_statement`/`problem_validated` and
|
| 192 |
+
does NOT yet have `objective`/`business_questions` β Harry's #3 drops the former + adds
|
| 193 |
+
the latter. Readers are getattr-tolerant so the eventual swap stays non-breaking.
|
| 194 |
+
|
| 195 |
+
`analysis` (singular) is the deprecated DUPLICATE table Harry will drop β never use it.
|
| 196 |
+
Class name kept as `AnalysisStateRow`.
|
| 197 |
"""
|
| 198 |
+
__tablename__ = "analyses"
|
| 199 |
|
| 200 |
id = Column(UUID(as_uuid=False), primary_key=True) # shared session id (uuid)
|
| 201 |
analysis_title = Column(String, nullable=False, default="New analysis")
|