Rifqi Hafizuddin commited on
Commit
d953ce6
Β·
1 Parent(s): 6a3b169

[NOTICKET] update table names to dedorch db

Browse files
Files changed (2) hide show
  1. DEV_PLAN.md +11 -11
  2. src/db/postgres/models.py +25 -8
DEV_PLAN.md CHANGED
@@ -29,15 +29,15 @@ describes the *in-flight plan* that changes it. New decisions from the 2026-06-2
29
  Confirms the 2026-06-24 direction and adds these concrete changes (folded into Β§4 as tasks 21–28):
30
 
31
  1. **Rename `analysis_records` β†’ `agent_observability`** β€” no "record" in the name (avoids clashing
32
- with Go's `analysis_message`). It **stays Python-owned** for now; once the columns/types are fixed,
33
  **hand the schema to Harry** so his dedorch migration creates it on env-move β€” this is how #16 is
34
  resolved (the table exists post-`SKIP_INIT_DB`). Open: does the write scope broaden to *every agent
35
  call* (true observability) or stay *one row per slow-path analysis*?
36
- 2. **`analysis` table (Go) gains columns:** `status` (active/inactive soft-delete, like chat rooms),
37
- `data_bind` + `data_bind_version` (per-analysis binding; version bumps on change), `report_collection`
38
- (generated reports by id + version). Plus Harry drops the duplicate/wrong `analysis` table. (β†’ #3)
39
- 3. **`analysis_message` (Go) = the analysis chat room** (user Q + agent A) β€” the new `chat_messages`
40
- equivalent; Python reads conversation history from here. (β†’ #25)
41
  4. **Reports: Go owns ALL writes.** Report stays a **skill** (no router intent): FE β†’ Go β†’ Python;
42
  Python only returns content. Input = the records table (now `agent_observability`); edit-mode may
43
  also need the last report. (β†’ #7/#18/#24)
@@ -112,8 +112,8 @@ Status legend: ⬜ not started Β· πŸ”„ in progress Β· βœ… done Β· β›” blocked Β·
112
  |---|---|---|---|---|
113
  | 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | βœ… | Done 2026-06-25 (one commit w/ #2). Unwired in `orchestration.py`, `intent_router.md`, `chat_handler.py`, `tools.py`; `problem_statement.py` kept intact |
114
  | 2 | Drop `problem_validated`: gate neutered; `is_report_ready`/`report_floor` β†’ **β‰₯1 completed analysis** only, no-LLM | Rifqi | βœ… | Done 2026-06-25. `gate.py` no-op, gate call site commented in `chat_handler.py`, `report_floor` drops the goal check. Tests updated (`test_gate`/`test_chat_handler`/`test_readiness`). Suite: **284 passed, 7 skipped**; ruff clean |
115
- | 3 | dedorch `analysis` migration: drop `problem_statement`/`problem_validated`, add `objective` + `business_questions`, rename `owner_id`β†’`user_id` | Harry | ⬜ | Blocks #4. **Checkpoint expands it:** also add `status` (soft-delete), `data_bind` + `data_bind_version`, `report_collection` (id+version); drop the duplicate/wrong `analysis` table |
116
- | 4 | Update Python `analysis` model + `state_store` + `analysis.py` to match #3; `owner_id`β†’`user_id` | Rifqi/Sofhia | β›” after #3 | getattr-tolerant so deploy order isn't a hard blocker |
117
  | 5 | Report generator β†’ `objective`+`business_questions`, "generated by {user}", formal outline | Sofhia | βœ… | Goal-shape (KM-674) + author name (#19) + outline (KM-680): Objective β†’ Business Questions β†’ Executive Summary β†’ Key Findings β†’ EDA β†’ Notes & Limitations β†’ How This Was Analyzed |
118
  | 6 | Report skill input contract: `analysis_id` + `user_id` (no `chat_history`) | Sofhia/Rifqi | βœ… | No-op: `POST /report` already takes only analysis_id + user_id (records-based). Documented in API_ENDPOINTS.md Β§5. *(Edit-mode input revisited in #24.)* |
119
  | 7 | `report_id` state update via request to Go, not direct DB | Sofhia + Harry | ⬜ | Needs Go endpoint. **Checkpoint:** Go owns ALL `reports` writes; Python stops any direct insert/update and only returns content; report stays a **skill** (no intent). See #18 |
@@ -134,7 +134,7 @@ Status legend: ⬜ not started Β· πŸ”„ in progress Β· βœ… done Β· β›” blocked Β·
134
  | 22 | Finalize `agent_observability` schema β†’ hand to Harry for the dedorch migration/startup | Rifqi β†’ Harry | ⬜ new | Resolves #16 (table created post-cutover). Python still creates it locally until then |
135
  | 23 | Report markdown formatting: tables, **bold**, *italic*, horizontal separators | Sofhia | βœ… | Done 2026-06-25. Added `---` separators between header + each section in `_render_markdown`. Tables (EDA) / bold (method labels) / italic (meta + citations) already emitted. Relaxed `report_summary.md` to allow inline `**bold**`/`*italic*` for emphasis (kept no-headings/no-bullets so it doesn't duplicate the section structure / Key Findings). Compile + ruff clean |
136
  | 24 | Clarify report input contract: records table (+ `last_report` for edit mode?) | Rifqi/Sofhia ↔ Harry | ⬜ new | Edit-mode input left open at the checkpoint |
137
- | 25 | Confirm Python history source = Go `analysis_message` | Rifqi ↔ Harry | ⬜ new | New chat-room table (Q + A); replaces `chat_messages` for analyses |
138
  | 26 | **Charts (DEFERRED):** store Plotly JSON in a future `chart` table (not matplotlib PNG) | β€” | ⏸️ | After the markdown path is done end-to-end |
139
  | 27 | **Images (DEFERRED):** image table (id, analysis_id, msg/report ref, order) + originals in a bucket | β€” | ⏸️ | Maintenance-heavy; parked |
140
  | 28 | **UI research** (FE): new-analysis form, knowledge menu (user vs analysis level), report artifacts + version selector | Team | ⬜ new | No dedicated UI person; interview + old analysis UI removed |
@@ -142,8 +142,8 @@ Status legend: ⬜ not started Β· πŸ”„ in progress Β· βœ… done Β· β›” blocked Β·
142
  ## 5. Critical path & sequencing
143
 
144
  - **Critical path:** #3 β†’ #4 (de-risked by getattr tolerance). #21 β†’ #22 (rename + hand schema to Harry). HF deploy (#13) for the playground.
145
- - **Parallelizable now (no blockers):** #11, #21, #23.
146
- - **Harry-blocked:** #3 (blocks #4), #7 (Go endpoint), #18 (Go state ownership), #24/#25 (contracts).
147
  - **Demo gate (playground, #13):** deploy the remove-`problem_statement` work to HF β€” slow path (#15 βœ…)
148
  and the report path are verified locally, and #16 is resolved (#22 hands Harry the schema). **Keep it
149
  playground-able.**
 
29
  Confirms the 2026-06-24 direction and adds these concrete changes (folded into Β§4 as tasks 21–28):
30
 
31
  1. **Rename `analysis_records` β†’ `agent_observability`** β€” no "record" in the name (avoids clashing
32
+ with Go's `analyses_messages`). It **stays Python-owned** for now; once the columns/types are fixed,
33
  **hand the schema to Harry** so his dedorch migration creates it on env-move β€” this is how #16 is
34
  resolved (the table exists post-`SKIP_INIT_DB`). Open: does the write scope broaden to *every agent
35
  call* (true observability) or stay *one row per slow-path analysis*?
36
+ 2. **`analyses` table (Go) β€” `status`, `data_bind` + `data_bind_version`, `report_collection`** (id+version).
37
+ **Verified 2026-06-25: these + `user_id` are ALREADY present in dedorch `analyses`.** Plus Harry drops
38
+ the duplicate/wrong singular `analysis` table. (β†’ #3)
39
+ 3. **`analyses_messages` (Go) = the analysis chat room** (user Q + agent A) β€” replaces the now-**deprecated**
40
+ `chat_messages`/`rooms`; Python's chat read/write must migrate here before cutover. (β†’ #25)
41
  4. **Reports: Go owns ALL writes.** Report stays a **skill** (no router intent): FE β†’ Go β†’ Python;
42
  Python only returns content. Input = the records table (now `agent_observability`); edit-mode may
43
  also need the last report. (β†’ #7/#18/#24)
 
112
  |---|---|---|---|---|
113
  | 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | βœ… | Done 2026-06-25 (one commit w/ #2). Unwired in `orchestration.py`, `intent_router.md`, `chat_handler.py`, `tools.py`; `problem_statement.py` kept intact |
114
  | 2 | Drop `problem_validated`: gate neutered; `is_report_ready`/`report_floor` β†’ **β‰₯1 completed analysis** only, no-LLM | Rifqi | βœ… | Done 2026-06-25. `gate.py` no-op, gate call site commented in `chat_handler.py`, `report_floor` drops the goal check. Tests updated (`test_gate`/`test_chat_handler`/`test_readiness`). Suite: **284 passed, 7 skipped**; ruff clean |
115
+ | 3 | dedorch `analyses` migration: drop `problem_statement`/`problem_validated`, add `objective` + `business_questions` | Harry | πŸ”„ | **Verified dedorch 2026-06-25:** `analyses` (plural) ALREADY has `user_id` + `status` + `data_bind` + `data_bind_version` + `report_collection` β†’ those parts done. **Remaining:** drop `problem_statement`/`problem_validated` + add `objective`/`business_questions`. Singular `analysis` = deprecated duplicate to drop |
116
+ | 4 | Update Python `analyses` model + `state_store` + `analysis.py` to match dedorch; `owner_id`β†’`user_id` | Rifqi/Sofhia | πŸ”„ | **Tablename done** (`models.py` `analysis`β†’`analyses`, uncommitted). **Remaining:** `owner_id`β†’`user_id` + add `status`/`data_bind`/`data_bind_version`/`report_collection` across `models.py`/`gate.py`/`state_store.py`/`analysis.py` (getattr-tolerant) + a test run before HF |
117
  | 5 | Report generator β†’ `objective`+`business_questions`, "generated by {user}", formal outline | Sofhia | βœ… | Goal-shape (KM-674) + author name (#19) + outline (KM-680): Objective β†’ Business Questions β†’ Executive Summary β†’ Key Findings β†’ EDA β†’ Notes & Limitations β†’ How This Was Analyzed |
118
  | 6 | Report skill input contract: `analysis_id` + `user_id` (no `chat_history`) | Sofhia/Rifqi | βœ… | No-op: `POST /report` already takes only analysis_id + user_id (records-based). Documented in API_ENDPOINTS.md Β§5. *(Edit-mode input revisited in #24.)* |
119
  | 7 | `report_id` state update via request to Go, not direct DB | Sofhia + Harry | ⬜ | Needs Go endpoint. **Checkpoint:** Go owns ALL `reports` writes; Python stops any direct insert/update and only returns content; report stays a **skill** (no intent). See #18 |
 
134
  | 22 | Finalize `agent_observability` schema β†’ hand to Harry for the dedorch migration/startup | Rifqi β†’ Harry | ⬜ new | Resolves #16 (table created post-cutover). Python still creates it locally until then |
135
  | 23 | Report markdown formatting: tables, **bold**, *italic*, horizontal separators | Sofhia | βœ… | Done 2026-06-25. Added `---` separators between header + each section in `_render_markdown`. Tables (EDA) / bold (method labels) / italic (meta + citations) already emitted. Relaxed `report_summary.md` to allow inline `**bold**`/`*italic*` for emphasis (kept no-headings/no-bullets so it doesn't duplicate the section structure / Key Findings). Compile + ruff clean |
136
  | 24 | Clarify report input contract: records table (+ `last_report` for edit mode?) | Rifqi/Sofhia ↔ Harry | ⬜ new | Edit-mode input left open at the checkpoint |
137
+ | 25 | Migrate Python chat path to Go `analyses_messages` (+ `analyses`) | Rifqi ↔ Harry | ⬜ | **Bigger than "confirm" (verified 2026-06-25):** dedorch `rooms` + `chat_messages` are **deprecated** (`zdeprecated_*`). Python's `Room`/`ChatMessage` models + `chat.py` `load_history`/`save_messages` target them β†’ **break post-cutover**. Move history read/write to `analyses_messages` before the conn-string cutover |
138
  | 26 | **Charts (DEFERRED):** store Plotly JSON in a future `chart` table (not matplotlib PNG) | β€” | ⏸️ | After the markdown path is done end-to-end |
139
  | 27 | **Images (DEFERRED):** image table (id, analysis_id, msg/report ref, order) + originals in a bucket | β€” | ⏸️ | Maintenance-heavy; parked |
140
  | 28 | **UI research** (FE): new-analysis form, knowledge menu (user vs analysis level), report artifacts + version selector | Team | ⬜ new | No dedicated UI person; interview + old analysis UI removed |
 
142
  ## 5. Critical path & sequencing
143
 
144
  - **Critical path:** #3 β†’ #4 (de-risked by getattr tolerance). #21 β†’ #22 (rename + hand schema to Harry). HF deploy (#13) for the playground.
145
+ - **Parallelizable now:** #11, #21. #4 in progress (tablename done; columns + readers next).
146
+ - **Harry-blocked / coordinated:** #3 (now πŸ”„, blocks #4), #7 (Go endpoint), #18 (Go state ownership), #24 (contract). **#25 = chat-path migration to `analyses_messages` β€” a cutover blocker.**
147
  - **Demo gate (playground, #13):** deploy the remove-`problem_statement` work to HF β€” slow path (#15 βœ…)
148
  and the report path are verified locally, and #16 is resolved (#22 hands Harry the schema). **Keep it
149
  playground-able.**
src/db/postgres/models.py CHANGED
@@ -138,8 +138,15 @@ class AnalysisRecordRow(Base):
138
 
139
  `analysis_id` is nullable until the Analysis State (owned upstream) is wired
140
  into the slow path; records still persist (and carry `user_id`) before then.
 
 
 
 
 
 
 
141
  """
142
- __tablename__ = "analysis_records"
143
 
144
  id = Column(String, primary_key=True) # AnalysisRecord.record_id
145
  analysis_id = Column(String, index=True) # FK to the analysis session (nullable for now)
@@ -169,16 +176,26 @@ class AnalysisReportRow(Base):
169
 
170
 
171
  class AnalysisStateRow(Base):
172
- """Per-analysis session state β€” the dedorch `analysis` table (Go-owned migration).
173
 
174
  One session = one analysis = one conversation; `id` is the shared session id
175
- (canonical UUID). The orchestrator gate + Help skill read this every turn;
176
- `problem_validated` gates structured analysis; the Problem Statement skill flips
177
- it; `report_id` is null until a report exists. `id`/`report_id` are Postgres
178
- `uuid` in dedorch, so they bind as UUID (canonical-string in/out). Class name
179
- kept as `AnalysisStateRow`; only the table + id types changed for dedorch.
 
 
 
 
 
 
 
 
 
 
180
  """
181
- __tablename__ = "analysis"
182
 
183
  id = Column(UUID(as_uuid=False), primary_key=True) # shared session id (uuid)
184
  analysis_title = Column(String, nullable=False, default="New analysis")
 
138
 
139
  `analysis_id` is nullable until the Analysis State (owned upstream) is wired
140
  into the slow path; records still persist (and carry `user_id`) before then.
141
+
142
+ OWNERSHIP / HANDOFF (2026-06-25 checkpoint): **Python-owned for now**, but to be
143
+ **renamed `agent_observability`** (task #21 β€” Harry doesn't want "record" in the name;
144
+ it clashes with Go's `analyses_messages`) with the finalized schema handed to Harry so
145
+ the dedorch migration creates it (task #22). Slated to **migrate to Go ownership later** β€”
146
+ keep this docstring + DEV_PLAN #21/#22 as the handoff record. NOTE: `analysis_records`
147
+ already exists (empty) in dedorch with this exact shape.
148
  """
149
+ __tablename__ = "analysis_records" # β†’ rename to `agent_observability` (#21)
150
 
151
  id = Column(String, primary_key=True) # AnalysisRecord.record_id
152
  analysis_id = Column(String, index=True) # FK to the analysis session (nullable for now)
 
176
 
177
 
178
  class AnalysisStateRow(Base):
179
+ """Per-analysis session state β€” the dedorch **`analyses`** table (plural; Go-owned).
180
 
181
  One session = one analysis = one conversation; `id` is the shared session id
182
+ (canonical UUID). Verified against the dedorch DB 2026-06-25.
183
+
184
+ dedorch `analyses` ACTUAL columns: `id` (uuid), `analysis_title`, `user_id` (text),
185
+ `report_id` (uuid), `created_at`, `updated_at`, `problem_statement`,
186
+ `problem_validated`, `status` (text 'active'|'inactive' β€” soft-delete),
187
+ `data_bind` (jsonb), `data_bind_version` (int), `report_collection` (jsonb).
188
+
189
+ ⚠️ This model is NOT yet column-reconciled to that shape (task #4): it still declares
190
+ `owner_id` (dedorch uses `user_id`) and omits `status`/`data_bind`/`data_bind_version`/
191
+ `report_collection`. dedorch still carries `problem_statement`/`problem_validated` and
192
+ does NOT yet have `objective`/`business_questions` β€” Harry's #3 drops the former + adds
193
+ the latter. Readers are getattr-tolerant so the eventual swap stays non-breaking.
194
+
195
+ `analysis` (singular) is the deprecated DUPLICATE table Harry will drop β€” never use it.
196
+ Class name kept as `AnalysisStateRow`.
197
  """
198
+ __tablename__ = "analyses"
199
 
200
  id = Column(UUID(as_uuid=False), primary_key=True) # shared session id (uuid)
201
  analysis_title = Column(String, nullable=False, default="New analysis")