# Backend Agentic Service — API Endpoint Docs (endpoint restructure) **Status:** contract draft for FE/Go integration (2026-06-30). Covers the AI-only surface after the restructure. Sections marked **TENTATIVE** (observability) may still change — send feedback before we lock them. **What changed** - **Only the chat pilot moves to `/api/v2`.** Everything else stays on `/api/v1`, regrouped under `/tools`. - **Chat pilot (`/api/v2/chat/stream`) uses `analysis_id`, not `room_id`.** - **Skills are grouped under `/api/v1/tools`:** `list` / `help` / `report`. - **New:** `GET /api/v1/observability` — Responsible-AI provenance per chat answer. - Python is **generative-AI only.** It never creates/updates an analysis, room, document, DB client, or catalog — Go owns those. Python just receives `analysis_id`. Those v1 routers are unwired from `main` + Swagger (not deleted). **Open coordination questions (need a decision with Harry) — flagged inline as ⚠️:** 1. ~~**`message_id` origin** — who mints the assistant turn id used to correlate stream ↔ observability?~~ **RESOLVED (pr/6):** **Python is the sole minter.** It is not a request field; Python always mints it (server-authoritative — keeps the `/observability` correlation key out of client control, for FE-security) and returns it on the `done` event. Go/FE read it off the stream. *(The `/observability` consumer itself is future work — a later PR.)* 2. **Deterministic `/help` dispatch** — dedicated endpoint (recommended below) vs router classification. 3. **Observability storage** — single JSONB row per message (recommended) vs 3 normalized tables. --- ## 1. call_agent — `POST /api/v2/chat/stream` The only FE→Python call in normal operation. Same as v1 except **`room_id` → `analysis_id`**, and the `done` event now carries the assistant `message_id` for observability correlation. **Request body** (`application/json`) — `ChatRequest`: ```json { "user_id": "u_1a2b3c", "analysis_id": "an_42", "message": "What were total sales by region last quarter?" } ``` - `analysis_id` is the analysis-session id (replaces `room_id`). No auth header (handled by Go). - `message_id` is **not** a request field. Python always mints the assistant turn id (server-authoritative, for FE-security) and returns it on `done`; the FE reads it there and passes it to `/api/v1/observability?message_id=...`. Any `message_id` a caller sends is ignored. *(The `/observability` endpoint is future work — a later PR; §7 is a forward-looking sketch.)* **Response:** `text/event-stream`. Events arrive in this order: | event | data | notes | |---|---|---| | `sources` | JSON array of `{document_id, filename, page_label}` | structured: one per executed table; unstructured: deduped doc/page; chat/help/error: `[]`. | | `status` | text | slow-path only — progress pings ("Planning…", "Running N steps…"). Safe to surface or ignore. | | `chunk` | text fragment | concatenate in order to form the answer. | | `done` | `{"message_id": "..."}` | **v2 change:** was empty; now returns the turn id for the observability lookup. | | `error` | text | terminal error; stream stops after this. | The internal `intent` event is consumed inside Python (gates caching) and **not** forwarded. **Stream carries the answer text ONLY.** Planning / tool calls / sources detail are **not** in the stream (it would slow it down) — fetch them from `/observability` (§7), called in parallel. **Example — `structured_flow` answer** (raw SSE; blank line separates events): ``` event: sources data: [{"document_id":"u_1a2b3c_orders","filename":"orders","page_label":null}] event: status data: Planning analysis… event: status data: Running 3 steps… event: chunk data: Total sales by region last quarter: event: chunk data: Central led at $1.21M (38%), East $0.74M, West $0.55M (down 12% QoQ). event: done data: {"message_id":"msg_88f1"} ``` **Example — simple chat reply** (no status pings, empty sources): ``` event: sources data: [] event: chunk data: I'm your AI data analyst — connect a source or ask a question to get started. event: done data: {"message_id":"msg_12"} ``` Behavior unchanged from v1: 1h Redis response-cache on the stateless `chat` intent only; greeting/farewell fast-path (canned, no LLM); LLM router classifies every message into one of 5 intents (`chat · help · check · unstructured_flow · structured_flow`); messages persist on `done`. --- ## 2. list_skills — `GET /api/v1/tools/list` Static, deterministic, safe for Go to cache. (Was `GET /api/v1/tools`.) **Request:** none. **Response 200** (`ListToolsResponse`): ```json { "count": 2, "tools": [ { "command": "/help", "name": "help", "type": "skill", "description": "Show what the assistant can do and guide your next step." }, { "command": "/report", "name": "report", "type": "skill", "description": "Generate a versioned analysis report (background, EDA, key findings, insights)." } ] } ``` `CommandResponse = { command, name, type, description }`, `type ∈ {skill, analytics, data_access}`. Catalog is `/help` + `/report` only; the `analyze_*` / `check_*` / `retrieve_*` and retired `/problem-statement` entries are commented out (kept for restorability), not deleted. **FE behavior:** the `/` slash menu surfaces **`/help` only**. **Report is a right-side button, not a slash command** (it fires only when an analysis is finished — saves tokens). --- ## 3. skill: help — `POST /api/v1/tools/help` ⚠️ **Proposed dedicated endpoint** (new in v2). In v1 there was no `/help` endpoint — help was reached only by letting the LLM router classify a chat message. A dedicated endpoint makes `/help` dispatch **deterministic** (no risk the router mis-classifies the slash command) and gives it a clean home in the tools group. State-aware: reads analysis state + history to guide the next step. > Alternative if we *don't* add this endpoint: FE keeps calling `POST /chat/stream` and trusts the > router to classify the help intent. We recommend the dedicated endpoint — decision pending (open > question #2). **Request body** (`application/json`): ```json { "user_id": "u_1a2b3c", "analysis_id": "an_42" } ``` **Response:** `text/event-stream` — same SSE shape as chat, with `sources: []` and no `status` pings (help never references documents). Streams a next-step guidance reply. ``` event: sources data: [] event: chunk data: Your goal is set — you can start exploring now. Try a question like "average order value by month", then I can generate a report. event: done data: {"message_id":"msg_h7"} ``` --- ## 4. skill: report — `POST /api/v1/tools/report` The "Generate Report" button. Same as v1, moved under `/tools`. Generate, persist, and return a new report version. Currently renders **Markdown** (FE preview); PPT/PDF/infographic export is future work (triggered on a download button, not here). **Query params:** `analysis_id` (required), `user_id` (required). No request body. ``` POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c ``` | status | meaning | |---|---| | 201 | new version generated → `AnalysisReport` body. | | 409 | floor not met — no recorded analyses yet for this session, nothing to report. | | 500 | generation or persistence failed. | **201 response** (`AnalysisReport`): ```json { "report_id": "8f3a2b1c9d4e4f6a8b0c1d2e3f4a5b6c", "analysis_id": "an_42", "user_id": "u_1a2b3c", "version": 2, "generated_at": "2026-06-30T09:14:33.512Z", "problem_statement": { "objective": "Understand which regions drive revenue and why Q1 dipped.", "business_questions": [ "Which regions contribute most to total revenue?", "Did any region decline quarter-over-quarter?" ] }, "record_ids": ["rec_a1", "rec_b2"], "executive_summary": "Revenue is concentrated in the Central region (38% of total). The West was the only region to contract, down 12% QoQ — the main driver of the Q1 dip.", "findings": [ { "text": "Central region contributed 38% of total revenue, the largest share.", "record_ids": ["rec_a1"], "supporting_data": null }, { "text": "West region revenue fell 12% quarter-over-quarter.", "record_ids": ["rec_b2"], "supporting_data": null } ], "caveats": [ { "text": "March data for the East region was partially missing (~6% of rows).", "record_ids": ["rec_b2"] } ], "open_questions": [ { "text": "What drove the West region's QoQ decline?", "record_ids": ["rec_b2"] } ], "data_sources": [ { "source_id": "src_sales_db", "name": "orders", "source_type": "postgres", "detail": { "tables": ["orders"], "row_count": 48213, "columns": ["region", "amount", "ordered_at"] } } ], "method_steps": [ { "task_id": "t1", "stage": "data_understanding", "objective": "Inventory the sales source", "status": "success", "tools_used": ["check_data"] }, { "task_id": "t2", "stage": "modeling", "objective": "Aggregate revenue by region", "status": "success", "tools_used": ["analyze_aggregate"] } ], "rendered_markdown": "# Analysis Report\n\n*Generated 2026-06-30 by u_1a2b3c · 2 analyses · 1 source(s)*\n\n## Objective\nUnderstand which regions drive revenue…\n\n## Key Findings\n1. Central region contributed 38%…" } ``` **409 response** (floor not met — the demo's most common error): ```json { "detail": "Not ready to generate a report — still needs at least one completed analysis." } ``` ⚠️ **Precondition:** `AnalysisRecord`s persist only on the slow path, so reports require `ENABLE_SLOW_PATH=true` on the Python deployment and ≥1 prior `structured_flow` question in the session. With slow path off, `POST` 409s by design. --- ## 5. report versions — `GET /api/v1/tools/report/{analysis_id}` and `/{analysis_id}/{version}` List a session's report versions (oldest-first). Returns `[ReportVersionEntry]`; `[]` if none. ```json [ { "report_id": "1b2c3d4e…", "version": 1, "generated_at": "2026-06-24T15:02:11Z", "record_count": 1 }, { "report_id": "8f3a2b1c…", "version": 2, "generated_at": "2026-06-25T09:14:33Z", "record_count": 2 } ] ``` `GET /api/v1/tools/report/{analysis_id}/{version}` → one `AnalysisReport` (same shape as the POST 201 body); 404 if that version doesn't exist: ```json { "detail": "No report v3 for analysis 'an_42'." } ``` --- ## 6. Unwired in v2 (mounted in v1, OFF in v2) Commented out of `main` + Swagger, **files kept**. Go owns these; Python is generative-only: `POST /analysis/create` + analysis CRUD · `room` · `db_client` · `document` · `data_catalog` · `users`/login. Re-mounting is a one-line `include_router` if ever needed. --- ## 7. observability — `GET /api/v1/observability` **(NEW · TENTATIVE)** Responsible-AI provenance for **one chat answer**. Separate endpoint, called **in parallel with the stream** — never embedded in it. The FE renders it as a collapsed dropdown the user can expand (planning / tool calls / sources), Claude/Codex-style. **Design (recommended):** one endpoint returns one merged object, backed by **one JSONB row per message** written by an accumulating "scratchpad" decorator inside the chat agent and flushed on `done`. The 3 facets (`planning` / `tool_calls` / `sources`) are **logical sections of the JSON**, not separate tables — so the shape can evolve without a dedorch migration each time. (Storage is open question #3.) **Query params:** `analysis_id` (required), `message_id` (required). ``` GET /api/v1/observability?analysis_id=an_42&message_id=msg_88f1 ``` **Timing:** the row is written when the turn finishes, so call this **after** the stream's `done` event (or poll until 200). "Parallel" = a separate call the FE fires alongside the stream, not data embedded in the stream. **Field rules (by intent):** - `planning` — present **only when the planner ran** (slow path); `null` otherwise. - `tool_calls` — every tool invoked, with input + output. `[]` for pure chat / greeting / help. - `sources` — **required for retrieve flows** (`structured_flow`, `unstructured_flow`). **Empty for greeting / `chat` / `help`** (they don't reference documents). - `thinking` — optional reasoning text; `null` if none. **200 response — full `structured_flow` turn** (planner ran → all sections present): ```json { "analysis_id": "an_42", "message_id": "msg_88f1", "intent": "structured_flow", "generated_at": "2026-06-30T03:21:09.114Z", "planning": { "goal_restated": "Find which regions drive revenue and why Q1 dipped.", "assumptions": ["'last quarter' = Q1 2026"], "steps": [ { "step": 1, "stage": "data_understanding", "objective": "Inventory the sales source" }, { "step": 2, "stage": "modeling", "objective": "Aggregate revenue by region" } ] }, "thinking": "The question needs a per-region breakdown plus a cause, so I inventory the source, aggregate revenue by region, then compare quarters.", "tool_calls": [ { "order": 1, "name": "check_data", "input": { "source_hint": "structured" }, "output": { "kind": "table", "summary": "1 source · 1 table (orders) · 48,213 rows" }, "status": "success" }, { "order": 2, "name": "retrieve_data", "input": { "source_id": "src_sales_db", "table_id": "orders", "select": ["region", "amount"], "group_by": ["region"] }, "output": { "kind": "table", "columns": ["region", "total"], "row_count": 4, "preview": [["Central", 1210000], ["East", 740000]] }, "status": "success" } ], "sources": [ { "type": "database", "source_id": "src_sales_db", "name": "orders", "query": "SELECT region, SUM(amount) AS total FROM orders GROUP BY region", "detail": { "tables": ["orders"], "row_count": 48213 } } ] } ``` **200 response — `unstructured_flow` turn** (no planner; source = document, with the retrieval query): ```json { "analysis_id": "an_42", "message_id": "msg_55", "intent": "unstructured_flow", "generated_at": "2026-06-30T03:40:02.001Z", "planning": null, "thinking": null, "tool_calls": [ { "order": 1, "name": "retrieve_knowledge", "input": { "query": "technology stack used in this project", "top_k": 4 }, "output": { "kind": "documents", "row_count": 4 }, "status": "success" } ], "sources": [ { "type": "document", "document_id": "doc_7", "filename": "tech_handbook.pdf", "page_label": "12", "query": "technology stack used in this project", "snippet": "The backend is built on FastAPI with async SQLAlchemy…", "score": 0.83 } ] } ``` **200 response — simple `chat` / greeting turn** (nothing to trace): ```json { "analysis_id": "an_42", "message_id": "msg_12", "intent": "chat", "generated_at": "2026-06-30T03:05:00.000Z", "planning": null, "thinking": null, "tool_calls": [], "sources": [] } ``` **404** — no provenance for that message yet (turn still running or unknown id): ```json { "detail": "No observability for message 'msg_88f1' yet." } ``` > ⚠️ **Richness is path-dependent.** Full `planning` + tool I/O exist only when > `ENABLE_SLOW_PATH=true`. Fast chat / single-query / help still record `sources` + the single > tool call but have `planning: null`. This matches the rule "planning only when the planner runs."