harryagasi commited on
Commit
b5e46e4
·
1 Parent(s): 1db9b28

feat: fetch and render charts for chat answers via /api/v1/charts

Browse files

- Add getCharts client and ChartSpec/ChartItem/ChartsResponse types
- Auto-fetch charts for each completed AI message once message_id is known, mirroring the existing traceability fetch pattern
- Treat empty/not_found chart status as silent no-op (no error shown); log not_found for debugging only
- Render fetched charts under the assistant bubble reusing PlotlyChartBlock from report rendering

docs/API_CONTRACT_BE_PYTHON.md CHANGED
@@ -1,8 +1,8 @@
1
  # Backend Agentic Service API Contract
2
 
3
- **Last updated**: 2026-07-13
4
 
5
- This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, and traceability data shown alongside chat answers.
6
 
7
  Base path examples use relative URLs. Configure the frontend with the deployed Python service base URL.
8
 
@@ -12,8 +12,9 @@ The Python backend owns the generative AI interaction surface:
12
 
13
  1. Stream chat answers from the AI agent.
14
  2. Execute tool-style actions for help and report generation.
15
- 3. Return report versions and report details.
16
  4. Return traceability for a completed assistant answer.
 
17
 
18
  The frontend uses this service during the analysis conversation flow:
19
 
@@ -21,7 +22,8 @@ The frontend uses this service during the analysis conversation flow:
21
  2. Frontend calls `POST /api/v2/chat/stream` and renders the streamed answer.
22
  3. When the stream emits `done`, frontend uses the returned `message_id` as the assistant answer correlation id.
23
  4. Frontend calls `GET /api/v1/traceability` for planning, tool calls, and source provenance.
24
- 5. Frontend calls `/api/v1/tools/help` for guided help and `/api/v1/tools/report` for report generation.
 
25
 
26
  ## Endpoint Summary
27
 
@@ -32,8 +34,11 @@ The frontend uses this service during the analysis conversation flow:
32
  | `POST` | `/api/v1/tools/help` | Stream contextual help for the current analysis conversation. |
33
  | `POST` | `/api/v1/tools/report` | Generate and persist a new report version. |
34
  | `GET` | `/api/v1/tools/report/{analysis_id}` | List report versions for an analysis. |
 
 
35
  | `GET` | `/api/v1/tools/report/{analysis_id}/{version}` | Retrieve one report version. |
36
  | `GET` | `/api/v1/traceability` | Retrieve provenance for one assistant answer. |
 
37
 
38
  ## Common Concepts
39
 
@@ -41,7 +46,7 @@ The frontend uses this service during the analysis conversation flow:
41
 
42
  - `user_id`: user identifier passed by the frontend.
43
  - `analysis_id`: analysis conversation identifier.
44
- - `message_id`: assistant answer identifier generated by Python and returned in the stream `done` event; used to correlate chat streaming, Golang message persistence, and traceability.
45
 
46
  ### Server-Sent Events
47
 
@@ -53,13 +58,13 @@ Common event types:
53
 
54
  | Event | Data | Meaning |
55
  | --- | --- | --- |
56
- | `sources` | JSON array | Sources available early in the stream. May be empty. |
57
  | `status` | text | Optional progress update for slower paths. |
58
  | `chunk` | text | Answer text fragment. Concatenate chunks in order. |
59
  | `done` | JSON object | Terminal success event. Includes `message_id`. |
60
  | `error` | text | Terminal error event. Stream stops after this. |
61
 
62
- The stream carries answer text only. Planning, tool call details, and full provenance are fetched from `GET /api/v1/traceability` after the stream is done.
63
 
64
  ## Chat
65
 
@@ -91,7 +96,7 @@ Example structured answer:
91
 
92
  ```text
93
  event: sources
94
- data: [{"document_id":"u_1a2b3c_orders","filename":"orders","page_label":null}]
95
 
96
  event: status
97
  data: Planning analysis...
@@ -126,9 +131,10 @@ Behavior notes:
126
 
127
  - Greeting and farewell messages may use a fast canned path.
128
  - Stateless `chat` intent may use a 1-hour Redis response cache.
129
- - The router may classify messages into intents such as `chat`, `help`, `check`, `unstructured_flow`, or `structured_flow`.
130
- - `sources` can be empty for chat/help/error paths.
131
  - `status` events are optional and should be safe for the frontend to ignore.
 
132
 
133
  ## Tools
134
 
@@ -142,24 +148,20 @@ Response `200`:
142
 
143
  ```json
144
  {
145
- "count": 2,
146
  "tools": [
147
  {
148
  "command": "/help",
149
  "name": "help",
150
  "type": "skill",
151
  "description": "Show what the assistant can do and guide your next step."
152
- },
153
- {
154
- "command": "/report",
155
- "name": "report",
156
- "type": "skill",
157
- "description": "Generate a versioned analysis report with background, EDA, key findings, and insights."
158
  }
159
  ]
160
  }
161
  ```
162
 
 
 
163
  Tool item shape:
164
 
165
  ```json
@@ -174,7 +176,7 @@ Tool item shape:
174
  Frontend behavior:
175
 
176
  - Surface `/help` in the slash menu.
177
- - Surface report generation as a button or explicit UI action.
178
 
179
  ### `POST /api/v1/tools/help`
180
 
@@ -218,11 +220,13 @@ Query params:
218
  | --- | --- | --- |
219
  | `analysis_id` | Yes | Analysis identifier. |
220
  | `user_id` | Yes | User identifier. |
 
221
 
222
  Example:
223
 
224
  ```text
225
  POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c
 
226
  ```
227
 
228
  Status codes:
@@ -251,6 +255,20 @@ Response `201`:
251
  },
252
  "record_ids": ["rec_a1", "rec_b2"],
253
  "executive_summary": "Revenue is concentrated in the Central region (38% of total). The West was the only region to contract, down 12% QoQ, the main driver of the Q1 dip.",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
254
  "findings": [
255
  {
256
  "text": "Central region contributed 38% of total revenue, the largest share.",
@@ -275,6 +293,36 @@ Response `201`:
275
  "record_ids": ["rec_b2"]
276
  }
277
  ],
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
278
  "data_sources": [
279
  {
280
  "source_id": "src_sales_db",
@@ -307,6 +355,32 @@ Response `201`:
307
  }
308
  ```
309
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
  Response `409`:
311
 
312
  ```json
@@ -315,11 +389,6 @@ Response `409`:
315
  }
316
  ```
317
 
318
- Precondition:
319
-
320
- - Reports require at least one completed analysis record for the session.
321
- - If slow-path analysis recording is disabled, report generation can return `409` by design.
322
-
323
  ### `GET /api/v1/tools/report/{analysis_id}`
324
 
325
  Lists report versions for one analysis, oldest first.
@@ -345,6 +414,48 @@ Response `200`:
345
 
346
  If no reports exist, returns `[]`.
347
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
348
  ### `GET /api/v1/tools/report/{analysis_id}/{version}`
349
 
350
  Returns one report version. Shape is the same as the `201` response from `POST /api/v1/tools/report`.
@@ -357,6 +468,106 @@ Response `404`:
357
  }
358
  ```
359
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
360
  ## Traceability
361
 
362
  ### `GET /api/v1/traceability`
@@ -378,16 +589,20 @@ Example:
378
  GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1
379
  ```
380
 
381
- `intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or Azure content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
382
 
383
  Field rules:
384
 
385
  - `planning`: present only when the planner ran (`structured_flow`); otherwise `null`.
386
- - `thinking`: **always `null` in v1** — our agents are plain chat completions with no native reasoning output, and synthesizing it post-hoc would be unfaithful. The field stays in the payload so it can be populated later without a contract change.
387
- - `tool_calls`: every invoked tool with `input`, `output`, `status`, `task_id` (nullable), and `error` (nullable); empty for chat / help / greeting / refusal paths.
388
- - `sources`: required for retrieval flows; empty for chat / help / refusal paths and for `check`.
 
 
 
 
389
  - The payload also carries an internal `user_id` (ownership); the frontend may ignore it.
390
- - Truncation: `preview` ≤ 5 rows; any string inside `input`/`output`/`preview`/`snippet` ≤ 300 chars; rows beyond the preview are dropped (`row_count` is preserved).
391
 
392
  Response `200` for `structured_flow`:
393
 
@@ -556,4 +771,4 @@ Frontend rendering guidance:
556
  - Render traceability separately from the streamed answer.
557
  - Default state can be collapsed.
558
  - Show planning, tool calls, and sources as separate sections.
559
- - Treat `planning: null`, `tool_calls: []`, and `sources: []` as valid states.
 
1
  # Backend Agentic Service API Contract
2
 
3
+ **Last updated**: 2026-07-14
4
 
5
+ This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, charts, and traceability data shown alongside chat answers.
6
 
7
  Base path examples use relative URLs. Configure the frontend with the deployed Python service base URL.
8
 
 
12
 
13
  1. Stream chat answers from the AI agent.
14
  2. Execute tool-style actions for help and report generation.
15
+ 3. Return report versions, report details, and report-readiness signals.
16
  4. Return traceability for a completed assistant answer.
17
+ 5. Return chart specifications produced for a completed assistant answer.
18
 
19
  The frontend uses this service during the analysis conversation flow:
20
 
 
22
  2. Frontend calls `POST /api/v2/chat/stream` and renders the streamed answer.
23
  3. When the stream emits `done`, frontend uses the returned `message_id` as the assistant answer correlation id.
24
  4. Frontend calls `GET /api/v1/traceability` for planning, tool calls, and source provenance.
25
+ 5. Frontend calls `GET /api/v1/charts` with the same `message_id` and renders any returned charts under the answer.
26
+ 6. Frontend calls `/api/v1/tools/help` for guided help and `/api/v1/tools/report` for report generation (the Generate-Report button; `GET …/readiness` drives the button state).
27
 
28
  ## Endpoint Summary
29
 
 
34
  | `POST` | `/api/v1/tools/help` | Stream contextual help for the current analysis conversation. |
35
  | `POST` | `/api/v1/tools/report` | Generate and persist a new report version. |
36
  | `GET` | `/api/v1/tools/report/{analysis_id}` | List report versions for an analysis. |
37
+ | `GET` | `/api/v1/tools/report/{analysis_id}/records` | List analysis records for report curation. |
38
+ | `GET` | `/api/v1/tools/report/{analysis_id}/readiness` | Report-readiness signal for the Generate-Report button. |
39
  | `GET` | `/api/v1/tools/report/{analysis_id}/{version}` | Retrieve one report version. |
40
  | `GET` | `/api/v1/traceability` | Retrieve provenance for one assistant answer. |
41
+ | `GET` | `/api/v1/charts` | Retrieve chart(s) produced for one assistant answer. |
42
 
43
  ## Common Concepts
44
 
 
46
 
47
  - `user_id`: user identifier passed by the frontend.
48
  - `analysis_id`: analysis conversation identifier.
49
+ - `message_id`: assistant answer identifier generated by Python and returned in the stream `done` event; used to correlate chat streaming, Golang message persistence, traceability, and charts. It is a UUID string (e.g. `77f06761-0fdf-4cc5-84f8-5f81bcbb6f84`); the `msg_…` values in the examples below are illustrative placeholders only. Never generate or send it from the frontend.
50
 
51
  ### Server-Sent Events
52
 
 
58
 
59
  | Event | Data | Meaning |
60
  | --- | --- | --- |
61
+ | `sources` | JSON array | Always `[]` real sources moved to `GET /api/v1/traceability`. Event kept for backward compatibility. |
62
  | `status` | text | Optional progress update for slower paths. |
63
  | `chunk` | text | Answer text fragment. Concatenate chunks in order. |
64
  | `done` | JSON object | Terminal success event. Includes `message_id`. |
65
  | `error` | text | Terminal error event. Stream stops after this. |
66
 
67
+ The stream carries answer text only. Planning, tool call details, and full provenance are fetched from `GET /api/v1/traceability` after the stream is done; charts are fetched from `GET /api/v1/charts`. The `done` event carries no chart hint — fetch `GET /api/v1/charts` unconditionally on every `done` (the response tells you if there is nothing to render).
68
 
69
  ## Chat
70
 
 
96
 
97
  ```text
98
  event: sources
99
+ data: []
100
 
101
  event: status
102
  data: Planning analysis...
 
131
 
132
  - Greeting and farewell messages may use a fast canned path.
133
  - Stateless `chat` intent may use a 1-hour Redis response cache.
134
+ - The router may classify messages into intents such as `chat`, `help`, `check`, `unstructured_flow`, `structured_flow`, or `out_of_scope`.
135
+ - `sources` in the stream is **always `[]`** — read the real `sources[]` from `GET /api/v1/traceability` after `done`.
136
  - `status` events are optional and should be safe for the frontend to ignore.
137
+ - When the user explicitly asks to plot/visualize ("show me a bar chart of…", "buatkan grafik…"), the answer text describes the result and the chart itself is delivered via `GET /api/v1/charts` — it is never embedded in `chunk` text.
138
 
139
  ## Tools
140
 
 
148
 
149
  ```json
150
  {
151
+ "count": 1,
152
  "tools": [
153
  {
154
  "command": "/help",
155
  "name": "help",
156
  "type": "skill",
157
  "description": "Show what the assistant can do and guide your next step."
 
 
 
 
 
 
158
  }
159
  ]
160
  }
161
  ```
162
 
163
+ The catalog is `/help` only. `/report` is not a slash command — report generation is a right-side **Generate** button; the button calls `POST /api/v1/tools/report`.
164
+
165
  Tool item shape:
166
 
167
  ```json
 
176
  Frontend behavior:
177
 
178
  - Surface `/help` in the slash menu.
179
+ - Surface report generation as a button or explicit UI action, driven by `GET /tools/report/{analysis_id}/readiness`.
180
 
181
  ### `POST /api/v1/tools/help`
182
 
 
220
  | --- | --- | --- |
221
  | `analysis_id` | Yes | Analysis identifier. |
222
  | `user_id` | Yes | User identifier. |
223
+ | `exclude_record_ids` | No | Record ids to leave out of this version (repeat the param per id). Get ids from `GET /tools/report/{analysis_id}/records`. Excluded runs are listed in the report's "Excluded Analyses" section. Excluding every substantive record returns `409`. |
224
 
225
  Example:
226
 
227
  ```text
228
  POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c
229
+ POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c&exclude_record_ids=rec_a1&exclude_record_ids=rec_c3
230
  ```
231
 
232
  Status codes:
 
255
  },
256
  "record_ids": ["rec_a1", "rec_b2"],
257
  "executive_summary": "Revenue is concentrated in the Central region (38% of total). The West was the only region to contract, down 12% QoQ, the main driver of the Q1 dip.",
258
+ "bq_answers": [
259
+ {
260
+ "question": "Which regions contribute most to total revenue?",
261
+ "answer": "The Central region leads with 38% of total revenue.",
262
+ "status": "answered",
263
+ "record_ids": ["rec_a1"]
264
+ },
265
+ {
266
+ "question": "Did any region decline quarter-over-quarter?",
267
+ "answer": "Yes — the West region fell 12% QoQ.",
268
+ "status": "answered",
269
+ "record_ids": ["rec_b2"]
270
+ }
271
+ ],
272
  "findings": [
273
  {
274
  "text": "Central region contributed 38% of total revenue, the largest share.",
 
293
  "record_ids": ["rec_b2"]
294
  }
295
  ],
296
+ "unresolved": [
297
+ {
298
+ "text": "Correlate churn with tenure — churn column not found in the source.",
299
+ "record_ids": ["rec_d4"]
300
+ }
301
+ ],
302
+ "excluded": [],
303
+ "evidence_tables": {
304
+ "rec_a1": [
305
+ {
306
+ "title": "Aggregate revenue by region",
307
+ "columns": ["region", "total_revenue"],
308
+ "rows": [["Central", "18321"], ["West", "9954"]],
309
+ "truncated": false
310
+ }
311
+ ]
312
+ },
313
+ "charts": {
314
+ "rec_a1": [
315
+ {
316
+ "schema": "dataeyond.chart.v1",
317
+ "chart_type": "bar",
318
+ "title": "Revenue by region",
319
+ "plotly": {
320
+ "data": [{ "type": "bar", "x": ["Central", "East", "West"], "y": [1210000, 740000, 550000], "name": "revenue" }],
321
+ "layout": { "title": { "text": "Revenue by region" }, "xaxis": { "title": { "text": "region" } }, "yaxis": { "title": { "text": "revenue" } } }
322
+ }
323
+ }
324
+ ]
325
+ },
326
  "data_sources": [
327
  {
328
  "source_id": "src_sales_db",
 
355
  }
356
  ```
357
 
358
+ Field notes:
359
+
360
+ - `bq_answers` — one entry per business question. `status` is `answered` | `partial` | `unanswered`; `record_ids` cite the backing analyses. Written in the analysis's language (Indonesian objective → Indonesian answers).
361
+ - `unresolved` — runs that were attempted but produced no usable evidence. Not part of the findings body.
362
+ - `excluded` — runs the caller excluded via `exclude_record_ids`.
363
+ - `evidence_tables` — `record_id` → small result tables copied from the run's stored outputs (max 3 tables per record, max 10 rows each; `truncated: true` when rows were capped). Rendered as markdown tables under the matching Key Findings group in `rendered_markdown`.
364
+ - `charts` — `record_id` → chart envelopes (same `dataeyond.chart.v1` shape as `GET /api/v1/charts` returns; max 3 per record) copied from the run's stored outputs. `rendered_markdown` contains an `## EDA` section where each chart appears as a fenced block:
365
+
366
+ ````text
367
+ ```plotly
368
+ {
369
+ "schema": "dataeyond.chart.v1",
370
+ "chart_type": "bar",
371
+ "title": "…",
372
+ "plotly": { "data": [ … ], "layout": { … } }
373
+ }
374
+ ```
375
+ ````
376
+
377
+ The fence content is the full envelope — parse it and render `Plotly.newPlot(el, parsed.plotly.data, parsed.plotly.layout)`. A bold caption line (the chart title) precedes each fence.
378
+
379
+ Precondition:
380
+
381
+ - Reports require at least one completed analysis record for the session. A run counts when an analysis step succeeded **or** a chart was produced — a chart-only session can generate a report.
382
+ - If slow-path analysis recording is disabled, report generation can return `409` by design.
383
+
384
  Response `409`:
385
 
386
  ```json
 
389
  }
390
  ```
391
 
 
 
 
 
 
392
  ### `GET /api/v1/tools/report/{analysis_id}`
393
 
394
  Lists report versions for one analysis, oldest first.
 
414
 
415
  If no reports exist, returns `[]`.
416
 
417
+ ### `GET /api/v1/tools/report/{analysis_id}/records`
418
+
419
+ Lists the persisted analysis runs a report would be built from, oldest first. The frontend shows this before generating so the user can deselect runs; the chosen ids go to `POST /tools/report` as `exclude_record_ids`.
420
+
421
+ Response `200`:
422
+
423
+ ```json
424
+ [
425
+ {
426
+ "record_id": "rec_a1",
427
+ "goal_restated": "Rank regions by total revenue",
428
+ "created_at": "2026-06-30T08:55:02Z",
429
+ "substantive": true,
430
+ "findings_count": 2
431
+ },
432
+ {
433
+ "record_id": "rec_d4",
434
+ "goal_restated": "Correlate churn with tenure",
435
+ "created_at": "2026-06-30T09:01:47Z",
436
+ "substantive": false,
437
+ "findings_count": 1
438
+ }
439
+ ]
440
+ ```
441
+
442
+ `substantive: false` means the run produced no usable result (no analysis step or chart succeeded) — that run is listed in the report's `unresolved` JSON field rather than the findings body. If no runs exist, returns `[]`.
443
+
444
+ ### `GET /api/v1/tools/report/{analysis_id}/readiness`
445
+
446
+ Deterministic report-readiness signal for the Generate-Report button — the same producer as Help's readiness signal, so the button and Help never disagree.
447
+
448
+ Response `200`:
449
+
450
+ ```json
451
+ {
452
+ "ready": false,
453
+ "missing": ["a new analysis since the last report"]
454
+ }
455
+ ```
456
+
457
+ Note: `POST /tools/report` itself only enforces the floor (at least one completed analysis). The delta gap in `missing` is a soft warning the frontend can surface ("nothing new since the last report") without blocking the button.
458
+
459
  ### `GET /api/v1/tools/report/{analysis_id}/{version}`
460
 
461
  Returns one report version. Shape is the same as the `201` response from `POST /api/v1/tools/report`.
 
468
  }
469
  ```
470
 
471
+ ## Charts
472
+
473
+ A chart is produced when the user explicitly asks to plot/visualize something in chat. The chart is never embedded in the streamed text — it is fetched separately after the stream completes, then rendered with plotly.js under the assistant message.
474
+
475
+ ### `GET /api/v1/charts`
476
+
477
+ Returns every chart produced during one assistant answer.
478
+
479
+ Call this after the chat stream emits `done`, using the `message_id` from the `done` event (same fetch-on-`done` pattern as traceability). Chart rows are written before `done`, so there is no polling race. Every response is HTTP `200` — branch on the `status` field, not the HTTP code.
480
+
481
+ Query params:
482
+
483
+ | Query | Required | Description |
484
+ | --- | --- | --- |
485
+ | `message_id` | Yes | Assistant answer identifier returned by the stream's `done` event. |
486
+
487
+ Example:
488
+
489
+ ```text
490
+ GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2
491
+ ```
492
+
493
+ Response `200` — `status: "success"` (≥1 chart to render):
494
+
495
+ ```json
496
+ {
497
+ "status": "success",
498
+ "message": "1 chart(s) for this message.",
499
+ "count": 1,
500
+ "charts": [
501
+ {
502
+ "chart_id": "3fbd8e2e-8e21-4d4b-9b21-9e6b6a0a6a6e",
503
+ "chart_type": "bar",
504
+ "title": "Revenue by region",
505
+ "spec": {
506
+ "schema": "dataeyond.chart.v1",
507
+ "chart_type": "bar",
508
+ "title": "Revenue by region",
509
+ "plotly": {
510
+ "data": [{ "type": "bar", "x": ["Central", "East", "West"], "y": [1210000, 740000, 550000], "name": "revenue" }],
511
+ "layout": { "title": { "text": "Revenue by region" }, "xaxis": { "title": { "text": "region" } }, "yaxis": { "title": { "text": "revenue" } } }
512
+ }
513
+ },
514
+ "created_at": "2026-07-13T03:21:09.114Z"
515
+ }
516
+ ]
517
+ }
518
+ ```
519
+
520
+ Response `200` — `status: "empty"` (the turn completed but produced no charts; the common case, not an error):
521
+
522
+ ```json
523
+ {
524
+ "status": "empty",
525
+ "message": "This message completed without producing charts.",
526
+ "count": 0,
527
+ "charts": []
528
+ }
529
+ ```
530
+
531
+ Response `200` — `status: "not_found"` (no completed turn is known for this `message_id`; usually a stale or mistyped id):
532
+
533
+ ```json
534
+ {
535
+ "status": "not_found",
536
+ "message": "No completed turn is known for this message_id.",
537
+ "count": 0,
538
+ "charts": []
539
+ }
540
+ ```
541
+
542
+ The `dataeyond.chart.v1` envelope (the shape of `charts[].spec`):
543
+
544
+ ```json
545
+ {
546
+ "schema": "dataeyond.chart.v1",
547
+ "chart_type": "bar",
548
+ "title": "Revenue by region",
549
+ "plotly": {
550
+ "data": [{ "type": "bar", "x": ["A", "B"], "y": [1, 2], "name": "revenue" }],
551
+ "layout": { "title": { "text": "Revenue by region" } }
552
+ }
553
+ }
554
+ ```
555
+
556
+ Field rules:
557
+
558
+ - `status` is the outcome marker: `success` | `empty` | `not_found`. Branch on it; do not parse `message` (human-readable, for logs only).
559
+ - `spec` is the full envelope, unmodified — render straight from it: `Plotly.newPlot(el, spec.plotly.data, spec.plotly.layout)`.
560
+ - `chart_type` / `title` are copied out of `spec` for convenience (list rendering without parsing `spec`); `title` may be `null`. Chart types: `bar`, `line`, `pie`, `scatter`.
561
+ - A turn can produce more than one chart; `charts` is ordered by creation time.
562
+
563
+ Frontend rendering guidance:
564
+
565
+ - Fetch unconditionally on every `done`; `status: "empty"` means render nothing extra.
566
+ - Render each chart under the assistant message it belongs to.
567
+ - Treat `status: "not_found"` as a signal worth logging (stale id or fetch bug) — not as a user-facing error.
568
+ - Chart iteration is a follow-up chat turn (e.g. "make it a line chart") — there is no separate edit endpoint.
569
+ - The same envelope shape appears inside report markdown as ` ```plotly ` fenced blocks (see Reports → `charts`), so one renderer can serve both surfaces.
570
+
571
  ## Traceability
572
 
573
  ### `GET /api/v1/traceability`
 
589
  GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1
590
  ```
591
 
592
+ `intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
593
 
594
  Field rules:
595
 
596
  - `planning`: present only when the planner ran (`structured_flow`); otherwise `null`.
597
+ - `thinking`: **always `null` in v1** — the field stays in the payload so it can be populated later without a contract change.
598
+ - `tool_calls`: every invoked tool with `summary` (plain-English one-liner), `input`, `output`, `status`, `task_id` (nullable), and `error` (nullable); empty for chat / help / greeting / refusal paths. `input`/`output` are the raw tool I/O (opaque ids) — render them in a collapsible "technical details" section, not the headline; use `summary` for the headline.
599
+ - A `render_chart` tool call reports a compact chart summary in `output` (`chart_type`, `title`, `trace_count`, `point_count`) the full spec is served by `GET /api/v1/charts`, not here.
600
+ - `data_used`: one entry per structured data pull, resolved to **real names** for display (empty when no structured pull ran). Split into `columns_read` (columns read straight from the user's data, each tagged with its `roles`) and `output_columns` (`kind: "column"` = read from data, `kind: "computed"` = calculated, carrying a `formula` and no id). Also carries `tables` (all touched, including join targets), `joins`, `filters` (with a plain-language `description`), `group_by`, `order_by`, `limit`, `rows_returned`, and the executed `query`.
601
+ - **`id` fields are machine-only.** Every `id` in `data_used` (`source.id`, `tables[].id`, `columns_read[].id`) is for linking/audit — **the frontend must never render it.** Show `name` (qualified as `table.name`). A `computed` output column has no id by design.
602
+ - `sources`: required for retrieval flows; empty for chat / help / refusal paths and for `check`. Database sources also carry `source_name` (the DB's real name) and `tables` (every table touched).
603
+ - Summaries and filter descriptions are built from fixed templates, never an LLM — traceability adds no latency and cannot hallucinate.
604
  - The payload also carries an internal `user_id` (ownership); the frontend may ignore it.
605
+ - Truncation: `preview` ≤ 5 rows; any string inside `input`/`output`/`preview`/`snippet` ≤ 300 chars (executed `query` ≤ 2000); rows beyond the preview are dropped (`row_count` is preserved).
606
 
607
  Response `200` for `structured_flow`:
608
 
 
771
  - Render traceability separately from the streamed answer.
772
  - Default state can be collapsed.
773
  - Show planning, tool calls, and sources as separate sections.
774
+ - Treat `planning: null`, `tool_calls: []`, and `sources: []` as valid states.
src/app/components/analysis/AnalysisShell.tsx CHANGED
@@ -17,7 +17,7 @@ import {
17
  type AnalysisMessage,
18
  type DataBindItem,
19
  } from "@/services/orchestrationApi";
20
- import { AgenticApiError, getObservability, streamChat, streamHelp, type AgentStreamEvent } from "@/services/agenticApi";
21
  import { AnalysisHeader } from "./AnalysisHeader";
22
  import { AppNavigation, type AppMenuKey } from "./AppNavigation";
23
  import { ChatInput } from "./ChatInput";
@@ -176,6 +176,37 @@ export function AnalysisShell() {
176
  });
177
  }, [activeAnalysis?.id, loadingMessages, messages]);
178
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
179
  const runAgentStream = async (kind: "chat" | "help", prompt?: string) => {
180
  if (!session?.user_id || !activeAnalysis || streamState !== "idle") return;
181
 
 
17
  type AnalysisMessage,
18
  type DataBindItem,
19
  } from "@/services/orchestrationApi";
20
+ import { AgenticApiError, getCharts, getObservability, streamChat, streamHelp, type AgentStreamEvent } from "@/services/agenticApi";
21
  import { AnalysisHeader } from "./AnalysisHeader";
22
  import { AppNavigation, type AppMenuKey } from "./AppNavigation";
23
  import { ChatInput } from "./ChatInput";
 
176
  });
177
  }, [activeAnalysis?.id, loadingMessages, messages]);
178
 
179
+ const fetchCharts = async (messageId: string | undefined, uiMessageId: string) => {
180
+ if (!messageId) return;
181
+ try {
182
+ const result = await getCharts(messageId);
183
+ if (result.status === "not_found") {
184
+ console.warn(`No completed turn found for chart lookup of message ${messageId}`);
185
+ }
186
+ const charts = result.status === "success" ? result.charts : [];
187
+ setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, charts } : msg)));
188
+ } catch {
189
+ setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, charts: [] } : msg)));
190
+ }
191
+ };
192
+
193
+ useEffect(() => {
194
+ if (!activeAnalysis?.id || loadingMessages) return;
195
+
196
+ messages
197
+ .filter(
198
+ (message) =>
199
+ message.role === "ai" &&
200
+ message.status === "complete" &&
201
+ Boolean(message.messageId) &&
202
+ message.messageId !== RESERVED_FAILED_MESSAGE_ID &&
203
+ !message.charts
204
+ )
205
+ .forEach((message) => {
206
+ fetchCharts(message.messageId, message.id);
207
+ });
208
+ }, [activeAnalysis?.id, loadingMessages, messages]);
209
+
210
  const runAgentStream = async (kind: "chat" | "help", prompt?: string) => {
211
  if (!session?.user_id || !activeAnalysis || streamState !== "idle") return;
212
 
src/app/components/analysis/MessageList.tsx CHANGED
@@ -3,6 +3,7 @@ import { Bot, Sparkles, User } from "lucide-react";
3
  import type { UiMessage } from "./types";
4
  import { MarkdownContent } from "./MarkdownContent";
5
  import { MessageTraceability } from "./MessageTraceability";
 
6
  import { cx, compactQuestions, formatDateTime } from "./utils";
7
 
8
  export function MessageList({
@@ -86,6 +87,16 @@ export function MessageList({
86
  {isUser ? <p className="whitespace-pre-wrap break-words text-sm leading-6 text-slate-800 [overflow-wrap:anywhere]">{message.content}</p> : <MarkdownContent content={message.content || "..."} />}
87
  {message.status === "streaming" && message.statusText && <p className="mt-2 text-xs text-slate-500">{message.statusText}</p>}
88
  {message.status === "error" && <p className="mt-2 text-xs text-red-600">{message.statusText ?? "Stream failed"}</p>}
 
 
 
 
 
 
 
 
 
 
89
  {!isUser && (
90
  <MessageTraceability
91
  observability={message.traceability}
 
3
  import type { UiMessage } from "./types";
4
  import { MarkdownContent } from "./MarkdownContent";
5
  import { MessageTraceability } from "./MessageTraceability";
6
+ import { PlotlyChartBlock } from "./PlotlyChartBlock";
7
  import { cx, compactQuestions, formatDateTime } from "./utils";
8
 
9
  export function MessageList({
 
87
  {isUser ? <p className="whitespace-pre-wrap break-words text-sm leading-6 text-slate-800 [overflow-wrap:anywhere]">{message.content}</p> : <MarkdownContent content={message.content || "..."} />}
88
  {message.status === "streaming" && message.statusText && <p className="mt-2 text-xs text-slate-500">{message.statusText}</p>}
89
  {message.status === "error" && <p className="mt-2 text-xs text-red-600">{message.statusText ?? "Stream failed"}</p>}
90
+ {!isUser && message.charts && message.charts.length > 0 && (
91
+ <div className="mt-2 space-y-3">
92
+ {message.charts.map((chart) => (
93
+ <div key={chart.chart_id}>
94
+ {chart.title && <p className="mb-1 text-xs font-semibold text-slate-700">{chart.title}</p>}
95
+ <PlotlyChartBlock spec={chart.spec} />
96
+ </div>
97
+ ))}
98
+ </div>
99
+ )}
100
  {!isUser && (
101
  <MessageTraceability
102
  observability={message.traceability}
src/app/components/analysis/types.ts CHANGED
@@ -1,4 +1,4 @@
1
- import type { Observability } from "@/services/agenticApi";
2
  import type { AnalysisMessage } from "@/services/orchestrationApi";
3
 
4
  export type StreamState = "idle" | "streaming-chat" | "streaming-help";
@@ -17,6 +17,7 @@ export interface UiMessage {
17
  traceability?: Observability | null;
18
  traceabilityLoading?: boolean;
19
  traceabilityError?: string | null;
 
20
  }
21
 
22
  export interface AnalysisDraft {
 
1
+ import type { ChartItem, Observability } from "@/services/agenticApi";
2
  import type { AnalysisMessage } from "@/services/orchestrationApi";
3
 
4
  export type StreamState = "idle" | "streaming-chat" | "streaming-help";
 
17
  traceability?: Observability | null;
18
  traceabilityLoading?: boolean;
19
  traceabilityError?: string | null;
20
+ charts?: ChartItem[];
21
  }
22
 
23
  export interface AnalysisDraft {
src/services/agenticApi.ts CHANGED
@@ -1,3 +1,4 @@
 
1
  import { getEnv } from "@/env";
2
 
3
  const AGENTIC_BASE_URL = getEnv("VITE_AGENTIC_API_BASE_URL");
@@ -99,6 +100,31 @@ export interface ObservabilityDataUsed {
99
  query?: string;
100
  }
101
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
102
  export interface Observability {
103
  analysis_id: string;
104
  message_id: string;
@@ -254,6 +280,9 @@ export const getObservability = (analysisId: string, messageId: string): Promise
254
  `/api/v1/traceability?analysis_id=${encodeURIComponent(analysisId)}&message_id=${encodeURIComponent(messageId)}`
255
  );
256
 
 
 
 
257
  export const generateReport = (analysisId: string, userId: string): Promise<ReportDetail> =>
258
  agenticJson(
259
  `/api/v1/tools/report?analysis_id=${encodeURIComponent(analysisId)}&user_id=${encodeURIComponent(userId)}`,
 
1
+ import type { Data, Layout } from "plotly.js";
2
  import { getEnv } from "@/env";
3
 
4
  const AGENTIC_BASE_URL = getEnv("VITE_AGENTIC_API_BASE_URL");
 
100
  query?: string;
101
  }
102
 
103
+ export interface ChartSpec {
104
+ schema: string;
105
+ chart_type: string;
106
+ title?: string | null;
107
+ plotly: {
108
+ data: Data[];
109
+ layout?: Partial<Layout>;
110
+ };
111
+ }
112
+
113
+ export interface ChartItem {
114
+ chart_id: string;
115
+ chart_type: string;
116
+ title?: string | null;
117
+ spec: ChartSpec;
118
+ created_at: string;
119
+ }
120
+
121
+ export interface ChartsResponse {
122
+ status: "success" | "empty" | "not_found";
123
+ message: string;
124
+ count: number;
125
+ charts: ChartItem[];
126
+ }
127
+
128
  export interface Observability {
129
  analysis_id: string;
130
  message_id: string;
 
280
  `/api/v1/traceability?analysis_id=${encodeURIComponent(analysisId)}&message_id=${encodeURIComponent(messageId)}`
281
  );
282
 
283
+ export const getCharts = (messageId: string): Promise<ChartsResponse> =>
284
+ agenticJson(`/api/v1/charts?message_id=${encodeURIComponent(messageId)}`);
285
+
286
  export const generateReport = (analysisId: string, userId: string): Promise<ReportDetail> =>
287
  agenticJson(
288
  `/api/v1/tools/report?analysis_id=${encodeURIComponent(analysisId)}&user_id=${encodeURIComponent(userId)}`,