API_CONTRACT_BE_PYTHON.md

# Backend Agentic Service API Contract

This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, and traceability data shown alongside chat answers.

Base path examples use relative URLs. Configure the frontend with the deployed Python service base URL.

Overview

The Python backend owns the generative AI interaction surface:

1. Stream chat answers from the AI agent.
2. Execute tool-style actions for help and report generation.
3. Return report versions and report details.
4. Return traceability/provenance for a completed assistant answer.

The frontend uses this service during the analysis conversation flow:

1. User sends a chat message.
2. Frontend calls POST /api/v2/chat/stream and renders the streamed answer.
3. When the stream emits done, frontend stores or reads the returned message_id.
4. Frontend calls GET /api/v1/traceability for planning, tool calls, and source provenance.
5. Frontend calls /api/v1/tools/help for guided help and /api/v1/tools/report for report generation.

Endpoint Summary

Method	Path	Purpose
POST	/api/v2/chat/stream	Stream an AI chat answer for one analysis conversation.
GET	/api/v1/tools/list	List available frontend tools.
POST	/api/v1/tools/help	Stream contextual help for the current analysis conversation.
POST	/api/v1/tools/report	Generate and persist a new report version.
GET	/api/v1/tools/report/{analysis_id}	List report versions for an analysis.
GET	/api/v1/tools/report/{analysis_id}/{version}	Retrieve one report version.
GET	/api/v1/traceability	Retrieve provenance for one assistant answer.

Common Concepts

Identifiers

• user_id: user identifier passed by the frontend.
• analysis_id: analysis conversation identifier.
• message_id: assistant answer identifier used to correlate chat streaming with traceability.

Server-Sent Events

Chat and help endpoints return text/event-stream.

Frontend should parse events by event name and data payload. Blank lines separate SSE events.

Common event types:

Event	Data	Meaning
sources	JSON array	Always [] — sources moved to GET /api/v1/traceability (KM-691). Event kept for backward-compat; read sources[] from the traceability call.
status	text	Optional progress update for slower paths.
chunk	text	Answer text fragment. Concatenate chunks in order.
done	JSON object	Terminal success event. Includes message_id.
error	text	Terminal error event. Stream stops after this.

The stream carries answer text only. Planning, tool call details, and full provenance are fetched from GET /api/v1/traceability after the stream is done.

Chat

POST /api/v2/chat/stream

Streams an AI answer for one user message in an analysis conversation.

Request body:

{
  "user_id": "u_1a2b3c",
  "analysis_id": "an_42",
  "message_id": "msg_88f1",
  "message": "What were total sales by region last quarter?"
}

Fields:

Field	Required	Description
user_id	Yes	User identifier.
analysis_id	Yes	Analysis conversation identifier.
message_id	No	Assistant answer id for traceability correlation. If omitted, Python returns one in done.
message	Yes	User message text.

Response: text/event-stream.

Example structured answer:

event: sources
data: []

event: status
data: Planning analysis...

event: status
data: Running 3 steps...

event: chunk
data: Total sales by region last quarter: 

event: chunk
data: Central led at $1.21M (38%), East $0.74M, West $0.55M (down 12% QoQ).

event: done
data: {"message_id":"msg_88f1"}

Example simple chat answer:

event: sources
data: []

event: chunk
data: I'm your AI data analyst. Connect a source or ask a question to get started.

event: done
data: {"message_id":"msg_12"}

Behavior notes:

• Greeting and farewell messages may use a fast canned path.
• Stateless chat intent may use a 1-hour Redis response cache.
• The router may classify messages into intents such as chat, help, check, unstructured_flow, or structured_flow.
• sources in the stream is always [] (KM-691) — read the real sources[] from GET /api/v1/traceability after done.
• status events are optional and should be safe for the frontend to ignore.

Tools

GET /api/v1/tools/list

Returns the deterministic list of tools available to the frontend.

Request: none.

Response 200:

{
  "count": 2,
  "tools": [
    {
      "command": "/help",
      "name": "help",
      "type": "skill",
      "description": "Show what the assistant can do and guide your next step."
    },
    {
      "command": "/report",
      "name": "report",
      "type": "skill",
      "description": "Generate a versioned analysis report with background, EDA, key findings, and insights."
    }
  ]
}

Tool item shape:

{
  "command": "/help",
  "name": "help",
  "type": "skill",
  "description": "Show what the assistant can do and guide your next step."
}

Frontend behavior:

• Surface /help in the slash menu.
• Surface report generation as a button or explicit UI action.

POST /api/v1/tools/help

Streams contextual guidance for the current analysis conversation.

Request body:

{
  "user_id": "u_1a2b3c",
  "analysis_id": "an_42"
}

Response: text/event-stream using the same event shape as chat.

Help responses usually emit sources: [] and no status pings.

Example:

event: sources
data: []

event: chunk
data: Your goal is set. You can start exploring now. Try a question like "average order value by month", then I can generate a report.

event: done
data: {"message_id":"msg_h7"}

Reports

POST /api/v1/tools/report

Generates, persists, and returns a new report version for an analysis.

Query params:

Query	Required	Description
analysis_id	Yes	Analysis identifier.
user_id	Yes	User identifier.

Example:

POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c

Status codes:

Status	Meaning
201	New report version generated.
409	Report floor/precondition not met.
500	Generation or persistence failed.

Response 201:

{
  "report_id": "8f3a2b1c9d4e4f6a8b0c1d2e3f4a5b6c",
  "analysis_id": "an_42",
  "user_id": "u_1a2b3c",
  "version": 2,
  "generated_at": "2026-06-30T09:14:33.512Z",
  "problem_statement": {
    "objective": "Understand which regions drive revenue and why Q1 dipped.",
    "business_questions": [
      "Which regions contribute most to total revenue?",
      "Did any region decline quarter-over-quarter?"
    ]
  },
  "record_ids": ["rec_a1", "rec_b2"],
  "executive_summary": "Revenue is concentrated in the Central region (38% of total). The West was the only region to contract, down 12% QoQ, the main driver of the Q1 dip.",
  "findings": [
    {
      "text": "Central region contributed 38% of total revenue, the largest share.",
      "record_ids": ["rec_a1"],
      "supporting_data": null
    },
    {
      "text": "West region revenue fell 12% quarter-over-quarter.",
      "record_ids": ["rec_b2"],
      "supporting_data": null
    }
  ],
  "caveats": [
    {
      "text": "March data for the East region was partially missing, around 6% of rows.",
      "record_ids": ["rec_b2"]
    }
  ],
  "open_questions": [
    {
      "text": "What drove the West region's QoQ decline?",
      "record_ids": ["rec_b2"]
    }
  ],
  "data_sources": [
    {
      "source_id": "src_sales_db",
      "name": "orders",
      "source_type": "postgres",
      "detail": {
        "tables": ["orders"],
        "row_count": 48213,
        "columns": ["region", "amount", "ordered_at"]
      }
    }
  ],
  "method_steps": [
    {
      "task_id": "t1",
      "stage": "data_understanding",
      "objective": "Inventory the sales source",
      "status": "success",
      "tools_used": ["check_data"]
    },
    {
      "task_id": "t2",
      "stage": "modeling",
      "objective": "Aggregate revenue by region",
      "status": "success",
      "tools_used": ["analyze_aggregate"]
    }
  ],
  "rendered_markdown": "# Analysis Report\n\n*Generated 2026-06-30 by u_1a2b3c*\n\n## Objective\nUnderstand which regions drive revenue..."
}

Response 409:

{
  "detail": "Not ready to generate a report - still needs at least one completed analysis."
}

Precondition:

• Reports require at least one completed analysis record for the session.
• If slow-path analysis recording is disabled, report generation can return 409 by design.

GET /api/v1/tools/report/{analysis_id}

Lists report versions for one analysis, oldest first.

Response 200:

[
  {
    "report_id": "1b2c3d4e",
    "version": 1,
    "generated_at": "2026-06-24T15:02:11Z",
    "record_count": 1
  },
  {
    "report_id": "8f3a2b1c",
    "version": 2,
    "generated_at": "2026-06-25T09:14:33Z",
    "record_count": 2
  }
]

If no reports exist, returns [].

GET /api/v1/tools/report/{analysis_id}/{version}

Returns one report version. Shape is the same as the 201 response from POST /api/v1/tools/report.

Response 404:

{
  "detail": "No report v3 for analysis 'an_42'."
}

Traceability

Renamed from observability (team decision 2026-07-06) so it is never confused with the internal Langfuse observability stack (engineering telemetry, PII-masked). Traceability is user-facing provenance — real tool args, output previews, and the executed query — shown alongside the answer.

GET /api/v1/traceability

Returns user-facing provenance for one assistant answer.

The frontend should call this after the chat/help stream emits done, using the message_id from the done event. The row is written before done, so an immediate GET returns 200 (no polling race). A 404 means the id is unknown or the turn errored before completing (error turns never produce a row).

Query params:

Query	Required	Description
analysis_id	Yes	Analysis identifier.
message_id	Yes	Assistant answer identifier returned by the stream.

Example:

GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1

intent values the frontend may see: chat · help · check · unstructured_flow · structured_flow · out_of_scope · blocked (blocked = input-guard or Azure content-filter refusal; chat also covers the greeting fast-path and cache replays).

Field rules:

• planning: present only when the planner ran (structured_flow); otherwise null.
• thinking: always null in v1 — our agents are plain chat completions with no native reasoning output, and synthesizing it post-hoc would be unfaithful. The field stays in the payload so it can be populated later without a contract change.
• tool_calls: every invoked tool with input, output, status, task_id (nullable), and error (nullable); empty for chat / help / greeting / refusal paths.
• sources: required for retrieval flows; empty for chat / help / refusal paths and for check.
• The payload also carries an internal user_id (ownership); the frontend may ignore it.
• Truncation: preview ≤ 5 rows; any string inside input/output/preview/snippet ≤ 300 chars; rows beyond the preview are dropped (row_count is preserved).

Response 200 for structured_flow:

{
  "analysis_id": "an_42",
  "message_id": "msg_88f1",
  "user_id": "user_7",
  "intent": "structured_flow",
  "generated_at": "2026-07-06T03:21:09.114Z",
  "planning": {
    "goal_restated": "Find which regions drive revenue and why Q1 dipped.",
    "assumptions": [],
    "steps": [
      {
        "step": 1,
        "stage": "data_understanding",
        "objective": "Inventory the sales source",
        "status": "success",
        "tools_used": ["check_data"]
      },
      {
        "step": 2,
        "stage": "modeling",
        "objective": "Aggregate revenue by region",
        "status": "success",
        "tools_used": ["retrieve_data", "analyze_aggregate"]
      }
    ]
  },
  "thinking": null,
  "tool_calls": [
    {
      "order": 1,
      "task_id": null,
      "name": "check_data",
      "input": { "source_hint": "structured" },
      "output": {
        "kind": "table",
        "columns": ["source_id", "name", "source_type", "table_count"],
        "row_count": 1,
        "preview": [["src_sales_db", "orders", "schema", 1]]
      },
      "status": "success",
      "error": null
    },
    {
      "order": 2,
      "task_id": null,
      "name": "retrieve_data",
      "input": { "ir": { "source_id": "src_sales_db", "table_id": "orders", "select": ["region", "amount"], "group_by": ["region"] } },
      "output": {
        "kind": "table",
        "columns": ["region", "total"],
        "row_count": 4,
        "preview": [["Central", 1210000], ["East", 740000]]
      },
      "status": "success",
      "error": null
    }
  ],
  "sources": [
    {
      "type": "database",
      "source_id": "src_sales_db",
      "name": "orders",
      "query": "SELECT region, SUM(amount) AS total FROM orders GROUP BY region",
      "detail": {
        "table": "orders",
        "row_count": 4
      }
    }
  ]
}

Note: retrieve_data's real input is the compiled query IR under an ir key (the planner builds an IR, never raw SQL). The executed SQL/rendered query appears on the corresponding sources[].query.

Response 200 for unstructured_flow:

{
  "analysis_id": "an_42",
  "message_id": "msg_55",
  "user_id": "user_7",
  "intent": "unstructured_flow",
  "generated_at": "2026-07-06T03:40:02.001Z",
  "planning": null,
  "thinking": null,
  "tool_calls": [
    {
      "order": 1,
      "task_id": null,
      "name": "retrieve_knowledge",
      "input": { "query": "technology stack used in this project" },
      "output": { "kind": "documents", "row_count": 4 },
      "status": "success",
      "error": null
    }
  ],
  "sources": [
    {
      "type": "document",
      "document_id": "doc_7",
      "filename": "tech_handbook.pdf",
      "page_label": "12",
      "query": "technology stack used in this project",
      "snippet": "The backend is built on FastAPI with async SQLAlchemy...",
      "score": 0.83
    }
  ]
}

Response 200 for chat / greeting / help / refusals (out_of_scope, blocked):

{
  "analysis_id": "an_42",
  "message_id": "msg_12",
  "user_id": "user_7",
  "intent": "chat",
  "generated_at": "2026-07-06T03:05:00.000Z",
  "planning": null,
  "thinking": null,
  "tool_calls": [],
  "sources": []
}

Response 404:

{
  "detail": "No traceability for message 'msg_88f1' yet."
}

Frontend rendering guidance:

• Render traceability separately from the streamed answer.
• Default state can be collapsed.
• Show planning, tool calls, and sources as separate sections.
• Treat planning: null, tool_calls: [], and sources: [] as valid states.