Commit ·
c38c0c2
1
Parent(s): 8c1bb63
[KM-642] refactor(tools): rename data-access tools to data/knowledge naming
Browse filesMechanical rename across the tool layer, the planner stub/validator/prompt/
few-shots, and docstrings: query_structured -> data_retrieve and
retrieve_documents -> knowledge_retrieve (data/knowledge taxonomy from the
2026-06-11 checkpoint). No behavior change; the data_check/knowledge_check
merge lands in a follow-up. Tests green (206 passed); all changed files pass
ruff, data_access.py clean under mypy.
Touches the planner area (prompt, validator, examples) — rename only,
wording preserved. Flagged for Rifqi review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- src/agents/planner/examples.py +9 -9
- src/agents/planner/inputs.py +3 -3
- src/agents/planner/registry.py +8 -8
- src/agents/planner/service.py +1 -1
- src/agents/planner/validator.py +5 -5
- src/catalog/reader.py +1 -1
- src/config/prompts/planner.md +4 -4
- src/query/executor/db.py +1 -1
- src/tools/analytics/aggregation.py +1 -1
- src/tools/data_access.py +19 -19
- src/tools/registry.py +2 -2
src/agents/planner/examples.py
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
Two illustrative (question -> TaskList) pairs that teach the OUTPUT SHAPE:
|
| 4 |
stages, dependency edges, ordered tool-call chains, inline QueryIR,
|
| 5 |
-
"${t<id>}" placeholders, and the assumed data-flow convention — `
|
| 6 |
pulls rows, then a composite `analyze_*` tool consumes them via a `data` placeholder
|
| 7 |
referencing the upstream result's column aliases (Pattern A; the tool team may
|
| 8 |
instead pick self-fetch by `source_id`, in which case these examples are reshaped
|
|
@@ -21,7 +21,7 @@ from .schemas import Task, TaskList, ToolCall
|
|
| 21 |
# --------------------------------------------------------------------------- #
|
| 22 |
# Example A — exploratory, no modeling.
|
| 23 |
# "Which product categories drove last quarter's revenue?"
|
| 24 |
-
# Shows:
|
| 25 |
# category in one call (no manual per-category queries).
|
| 26 |
# --------------------------------------------------------------------------- #
|
| 27 |
|
|
@@ -47,7 +47,7 @@ _EXAMPLE_A = TaskList(
|
|
| 47 |
objective="Pull last quarter's order-level category and revenue rows.",
|
| 48 |
tool_calls=[
|
| 49 |
ToolCall(
|
| 50 |
-
tool="
|
| 51 |
args={
|
| 52 |
"ir": {
|
| 53 |
"source_id": "src_sales",
|
|
@@ -123,7 +123,7 @@ _EXAMPLE_B = TaskList(
|
|
| 123 |
objective="Pull this year's order dates, revenue, and region.",
|
| 124 |
tool_calls=[
|
| 125 |
ToolCall(
|
| 126 |
-
tool="
|
| 127 |
args={
|
| 128 |
"ir": {
|
| 129 |
"source_id": "src_sales",
|
|
@@ -187,8 +187,8 @@ _EXAMPLE_B = TaskList(
|
|
| 187 |
# Example C — mixed structured + unstructured.
|
| 188 |
# "Revenue dipped in Q1 — what happened?"
|
| 189 |
# Shows: a structured branch (query -> analyze_trend) runs alongside an
|
| 190 |
-
# INDEPENDENT
|
| 191 |
-
#
|
| 192 |
# placeholder — it is a source, not a consumer) and can run in parallel; the
|
| 193 |
# Assembler folds the document context into the explanation.
|
| 194 |
# --------------------------------------------------------------------------- #
|
|
@@ -215,7 +215,7 @@ _EXAMPLE_C = TaskList(
|
|
| 215 |
objective="Pull Q1 order dates and revenue.",
|
| 216 |
tool_calls=[
|
| 217 |
ToolCall(
|
| 218 |
-
tool="
|
| 219 |
args={
|
| 220 |
"ir": {
|
| 221 |
"source_id": "src_sales",
|
|
@@ -273,7 +273,7 @@ _EXAMPLE_C = TaskList(
|
|
| 273 |
objective="Retrieve qualitative context on Q1 operational events behind a dip.",
|
| 274 |
tool_calls=[
|
| 275 |
ToolCall(
|
| 276 |
-
tool="
|
| 277 |
args={
|
| 278 |
"query": "operational issues, outages, or notable events in Q1 2026",
|
| 279 |
"top_k": 5,
|
|
@@ -320,7 +320,7 @@ _EXAMPLE_D = TaskList(
|
|
| 320 |
objective="Pull order-level region and revenue.",
|
| 321 |
tool_calls=[
|
| 322 |
ToolCall(
|
| 323 |
-
tool="
|
| 324 |
args={
|
| 325 |
"ir": {
|
| 326 |
"source_id": "src_sales",
|
|
|
|
| 2 |
|
| 3 |
Two illustrative (question -> TaskList) pairs that teach the OUTPUT SHAPE:
|
| 4 |
stages, dependency edges, ordered tool-call chains, inline QueryIR,
|
| 5 |
+
"${t<id>}" placeholders, and the assumed data-flow convention — `data_retrieve`
|
| 6 |
pulls rows, then a composite `analyze_*` tool consumes them via a `data` placeholder
|
| 7 |
referencing the upstream result's column aliases (Pattern A; the tool team may
|
| 8 |
instead pick self-fetch by `source_id`, in which case these examples are reshaped
|
|
|
|
| 21 |
# --------------------------------------------------------------------------- #
|
| 22 |
# Example A — exploratory, no modeling.
|
| 23 |
# "Which product categories drove last quarter's revenue?"
|
| 24 |
+
# Shows: data_retrieve pulls rows -> analyze_aggregate sums revenue per
|
| 25 |
# category in one call (no manual per-category queries).
|
| 26 |
# --------------------------------------------------------------------------- #
|
| 27 |
|
|
|
|
| 47 |
objective="Pull last quarter's order-level category and revenue rows.",
|
| 48 |
tool_calls=[
|
| 49 |
ToolCall(
|
| 50 |
+
tool="data_retrieve",
|
| 51 |
args={
|
| 52 |
"ir": {
|
| 53 |
"source_id": "src_sales",
|
|
|
|
| 123 |
objective="Pull this year's order dates, revenue, and region.",
|
| 124 |
tool_calls=[
|
| 125 |
ToolCall(
|
| 126 |
+
tool="data_retrieve",
|
| 127 |
args={
|
| 128 |
"ir": {
|
| 129 |
"source_id": "src_sales",
|
|
|
|
| 187 |
# Example C — mixed structured + unstructured.
|
| 188 |
# "Revenue dipped in Q1 — what happened?"
|
| 189 |
# Shows: a structured branch (query -> analyze_trend) runs alongside an
|
| 190 |
+
# INDEPENDENT knowledge_retrieve branch that pulls qualitative context. Note
|
| 191 |
+
# knowledge_retrieve takes a natural-language `query` (NOT a `${t<id>}` data
|
| 192 |
# placeholder — it is a source, not a consumer) and can run in parallel; the
|
| 193 |
# Assembler folds the document context into the explanation.
|
| 194 |
# --------------------------------------------------------------------------- #
|
|
|
|
| 215 |
objective="Pull Q1 order dates and revenue.",
|
| 216 |
tool_calls=[
|
| 217 |
ToolCall(
|
| 218 |
+
tool="data_retrieve",
|
| 219 |
args={
|
| 220 |
"ir": {
|
| 221 |
"source_id": "src_sales",
|
|
|
|
| 273 |
objective="Retrieve qualitative context on Q1 operational events behind a dip.",
|
| 274 |
tool_calls=[
|
| 275 |
ToolCall(
|
| 276 |
+
tool="knowledge_retrieve",
|
| 277 |
args={
|
| 278 |
"query": "operational issues, outages, or notable events in Q1 2026",
|
| 279 |
"top_k": 5,
|
|
|
|
| 320 |
objective="Pull order-level region and revenue.",
|
| 321 |
tool_calls=[
|
| 322 |
ToolCall(
|
| 323 |
+
tool="data_retrieve",
|
| 324 |
args={
|
| 325 |
"ir": {
|
| 326 |
"source_id": "src_sales",
|
src/agents/planner/inputs.py
CHANGED
|
@@ -4,9 +4,9 @@
|
|
| 4 |
for the planner prompt. It carries every table + column id/type/PII flag + row
|
| 5 |
counts + low-cardinality top_values, with `sample_values` nulled on PII columns
|
| 6 |
(INV: no PII sample values into the prompt, see doc §13). It also lists the
|
| 7 |
-
available unstructured sources so the planner can plan `
|
| 8 |
|
| 9 |
-
The planner *validator* still checks inline `
|
| 10 |
full `Catalog` via the existing IRValidator — the summary is a prompt input, not
|
| 11 |
the validation source of truth.
|
| 12 |
|
|
@@ -124,7 +124,7 @@ class CatalogSummary(BaseModel):
|
|
| 124 |
lines.append("")
|
| 125 |
|
| 126 |
if self.unstructured_sources:
|
| 127 |
-
lines.append("Unstructured sources (for
|
| 128 |
for src in self.unstructured_sources:
|
| 129 |
lines.append(f" - {src.name} — id={src.source_id}")
|
| 130 |
|
|
|
|
| 4 |
for the planner prompt. It carries every table + column id/type/PII flag + row
|
| 5 |
counts + low-cardinality top_values, with `sample_values` nulled on PII columns
|
| 6 |
(INV: no PII sample values into the prompt, see doc §13). It also lists the
|
| 7 |
+
available unstructured sources so the planner can plan `knowledge_retrieve`.
|
| 8 |
|
| 9 |
+
The planner *validator* still checks inline `data_retrieve` IRs against the
|
| 10 |
full `Catalog` via the existing IRValidator — the summary is a prompt input, not
|
| 11 |
the validation source of truth.
|
| 12 |
|
|
|
|
| 124 |
lines.append("")
|
| 125 |
|
| 126 |
if self.unstructured_sources:
|
| 127 |
+
lines.append("Unstructured sources (for knowledge_retrieve):")
|
| 128 |
for src in self.unstructured_sources:
|
| 129 |
lines.append(f" - {src.name} — id={src.source_id}")
|
| 130 |
|
src/agents/planner/registry.py
CHANGED
|
@@ -7,7 +7,7 @@ outside it).
|
|
| 7 |
`src/tools/registry.py::analytics_registry()` (KM-628), built on the canonical
|
| 8 |
`ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
|
| 9 |
descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
|
| 10 |
-
- **Data access (`
|
| 11 |
`describe_source`) — spec BODIES still a local stub.** The tool team owns these too,
|
| 12 |
but their wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
|
| 13 |
spec bodies here so the Planner can plan end-to-end — but the NAMES derive from
|
|
@@ -16,10 +16,10 @@ outside it).
|
|
| 16 |
this slice and swap `default_registry()` for the tool team's full composition.
|
| 17 |
|
| 18 |
**Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
|
| 19 |
-
`"${t<id>}"` placeholder pointing at an upstream `
|
| 20 |
self-fetch); resolved to a DataFrame at execution time. `input_schema` is the
|
| 21 |
lightweight `{required, properties}` dict the planner validator (check #8) reads;
|
| 22 |
-
`
|
| 23 |
catalog by the existing IRValidator.
|
| 24 |
|
| 25 |
See AGENT_ARCHITECTURE_CONTEXT_new.md §9.2 / §9.3.
|
|
@@ -38,7 +38,7 @@ from .contracts import ToolRegistry, ToolSpec
|
|
| 38 |
# --------------------------------------------------------------------------- #
|
| 39 |
_DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
| 40 |
ToolSpec(
|
| 41 |
-
name="
|
| 42 |
category="analytics.query",
|
| 43 |
input_schema={"required": ["ir"], "properties": {"ir": {"type": "object"}}},
|
| 44 |
output_kind="table",
|
|
@@ -52,11 +52,11 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
|
| 52 |
"(count/sum/avg/min/max/count_distinct). Do NOT use it for richer statistics "
|
| 53 |
"(median/percentile/mode/stddev/skew → analyze_descriptive), trends "
|
| 54 |
"(analyze_trend), correlation, segmentation, or share-of-total; and do NOT "
|
| 55 |
-
"use it to read documents (use
|
| 56 |
),
|
| 57 |
),
|
| 58 |
ToolSpec(
|
| 59 |
-
name="
|
| 60 |
category="retrieval.documents",
|
| 61 |
input_schema={
|
| 62 |
"required": ["query"],
|
|
@@ -71,7 +71,7 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
|
| 71 |
"Dense-retrieve the most relevant chunks from the user's unstructured "
|
| 72 |
"sources (PDF/DOCX/TXT) for a natural-language `query`. Use this to pull "
|
| 73 |
"qualitative context into an analysis. Optionally scope to one `source_id`. "
|
| 74 |
-
"Do NOT use it for numbers in tables — that is
|
| 75 |
),
|
| 76 |
),
|
| 77 |
ToolSpec(
|
|
@@ -96,7 +96,7 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
|
| 96 |
description=(
|
| 97 |
"Return the tables and columns (names, types, row counts) of one source by "
|
| 98 |
"`source_id`. Use in data_understanding to confirm the shape of a source "
|
| 99 |
-
"before querying it. Do NOT use it to fetch data rows (use
|
| 100 |
),
|
| 101 |
),
|
| 102 |
)
|
|
|
|
| 7 |
`src/tools/registry.py::analytics_registry()` (KM-628), built on the canonical
|
| 8 |
`ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
|
| 9 |
descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
|
| 10 |
+
- **Data access (`data_retrieve` / `knowledge_retrieve` / `list_sources` /
|
| 11 |
`describe_source`) — spec BODIES still a local stub.** The tool team owns these too,
|
| 12 |
but their wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
|
| 13 |
spec bodies here so the Planner can plan end-to-end — but the NAMES derive from
|
|
|
|
| 16 |
this slice and swap `default_registry()` for the tool team's full composition.
|
| 17 |
|
| 18 |
**Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
|
| 19 |
+
`"${t<id>}"` placeholder pointing at an upstream `data_retrieve` output (no
|
| 20 |
self-fetch); resolved to a DataFrame at execution time. `input_schema` is the
|
| 21 |
lightweight `{required, properties}` dict the planner validator (check #8) reads;
|
| 22 |
+
`data_retrieve.args["ir"]` carries an inline QueryIR validated against the
|
| 23 |
catalog by the existing IRValidator.
|
| 24 |
|
| 25 |
See AGENT_ARCHITECTURE_CONTEXT_new.md §9.2 / §9.3.
|
|
|
|
| 38 |
# --------------------------------------------------------------------------- #
|
| 39 |
_DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
| 40 |
ToolSpec(
|
| 41 |
+
name="data_retrieve",
|
| 42 |
category="analytics.query",
|
| 43 |
input_schema={"required": ["ir"], "properties": {"ir": {"type": "object"}}},
|
| 44 |
output_kind="table",
|
|
|
|
| 52 |
"(count/sum/avg/min/max/count_distinct). Do NOT use it for richer statistics "
|
| 53 |
"(median/percentile/mode/stddev/skew → analyze_descriptive), trends "
|
| 54 |
"(analyze_trend), correlation, segmentation, or share-of-total; and do NOT "
|
| 55 |
+
"use it to read documents (use knowledge_retrieve)."
|
| 56 |
),
|
| 57 |
),
|
| 58 |
ToolSpec(
|
| 59 |
+
name="knowledge_retrieve",
|
| 60 |
category="retrieval.documents",
|
| 61 |
input_schema={
|
| 62 |
"required": ["query"],
|
|
|
|
| 71 |
"Dense-retrieve the most relevant chunks from the user's unstructured "
|
| 72 |
"sources (PDF/DOCX/TXT) for a natural-language `query`. Use this to pull "
|
| 73 |
"qualitative context into an analysis. Optionally scope to one `source_id`. "
|
| 74 |
+
"Do NOT use it for numbers in tables — that is data_retrieve's job."
|
| 75 |
),
|
| 76 |
),
|
| 77 |
ToolSpec(
|
|
|
|
| 96 |
description=(
|
| 97 |
"Return the tables and columns (names, types, row counts) of one source by "
|
| 98 |
"`source_id`. Use in data_understanding to confirm the shape of a source "
|
| 99 |
+
"before querying it. Do NOT use it to fetch data rows (use data_retrieve)."
|
| 100 |
),
|
| 101 |
),
|
| 102 |
)
|
src/agents/planner/service.py
CHANGED
|
@@ -9,7 +9,7 @@ static plan.
|
|
| 9 |
|
| 10 |
The service takes the full `Catalog` (not just a `CatalogSummary`): it derives
|
| 11 |
the PII-safe `CatalogSummary` for the prompt, but validation needs the full
|
| 12 |
-
catalog so the existing `IRValidator` can check inline `
|
| 13 |
|
| 14 |
See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3.
|
| 15 |
"""
|
|
|
|
| 9 |
|
| 10 |
The service takes the full `Catalog` (not just a `CatalogSummary`): it derives
|
| 11 |
the PII-safe `CatalogSummary` for the prompt, but validation needs the full
|
| 12 |
+
catalog so the existing `IRValidator` can check inline `data_retrieve` IRs.
|
| 13 |
|
| 14 |
See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3.
|
| 15 |
"""
|
src/agents/planner/validator.py
CHANGED
|
@@ -95,8 +95,8 @@ class PlannerValidator:
|
|
| 95 |
f"source_id {src!r} (known: {sorted(known_sources)})"
|
| 96 |
)
|
| 97 |
|
| 98 |
-
# Check 8b — inline
|
| 99 |
-
if call.tool == "
|
| 100 |
self._validate_inline_ir(task.id, call.args, catalog)
|
| 101 |
|
| 102 |
# Check 7 — success_criteria is checkable.
|
|
@@ -114,20 +114,20 @@ class PlannerValidator:
|
|
| 114 |
raw_ir = args.get("ir")
|
| 115 |
if not isinstance(raw_ir, dict):
|
| 116 |
raise PlannerValidationError(
|
| 117 |
-
f"task {task_id}:
|
| 118 |
f"object, got {type(raw_ir).__name__}"
|
| 119 |
)
|
| 120 |
try:
|
| 121 |
ir = QueryIR.model_validate(raw_ir)
|
| 122 |
except ValidationError as e:
|
| 123 |
raise PlannerValidationError(
|
| 124 |
-
f"task {task_id}:
|
| 125 |
) from e
|
| 126 |
try:
|
| 127 |
self._ir_validator.validate(ir, catalog)
|
| 128 |
except IRValidationError as e:
|
| 129 |
raise PlannerValidationError(
|
| 130 |
-
f"task {task_id}:
|
| 131 |
) from e
|
| 132 |
|
| 133 |
@staticmethod
|
|
|
|
| 95 |
f"source_id {src!r} (known: {sorted(known_sources)})"
|
| 96 |
)
|
| 97 |
|
| 98 |
+
# Check 8b — inline data_retrieve IR validates against the catalog.
|
| 99 |
+
if call.tool == "data_retrieve":
|
| 100 |
self._validate_inline_ir(task.id, call.args, catalog)
|
| 101 |
|
| 102 |
# Check 7 — success_criteria is checkable.
|
|
|
|
| 114 |
raw_ir = args.get("ir")
|
| 115 |
if not isinstance(raw_ir, dict):
|
| 116 |
raise PlannerValidationError(
|
| 117 |
+
f"task {task_id}: data_retrieve.args.ir must be an inline QueryIR "
|
| 118 |
f"object, got {type(raw_ir).__name__}"
|
| 119 |
)
|
| 120 |
try:
|
| 121 |
ir = QueryIR.model_validate(raw_ir)
|
| 122 |
except ValidationError as e:
|
| 123 |
raise PlannerValidationError(
|
| 124 |
+
f"task {task_id}: data_retrieve.args.ir is not a valid QueryIR: {e}"
|
| 125 |
) from e
|
| 126 |
try:
|
| 127 |
self._ir_validator.validate(ir, catalog)
|
| 128 |
except IRValidationError as e:
|
| 129 |
raise PlannerValidationError(
|
| 130 |
+
f"task {task_id}: data_retrieve IR failed catalog validation: {e}"
|
| 131 |
) from e
|
| 132 |
|
| 133 |
@staticmethod
|
src/catalog/reader.py
CHANGED
|
@@ -45,7 +45,7 @@ class MemoizingCatalogReader(CatalogReader):
|
|
| 45 |
|
| 46 |
One per request. The same per-user catalog is otherwise fetched from the
|
| 47 |
catalog DB 4-5x during a single slow-path run (planner load, then
|
| 48 |
-
describe_source's structured+unstructured reads, then
|
| 49 |
structured read). Wrapping the base reader collapses those to one round-trip
|
| 50 |
per distinct source_hint and pins a single consistent snapshot for the whole
|
| 51 |
request (plan-time and execution-time catalogs can no longer diverge).
|
|
|
|
| 45 |
|
| 46 |
One per request. The same per-user catalog is otherwise fetched from the
|
| 47 |
catalog DB 4-5x during a single slow-path run (planner load, then
|
| 48 |
+
describe_source's structured+unstructured reads, then data_retrieve's
|
| 49 |
structured read). Wrapping the base reader collapses those to one round-trip
|
| 50 |
per distinct source_hint and pins a single consistent snapshot for the whole
|
| 51 |
request (plan-time and execution-time catalogs can no longer diverge).
|
src/config/prompts/planner.md
CHANGED
|
@@ -10,7 +10,7 @@ only a `TaskList` object that conforms to the provided schema.
|
|
| 10 |
|
| 11 |
1. **Emit intent, never code.** Never write SQL, pandas, or any code. The only
|
| 12 |
query you express is an inline `QueryIR` (a JSON intent object) inside a
|
| 13 |
-
`
|
| 14 |
2. **The plan is static.** There is no replanning and no execution feedback. Plan
|
| 15 |
the whole analysis up front; assume each task runs once, in dependency order.
|
| 16 |
3. **Use only tools from the "Available tools" list.** Never invent a tool name.
|
|
@@ -31,16 +31,16 @@ only a `TaskList` object that conforms to the provided schema.
|
|
| 31 |
- **Wire data between tasks with placeholders.** When a task needs an upstream
|
| 32 |
task's output as an argument, use the string `"${t<id>}"` (e.g. `"${t2}"`) as
|
| 33 |
the argument value. Set `depends_on` accordingly.
|
| 34 |
-
- **Data access vs analytics tools.** `
|
| 35 |
point: use it to select, filter, and pull rows (and simple built-in
|
| 36 |
count/sum/avg/min/max/count_distinct the IR can express). For anything richer —
|
| 37 |
descriptive statistics (median/percentile/mode/std/skew), time trends, group
|
| 38 |
comparisons, share-of-total, correlation, segmentation, or data-quality
|
| 39 |
-
profiling — run `
|
| 40 |
the matching composite `analyze_*` tool via a `"${t<id>}"` `data` argument
|
| 41 |
(referencing the upstream result's column aliases).
|
| 42 |
- **Mixing structured + unstructured.** If qualitative context helps, add a
|
| 43 |
-
`
|
| 44 |
- **CRISP-DM stages.** Tag each task with the stage it serves:
|
| 45 |
`data_understanding`, `data_preparation`, or `evaluation`. (Never `modeling`.)
|
| 46 |
- **success_criteria is a reporting signal**, not a control trigger. State, in
|
|
|
|
| 10 |
|
| 11 |
1. **Emit intent, never code.** Never write SQL, pandas, or any code. The only
|
| 12 |
query you express is an inline `QueryIR` (a JSON intent object) inside a
|
| 13 |
+
`data_retrieve` tool call's `args.ir`.
|
| 14 |
2. **The plan is static.** There is no replanning and no execution feedback. Plan
|
| 15 |
the whole analysis up front; assume each task runs once, in dependency order.
|
| 16 |
3. **Use only tools from the "Available tools" list.** Never invent a tool name.
|
|
|
|
| 31 |
- **Wire data between tasks with placeholders.** When a task needs an upstream
|
| 32 |
task's output as an argument, use the string `"${t<id>}"` (e.g. `"${t2}"`) as
|
| 33 |
the argument value. Set `depends_on` accordingly.
|
| 34 |
+
- **Data access vs analytics tools.** `data_retrieve` is the data-access entry
|
| 35 |
point: use it to select, filter, and pull rows (and simple built-in
|
| 36 |
count/sum/avg/min/max/count_distinct the IR can express). For anything richer —
|
| 37 |
descriptive statistics (median/percentile/mode/std/skew), time trends, group
|
| 38 |
comparisons, share-of-total, correlation, segmentation, or data-quality
|
| 39 |
+
profiling — run `data_retrieve` to fetch the rows, then pass its output to
|
| 40 |
the matching composite `analyze_*` tool via a `"${t<id>}"` `data` argument
|
| 41 |
(referencing the upstream result's column aliases).
|
| 42 |
- **Mixing structured + unstructured.** If qualitative context helps, add a
|
| 43 |
+
`knowledge_retrieve` task against an unstructured source listed in the catalog.
|
| 44 |
- **CRISP-DM stages.** Tag each task with the stage it serves:
|
| 45 |
`data_understanding`, `data_preparation`, or `evaluation`. (Never `modeling`.)
|
| 46 |
- **success_criteria is a reporting signal**, not a control trigger. State, in
|
src/query/executor/db.py
CHANGED
|
@@ -220,7 +220,7 @@ class DbExecutor(BaseExecutor):
|
|
| 220 |
"""Best-effort: warm pooled engines for the catalog's schema sources.
|
| 221 |
|
| 222 |
Called at slow-path entry so the TCP+TLS+auth handshake overlaps the ~4s
|
| 223 |
-
Planner LLM call — by the time `
|
| 224 |
already established. Warming is an optimization, never a requirement, so
|
| 225 |
this never raises and per-source failures are swallowed.
|
| 226 |
"""
|
|
|
|
| 220 |
"""Best-effort: warm pooled engines for the catalog's schema sources.
|
| 221 |
|
| 222 |
Called at slow-path entry so the TCP+TLS+auth handshake overlaps the ~4s
|
| 223 |
+
Planner LLM call — by the time `data_retrieve` runs, the connection is
|
| 224 |
already established. Warming is an optimization, never a requirement, so
|
| 225 |
this never raises and per-source failures are swallowed.
|
| 226 |
"""
|
src/tools/analytics/aggregation.py
CHANGED
|
@@ -3,7 +3,7 @@
|
|
| 3 |
An analytical "family" tool: in ONE call it groups rows by one or more keys
|
| 4 |
and computes aggregates (sum, mean, count, min, max, median, nunique) per
|
| 5 |
group. This is the deterministic compute twin of the Planner's
|
| 6 |
-
`
|
| 7 |
|
| 8 |
STATUS: compute layer only — the function takes an already-materialized
|
| 9 |
DataFrame. The wrapper layer (fetching data from the catalog via source_id,
|
|
|
|
| 3 |
An analytical "family" tool: in ONE call it groups rows by one or more keys
|
| 4 |
and computes aggregates (sum, mean, count, min, max, median, nunique) per
|
| 5 |
group. This is the deterministic compute twin of the Planner's
|
| 6 |
+
`data_retrieve` step — the wrapper layer later maps a QueryIR onto this.
|
| 7 |
|
| 8 |
STATUS: compute layer only — the function takes an already-materialized
|
| 9 |
DataFrame. The wrapper layer (fetching data from the catalog via source_id,
|
src/tools/data_access.py
CHANGED
|
@@ -11,11 +11,11 @@ Tools implemented here:
|
|
| 11 |
- `list_sources` — the user's data sources (id, name, type, table count).
|
| 12 |
- `describe_source` — tables/columns of one source (one row per column,
|
| 13 |
metadata only — exposes `pii_flag`, never sample values).
|
| 14 |
-
- `
|
| 15 |
execute, skipping the planner) and returns rows as
|
| 16 |
`ToolOutput(kind="table")` — the Pattern A handoff the
|
| 17 |
`analyze_*` tools consume.
|
| 18 |
-
- `
|
| 19 |
`ToolOutput(kind="documents")`.
|
| 20 |
|
| 21 |
Frozen guarantee (§8.4): **never throws.** Any failure returns
|
|
@@ -47,7 +47,7 @@ DispatcherFactory = Callable[[Catalog], ExecutorDispatcher]
|
|
| 47 |
# adding/renaming a data-access tool can't silently drift the router out of sync
|
| 48 |
# from the registry (R11). Must match the names in `DataAccessToolInvoker.invoke`.
|
| 49 |
DATA_ACCESS_TOOLS: frozenset[str] = frozenset(
|
| 50 |
-
{"list_sources", "describe_source", "
|
| 51 |
)
|
| 52 |
|
| 53 |
|
|
@@ -73,14 +73,14 @@ class DataAccessToolInvoker:
|
|
| 73 |
) -> None:
|
| 74 |
self._user_id = user_id
|
| 75 |
self._reader = catalog_reader
|
| 76 |
-
#
|
| 77 |
# validator is stateless; the dispatcher is built per-call from the
|
| 78 |
# request's catalog (executors are picked by source_type).
|
| 79 |
self._validator = ir_validator or IRValidator()
|
| 80 |
self._dispatcher_factory: DispatcherFactory = (
|
| 81 |
dispatcher_factory or ExecutorDispatcher
|
| 82 |
)
|
| 83 |
-
#
|
| 84 |
# for tests (the real one pulls PGVector + Redis). Lazy-imported on first
|
| 85 |
# use so importing this module stays cheap.
|
| 86 |
self._retriever = document_retriever
|
|
@@ -91,10 +91,10 @@ class DataAccessToolInvoker:
|
|
| 91 |
return await self._list_sources()
|
| 92 |
if tool_name == "describe_source":
|
| 93 |
return await self._describe_source(args)
|
| 94 |
-
if tool_name == "
|
| 95 |
-
return await self.
|
| 96 |
-
if tool_name == "
|
| 97 |
-
return await self.
|
| 98 |
return ToolOutput(
|
| 99 |
tool=tool_name, kind="error", error=f"unknown tool {tool_name!r}"
|
| 100 |
)
|
|
@@ -183,7 +183,7 @@ class DataAccessToolInvoker:
|
|
| 183 |
},
|
| 184 |
)
|
| 185 |
|
| 186 |
-
async def
|
| 187 |
"""Run one validated, single-table QueryIR and return rows as a table.
|
| 188 |
|
| 189 |
This is the spine of the slow path (Pattern A): the `analyze_*` tools
|
|
@@ -196,14 +196,14 @@ class DataAccessToolInvoker:
|
|
| 196 |
raw = args.get("ir")
|
| 197 |
if raw is None:
|
| 198 |
return ToolOutput(
|
| 199 |
-
tool="
|
| 200 |
)
|
| 201 |
|
| 202 |
try:
|
| 203 |
ir = raw if isinstance(raw, QueryIR) else QueryIR.model_validate(raw)
|
| 204 |
except ValidationError as exc:
|
| 205 |
return ToolOutput(
|
| 206 |
-
tool="
|
| 207 |
)
|
| 208 |
|
| 209 |
catalog = await self._reader.read(self._user_id, "structured")
|
|
@@ -212,7 +212,7 @@ class DataAccessToolInvoker:
|
|
| 212 |
self._validator.validate(ir, catalog)
|
| 213 |
except IRValidationError as exc:
|
| 214 |
return ToolOutput(
|
| 215 |
-
tool="
|
| 216 |
kind="error",
|
| 217 |
error=f"IR validation failed: {exc}",
|
| 218 |
)
|
|
@@ -223,7 +223,7 @@ class DataAccessToolInvoker:
|
|
| 223 |
|
| 224 |
if result.error:
|
| 225 |
return ToolOutput(
|
| 226 |
-
tool="
|
| 227 |
)
|
| 228 |
|
| 229 |
# QueryResult.rows is list[dict]; ToolOutput.rows is list[list] ordered
|
|
@@ -235,7 +235,7 @@ class DataAccessToolInvoker:
|
|
| 235 |
[_json_safe(row.get(c)) for c in result.columns] for row in result.rows
|
| 236 |
]
|
| 237 |
return ToolOutput(
|
| 238 |
-
tool="
|
| 239 |
kind="table",
|
| 240 |
columns=result.columns,
|
| 241 |
rows=rows,
|
|
@@ -251,7 +251,7 @@ class DataAccessToolInvoker:
|
|
| 251 |
},
|
| 252 |
)
|
| 253 |
|
| 254 |
-
async def
|
| 255 |
"""Dense-retrieve relevant chunks from the user's unstructured sources.
|
| 256 |
|
| 257 |
Pulls qualitative context (PDF/DOCX/TXT) for a natural-language `query`
|
|
@@ -259,7 +259,7 @@ class DataAccessToolInvoker:
|
|
| 259 |
`source_id` scopes to one source (best-effort metadata filter — the
|
| 260 |
router itself does not yet scope by source, so this prunes the results).
|
| 261 |
|
| 262 |
-
TODO(retrieval scoping): the Planner few-shot has no `
|
| 263 |
example, so `source_id` is rarely emitted today and this post-filter is
|
| 264 |
adequate. If source-scoped retrieval becomes common, push scoping down
|
| 265 |
into RetrievalRouter.retrieve()/DocumentRetriever (WHERE
|
|
@@ -269,7 +269,7 @@ class DataAccessToolInvoker:
|
|
| 269 |
query = args.get("query")
|
| 270 |
if not isinstance(query, str) or not query.strip():
|
| 271 |
return ToolOutput(
|
| 272 |
-
tool="
|
| 273 |
kind="error",
|
| 274 |
error="missing 'query' argument",
|
| 275 |
)
|
|
@@ -300,7 +300,7 @@ class DataAccessToolInvoker:
|
|
| 300 |
for r in results
|
| 301 |
]
|
| 302 |
return ToolOutput(
|
| 303 |
-
tool="
|
| 304 |
kind="documents",
|
| 305 |
value=documents,
|
| 306 |
meta={
|
|
|
|
| 11 |
- `list_sources` — the user's data sources (id, name, type, table count).
|
| 12 |
- `describe_source` — tables/columns of one source (one row per column,
|
| 13 |
metadata only — exposes `pii_flag`, never sample values).
|
| 14 |
+
- `data_retrieve` — runs a pre-built `QueryIR` (validate -> dispatch ->
|
| 15 |
execute, skipping the planner) and returns rows as
|
| 16 |
`ToolOutput(kind="table")` — the Pattern A handoff the
|
| 17 |
`analyze_*` tools consume.
|
| 18 |
+
- `knowledge_retrieve` — dense retrieval over unstructured sources, returns
|
| 19 |
`ToolOutput(kind="documents")`.
|
| 20 |
|
| 21 |
Frozen guarantee (§8.4): **never throws.** Any failure returns
|
|
|
|
| 47 |
# adding/renaming a data-access tool can't silently drift the router out of sync
|
| 48 |
# from the registry (R11). Must match the names in `DataAccessToolInvoker.invoke`.
|
| 49 |
DATA_ACCESS_TOOLS: frozenset[str] = frozenset(
|
| 50 |
+
{"list_sources", "describe_source", "data_retrieve", "knowledge_retrieve"}
|
| 51 |
)
|
| 52 |
|
| 53 |
|
|
|
|
| 73 |
) -> None:
|
| 74 |
self._user_id = user_id
|
| 75 |
self._reader = catalog_reader
|
| 76 |
+
# data_retrieve deps — injectable so tests need no real LLM/DB. The
|
| 77 |
# validator is stateless; the dispatcher is built per-call from the
|
| 78 |
# request's catalog (executors are picked by source_type).
|
| 79 |
self._validator = ir_validator or IRValidator()
|
| 80 |
self._dispatcher_factory: DispatcherFactory = (
|
| 81 |
dispatcher_factory or ExecutorDispatcher
|
| 82 |
)
|
| 83 |
+
# knowledge_retrieve dep — the module singleton by default, injectable
|
| 84 |
# for tests (the real one pulls PGVector + Redis). Lazy-imported on first
|
| 85 |
# use so importing this module stays cheap.
|
| 86 |
self._retriever = document_retriever
|
|
|
|
| 91 |
return await self._list_sources()
|
| 92 |
if tool_name == "describe_source":
|
| 93 |
return await self._describe_source(args)
|
| 94 |
+
if tool_name == "data_retrieve":
|
| 95 |
+
return await self._data_retrieve(args)
|
| 96 |
+
if tool_name == "knowledge_retrieve":
|
| 97 |
+
return await self._knowledge_retrieve(args)
|
| 98 |
return ToolOutput(
|
| 99 |
tool=tool_name, kind="error", error=f"unknown tool {tool_name!r}"
|
| 100 |
)
|
|
|
|
| 183 |
},
|
| 184 |
)
|
| 185 |
|
| 186 |
+
async def _data_retrieve(self, args: dict[str, Any]) -> ToolOutput:
|
| 187 |
"""Run one validated, single-table QueryIR and return rows as a table.
|
| 188 |
|
| 189 |
This is the spine of the slow path (Pattern A): the `analyze_*` tools
|
|
|
|
| 196 |
raw = args.get("ir")
|
| 197 |
if raw is None:
|
| 198 |
return ToolOutput(
|
| 199 |
+
tool="data_retrieve", kind="error", error="missing 'ir' argument"
|
| 200 |
)
|
| 201 |
|
| 202 |
try:
|
| 203 |
ir = raw if isinstance(raw, QueryIR) else QueryIR.model_validate(raw)
|
| 204 |
except ValidationError as exc:
|
| 205 |
return ToolOutput(
|
| 206 |
+
tool="data_retrieve", kind="error", error=f"invalid IR: {exc}"
|
| 207 |
)
|
| 208 |
|
| 209 |
catalog = await self._reader.read(self._user_id, "structured")
|
|
|
|
| 212 |
self._validator.validate(ir, catalog)
|
| 213 |
except IRValidationError as exc:
|
| 214 |
return ToolOutput(
|
| 215 |
+
tool="data_retrieve",
|
| 216 |
kind="error",
|
| 217 |
error=f"IR validation failed: {exc}",
|
| 218 |
)
|
|
|
|
| 223 |
|
| 224 |
if result.error:
|
| 225 |
return ToolOutput(
|
| 226 |
+
tool="data_retrieve", kind="error", error=result.error
|
| 227 |
)
|
| 228 |
|
| 229 |
# QueryResult.rows is list[dict]; ToolOutput.rows is list[list] ordered
|
|
|
|
| 235 |
[_json_safe(row.get(c)) for c in result.columns] for row in result.rows
|
| 236 |
]
|
| 237 |
return ToolOutput(
|
| 238 |
+
tool="data_retrieve",
|
| 239 |
kind="table",
|
| 240 |
columns=result.columns,
|
| 241 |
rows=rows,
|
|
|
|
| 251 |
},
|
| 252 |
)
|
| 253 |
|
| 254 |
+
async def _knowledge_retrieve(self, args: dict[str, Any]) -> ToolOutput:
|
| 255 |
"""Dense-retrieve relevant chunks from the user's unstructured sources.
|
| 256 |
|
| 257 |
Pulls qualitative context (PDF/DOCX/TXT) for a natural-language `query`
|
|
|
|
| 259 |
`source_id` scopes to one source (best-effort metadata filter — the
|
| 260 |
router itself does not yet scope by source, so this prunes the results).
|
| 261 |
|
| 262 |
+
TODO(retrieval scoping): the Planner few-shot has no `knowledge_retrieve`
|
| 263 |
example, so `source_id` is rarely emitted today and this post-filter is
|
| 264 |
adequate. If source-scoped retrieval becomes common, push scoping down
|
| 265 |
into RetrievalRouter.retrieve()/DocumentRetriever (WHERE
|
|
|
|
| 269 |
query = args.get("query")
|
| 270 |
if not isinstance(query, str) or not query.strip():
|
| 271 |
return ToolOutput(
|
| 272 |
+
tool="knowledge_retrieve",
|
| 273 |
kind="error",
|
| 274 |
error="missing 'query' argument",
|
| 275 |
)
|
|
|
|
| 300 |
for r in results
|
| 301 |
]
|
| 302 |
return ToolOutput(
|
| 303 |
+
tool="knowledge_retrieve",
|
| 304 |
kind="documents",
|
| 305 |
value=documents,
|
| 306 |
meta={
|
src/tools/registry.py
CHANGED
|
@@ -8,7 +8,7 @@ reads to choose a tool (KM-625). This replaces the agent team's local stub in
|
|
| 8 |
Conventions (decided with the agent team, KM-465):
|
| 9 |
- **Pattern A** — `analyze_*` tools do NOT self-fetch by `source_id`. Each takes a
|
| 10 |
`data` argument that is a `"${t<id>}"` placeholder pointing at an upstream
|
| 11 |
-
`
|
| 12 |
Column arguments reference the aliases that upstream query produced.
|
| 13 |
- `input_schema` is the lightweight JSON-schema-ish dict the planner validator
|
| 14 |
consumes: `required` (arg names with no default) + `properties` (allowed args).
|
|
@@ -17,7 +17,7 @@ Conventions (decided with the agent team, KM-465):
|
|
| 17 |
- `output_kind` is the `ToolOutput.kind` each tool returns: stats (labelled-metric
|
| 18 |
dict) | table (rows×cols) | series (ordered periods).
|
| 19 |
|
| 20 |
-
The four data-access tools (
|
| 21 |
describe_source) are registered separately once their wrappers land (KM-465 #4);
|
| 22 |
`default_registry()` composes both slices.
|
| 23 |
"""
|
|
|
|
| 8 |
Conventions (decided with the agent team, KM-465):
|
| 9 |
- **Pattern A** — `analyze_*` tools do NOT self-fetch by `source_id`. Each takes a
|
| 10 |
`data` argument that is a `"${t<id>}"` placeholder pointing at an upstream
|
| 11 |
+
`data_retrieve` table output, resolved to a DataFrame at execution time.
|
| 12 |
Column arguments reference the aliases that upstream query produced.
|
| 13 |
- `input_schema` is the lightweight JSON-schema-ish dict the planner validator
|
| 14 |
consumes: `required` (arg names with no default) + `properties` (allowed args).
|
|
|
|
| 17 |
- `output_kind` is the `ToolOutput.kind` each tool returns: stats (labelled-metric
|
| 18 |
dict) | table (rows×cols) | series (ordered periods).
|
| 19 |
|
| 20 |
+
The four data-access tools (data_retrieve / knowledge_retrieve / list_sources /
|
| 21 |
describe_source) are registered separately once their wrappers land (KM-465 #4);
|
| 22 |
`default_registry()` composes both slices.
|
| 23 |
"""
|