sofhiaazzhr Claude Opus 4.8 commited on
Commit
c38c0c2
·
1 Parent(s): 8c1bb63

[KM-642] refactor(tools): rename data-access tools to data/knowledge naming

Browse files

Mechanical rename across the tool layer, the planner stub/validator/prompt/
few-shots, and docstrings: query_structured -> data_retrieve and
retrieve_documents -> knowledge_retrieve (data/knowledge taxonomy from the
2026-06-11 checkpoint). No behavior change; the data_check/knowledge_check
merge lands in a follow-up. Tests green (206 passed); all changed files pass
ruff, data_access.py clean under mypy.

Touches the planner area (prompt, validator, examples) — rename only,
wording preserved. Flagged for Rifqi review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

src/agents/planner/examples.py CHANGED
@@ -2,7 +2,7 @@
2
 
3
  Two illustrative (question -> TaskList) pairs that teach the OUTPUT SHAPE:
4
  stages, dependency edges, ordered tool-call chains, inline QueryIR,
5
- "${t<id>}" placeholders, and the assumed data-flow convention — `query_structured`
6
  pulls rows, then a composite `analyze_*` tool consumes them via a `data` placeholder
7
  referencing the upstream result's column aliases (Pattern A; the tool team may
8
  instead pick self-fetch by `source_id`, in which case these examples are reshaped
@@ -21,7 +21,7 @@ from .schemas import Task, TaskList, ToolCall
21
  # --------------------------------------------------------------------------- #
22
  # Example A — exploratory, no modeling.
23
  # "Which product categories drove last quarter's revenue?"
24
- # Shows: query_structured pulls rows -> analyze_aggregate sums revenue per
25
  # category in one call (no manual per-category queries).
26
  # --------------------------------------------------------------------------- #
27
 
@@ -47,7 +47,7 @@ _EXAMPLE_A = TaskList(
47
  objective="Pull last quarter's order-level category and revenue rows.",
48
  tool_calls=[
49
  ToolCall(
50
- tool="query_structured",
51
  args={
52
  "ir": {
53
  "source_id": "src_sales",
@@ -123,7 +123,7 @@ _EXAMPLE_B = TaskList(
123
  objective="Pull this year's order dates, revenue, and region.",
124
  tool_calls=[
125
  ToolCall(
126
- tool="query_structured",
127
  args={
128
  "ir": {
129
  "source_id": "src_sales",
@@ -187,8 +187,8 @@ _EXAMPLE_B = TaskList(
187
  # Example C — mixed structured + unstructured.
188
  # "Revenue dipped in Q1 — what happened?"
189
  # Shows: a structured branch (query -> analyze_trend) runs alongside an
190
- # INDEPENDENT retrieve_documents branch that pulls qualitative context. Note
191
- # retrieve_documents takes a natural-language `query` (NOT a `${t<id>}` data
192
  # placeholder — it is a source, not a consumer) and can run in parallel; the
193
  # Assembler folds the document context into the explanation.
194
  # --------------------------------------------------------------------------- #
@@ -215,7 +215,7 @@ _EXAMPLE_C = TaskList(
215
  objective="Pull Q1 order dates and revenue.",
216
  tool_calls=[
217
  ToolCall(
218
- tool="query_structured",
219
  args={
220
  "ir": {
221
  "source_id": "src_sales",
@@ -273,7 +273,7 @@ _EXAMPLE_C = TaskList(
273
  objective="Retrieve qualitative context on Q1 operational events behind a dip.",
274
  tool_calls=[
275
  ToolCall(
276
- tool="retrieve_documents",
277
  args={
278
  "query": "operational issues, outages, or notable events in Q1 2026",
279
  "top_k": 5,
@@ -320,7 +320,7 @@ _EXAMPLE_D = TaskList(
320
  objective="Pull order-level region and revenue.",
321
  tool_calls=[
322
  ToolCall(
323
- tool="query_structured",
324
  args={
325
  "ir": {
326
  "source_id": "src_sales",
 
2
 
3
  Two illustrative (question -> TaskList) pairs that teach the OUTPUT SHAPE:
4
  stages, dependency edges, ordered tool-call chains, inline QueryIR,
5
+ "${t<id>}" placeholders, and the assumed data-flow convention — `data_retrieve`
6
  pulls rows, then a composite `analyze_*` tool consumes them via a `data` placeholder
7
  referencing the upstream result's column aliases (Pattern A; the tool team may
8
  instead pick self-fetch by `source_id`, in which case these examples are reshaped
 
21
  # --------------------------------------------------------------------------- #
22
  # Example A — exploratory, no modeling.
23
  # "Which product categories drove last quarter's revenue?"
24
+ # Shows: data_retrieve pulls rows -> analyze_aggregate sums revenue per
25
  # category in one call (no manual per-category queries).
26
  # --------------------------------------------------------------------------- #
27
 
 
47
  objective="Pull last quarter's order-level category and revenue rows.",
48
  tool_calls=[
49
  ToolCall(
50
+ tool="data_retrieve",
51
  args={
52
  "ir": {
53
  "source_id": "src_sales",
 
123
  objective="Pull this year's order dates, revenue, and region.",
124
  tool_calls=[
125
  ToolCall(
126
+ tool="data_retrieve",
127
  args={
128
  "ir": {
129
  "source_id": "src_sales",
 
187
  # Example C — mixed structured + unstructured.
188
  # "Revenue dipped in Q1 — what happened?"
189
  # Shows: a structured branch (query -> analyze_trend) runs alongside an
190
+ # INDEPENDENT knowledge_retrieve branch that pulls qualitative context. Note
191
+ # knowledge_retrieve takes a natural-language `query` (NOT a `${t<id>}` data
192
  # placeholder — it is a source, not a consumer) and can run in parallel; the
193
  # Assembler folds the document context into the explanation.
194
  # --------------------------------------------------------------------------- #
 
215
  objective="Pull Q1 order dates and revenue.",
216
  tool_calls=[
217
  ToolCall(
218
+ tool="data_retrieve",
219
  args={
220
  "ir": {
221
  "source_id": "src_sales",
 
273
  objective="Retrieve qualitative context on Q1 operational events behind a dip.",
274
  tool_calls=[
275
  ToolCall(
276
+ tool="knowledge_retrieve",
277
  args={
278
  "query": "operational issues, outages, or notable events in Q1 2026",
279
  "top_k": 5,
 
320
  objective="Pull order-level region and revenue.",
321
  tool_calls=[
322
  ToolCall(
323
+ tool="data_retrieve",
324
  args={
325
  "ir": {
326
  "source_id": "src_sales",
src/agents/planner/inputs.py CHANGED
@@ -4,9 +4,9 @@
4
  for the planner prompt. It carries every table + column id/type/PII flag + row
5
  counts + low-cardinality top_values, with `sample_values` nulled on PII columns
6
  (INV: no PII sample values into the prompt, see doc §13). It also lists the
7
- available unstructured sources so the planner can plan `retrieve_documents`.
8
 
9
- The planner *validator* still checks inline `query_structured` IRs against the
10
  full `Catalog` via the existing IRValidator — the summary is a prompt input, not
11
  the validation source of truth.
12
 
@@ -124,7 +124,7 @@ class CatalogSummary(BaseModel):
124
  lines.append("")
125
 
126
  if self.unstructured_sources:
127
- lines.append("Unstructured sources (for retrieve_documents):")
128
  for src in self.unstructured_sources:
129
  lines.append(f" - {src.name} — id={src.source_id}")
130
 
 
4
  for the planner prompt. It carries every table + column id/type/PII flag + row
5
  counts + low-cardinality top_values, with `sample_values` nulled on PII columns
6
  (INV: no PII sample values into the prompt, see doc §13). It also lists the
7
+ available unstructured sources so the planner can plan `knowledge_retrieve`.
8
 
9
+ The planner *validator* still checks inline `data_retrieve` IRs against the
10
  full `Catalog` via the existing IRValidator — the summary is a prompt input, not
11
  the validation source of truth.
12
 
 
124
  lines.append("")
125
 
126
  if self.unstructured_sources:
127
+ lines.append("Unstructured sources (for knowledge_retrieve):")
128
  for src in self.unstructured_sources:
129
  lines.append(f" - {src.name} — id={src.source_id}")
130
 
src/agents/planner/registry.py CHANGED
@@ -7,7 +7,7 @@ outside it).
7
  `src/tools/registry.py::analytics_registry()` (KM-628), built on the canonical
8
  `ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
9
  descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
10
- - **Data access (`query_structured` / `retrieve_documents` / `list_sources` /
11
  `describe_source`) — spec BODIES still a local stub.** The tool team owns these too,
12
  but their wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
13
  spec bodies here so the Planner can plan end-to-end — but the NAMES derive from
@@ -16,10 +16,10 @@ outside it).
16
  this slice and swap `default_registry()` for the tool team's full composition.
17
 
18
  **Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
19
- `"${t<id>}"` placeholder pointing at an upstream `query_structured` output (no
20
  self-fetch); resolved to a DataFrame at execution time. `input_schema` is the
21
  lightweight `{required, properties}` dict the planner validator (check #8) reads;
22
- `query_structured.args["ir"]` carries an inline QueryIR validated against the
23
  catalog by the existing IRValidator.
24
 
25
  See AGENT_ARCHITECTURE_CONTEXT_new.md §9.2 / §9.3.
@@ -38,7 +38,7 @@ from .contracts import ToolRegistry, ToolSpec
38
  # --------------------------------------------------------------------------- #
39
  _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
40
  ToolSpec(
41
- name="query_structured",
42
  category="analytics.query",
43
  input_schema={"required": ["ir"], "properties": {"ir": {"type": "object"}}},
44
  output_kind="table",
@@ -52,11 +52,11 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
52
  "(count/sum/avg/min/max/count_distinct). Do NOT use it for richer statistics "
53
  "(median/percentile/mode/stddev/skew → analyze_descriptive), trends "
54
  "(analyze_trend), correlation, segmentation, or share-of-total; and do NOT "
55
- "use it to read documents (use retrieve_documents)."
56
  ),
57
  ),
58
  ToolSpec(
59
- name="retrieve_documents",
60
  category="retrieval.documents",
61
  input_schema={
62
  "required": ["query"],
@@ -71,7 +71,7 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
71
  "Dense-retrieve the most relevant chunks from the user's unstructured "
72
  "sources (PDF/DOCX/TXT) for a natural-language `query`. Use this to pull "
73
  "qualitative context into an analysis. Optionally scope to one `source_id`. "
74
- "Do NOT use it for numbers in tables — that is query_structured's job."
75
  ),
76
  ),
77
  ToolSpec(
@@ -96,7 +96,7 @@ _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
96
  description=(
97
  "Return the tables and columns (names, types, row counts) of one source by "
98
  "`source_id`. Use in data_understanding to confirm the shape of a source "
99
- "before querying it. Do NOT use it to fetch data rows (use query_structured)."
100
  ),
101
  ),
102
  )
 
7
  `src/tools/registry.py::analytics_registry()` (KM-628), built on the canonical
8
  `ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
9
  descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
10
+ - **Data access (`data_retrieve` / `knowledge_retrieve` / `list_sources` /
11
  `describe_source`) — spec BODIES still a local stub.** The tool team owns these too,
12
  but their wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
13
  spec bodies here so the Planner can plan end-to-end — but the NAMES derive from
 
16
  this slice and swap `default_registry()` for the tool team's full composition.
17
 
18
  **Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
19
+ `"${t<id>}"` placeholder pointing at an upstream `data_retrieve` output (no
20
  self-fetch); resolved to a DataFrame at execution time. `input_schema` is the
21
  lightweight `{required, properties}` dict the planner validator (check #8) reads;
22
+ `data_retrieve.args["ir"]` carries an inline QueryIR validated against the
23
  catalog by the existing IRValidator.
24
 
25
  See AGENT_ARCHITECTURE_CONTEXT_new.md §9.2 / §9.3.
 
38
  # --------------------------------------------------------------------------- #
39
  _DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
40
  ToolSpec(
41
+ name="data_retrieve",
42
  category="analytics.query",
43
  input_schema={"required": ["ir"], "properties": {"ir": {"type": "object"}}},
44
  output_kind="table",
 
52
  "(count/sum/avg/min/max/count_distinct). Do NOT use it for richer statistics "
53
  "(median/percentile/mode/stddev/skew → analyze_descriptive), trends "
54
  "(analyze_trend), correlation, segmentation, or share-of-total; and do NOT "
55
+ "use it to read documents (use knowledge_retrieve)."
56
  ),
57
  ),
58
  ToolSpec(
59
+ name="knowledge_retrieve",
60
  category="retrieval.documents",
61
  input_schema={
62
  "required": ["query"],
 
71
  "Dense-retrieve the most relevant chunks from the user's unstructured "
72
  "sources (PDF/DOCX/TXT) for a natural-language `query`. Use this to pull "
73
  "qualitative context into an analysis. Optionally scope to one `source_id`. "
74
+ "Do NOT use it for numbers in tables — that is data_retrieve's job."
75
  ),
76
  ),
77
  ToolSpec(
 
96
  description=(
97
  "Return the tables and columns (names, types, row counts) of one source by "
98
  "`source_id`. Use in data_understanding to confirm the shape of a source "
99
+ "before querying it. Do NOT use it to fetch data rows (use data_retrieve)."
100
  ),
101
  ),
102
  )
src/agents/planner/service.py CHANGED
@@ -9,7 +9,7 @@ static plan.
9
 
10
  The service takes the full `Catalog` (not just a `CatalogSummary`): it derives
11
  the PII-safe `CatalogSummary` for the prompt, but validation needs the full
12
- catalog so the existing `IRValidator` can check inline `query_structured` IRs.
13
 
14
  See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3.
15
  """
 
9
 
10
  The service takes the full `Catalog` (not just a `CatalogSummary`): it derives
11
  the PII-safe `CatalogSummary` for the prompt, but validation needs the full
12
+ catalog so the existing `IRValidator` can check inline `data_retrieve` IRs.
13
 
14
  See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3.
15
  """
src/agents/planner/validator.py CHANGED
@@ -95,8 +95,8 @@ class PlannerValidator:
95
  f"source_id {src!r} (known: {sorted(known_sources)})"
96
  )
97
 
98
- # Check 8b — inline query_structured IR validates against the catalog.
99
- if call.tool == "query_structured":
100
  self._validate_inline_ir(task.id, call.args, catalog)
101
 
102
  # Check 7 — success_criteria is checkable.
@@ -114,20 +114,20 @@ class PlannerValidator:
114
  raw_ir = args.get("ir")
115
  if not isinstance(raw_ir, dict):
116
  raise PlannerValidationError(
117
- f"task {task_id}: query_structured.args.ir must be an inline QueryIR "
118
  f"object, got {type(raw_ir).__name__}"
119
  )
120
  try:
121
  ir = QueryIR.model_validate(raw_ir)
122
  except ValidationError as e:
123
  raise PlannerValidationError(
124
- f"task {task_id}: query_structured.args.ir is not a valid QueryIR: {e}"
125
  ) from e
126
  try:
127
  self._ir_validator.validate(ir, catalog)
128
  except IRValidationError as e:
129
  raise PlannerValidationError(
130
- f"task {task_id}: query_structured IR failed catalog validation: {e}"
131
  ) from e
132
 
133
  @staticmethod
 
95
  f"source_id {src!r} (known: {sorted(known_sources)})"
96
  )
97
 
98
+ # Check 8b — inline data_retrieve IR validates against the catalog.
99
+ if call.tool == "data_retrieve":
100
  self._validate_inline_ir(task.id, call.args, catalog)
101
 
102
  # Check 7 — success_criteria is checkable.
 
114
  raw_ir = args.get("ir")
115
  if not isinstance(raw_ir, dict):
116
  raise PlannerValidationError(
117
+ f"task {task_id}: data_retrieve.args.ir must be an inline QueryIR "
118
  f"object, got {type(raw_ir).__name__}"
119
  )
120
  try:
121
  ir = QueryIR.model_validate(raw_ir)
122
  except ValidationError as e:
123
  raise PlannerValidationError(
124
+ f"task {task_id}: data_retrieve.args.ir is not a valid QueryIR: {e}"
125
  ) from e
126
  try:
127
  self._ir_validator.validate(ir, catalog)
128
  except IRValidationError as e:
129
  raise PlannerValidationError(
130
+ f"task {task_id}: data_retrieve IR failed catalog validation: {e}"
131
  ) from e
132
 
133
  @staticmethod
src/catalog/reader.py CHANGED
@@ -45,7 +45,7 @@ class MemoizingCatalogReader(CatalogReader):
45
 
46
  One per request. The same per-user catalog is otherwise fetched from the
47
  catalog DB 4-5x during a single slow-path run (planner load, then
48
- describe_source's structured+unstructured reads, then query_structured's
49
  structured read). Wrapping the base reader collapses those to one round-trip
50
  per distinct source_hint and pins a single consistent snapshot for the whole
51
  request (plan-time and execution-time catalogs can no longer diverge).
 
45
 
46
  One per request. The same per-user catalog is otherwise fetched from the
47
  catalog DB 4-5x during a single slow-path run (planner load, then
48
+ describe_source's structured+unstructured reads, then data_retrieve's
49
  structured read). Wrapping the base reader collapses those to one round-trip
50
  per distinct source_hint and pins a single consistent snapshot for the whole
51
  request (plan-time and execution-time catalogs can no longer diverge).
src/config/prompts/planner.md CHANGED
@@ -10,7 +10,7 @@ only a `TaskList` object that conforms to the provided schema.
10
 
11
  1. **Emit intent, never code.** Never write SQL, pandas, or any code. The only
12
  query you express is an inline `QueryIR` (a JSON intent object) inside a
13
- `query_structured` tool call's `args.ir`.
14
  2. **The plan is static.** There is no replanning and no execution feedback. Plan
15
  the whole analysis up front; assume each task runs once, in dependency order.
16
  3. **Use only tools from the "Available tools" list.** Never invent a tool name.
@@ -31,16 +31,16 @@ only a `TaskList` object that conforms to the provided schema.
31
  - **Wire data between tasks with placeholders.** When a task needs an upstream
32
  task's output as an argument, use the string `"${t<id>}"` (e.g. `"${t2}"`) as
33
  the argument value. Set `depends_on` accordingly.
34
- - **Data access vs analytics tools.** `query_structured` is the data-access entry
35
  point: use it to select, filter, and pull rows (and simple built-in
36
  count/sum/avg/min/max/count_distinct the IR can express). For anything richer —
37
  descriptive statistics (median/percentile/mode/std/skew), time trends, group
38
  comparisons, share-of-total, correlation, segmentation, or data-quality
39
- profiling — run `query_structured` to fetch the rows, then pass its output to
40
  the matching composite `analyze_*` tool via a `"${t<id>}"` `data` argument
41
  (referencing the upstream result's column aliases).
42
  - **Mixing structured + unstructured.** If qualitative context helps, add a
43
- `retrieve_documents` task against an unstructured source listed in the catalog.
44
  - **CRISP-DM stages.** Tag each task with the stage it serves:
45
  `data_understanding`, `data_preparation`, or `evaluation`. (Never `modeling`.)
46
  - **success_criteria is a reporting signal**, not a control trigger. State, in
 
10
 
11
  1. **Emit intent, never code.** Never write SQL, pandas, or any code. The only
12
  query you express is an inline `QueryIR` (a JSON intent object) inside a
13
+ `data_retrieve` tool call's `args.ir`.
14
  2. **The plan is static.** There is no replanning and no execution feedback. Plan
15
  the whole analysis up front; assume each task runs once, in dependency order.
16
  3. **Use only tools from the "Available tools" list.** Never invent a tool name.
 
31
  - **Wire data between tasks with placeholders.** When a task needs an upstream
32
  task's output as an argument, use the string `"${t<id>}"` (e.g. `"${t2}"`) as
33
  the argument value. Set `depends_on` accordingly.
34
+ - **Data access vs analytics tools.** `data_retrieve` is the data-access entry
35
  point: use it to select, filter, and pull rows (and simple built-in
36
  count/sum/avg/min/max/count_distinct the IR can express). For anything richer —
37
  descriptive statistics (median/percentile/mode/std/skew), time trends, group
38
  comparisons, share-of-total, correlation, segmentation, or data-quality
39
+ profiling — run `data_retrieve` to fetch the rows, then pass its output to
40
  the matching composite `analyze_*` tool via a `"${t<id>}"` `data` argument
41
  (referencing the upstream result's column aliases).
42
  - **Mixing structured + unstructured.** If qualitative context helps, add a
43
+ `knowledge_retrieve` task against an unstructured source listed in the catalog.
44
  - **CRISP-DM stages.** Tag each task with the stage it serves:
45
  `data_understanding`, `data_preparation`, or `evaluation`. (Never `modeling`.)
46
  - **success_criteria is a reporting signal**, not a control trigger. State, in
src/query/executor/db.py CHANGED
@@ -220,7 +220,7 @@ class DbExecutor(BaseExecutor):
220
  """Best-effort: warm pooled engines for the catalog's schema sources.
221
 
222
  Called at slow-path entry so the TCP+TLS+auth handshake overlaps the ~4s
223
- Planner LLM call — by the time `query_structured` runs, the connection is
224
  already established. Warming is an optimization, never a requirement, so
225
  this never raises and per-source failures are swallowed.
226
  """
 
220
  """Best-effort: warm pooled engines for the catalog's schema sources.
221
 
222
  Called at slow-path entry so the TCP+TLS+auth handshake overlaps the ~4s
223
+ Planner LLM call — by the time `data_retrieve` runs, the connection is
224
  already established. Warming is an optimization, never a requirement, so
225
  this never raises and per-source failures are swallowed.
226
  """
src/tools/analytics/aggregation.py CHANGED
@@ -3,7 +3,7 @@
3
  An analytical "family" tool: in ONE call it groups rows by one or more keys
4
  and computes aggregates (sum, mean, count, min, max, median, nunique) per
5
  group. This is the deterministic compute twin of the Planner's
6
- `query_structured` step — the wrapper layer later maps a QueryIR onto this.
7
 
8
  STATUS: compute layer only — the function takes an already-materialized
9
  DataFrame. The wrapper layer (fetching data from the catalog via source_id,
 
3
  An analytical "family" tool: in ONE call it groups rows by one or more keys
4
  and computes aggregates (sum, mean, count, min, max, median, nunique) per
5
  group. This is the deterministic compute twin of the Planner's
6
+ `data_retrieve` step — the wrapper layer later maps a QueryIR onto this.
7
 
8
  STATUS: compute layer only — the function takes an already-materialized
9
  DataFrame. The wrapper layer (fetching data from the catalog via source_id,
src/tools/data_access.py CHANGED
@@ -11,11 +11,11 @@ Tools implemented here:
11
  - `list_sources` — the user's data sources (id, name, type, table count).
12
  - `describe_source` — tables/columns of one source (one row per column,
13
  metadata only — exposes `pii_flag`, never sample values).
14
- - `query_structured` — runs a pre-built `QueryIR` (validate -> dispatch ->
15
  execute, skipping the planner) and returns rows as
16
  `ToolOutput(kind="table")` — the Pattern A handoff the
17
  `analyze_*` tools consume.
18
- - `retrieve_documents` — dense retrieval over unstructured sources, returns
19
  `ToolOutput(kind="documents")`.
20
 
21
  Frozen guarantee (§8.4): **never throws.** Any failure returns
@@ -47,7 +47,7 @@ DispatcherFactory = Callable[[Catalog], ExecutorDispatcher]
47
  # adding/renaming a data-access tool can't silently drift the router out of sync
48
  # from the registry (R11). Must match the names in `DataAccessToolInvoker.invoke`.
49
  DATA_ACCESS_TOOLS: frozenset[str] = frozenset(
50
- {"list_sources", "describe_source", "query_structured", "retrieve_documents"}
51
  )
52
 
53
 
@@ -73,14 +73,14 @@ class DataAccessToolInvoker:
73
  ) -> None:
74
  self._user_id = user_id
75
  self._reader = catalog_reader
76
- # query_structured deps — injectable so tests need no real LLM/DB. The
77
  # validator is stateless; the dispatcher is built per-call from the
78
  # request's catalog (executors are picked by source_type).
79
  self._validator = ir_validator or IRValidator()
80
  self._dispatcher_factory: DispatcherFactory = (
81
  dispatcher_factory or ExecutorDispatcher
82
  )
83
- # retrieve_documents dep — the module singleton by default, injectable
84
  # for tests (the real one pulls PGVector + Redis). Lazy-imported on first
85
  # use so importing this module stays cheap.
86
  self._retriever = document_retriever
@@ -91,10 +91,10 @@ class DataAccessToolInvoker:
91
  return await self._list_sources()
92
  if tool_name == "describe_source":
93
  return await self._describe_source(args)
94
- if tool_name == "query_structured":
95
- return await self._query_structured(args)
96
- if tool_name == "retrieve_documents":
97
- return await self._retrieve_documents(args)
98
  return ToolOutput(
99
  tool=tool_name, kind="error", error=f"unknown tool {tool_name!r}"
100
  )
@@ -183,7 +183,7 @@ class DataAccessToolInvoker:
183
  },
184
  )
185
 
186
- async def _query_structured(self, args: dict[str, Any]) -> ToolOutput:
187
  """Run one validated, single-table QueryIR and return rows as a table.
188
 
189
  This is the spine of the slow path (Pattern A): the `analyze_*` tools
@@ -196,14 +196,14 @@ class DataAccessToolInvoker:
196
  raw = args.get("ir")
197
  if raw is None:
198
  return ToolOutput(
199
- tool="query_structured", kind="error", error="missing 'ir' argument"
200
  )
201
 
202
  try:
203
  ir = raw if isinstance(raw, QueryIR) else QueryIR.model_validate(raw)
204
  except ValidationError as exc:
205
  return ToolOutput(
206
- tool="query_structured", kind="error", error=f"invalid IR: {exc}"
207
  )
208
 
209
  catalog = await self._reader.read(self._user_id, "structured")
@@ -212,7 +212,7 @@ class DataAccessToolInvoker:
212
  self._validator.validate(ir, catalog)
213
  except IRValidationError as exc:
214
  return ToolOutput(
215
- tool="query_structured",
216
  kind="error",
217
  error=f"IR validation failed: {exc}",
218
  )
@@ -223,7 +223,7 @@ class DataAccessToolInvoker:
223
 
224
  if result.error:
225
  return ToolOutput(
226
- tool="query_structured", kind="error", error=result.error
227
  )
228
 
229
  # QueryResult.rows is list[dict]; ToolOutput.rows is list[list] ordered
@@ -235,7 +235,7 @@ class DataAccessToolInvoker:
235
  [_json_safe(row.get(c)) for c in result.columns] for row in result.rows
236
  ]
237
  return ToolOutput(
238
- tool="query_structured",
239
  kind="table",
240
  columns=result.columns,
241
  rows=rows,
@@ -251,7 +251,7 @@ class DataAccessToolInvoker:
251
  },
252
  )
253
 
254
- async def _retrieve_documents(self, args: dict[str, Any]) -> ToolOutput:
255
  """Dense-retrieve relevant chunks from the user's unstructured sources.
256
 
257
  Pulls qualitative context (PDF/DOCX/TXT) for a natural-language `query`
@@ -259,7 +259,7 @@ class DataAccessToolInvoker:
259
  `source_id` scopes to one source (best-effort metadata filter — the
260
  router itself does not yet scope by source, so this prunes the results).
261
 
262
- TODO(retrieval scoping): the Planner few-shot has no `retrieve_documents`
263
  example, so `source_id` is rarely emitted today and this post-filter is
264
  adequate. If source-scoped retrieval becomes common, push scoping down
265
  into RetrievalRouter.retrieve()/DocumentRetriever (WHERE
@@ -269,7 +269,7 @@ class DataAccessToolInvoker:
269
  query = args.get("query")
270
  if not isinstance(query, str) or not query.strip():
271
  return ToolOutput(
272
- tool="retrieve_documents",
273
  kind="error",
274
  error="missing 'query' argument",
275
  )
@@ -300,7 +300,7 @@ class DataAccessToolInvoker:
300
  for r in results
301
  ]
302
  return ToolOutput(
303
- tool="retrieve_documents",
304
  kind="documents",
305
  value=documents,
306
  meta={
 
11
  - `list_sources` — the user's data sources (id, name, type, table count).
12
  - `describe_source` — tables/columns of one source (one row per column,
13
  metadata only — exposes `pii_flag`, never sample values).
14
+ - `data_retrieve` — runs a pre-built `QueryIR` (validate -> dispatch ->
15
  execute, skipping the planner) and returns rows as
16
  `ToolOutput(kind="table")` — the Pattern A handoff the
17
  `analyze_*` tools consume.
18
+ - `knowledge_retrieve` — dense retrieval over unstructured sources, returns
19
  `ToolOutput(kind="documents")`.
20
 
21
  Frozen guarantee (§8.4): **never throws.** Any failure returns
 
47
  # adding/renaming a data-access tool can't silently drift the router out of sync
48
  # from the registry (R11). Must match the names in `DataAccessToolInvoker.invoke`.
49
  DATA_ACCESS_TOOLS: frozenset[str] = frozenset(
50
+ {"list_sources", "describe_source", "data_retrieve", "knowledge_retrieve"}
51
  )
52
 
53
 
 
73
  ) -> None:
74
  self._user_id = user_id
75
  self._reader = catalog_reader
76
+ # data_retrieve deps — injectable so tests need no real LLM/DB. The
77
  # validator is stateless; the dispatcher is built per-call from the
78
  # request's catalog (executors are picked by source_type).
79
  self._validator = ir_validator or IRValidator()
80
  self._dispatcher_factory: DispatcherFactory = (
81
  dispatcher_factory or ExecutorDispatcher
82
  )
83
+ # knowledge_retrieve dep — the module singleton by default, injectable
84
  # for tests (the real one pulls PGVector + Redis). Lazy-imported on first
85
  # use so importing this module stays cheap.
86
  self._retriever = document_retriever
 
91
  return await self._list_sources()
92
  if tool_name == "describe_source":
93
  return await self._describe_source(args)
94
+ if tool_name == "data_retrieve":
95
+ return await self._data_retrieve(args)
96
+ if tool_name == "knowledge_retrieve":
97
+ return await self._knowledge_retrieve(args)
98
  return ToolOutput(
99
  tool=tool_name, kind="error", error=f"unknown tool {tool_name!r}"
100
  )
 
183
  },
184
  )
185
 
186
+ async def _data_retrieve(self, args: dict[str, Any]) -> ToolOutput:
187
  """Run one validated, single-table QueryIR and return rows as a table.
188
 
189
  This is the spine of the slow path (Pattern A): the `analyze_*` tools
 
196
  raw = args.get("ir")
197
  if raw is None:
198
  return ToolOutput(
199
+ tool="data_retrieve", kind="error", error="missing 'ir' argument"
200
  )
201
 
202
  try:
203
  ir = raw if isinstance(raw, QueryIR) else QueryIR.model_validate(raw)
204
  except ValidationError as exc:
205
  return ToolOutput(
206
+ tool="data_retrieve", kind="error", error=f"invalid IR: {exc}"
207
  )
208
 
209
  catalog = await self._reader.read(self._user_id, "structured")
 
212
  self._validator.validate(ir, catalog)
213
  except IRValidationError as exc:
214
  return ToolOutput(
215
+ tool="data_retrieve",
216
  kind="error",
217
  error=f"IR validation failed: {exc}",
218
  )
 
223
 
224
  if result.error:
225
  return ToolOutput(
226
+ tool="data_retrieve", kind="error", error=result.error
227
  )
228
 
229
  # QueryResult.rows is list[dict]; ToolOutput.rows is list[list] ordered
 
235
  [_json_safe(row.get(c)) for c in result.columns] for row in result.rows
236
  ]
237
  return ToolOutput(
238
+ tool="data_retrieve",
239
  kind="table",
240
  columns=result.columns,
241
  rows=rows,
 
251
  },
252
  )
253
 
254
+ async def _knowledge_retrieve(self, args: dict[str, Any]) -> ToolOutput:
255
  """Dense-retrieve relevant chunks from the user's unstructured sources.
256
 
257
  Pulls qualitative context (PDF/DOCX/TXT) for a natural-language `query`
 
259
  `source_id` scopes to one source (best-effort metadata filter — the
260
  router itself does not yet scope by source, so this prunes the results).
261
 
262
+ TODO(retrieval scoping): the Planner few-shot has no `knowledge_retrieve`
263
  example, so `source_id` is rarely emitted today and this post-filter is
264
  adequate. If source-scoped retrieval becomes common, push scoping down
265
  into RetrievalRouter.retrieve()/DocumentRetriever (WHERE
 
269
  query = args.get("query")
270
  if not isinstance(query, str) or not query.strip():
271
  return ToolOutput(
272
+ tool="knowledge_retrieve",
273
  kind="error",
274
  error="missing 'query' argument",
275
  )
 
300
  for r in results
301
  ]
302
  return ToolOutput(
303
+ tool="knowledge_retrieve",
304
  kind="documents",
305
  value=documents,
306
  meta={
src/tools/registry.py CHANGED
@@ -8,7 +8,7 @@ reads to choose a tool (KM-625). This replaces the agent team's local stub in
8
  Conventions (decided with the agent team, KM-465):
9
  - **Pattern A** — `analyze_*` tools do NOT self-fetch by `source_id`. Each takes a
10
  `data` argument that is a `"${t<id>}"` placeholder pointing at an upstream
11
- `query_structured` table output, resolved to a DataFrame at execution time.
12
  Column arguments reference the aliases that upstream query produced.
13
  - `input_schema` is the lightweight JSON-schema-ish dict the planner validator
14
  consumes: `required` (arg names with no default) + `properties` (allowed args).
@@ -17,7 +17,7 @@ Conventions (decided with the agent team, KM-465):
17
  - `output_kind` is the `ToolOutput.kind` each tool returns: stats (labelled-metric
18
  dict) | table (rows×cols) | series (ordered periods).
19
 
20
- The four data-access tools (query_structured / retrieve_documents / list_sources /
21
  describe_source) are registered separately once their wrappers land (KM-465 #4);
22
  `default_registry()` composes both slices.
23
  """
 
8
  Conventions (decided with the agent team, KM-465):
9
  - **Pattern A** — `analyze_*` tools do NOT self-fetch by `source_id`. Each takes a
10
  `data` argument that is a `"${t<id>}"` placeholder pointing at an upstream
11
+ `data_retrieve` table output, resolved to a DataFrame at execution time.
12
  Column arguments reference the aliases that upstream query produced.
13
  - `input_schema` is the lightweight JSON-schema-ish dict the planner validator
14
  consumes: `required` (arg names with no default) + `properties` (allowed args).
 
17
  - `output_kind` is the `ToolOutput.kind` each tool returns: stats (labelled-metric
18
  dict) | table (rows×cols) | series (ordered periods).
19
 
20
+ The four data-access tools (data_retrieve / knowledge_retrieve / list_sources /
21
  describe_source) are registered separately once their wrappers land (KM-465 #4);
22
  `default_registry()` composes both slices.
23
  """