[NOTICKET][AI] refactor(planner): R11 - single-source placeholder regex + name-checked data-access specs
Browse filesComments are not a sync mechanism. Remove the two duplicated definitions on the
agent side.
- PLACEHOLDER_RE: defined once in planner/schemas.py (it's part of the ToolCall
${t<id>} placeholder convention); planner/validator.py and slow_path/task_runner.py
now import it instead of each compiling their own copy "kept in sync" by comment.
- planner/registry.py still holds local data-access spec BODIES (stub pending
KM-465 #4) but derives/validates their NAMES against the canonical
src.tools.data_access.DATA_ACCESS_TOOLS: _data_access_slice() raises at
default_registry() if a tool is added/renamed/removed upstream, instead of
drifting silently. Registry output unchanged (same 12 tools, same order).
Tests: 51 passed (planner + slow_path); ruff clean; registry builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PROGRESS.md +1 -1
- src/agents/planner/registry.py +34 -10
- src/agents/planner/schemas.py +7 -0
- src/agents/planner/validator.py +3 -6
- src/agents/slow_path/task_runner.py +2 -6
|
@@ -50,7 +50,7 @@ Verified against code before logging. Severity: **critical** / important / nice-
|
|
| 50 |
| R8 | **Catalog upsert race** — per-user advisory lock around read-merge-upsert (`store.py`); concurrent uploads can drop a source. | important | DB | `[ ]` |
|
| 51 |
| R9 | **`extra="ignore"`** in `settings.py:15` (currently `allow` → typo'd env vars silently swallowed); require Azure keys in prod. | nice-to-have | B | `[ ]` |
|
| 52 |
| R10 | **Read-only enforcement is session-state, not a server role.** `REPO_CONTEXT.md` counts "read-only DB credentials" as a defense layer but nothing requests/verifies a read-only role. Either request read-only creds at registration (verify via `SELECT current_setting(...)`) or drop the claim. | important | DB | `[ ]` |
|
| 53 |
-
| R11 | **De-duplicate** `_PLACEHOLDER_RE` (`task_runner.py:31` vs validator) and `_DATA_ACCESS_TOOLS` (invoker vs planner registry) — import one from the other; comments aren't a sync mechanism. **TAB slice done (90e80f9):** canonical `DATA_ACCESS_TOOLS` now lives once in `tools/data_access.py`; `invoker.py` imports it (was a duplicated frozenset synced by comment). **Agent
|
| 54 |
| R12 | **Doc/process hygiene** — some code docstrings cite internal design specs that are not committed to the repo (design docs are kept out of version control), so the references dangle for anyone but the author; `CLAUDE.md` lists deleted modules (enricher, `pipeline/orchestrator.py`); `main` is 38 commits behind on a dead architecture. | nice-to-have | agent | `[ ]` |
|
| 55 |
| R13 | **Pre-existing test failure** (found during R2, NOT caused by it): `tests/query/planner/test_prompt.py::test_render_catalog_with_sources` fails — `query/planner/prompt.py::render_catalog` now renders stable IDs (`src_test_db`) the test asserts are absent. Old query-planner path; confirmed failing on a clean tree. | nice-to-have | DB | `[ ]` |
|
| 56 |
| T1 | **`input_schema` is presence-only, not type-checked** — `ToolSpec.input_schema` comment said "validates ToolCall.args", but `TaskRunner._validate_args` only enforces `required` presence; the `properties` types are documentation, never validated at runtime. Clarified the contract in `tools/contracts.py` so nobody assumes type-safety (a wrong-typed arg passes validation, surfaces only inside the compute fn). Doc-only, no behavior change (90e80f9). | nice-to-have | TAB | `[x]` |
|
|
|
|
| 50 |
| R8 | **Catalog upsert race** — per-user advisory lock around read-merge-upsert (`store.py`); concurrent uploads can drop a source. | important | DB | `[ ]` |
|
| 51 |
| R9 | **`extra="ignore"`** in `settings.py:15` (currently `allow` → typo'd env vars silently swallowed); require Azure keys in prod. | nice-to-have | B | `[ ]` |
|
| 52 |
| R10 | **Read-only enforcement is session-state, not a server role.** `REPO_CONTEXT.md` counts "read-only DB credentials" as a defense layer but nothing requests/verifies a read-only role. Either request read-only creds at registration (verify via `SELECT current_setting(...)`) or drop the claim. | important | DB | `[ ]` |
|
| 53 |
+
| R11 | **De-duplicate** `_PLACEHOLDER_RE` (`task_runner.py:31` vs validator) and `_DATA_ACCESS_TOOLS` (invoker vs planner registry) — import one from the other; comments aren't a sync mechanism. **TAB slice done (90e80f9):** canonical `DATA_ACCESS_TOOLS` now lives once in `tools/data_access.py`; `invoker.py` imports it (was a duplicated frozenset synced by comment). **Agent slice done (2026-06-10):** `PLACEHOLDER_RE` single-sourced in `planner/schemas.py` (part of the ToolCall placeholder convention); validator + task_runner import it. `planner/registry.py` keeps local spec *bodies* (stub pending KM-465 #4) but name-checks them against `DATA_ACCESS_TOOLS` in `_data_access_slice()` — upstream rename/add now raises at `default_registry()` instead of drifting silently. Registry output unchanged (same 12 tools, same order). | nice-to-have | agent/tool | `[x]` |
|
| 54 |
| R12 | **Doc/process hygiene** — some code docstrings cite internal design specs that are not committed to the repo (design docs are kept out of version control), so the references dangle for anyone but the author; `CLAUDE.md` lists deleted modules (enricher, `pipeline/orchestrator.py`); `main` is 38 commits behind on a dead architecture. | nice-to-have | agent | `[ ]` |
|
| 55 |
| R13 | **Pre-existing test failure** (found during R2, NOT caused by it): `tests/query/planner/test_prompt.py::test_render_catalog_with_sources` fails — `query/planner/prompt.py::render_catalog` now renders stable IDs (`src_test_db`) the test asserts are absent. Old query-planner path; confirmed failing on a clean tree. | nice-to-have | DB | `[ ]` |
|
| 56 |
| T1 | **`input_schema` is presence-only, not type-checked** — `ToolSpec.input_schema` comment said "validates ToolCall.args", but `TaskRunner._validate_args` only enforces `required` presence; the `properties` types are documentation, never validated at runtime. Clarified the contract in `tools/contracts.py` so nobody assumes type-safety (a wrong-typed arg passes validation, surfaces only inside the compute fn). Doc-only, no behavior change (90e80f9). | nice-to-have | TAB | `[x]` |
|
|
@@ -8,10 +8,12 @@ outside it).
|
|
| 8 |
`ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
|
| 9 |
descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
|
| 10 |
- **Data access (`query_structured` / `retrieve_documents` / `list_sources` /
|
| 11 |
-
`describe_source`) —
|
| 12 |
-
wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
|
| 13 |
-
here so the Planner can plan end-to-end
|
| 14 |
-
|
|
|
|
|
|
|
| 15 |
|
| 16 |
**Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
|
| 17 |
`"${t<id>}"` placeholder pointing at an upstream `query_structured` output (no
|
|
@@ -25,14 +27,16 @@ See AGENT_ARCHITECTURE_CONTEXT_new.md §9.2 / §9.3.
|
|
| 25 |
|
| 26 |
from __future__ import annotations
|
| 27 |
|
|
|
|
| 28 |
from src.tools.registry import analytics_registry
|
| 29 |
|
| 30 |
from .contracts import ToolRegistry, ToolSpec
|
| 31 |
|
| 32 |
# --------------------------------------------------------------------------- #
|
| 33 |
-
# Data-access slice — LOCAL STUB pending the tool team's real
|
|
|
|
| 34 |
# --------------------------------------------------------------------------- #
|
| 35 |
-
|
| 36 |
ToolSpec(
|
| 37 |
name="query_structured",
|
| 38 |
category="analytics.query",
|
|
@@ -95,14 +99,34 @@ _DATA_ACCESS_TOOLS: list[ToolSpec] = [
|
|
| 95 |
"before querying it. Do NOT use it to fetch data rows (use query_structured)."
|
| 96 |
),
|
| 97 |
),
|
| 98 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
|
| 100 |
|
| 101 |
def default_registry() -> ToolRegistry:
|
| 102 |
"""The v1 registry: stub data-access slice + the real analytics slice.
|
| 103 |
|
| 104 |
The analytics tools come live from `src.tools.registry` (the tool team's real
|
| 105 |
-
registry);
|
| 106 |
-
per call.
|
| 107 |
"""
|
| 108 |
-
return ToolRegistry(tools=[*
|
|
|
|
| 8 |
`ToolSpec` (`src/tools/contracts.py`, KM-465/KM-627) and the prompt-style tool
|
| 9 |
descriptions (KM-625). No longer a stub on our side — it tracks the real registry.
|
| 10 |
- **Data access (`query_structured` / `retrieve_documents` / `list_sources` /
|
| 11 |
+
`describe_source`) — spec BODIES still a local stub.** The tool team owns these too,
|
| 12 |
+
but their wrappers + `ToolSpec`s haven't landed yet (KM-465 #4). We keep best-guess
|
| 13 |
+
spec bodies here so the Planner can plan end-to-end — but the NAMES derive from
|
| 14 |
+
`src.tools.data_access.DATA_ACCESS_TOOLS` (R11), so a tool rename/addition upstream
|
| 15 |
+
fails loudly here instead of drifting silently. When the real specs ship, delete
|
| 16 |
+
this slice and swap `default_registry()` for the tool team's full composition.
|
| 17 |
|
| 18 |
**Confirmed conventions (KM-465):** Pattern A — `analyze_*` tools take a `data`
|
| 19 |
`"${t<id>}"` placeholder pointing at an upstream `query_structured` output (no
|
|
|
|
| 27 |
|
| 28 |
from __future__ import annotations
|
| 29 |
|
| 30 |
+
from src.tools.data_access import DATA_ACCESS_TOOLS
|
| 31 |
from src.tools.registry import analytics_registry
|
| 32 |
|
| 33 |
from .contracts import ToolRegistry, ToolSpec
|
| 34 |
|
| 35 |
# --------------------------------------------------------------------------- #
|
| 36 |
+
# Data-access slice — spec bodies are a LOCAL STUB pending the tool team's real
|
| 37 |
+
# specs (KM-465 #4); the canonical NAME SET is `DATA_ACCESS_TOOLS` (tool-owned).
|
| 38 |
# --------------------------------------------------------------------------- #
|
| 39 |
+
_DATA_ACCESS_SPEC_BODIES: tuple[ToolSpec, ...] = (
|
| 40 |
ToolSpec(
|
| 41 |
name="query_structured",
|
| 42 |
category="analytics.query",
|
|
|
|
| 99 |
"before querying it. Do NOT use it to fetch data rows (use query_structured)."
|
| 100 |
),
|
| 101 |
),
|
| 102 |
+
)
|
| 103 |
+
|
| 104 |
+
_DATA_ACCESS_SPECS: dict[str, ToolSpec] = {s.name: s for s in _DATA_ACCESS_SPEC_BODIES}
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def _data_access_slice() -> list[ToolSpec]:
|
| 108 |
+
"""Data-access specs in body order, with names checked against the tool layer.
|
| 109 |
+
|
| 110 |
+
`DATA_ACCESS_TOOLS` (src.tools.data_access) is the canonical name set; the
|
| 111 |
+
spec bodies above are still our local stub. Any mismatch (a tool added,
|
| 112 |
+
renamed, or removed upstream) raises here instead of drifting silently.
|
| 113 |
+
"""
|
| 114 |
+
if set(_DATA_ACCESS_SPECS) != DATA_ACCESS_TOOLS:
|
| 115 |
+
missing = sorted(DATA_ACCESS_TOOLS - _DATA_ACCESS_SPECS.keys())
|
| 116 |
+
stale = sorted(_DATA_ACCESS_SPECS.keys() - DATA_ACCESS_TOOLS)
|
| 117 |
+
raise RuntimeError(
|
| 118 |
+
"planner data-access specs out of sync with "
|
| 119 |
+
f"src.tools.data_access.DATA_ACCESS_TOOLS: missing spec for {missing}, "
|
| 120 |
+
f"stale spec for {stale}"
|
| 121 |
+
)
|
| 122 |
+
return list(_DATA_ACCESS_SPECS.values())
|
| 123 |
|
| 124 |
|
| 125 |
def default_registry() -> ToolRegistry:
|
| 126 |
"""The v1 registry: stub data-access slice + the real analytics slice.
|
| 127 |
|
| 128 |
The analytics tools come live from `src.tools.registry` (the tool team's real
|
| 129 |
+
registry); the data-access spec bodies are still a local stub, name-checked
|
| 130 |
+
against `DATA_ACCESS_TOOLS`. A fresh instance per call.
|
| 131 |
"""
|
| 132 |
+
return ToolRegistry(tools=[*_data_access_slice(), *analytics_registry().tools])
|
|
@@ -10,10 +10,17 @@ See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3.
|
|
| 10 |
|
| 11 |
from __future__ import annotations
|
| 12 |
|
|
|
|
| 13 |
from typing import Any, Literal
|
| 14 |
|
| 15 |
from pydantic import BaseModel, Field
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
CrispStage = Literal[
|
| 18 |
"data_understanding",
|
| 19 |
"data_preparation",
|
|
|
|
| 10 |
|
| 11 |
from __future__ import annotations
|
| 12 |
|
| 13 |
+
import re
|
| 14 |
from typing import Any, Literal
|
| 15 |
|
| 16 |
from pydantic import BaseModel, Field
|
| 17 |
|
| 18 |
+
# The "${t<id>}" placeholder convention (Pattern A): a ToolCall arg whose value
|
| 19 |
+
# matches this pattern refers to an upstream task's output, resolved by the
|
| 20 |
+
# TaskRunner at execution time. Single definition — the planner validator and
|
| 21 |
+
# the TaskRunner both import it (R11).
|
| 22 |
+
PLACEHOLDER_RE = re.compile(r"\$\{(t[^}]+)\}")
|
| 23 |
+
|
| 24 |
CrispStage = Literal[
|
| 25 |
"data_understanding",
|
| 26 |
"data_preparation",
|
|
@@ -11,8 +11,6 @@ rejects structurally-invalid plans (duplicate ids, dangling edges, cycles).
|
|
| 11 |
|
| 12 |
from __future__ import annotations
|
| 13 |
|
| 14 |
-
import re
|
| 15 |
-
|
| 16 |
from pydantic import ValidationError
|
| 17 |
|
| 18 |
from ...catalog.models import Catalog
|
|
@@ -21,11 +19,10 @@ from ...query.ir.validator import IRValidationError, IRValidator
|
|
| 21 |
from .contracts import ToolRegistry
|
| 22 |
from .errors import PlannerValidationError
|
| 23 |
from .inputs import Constraints
|
| 24 |
-
from .schemas import TaskList
|
| 25 |
|
| 26 |
# Heuristic: a checkable success_criteria mentions a measurable signal.
|
| 27 |
_CHECKABLE_TOKENS = ("rate", "count", "match", "produced", "above", "below", "equal")
|
| 28 |
-
_PLACEHOLDER_RE = re.compile(r"\$\{(t[^}]+)\}")
|
| 29 |
|
| 30 |
# DFS colors for cycle detection.
|
| 31 |
_WHITE, _GREY, _BLACK = 0, 1, 2
|
|
@@ -169,7 +166,7 @@ class PlannerValidator:
|
|
| 169 |
|
| 170 |
|
| 171 |
def _is_placeholder(value: str) -> bool:
|
| 172 |
-
return bool(
|
| 173 |
|
| 174 |
|
| 175 |
def _placeholder_refs(task) -> set[str]:
|
|
@@ -177,7 +174,7 @@ def _placeholder_refs(task) -> set[str]:
|
|
| 177 |
for call in task.tool_calls:
|
| 178 |
for value in call.args.values():
|
| 179 |
if isinstance(value, str):
|
| 180 |
-
refs.update(
|
| 181 |
return refs
|
| 182 |
|
| 183 |
|
|
|
|
| 11 |
|
| 12 |
from __future__ import annotations
|
| 13 |
|
|
|
|
|
|
|
| 14 |
from pydantic import ValidationError
|
| 15 |
|
| 16 |
from ...catalog.models import Catalog
|
|
|
|
| 19 |
from .contracts import ToolRegistry
|
| 20 |
from .errors import PlannerValidationError
|
| 21 |
from .inputs import Constraints
|
| 22 |
+
from .schemas import PLACEHOLDER_RE, TaskList
|
| 23 |
|
| 24 |
# Heuristic: a checkable success_criteria mentions a measurable signal.
|
| 25 |
_CHECKABLE_TOKENS = ("rate", "count", "match", "produced", "above", "below", "equal")
|
|
|
|
| 26 |
|
| 27 |
# DFS colors for cycle detection.
|
| 28 |
_WHITE, _GREY, _BLACK = 0, 1, 2
|
|
|
|
| 166 |
|
| 167 |
|
| 168 |
def _is_placeholder(value: str) -> bool:
|
| 169 |
+
return bool(PLACEHOLDER_RE.fullmatch(value.strip()))
|
| 170 |
|
| 171 |
|
| 172 |
def _placeholder_refs(task) -> set[str]:
|
|
|
|
| 174 |
for call in task.tool_calls:
|
| 175 |
for value in call.args.values():
|
| 176 |
if isinstance(value, str):
|
| 177 |
+
refs.update(PLACEHOLDER_RE.findall(value))
|
| 178 |
return refs
|
| 179 |
|
| 180 |
|
|
@@ -16,22 +16,18 @@ See AGENT_ARCHITECTURE_CONTEXT_new.md §7.4.
|
|
| 16 |
from __future__ import annotations
|
| 17 |
|
| 18 |
import asyncio
|
| 19 |
-
import re
|
| 20 |
from typing import Any
|
| 21 |
|
| 22 |
from src.middlewares.logging import get_logger
|
| 23 |
|
| 24 |
from ..planner.contracts import ToolOutput, ToolRegistry
|
| 25 |
-
from ..planner.schemas import Task
|
| 26 |
from ..planner.schemas import TaskList as PlanTaskList
|
| 27 |
from .invoker import ToolInvoker
|
| 28 |
from .schemas import RunState, TaskResult, TaskStatus
|
| 29 |
|
| 30 |
logger = get_logger("task_runner")
|
| 31 |
|
| 32 |
-
# Mirrors planner/validator.py:28 `_PLACEHOLDER_RE` — keep the two in sync.
|
| 33 |
-
_PLACEHOLDER_RE = re.compile(r"\$\{(t[^}]+)\}")
|
| 34 |
-
|
| 35 |
|
| 36 |
class TaskRunner:
|
| 37 |
"""Runs a `TaskList` against a `ToolInvoker`, producing a `RunState`."""
|
|
@@ -131,7 +127,7 @@ class TaskRunner:
|
|
| 131 |
# the referenced task's representative output (its last ToolOutput).
|
| 132 |
# Materializing that envelope into a DataFrame is the invoker's job.
|
| 133 |
if isinstance(value, str):
|
| 134 |
-
match =
|
| 135 |
if match:
|
| 136 |
upstream = results.get(match.group(1))
|
| 137 |
if upstream is None or not upstream.outputs:
|
|
|
|
| 16 |
from __future__ import annotations
|
| 17 |
|
| 18 |
import asyncio
|
|
|
|
| 19 |
from typing import Any
|
| 20 |
|
| 21 |
from src.middlewares.logging import get_logger
|
| 22 |
|
| 23 |
from ..planner.contracts import ToolOutput, ToolRegistry
|
| 24 |
+
from ..planner.schemas import PLACEHOLDER_RE, Task
|
| 25 |
from ..planner.schemas import TaskList as PlanTaskList
|
| 26 |
from .invoker import ToolInvoker
|
| 27 |
from .schemas import RunState, TaskResult, TaskStatus
|
| 28 |
|
| 29 |
logger = get_logger("task_runner")
|
| 30 |
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
class TaskRunner:
|
| 33 |
"""Runs a `TaskList` against a `ToolInvoker`, producing a `RunState`."""
|
|
|
|
| 127 |
# the referenced task's representative output (its last ToolOutput).
|
| 128 |
# Materializing that envelope into a DataFrame is the invoker's job.
|
| 129 |
if isinstance(value, str):
|
| 130 |
+
match = PLACEHOLDER_RE.fullmatch(value.strip())
|
| 131 |
if match:
|
| 132 |
upstream = results.get(match.group(1))
|
| 133 |
if upstream is None or not upstream.outputs:
|