Rifqi Hafizuddin Claude Opus 4.8 commited on
Commit
0e2bb5c
·
1 Parent(s): 272a186

[NOTICKET][AI] docs: update PROGRESS for tool layer + tracing + gated slow-path wiring

Browse files
Files changed (1) hide show
  1. PROGRESS.md +65 -2
PROGRESS.md CHANGED
@@ -2,8 +2,71 @@
2
 
3
  Persistent tracker mirroring the 42-item ownership table in `REPO_CONTEXT.md` "Team — division of work". Update as PRs land. Future Claude Code sessions read this to know what's already done.
4
 
5
- **Last updated**: 2026-06-08 (slow-path agent layer TaskRunner + Assembler + Coordinator)
6
- **Current open PR**: `pr/1` — active. Cleanup PR committed and pushed.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
 
8
  ---
9
 
 
2
 
3
  Persistent tracker mirroring the 42-item ownership table in `REPO_CONTEXT.md` "Team — division of work". Update as PRs land. Future Claude Code sessions read this to know what's already done.
4
 
5
+ **Last updated**: 2026-06-10 (tool layer complete + Langfuse tracing + gated slow-path wiring)
6
+ **Current open PR**: `pr/2` — active.
7
+
8
+ ---
9
+
10
+ ## What just shipped (2026-06-09/10 — tool layer, tracing, slow-path wiring)
11
+
12
+ Big stretch since the slow-path workers landed. The tool layer (teammate-owned) is
13
+ now **complete and real**, the slow path is **wired into `ChatHandler` behind a gate**,
14
+ and the whole chat pipeline is **traced**. Fast path still untouched; live behavior
15
+ unchanged (flags default off).
16
+
17
+ **Tool layer — COMPLETE (teammate, KM-624→630).** `src/tools/` was re-created (the
18
+ 2026-05-11 note about deleting it is superseded). Now teammate-owned:
19
+ - `src/tools/analytics/` — the 8 **composite** `analyze_*` computes (descriptive,
20
+ aggregate, comparison, contribution, profile, correlation, segment, trend) +
21
+ prompt-style DESCRIPTIONs (KM-624/625).
22
+ - `src/tools/contracts.py` — canonical `ToolSpec`/`ToolRegistry`/`ToolOutput` (KM-627).
23
+ `agents/planner/contracts.py` now just re-exports them + keeps the `BusinessContext`
24
+ stub (lead's).
25
+ - `src/tools/registry.py::analytics_registry()` (KM-628); `src/tools/invoker.py` +
26
+ `src/tools/data_access.py` — `AnalyticsToolInvoker` (KM-629), `DataAccessToolInvoker`
27
+ + `CompositeToolInvoker` (KM-630). All never-throw. **Pattern A confirmed** (`analyze_*`
28
+ take a `data` `${t<id>}` placeholder from an upstream `query_structured`).
29
+ - **Verified live E2E (2026-06-09):** real `query_structured` against a user's Neon
30
+ Postgres → `analyze_trend` → Assembler. `analyze_contribution` surfaced a real tool
31
+ bug (Decimal vs float in `decomposition.py`) — degrade-and-continue held; teammate's
32
+ domain to fix. **Directive:** agent side does NOT modify `src/tools/` without confirmation.
33
+
34
+ **Planner — realigned to the real tools (KM-626).** `registry.py::default_registry()`
35
+ composes the real `analytics_registry()` + a local stub for the 4 data-access tools.
36
+ Few-shots grown to **A–D**: A `analyze_contribution`, B `analyze_trend`, C mixed
37
+ structured+unstructured (`retrieve_documents`, independent branch), D `analyze_aggregate`.
38
+ `parallelizable_with` **removed** from `Task` (schema/validator/examples/prompt) —
39
+ TaskRunner derives parallelism from `depends_on` alone.
40
+
41
+ **Slow-path wiring — built, GATED OFF (KM-626).** `agents/chat_handler.py` gains a
42
+ `structured→slow` branch behind `ChatHandler(enable_slow_path=False)`: when on it builds
43
+ a per-request `CompositeToolInvoker` (composition root) + `SlowPathCoordinator`, streams
44
+ `chat_answer`, persists the `analysis_record`. Two seams isolate the remaining blockers:
45
+ - `agents/planner/business_context.py::get_business_context(user_id)` — async stub
46
+ `BusinessContext`; TODO(lead) swap for the real read.
47
+ - `agents/slow_path/store.py` — `AnalysisStore` Protocol + `NullAnalysisStore` (logs
48
+ only). Real store = `analysis_records` table in the catalog DB (Neon `dataeyond`) —
49
+ **table not created yet**. `chat_answer` still emitted as one chunk (not token-streamed).
50
+
51
+ **Observability — Langfuse tracing wired (KM-631).** `src/observability/langfuse/
52
+ tracing.py` — `RequestTracer`/`NullTracer`/`TracingToolInvoker` + `_redact`. One trace
53
+ per request groups Orchestrator.classify, Planner.plan (each retry = its own generation),
54
+ Assembler.assemble, Chatbot.astream + tool spans (latency/metadata only). Gated:
55
+ `ChatHandler(enable_tracing=False)`; `api/v1/chat.py` opts in (`=True`). PII policy:
56
+ Orchestrator+Planner unmasked (question + PII-safe summary); Assembler+Chatbot masked
57
+ (see real rows/chunks); tool spans carry name + arg keys + row count only. Zero added
58
+ LLM tokens; verified live to US Cloud.
59
+
60
+ **Live evals green (2026-06-09, real Azure 4o):** `RUN_PLANNER_EVAL=1` and
61
+ `RUN_SLOW_PATH_EVAL=1` both pass — Planner emits valid catalog-consistent `QueryIR` and
62
+ wires Pattern A correctly; self-corrects via retry.
63
+
64
+ **Open follow-ups:** real `BusinessContext` (lead); create `analysis_records` table +
65
+ real `AnalysisStore`; register data-access `ToolSpec`s upstream (`data_access_registry()`)
66
+ or keep the planner stub; 4o → GPT-mini deployment swap; flip `enable_slow_path` on once
67
+ `BusinessContext` is real. NOTE: 3 test files pre-existing broken from rename rot
68
+ (`test_chat_handler.py`, `test_intent_router.py`, `test_answer_agent.py` import the old
69
+ `answer_agent`/`intent_router` module names).
70
 
71
  ---
72