Commit History
28 days ago
-
[NOTICKET] planner: group-first tool selection (A17)
Rifqi Hafizuddin +1: Claude Opus 5A0 activated every analytics tool, which puts the planner-visible set at 18 -- inside the measured 15-20 band where tool-selection accuracy degrades, and this registry has exactly the semantic near-duplicates that drive it (trend/forecast, correlation/importance, segmentation/cluster). A0 chose to pay that cost with grouping rather than by turning tools off, so nothing goes dark. Three layers, and which does what is the whole design: router (LLM, already runs) -> emits group_hint on its existing structured output -> planner/groups.py (deterministic, 0 LLM) -> filters the registry -> planner (LLM) -> plans against the slice, unchanged The filter is deliberately NOT the planner's job: to choose a group it would have to be shown every group, which is the thing being avoided, and a second LLM call would break the two-call invariant for a benefit that only pays above ~60 tools. It is not the router's job either -- the router classifies, it does not filter. Groups are DERIVED…6371d85 -
[NOTICKET] tools: quality blocks for forecast + ab_test, forecast backtest, CK7 fix
Rifqi Hafizuddin +1: Claude Opus 5Extends the reserved `quality` convention to the tools where it means something, and fixes a CK7 message that was wrong for half its inputs. analyze_forecast -- the substantial one. It reported NO accuracy at all, so "how accurate is it?" had no answer. Now backtests: hold out the last h periods (bounded by a quarter of the series and by the fitting floor), refit, and score with MASE -- mean absolute error over the naive "next = last" error on the training span. MASE is scale-free (Hyndman & Koehler 2006) and, unlike MAPE, neither explodes near zero nor punishes over-prediction asymmetrically. Both fits run in ONE run_fit call, so a forecast still costs one thread hop and one timeout. A series too short to hold anything out, or one that never moves, ships UNVALIDATED -- also weak, with the two causes reported as distinct reasons rather than one wrong one. analyze_ab_test -- free and exact: the statistical verdict IS the quality verdict. A gap the test cannot separate from noise is we…2082eba
29 days ago
-
[NOTICKET] feat(tools): analyze_importance + analyze_cluster (unregistered) + CK7
Rifqi Hafizuddin +1: Claude Opus 5A8 and A10. Both files are INERT on purpose: not in _DISPATCH, not in either registry list. registry.py and invoker.py are the two files every tool addition touches, so they stay single-owner until Sofhia makes the call -- the DEFERRED mechanism means an unregistered tool costs nothing. analyze_importance -- rank what explains a target. Standardised coefficients, NOT permutation importance. For a linear model the coefficient IS the importance and comes free with the fit; permutation re-scores once per feature, which is the E4 latency tall pole, and buys nothing here. Ridge for a numeric target, Logistic for a categorical one, auto-detected. Leakage-safe: the scaler is fitted on the TRAIN split only, so no test statistic reaches the fit -- the thing R-3 and the survey literature both single out. analyze_cluster -- discover groups nobody named yet. KMeans with k chosen by silhouette over 2..8, every candidate score returned. Scatter plots two REAL columns rather than PC…9a29ca3 -
[NOTICKET] feat(report): embed analyze_* auto-charts, share one chart extractor (A21)
Rifqi Hafizuddin +1: Claude Opus 5Closes the gap Sofhia flagged at handoff: an analyze_* tool's auto-chart appeared under the chat answer and silently vanished from the downloaded report, because report/generator._collect_charts only ever looked at render_chart's chart-kind output and never at ToolOutput.meta["charts"]. Rather than copy the meta-charts logic into a second place, the extraction moved to one definition -- src/charts/extract.py::chart_envelopes -- now used by BOTH consumers: the per-turn ChartStore persistence and the report's ## EDA section. That is the actual fix; the missing charts were a symptom of having two implementations of "what charts does this output carry", only one of which was updated in A4. Also tightened, and this is a small behaviour change worth naming: an envelope must carry a `plotly` object to be emitted. The report path already required that; the persistence path accepted any dict, so a malformed envelope could be written as a message_charts row the frontend could not draw. Both pa…1c36d9a -
[NOTICKET] feat(slow-path): per-step SSE progress, bilingual (A22)
Rifqi Hafizuddin +1: Claude Opus 5A structured_flow turn took ~12s behind three stage pings -- "Planning", "Running N steps", "Composing". Past Miller's ~10s attention threshold what a waiting user needs is evidence that something is happening, not a faster answer, so the TaskRunner now announces each wave BEFORE running it. A 4-task plan emits ~6 status events, not 3. New src/agents/slow_path/progress.py holds the labels: - BILINGUAL. The three existing stage pings were English-only on a product whose traffic and eval set are majority Indonesian; they follow reply_language now, the same "id" if reply_language == "Indonesian" test refusals.py uses. - A bounded activity vocabulary keyed by tool ("retrieving data" / "mengambil data"), NOT the task's objective. Plan internals are deliberately English, so using objective as status text would show English prose to an Indonesian user. - Unknown tools fall back to "analysing" / "menganalisis", so the six tools still to land (DEV_PLAN 0.9) produce s…06aecc8
30 days ago
-
[NOTICKET] docs: analytical-tools foundation — conventions, DEV_PLAN 0.9, status sync
Rifqi Hafizuddin +1: Claude Opus 5Documents the foundation shipped on pr/22 (ee1a76a, 535f173, 522e582, 4d76bd0). No code touched. New: docs/tools/ANALYTICAL_TOOLS_CONVENTIONS.md — the authoring guide, written against the shipped modules: the InputModel -> ToolSpec flow via spec_builder, build_description's three-part format, the auto-chart "charts" key convention, run_fit, the three registrations a tool needs, and the non-negotiable rules. Indexed as a new docs/ workstream section, mirroring docs/knowledge/. DEV_PLAN: new 0.9 sprint section with the 24-row task table and owners. Records the locked decision that the tool contract is FUNCTIONAL, not a base class -- BaseAnalysisTool/ToolResult/mcp_schema were designed and dropped because ToolSpec is already MCP-shaped, ACTIVE/DEFERRED already gates exposure, the chart envelope already existed, and a second output envelope would have collided with everything keying off ToolOutput.kind. Section 6: W3 re-open struck as RESOLVED -- superseded by the per-tool P1 decision ra…a7f0bca