Commit History
23 days ago
-
[NOTICKET] anomaly: name the anomalous rows + a flag-rate quality verdict (D-3)
analyze_anomaly's records carried only the scored numeric columns plus a score, so "which units look unusual?" could never name a unit; and it was the only ML tool with no quality block, so an 86%-flagged run reached the assembler as silence. - Add optional label_columns: identifier columns carried into each record (identifiers first), never fed to the model. Planner guidance to pass the id column follows on Rifqi's side (decided shape (a), tool-side). - Add the reserved quality block: metric anomaly_rate, weak only ABOVE a 0.5 ceiling. A20's flag-rate finding was re-measured, not re-assumed -- clean noise flags 0.14-0.22 (mean 0.17, p95 0.22), a genuine 30%-outlier frame 0.30, the demo's degenerate run 0.86 -- so 0.5 clears the noise floor and a real finding yet still catches "most rows flagged"; the note states the rate plainly either way. Logic in _flag_rate_quality() so both branches are unit-testable (the fit can't be pushed past 0.5 via contamination). PR-0 from…ae50194 -
[NOTICKET] docs: PR-0/PR-1 spec for Sofhia, and the feedback-edge before/after
Rifqi Hafizuddin +1: Claude Opus 5docs/plans/PR0_PR1_TOOL_FIXES.md — six tool fixes, split into PR-0 (two that block QA) and PR-1 (four that block nothing). Both stay on pr/22: the scope is one directory. Every claim was RE-VERIFIED against the code today rather than taken from the review, which was read-only and says so. That changed two of them: - D-6 (pareto non-additive aggs) is already DOCUMENTED at pareto.py:24 — a deliberate trade, not an oversight. The spec says so, and narrows the ask to the two real gaps: the output still calls it "share" with no caveat, and negatives are unguarded. - D-5 was dropped from Sofhia's scope entirely. The docstring states the empty-set semantics are intended and correct; the actual gap is narrower (absent column vs empty result) and lives in task_runner, which is not her lane. D-12 also moved out — it is in planner/examples.py, Rifqi's file. Flagged for her to DECIDE before starting: D-3's fix may need the planner to pass an identifier column, which would c…9c0fba7
26 days ago
-
[NOTICKET] chore: untrack a render byproduct, and note what 7ffe004a swept in
Rifqi Hafizuddin +1: Claude Opus 5Two corrections to the previous commit, fixed forward rather than by rewriting pushed history. 1. docs/testing/agentic-flow.preview.html was tracked. It is a byproduct of the offline excalidraw preview, regenerated on every render. Untracked and ignored. 2. 7ffe004a's message describes only the _bound threshold change, but the commit also carried two files it never mentions: docs/reviews/AGENTIC_DESIGN_REVIEW_2026-09-18.md (559 lines) docs/testing/agentic-flow.excalidraw (80 elements) Neither is mine. They are from a concurrent session working in this same tree, and `git add -A` swept them into my commit. The work is legitimate and stays; what was wrong was attributing it to a commit about something else. Recording it here so the log is honest about which change is which. Worth a decision, not made here: docs/testing now holds TWO flow diagrams whose names differ only by a hyphen -- agentic_flow.excalidraw (mine, 109 elements, "T…171ac0c -
[NOTICKET] slow path: don't re-trim a payload the tool already bounded (A30 follow-up)
Rifqi Hafizuddin +1: Claude Opus 5Sofhia's a2089edb caps analyze_anomaly's sample at source (200 -> 50) and reports `anomalies_total` / `anomalies_shown` beside it. That is the right layer for it: my A30 cap protects what reaches the LLM, hers protects what gets persisted to results_snapshot and traced to Langfuse, where the 50k blob was still landing whole. But the two stacked badly. Her cap alone brings the padb payload to 1,180 tokens -- already sane. My generic list trim then cut it to 691 and left anomalies_shown: 36 beside a list of 20 rows + "… (+16 more)" The count is recoverable from the marker, and `anomalies_total` was exact throughout, so nothing was ever wrong -- but a narrative reads straight past that, and "a number in the payload that disagrees with the payload" is the exact class of bug this round has been chasing. Fixed at my end, not hers: `_bound` now leaves a value alone below _MIN_TRIM_CHARS (8,000). The bound exists to stop floods, not to second-guess a tool that already bounded itself …7ffe004
27 days ago
-
[NOTICKET] docs: diagram of the agentic path, drawn from the code
Rifqi Hafizuddin +1: Claude Opus 5docs/testing/agentic_flow.excalidraw. The argument the shape makes: six intents leave the router, five are shallow, and only structured_flow descends. Everything that guards correctness -- the validator's 10 checks, the retry loop, the S1a checkpoint, the infeasible exit -- sits on that one spine. Strip the labels and the structure still says it: a narrow entry, a diamond fanning to six, five short stubs, one long descending spine, converging to four outputs. Built from the code rather than memory: the six intent branches are the ones in chat_handler (lines 399-524), the slow-path order is the coordinator's Planner -> TaskRunner -> Assembler with the checkpoint between, and the SSE order is the contract's sources -> status -> chunk -> done. Carries three evidence artifacts -- a real TaskList shape with the ${t1} handoff, the SSE event order, and the 14 tool names. The rendered PNG is deliberately NOT committed. HF's pre-receive hook rejects binaries that have not gone through Xet, a…10f94f3 -
[NOTICKET] docs: fill bug 14's commit hash
Rifqi Hafizuddin +1: Claude Opus 5Third time this round a hash had to be written after the fact rather than guessed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>198b4b3 -
[NOTICKET] planner: surface numeric min/max, so thresholds aren't chosen blind
Rifqi Hafizuddin +1: Claude Opus 5BUG 14, found by giving analyze_segmentation its first end-to-end run — it was the last active tool never to have executed outside unit tests. Asked to band `pa` into low/medium/high, the planner used thirds of 0-100 and all 42 rows landed in "high". A segmentation where every row is in one bucket says nothing, and nothing in the pipeline flagged it: the tool did exactly what it was told. The column runs 68.8-95.9. Go has populated stats.min/max on 98 of 98 numeric columns all along; CatalogSummary dropped them, keeping only sample values. The planner did not guess badly — it could not see the range. Same root cause as A28 (constant columns): the catalog knew, the prompt discarded it. It is also the Python half of the date-range gap handed to Harry — his side populates dates, this side renders whatever is there, and neither half is useful alone. Date columns render automatically once he ships. Rendered as `range=<min>..<max>`, skipped for constant columns because their marker alrea…9155dcb -
[NOTICKET] docs: real commit hashes in the dry-run log
Rifqi Hafizuddin +1: Claude Opus 5Second time this round I wrote a hash from memory before the commit existed: the three validator checks are 1bf70654, not d0d18e8. Also replaced two 'this commit' placeholders with 59715a00 and ec492813, which only read correctly in the commit that added them. A wrong hash in a reference doc is worse than none — it sends the reader looking for something that was never there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>9d36f1e -
[NOTICKET] assembler: never state the finding a failed step was meant to produce
Rifqi Hafizuddin +1: Claude Opus 5BUG 13, found by the certification run (2026-09-17, analysis 62c34bc6). analyze_correlation refused a non-numeric column (eqm_no), so nothing was measured. The reply opened: "Ya, ada indikasi hubungan antara backlog dan bad actor score..." (Yes, there is an indication of a relationship...) assembler.md rule 2 already said "do not pretend they succeeded" AND "do not lead with the failure". The model obeyed both and invented the conclusion in between. Nothing in the rule forbade stating the very finding the failed step existed to produce. This is the dangerous class again: the analysis failed, the narrative claimed a result, and the hedge arrived two clauses later — where it does not help, because the reader takes the first clause. Rule 2 now says so explicitly: not "yes, there is an indication", not "X appears to drive Y", and not softened with "tends to" or "seems". Name the question that stays open, and answer only what the successful steps support. Verified with --turns…ec49281 -
[NOTICKET] eval(chat_sim): --turns, so verifying a late fix doesn't cost a whole suite
Rifqi Hafizuddin +1: Claude Opus 5--max-turns only ever takes a PREFIX, so checking a fix to the last turn of an 8-turn suite meant paying for all eight: ~290k to verify one ~30k turn. --turns 8 runs exactly that turn. First use, verifying the bug-12 fix: 57,734 tokens against ~290k for the full suite. History is still built from whatever runs, so a turn that depends on an earlier one is selected together with it (--turns 2,8). Noted in the flag's help and in the doc. Verified on the real suite: turn 8 alone now returns the honest refusal — it names both tables, says no foreign key is declared between them, and offers the per-table summaries it CAN produce. No validator vocabulary reaches the user. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>0f99ba0 -
[NOTICKET] planner: exhaustion refuses instead of crashing, + the dry-run findings log
Rifqi Hafizuddin +1: Claude Opus 5BUG 12 — retry exhaustion showed the user the validator's own words. Asked whether backlog relates to bad actor score, the planner burned all three attempts (cross-table select, then a join the schema does not declare) and the turn ended with Analysis failed: planner could not produce a usable plan after 3 attempts; last error: joins[0]: (...) is not a declared foreign key That is internal vocabulary about an internal contract, and it reads as a broken product rather than a limit of the data. The SAME question refused cleanly on the previous run, so this is reachable by chance, not by input. Two changes. On the final attempt the planner is now told that refusing is a valid outcome — until now the retry prompt only ever said "this plan was invalid, fix it", which pushes toward another plan even when none exists. And if it still exhausts, the service returns the infeasible sentinel instead of raising, reusing the path that already handles this well: the coordinator renders it…59715a0 -
[NOTICKET] planner: group-first tool selection (A17)
Rifqi Hafizuddin +1: Claude Opus 5A0 activated every analytics tool, which puts the planner-visible set at 18 -- inside the measured 15-20 band where tool-selection accuracy degrades, and this registry has exactly the semantic near-duplicates that drive it (trend/forecast, correlation/importance, segmentation/cluster). A0 chose to pay that cost with grouping rather than by turning tools off, so nothing goes dark. Three layers, and which does what is the whole design: router (LLM, already runs) -> emits group_hint on its existing structured output -> planner/groups.py (deterministic, 0 LLM) -> filters the registry -> planner (LLM) -> plans against the slice, unchanged The filter is deliberately NOT the planner's job: to choose a group it would have to be shown every group, which is the thing being avoided, and a second LLM call would break the two-call invariant for a benefit that only pays above ~60 tools. It is not the router's job either -- the router classifies, it does not filter. Groups are DERIVED…6371d85 -
[NOTICKET] tools: quality blocks for forecast + ab_test, forecast backtest, CK7 fix
Rifqi Hafizuddin +1: Claude Opus 5Extends the reserved `quality` convention to the tools where it means something, and fixes a CK7 message that was wrong for half its inputs. analyze_forecast -- the substantial one. It reported NO accuracy at all, so "how accurate is it?" had no answer. Now backtests: hold out the last h periods (bounded by a quarter of the series and by the fitting floor), refit, and score with MASE -- mean absolute error over the naive "next = last" error on the training span. MASE is scale-free (Hyndman & Koehler 2006) and, unlike MAPE, neither explodes near zero nor punishes over-prediction asymmetrically. Both fits run in ONE run_fit call, so a forecast still costs one thread hop and one timeout. A series too short to hold anything out, or one that never moves, ships UNVALIDATED -- also weak, with the two causes reported as distinct reasons rather than one wrong one. analyze_ab_test -- free and exact: the statistical verdict IS the quality verdict. A gap the test cannot separate from noise is we…2082eba
28 days ago
-
[NOTICKET] feat(tools): analyze_importance + analyze_cluster (unregistered) + CK7
Rifqi Hafizuddin +1: Claude Opus 5A8 and A10. Both files are INERT on purpose: not in _DISPATCH, not in either registry list. registry.py and invoker.py are the two files every tool addition touches, so they stay single-owner until Sofhia makes the call -- the DEFERRED mechanism means an unregistered tool costs nothing. analyze_importance -- rank what explains a target. Standardised coefficients, NOT permutation importance. For a linear model the coefficient IS the importance and comes free with the fit; permutation re-scores once per feature, which is the E4 latency tall pole, and buys nothing here. Ridge for a numeric target, Logistic for a categorical one, auto-detected. Leakage-safe: the scaler is fitted on the TRAIN split only, so no test statistic reaches the fit -- the thing R-3 and the survey literature both single out. analyze_cluster -- discover groups nobody named yet. KMeans with k chosen by silhouette over 2..8, every candidate score returned. Scatter plots two REAL columns rather than PC…9a29ca3 -
[NOTICKET] docs(knowledge): bring the persistence contract current so it can be sent
Rifqi Hafizuddin +1: Claude Opus 5This doc calls itself "the DDL handoff" and was the obvious thing to send Harry alongside the SQL - but it had drifted to the point where sending it would have told him the opposite of the truth. It described the 2026-09-07 five-table shape and knew nothing of the seven columns added 2026-09-14. Fixed: - Section 2 tables now match LIVE, each addition carrying a dated marker: entries gains doc_ids/content/provenance, nullable parents with the domain CHECK, 'document' in the kind CHECK, and the key index; documents gains withdrawn_at and the partial live index; reviews gains decided_by and authorised_by; jobs gains review_mode. The label column's CHANGED meaning is called out - it carries a readable key now, not the term, and a consumer rendering it as a display name will show GLOSSARY_PA. - Banner no longer contradicts itself. It opened with a green LIVE line and then carried, in bold, "Nothing here exists in dedorch yet and no ORM model has been written against it". …ff27029 -
[NOTICKET] feat(report): embed analyze_* auto-charts, share one chart extractor (A21)
Rifqi Hafizuddin +1: Claude Opus 5Closes the gap Sofhia flagged at handoff: an analyze_* tool's auto-chart appeared under the chat answer and silently vanished from the downloaded report, because report/generator._collect_charts only ever looked at render_chart's chart-kind output and never at ToolOutput.meta["charts"]. Rather than copy the meta-charts logic into a second place, the extraction moved to one definition -- src/charts/extract.py::chart_envelopes -- now used by BOTH consumers: the per-turn ChartStore persistence and the report's ## EDA section. That is the actual fix; the missing charts were a symptom of having two implementations of "what charts does this output carry", only one of which was updated in A4. Also tightened, and this is a small behaviour change worth naming: an envelope must carry a `plotly` object to be emitted. The report path already required that; the persistence path accepted any dict, so a malformed envelope could be written as a message_charts row the frontend could not draw. Both pa…1c36d9a -
[NOTICKET] feat(slow-path): per-step SSE progress, bilingual (A22)
Rifqi Hafizuddin +1: Claude Opus 5A structured_flow turn took ~12s behind three stage pings -- "Planning", "Running N steps", "Composing". Past Miller's ~10s attention threshold what a waiting user needs is evidence that something is happening, not a faster answer, so the TaskRunner now announces each wave BEFORE running it. A 4-task plan emits ~6 status events, not 3. New src/agents/slow_path/progress.py holds the labels: - BILINGUAL. The three existing stage pings were English-only on a product whose traffic and eval set are majority Indonesian; they follow reply_language now, the same "id" if reply_language == "Indonesian" test refusals.py uses. - A bounded activity vocabulary keyed by tool ("retrieving data" / "mengambil data"), NOT the task's objective. Plan internals are deliberately English, so using objective as status text would show English prose to an Indonesian user. - Unknown tools fall back to "analysing" / "menganalisis", so the six tools still to land (DEV_PLAN 0.9) produce s…06aecc8
29 days ago
-
[NOTICKET] docs: analytical-tools foundation — conventions, DEV_PLAN 0.9, status sync
Rifqi Hafizuddin +1: Claude Opus 5Documents the foundation shipped on pr/22 (ee1a76a, 535f173, 522e582, 4d76bd0). No code touched. New: docs/tools/ANALYTICAL_TOOLS_CONVENTIONS.md — the authoring guide, written against the shipped modules: the InputModel -> ToolSpec flow via spec_builder, build_description's three-part format, the auto-chart "charts" key convention, run_fit, the three registrations a tool needs, and the non-negotiable rules. Indexed as a new docs/ workstream section, mirroring docs/knowledge/. DEV_PLAN: new 0.9 sprint section with the 24-row task table and owners. Records the locked decision that the tool contract is FUNCTIONAL, not a base class -- BaseAnalysisTool/ToolResult/mcp_schema were designed and dropped because ToolSpec is already MCP-shaped, ACTIVE/DEFERRED already gates exposure, the chart envelope already existed, and a second output envelope would have collided with everything keying off ToolOutput.kind. Section 6: W3 re-open struck as RESOLVED -- superseded by the per-tool P1 decision ra…a7f0bca
30 days ago
-
/fix parsing and term extract (#21)
- [NOTICKET] fix: retire the fuzzy clustering pass; add E1c post-cluster recall (2cc21c92688d662c5cfdfeeb6784bb85b98575ca) - [NOTICKET] feat: parsing schema 0.4.0 — asset pointers for figures and tables (e662ac089b741c90e19b9f51f4933fb97a1225bc) - [NOTICKET] docs: sync REPO_STATUS / DEV_PLAN / contracts for pipeline v2 (c3c184d83bec4a4d42e9ad3f7265560802817e5b) - [NOTICKET] docs: close K15 with the paid v1-vs-v2 comparison (62782fed9292f67d7b376783920a07197d6776e9) - [NOTICKET] fix: test the caption hypothesis; ship the signal at weight 0.0 (36a166e1844d186d6cfb8fff9ea070f3dc69b220) - [NOTICKET] fix: cap key-parameter pre-emption in the review queue (Step 2) (4e3a6aefef7024db3ea4b7946fd7771a6c269601) - [NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5) (f1d64b113db0289601806635a9b154201a6127fc) - [NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6) (cdb5fe82cc5f2893ccbb5e28282e8dc46a27e061) - [NOTICKET] fix: retire escalation…f07443e
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f