Commit History
23 days ago
-
[NOTICKET] anomaly: name the anomalous rows + a flag-rate quality verdict (D-3)
analyze_anomaly's records carried only the scored numeric columns plus a score, so "which units look unusual?" could never name a unit; and it was the only ML tool with no quality block, so an 86%-flagged run reached the assembler as silence. - Add optional label_columns: identifier columns carried into each record (identifiers first), never fed to the model. Planner guidance to pass the id column follows on Rifqi's side (decided shape (a), tool-side). - Add the reserved quality block: metric anomaly_rate, weak only ABOVE a 0.5 ceiling. A20's flag-rate finding was re-measured, not re-assumed -- clean noise flags 0.14-0.22 (mean 0.17, p95 0.22), a genuine 30%-outlier frame 0.30, the demo's degenerate run 0.86 -- so 0.5 clears the noise floor and a real finding yet still catches "most rows flagged"; the note states the rate plainly either way. Logic in _flag_rate_quality() so both branches are unit-testable (the fit can't be pushed past 0.5 via contamination). PR-0 from…ae50194 -
[NOTICKET] data_quality: actually count the duplicate rows it advertises (D-1)
analyze_data_quality's use_when lists "duplikat"/duplicates, but the file had no duplicated()/drop_duplicates() anywhere -- asked "ada duplikat?" the user got a confident per-column profile with no answer, and the assembler would infer one from distinct_rate. Add duplicate_rows {count, rate} over the profiled columns (df[cols].duplicated(), keep="first" = the redundant copies; a column subset is a candidate-key check) and issues.duplicate_row_count. Always present (0 when none, never None/absent), so the advertised claim is now backed by a real count. PR-0 from docs/plans/PR0_PR1_TOOL_FIXES.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>7078816
26 days ago
-
[NOTICKET] slow path: don't re-trim a payload the tool already bounded (A30 follow-up)
Rifqi Hafizuddin +1: Claude Opus 5Sofhia's a2089edb caps analyze_anomaly's sample at source (200 -> 50) and reports `anomalies_total` / `anomalies_shown` beside it. That is the right layer for it: my A30 cap protects what reaches the LLM, hers protects what gets persisted to results_snapshot and traced to Langfuse, where the 50k blob was still landing whole. But the two stacked badly. Her cap alone brings the padb payload to 1,180 tokens -- already sane. My generic list trim then cut it to 691 and left anomalies_shown: 36 beside a list of 20 rows + "… (+16 more)" The count is recoverable from the marker, and `anomalies_total` was exact throughout, so nothing was ever wrong -- but a narrative reads straight past that, and "a number in the payload that disagrees with the payload" is the exact class of bug this round has been chasing. Fixed at my end, not hers: `_bound` now leaves a value alone below _MIN_TRIM_CHARS (8,000). The bound exists to stop floods, not to second-guess a tool that already bounded itself …7ffe004 -
[NOTICKET] anomaly: bound the returned sample at source, not just the assembler
analyze_anomaly's `anomalies` list could carry up to 200 records (all scored columns + score each). A30 capped the same bloat at the assembler (what reaches the LLM), but the tool output is ALSO persisted to results_snapshot and Langfuse-traced, so the 50k blob still landed there. Cap _MAX_ANOMALY_ROWS 200 -> 50 (a bounded top-by-score SAMPLE, not the full set) and add anomalies_total / anomalies_shown so "N of M" stays honest. The exact count (n_anomalies), rate, and chart are unchanged. Defense-in-depth with A30's assembler cap, not a duplicate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>a2089ed
27 days ago
-
[NOTICKET] planner: surface numeric min/max, so thresholds aren't chosen blind
Rifqi Hafizuddin +1: Claude Opus 5BUG 14, found by giving analyze_segmentation its first end-to-end run — it was the last active tool never to have executed outside unit tests. Asked to band `pa` into low/medium/high, the planner used thirds of 0-100 and all 42 rows landed in "high". A segmentation where every row is in one bucket says nothing, and nothing in the pipeline flagged it: the tool did exactly what it was told. The column runs 68.8-95.9. Go has populated stats.min/max on 98 of 98 numeric columns all along; CatalogSummary dropped them, keeping only sample values. The planner did not guess badly — it could not see the range. Same root cause as A28 (constant columns): the catalog knew, the prompt discarded it. It is also the Python half of the date-range gap handed to Harry — his side populates dates, this side renders whatever is there, and neither half is useful alone. Date columns render automatically once he ships. Rendered as `range=<min>..<max>`, skipped for constant columns because their marker alrea…9155dcb -
[NOTICKET] assembler: never state the finding a failed step was meant to produce
Rifqi Hafizuddin +1: Claude Opus 5BUG 13, found by the certification run (2026-09-17, analysis 62c34bc6). analyze_correlation refused a non-numeric column (eqm_no), so nothing was measured. The reply opened: "Ya, ada indikasi hubungan antara backlog dan bad actor score..." (Yes, there is an indication of a relationship...) assembler.md rule 2 already said "do not pretend they succeeded" AND "do not lead with the failure". The model obeyed both and invented the conclusion in between. Nothing in the rule forbade stating the very finding the failed step existed to produce. This is the dangerous class again: the analysis failed, the narrative claimed a result, and the hedge arrived two clauses later — where it does not help, because the reader takes the first clause. Rule 2 now says so explicitly: not "yes, there is an indication", not "X appears to drive Y", and not softened with "tends to" or "seems". Name the question that stays open, and answer only what the successful steps support. Verified with --turns…ec49281 -
[NOTICKET] planner: exhaustion refuses instead of crashing, + the dry-run findings log
Rifqi Hafizuddin +1: Claude Opus 5BUG 12 — retry exhaustion showed the user the validator's own words. Asked whether backlog relates to bad actor score, the planner burned all three attempts (cross-table select, then a join the schema does not declare) and the turn ended with Analysis failed: planner could not produce a usable plan after 3 attempts; last error: joins[0]: (...) is not a declared foreign key That is internal vocabulary about an internal contract, and it reads as a broken product rather than a limit of the data. The SAME question refused cleanly on the previous run, so this is reachable by chance, not by input. Two changes. On the final attempt the planner is now told that refusing is a valid outcome — until now the retry prompt only ever said "this plan was invalid, fix it", which pushes toward another plan even when none exists. And if it still exhausts, the service returns the infeasible sentinel instead of raising, reusing the path that already handles this well: the coordinator renders it…59715a0 -
[NOTICKET] fix(planner): three checks for plans that lie about what they selected
Rifqi Hafizuddin +1: Claude Opus 5All three come from the first pa_dummy run (2026-09-17, analysis 2ad2354c) -- the first dry-run ever to reach the SQL path. Two of them produced a confident ANSWER rather than an error, which is why nothing caught them before: nothing was looking. CHECK 8f -- an alias may not rename a column into another thing that exists here. Asked whether backlog relates to bad actor score -- two tables the schema declares no FK between, so unjoinable -- the planner emitted SELECT badactor.bad_actor_score, badactor.ma AS "backlog" FROM badactor and the answer read "Korelasi Pearson antara backlog dan bad actor score adalah -0,029". `ma` is Mechanical Availability. It is not backlog, and `backlog` is a separate TABLE it could not reach. Rather than refuse, the plan renamed a different column to match the user's word, and the alias made the wrong answer read as the right one: every layer downstream, the narrative included, saw only "backlog". This is the force-mapping recipes R6/R15 exist to …1bf7065 -
[NOTICKET] fix(planner/slow path): checkable criteria in Indonesian, bound meta too
Rifqi Hafizuddin +1: Claude Opus 5BUG 7 -- check 7 only spoke English, on a majority-Indonesian product. _CHECKABLE_TOKENS was ("rate", "count", "match", "produced", "above", "below", "equal"). The planner writes success_criteria in the language of the question, so an Indonesian criterion could not pass except by luck. Every Indonesian turn in the dry-run burned a retry here, and one degenerated past that: told "not checkable" twice, the model dropped `success_criteria` AND `expected_output` altogether and the turn died. The field-dropping looked like a schema bug (Bug 5) but this was upstream of it -- Bug 5's fix caught the fall, this removes the push. Added the Indonesian equivalents. Parity, not loosening: each mirrors an English token already trusted ("menghasilkan" = "produced", "jumlah"/"baris" = "count", "minimal"/"lebih"/"kurang" = "above"/"below"), so a criterion that fails in English still fails in Indonesian. Pinned by a test asserting a vague criterion is refused in BOTH languages. BUG 8 -- `meta` was st…a6c79c9 -
[NOTICKET] fix(slow path): retry malformed plans, bound stats in the assembler prompt
Rifqi Hafizuddin +1: Claude Opus 5Two bugs the 2026-09-17 dry-run surfaced, both found by running the thing rather than by reading it. BUG 5 -- a malformed plan crashed the turn instead of being retried. Structured output came back with task t1 missing `expected_output` and `success_criteria`. That fails inside `chain.ainvoke` -- pydantic, before PlannerValidator ever runs -- and the call sat OUTSIDE the try. So this one failure mode skipped the retry loop every other planner mistake gets, propagated raw, and reached the user as their literal answer: ERROR: Analysis failed: 2 validation errors for TaskList tasks.0.expected_output Field required A dropped field is exactly as retryable as a bad column_id. The call now sits inside the loop, and the pydantic message names the missing field, so it is fed back compacted (`loc: msg`, no docs URLs) and the next attempt is told what to fix. On exhaustion the caller gets a PlannerError, never a raw pydantic dump. BUG 6 -- `stats` outputs were rendered to the assemb…7a89a2b -
[NOTICKET] fix(router): route data-condition questions to analysis, not inventory (A23 Bug 4)
Rifqi Hafizuddin +1: Claude Opus 5"Data PA-nya bersih nggak? Ada kolom yang kosong?" routed to `check` at 0.96 confidence, so the slow path never ran and analyze_data_quality was never called. The reply was a 36-row schema table whose "Boleh kosong" column is the catalog's declared `nullable` SCHEMA FLAG -- confident, well formatted, and not an answer to the question the user asked about the actual rows. The prompt defined `check` as "what columns are in this table?", and "ada kolom yang kosong?" is lexically a columns question. The only boundary offered was "what do I have" -> check versus "analyze/compute/trend/correlate" -> structured_flow, and a data-CONDITION question is neither phrasing. Recast the boundary as EXISTENCE vs CONDITION: `check` answers what is there, from metadata alone; the moment the answer depends on the values in the rows it is `structured_flow`, because only computing over the data can answer it. Stated explicitly that naming a column does not make it `check`, and that a `nullable` flag says …e815376 -
[NOTICKET] fix(planner): surface constant columns, reject constant targets (A23 Bug 3)
Rifqi Hafizuddin +1: Claude Opus 5Asked "what's driving down hours the most?", the planner targeted Total_Hours -- a fixed 24-hour day -- instead of Total_Down_Hour, and analyze_importance correctly refused: nothing can explain a number that never moves. The catalog knew all along. Total_Hours carries distinct_count=1, min=24, max=24. But CatalogSummary carried only sample_values and top_values and dropped every other stat, so the fact never reached the prompt. The planner was not ignoring the evidence; it never had it. Two fixes, both deterministic: 1. The planner can now SEE it. ColumnSummary.is_constant, rendered as "[CONSTANT: one value in every row -- cannot be a target or a grouping key]" and ONLY when true, so a normal column costs nothing and the marker is conspicuous exactly where it matters. A bool rather than the raw count, because constancy is the only fact here that changes a planning decision -- a count on every column would spend tokens saying something true but useless about the other …688431c -
[NOTICKET] fix(tools): accept `avg` wherever `mean` works (A23 Bug 2)
Rifqi Hafizuddin +1: Claude Opus 5The planner and the tools disagreed about what "average" is called, and the planner lost. It fills in a QueryIR and a tool call from ONE prompt: QueryIR AggFn count, count_distinct, sum, avg, min, max tools SUPPORTED_AGGS sum, mean, count, min, max, median (+ nunique) planner.md teaches `avg` three times because the IR requires it -- line 331 even says "utilization, scores -> avg", which is exactly the PA case. So the planner wrote agg="avg" into analyze_forecast and the tool raised UnsupportedAggregationError. The dry-run lost its forecast turn that way. The planner was not wrong; it obeyed its own prompt. Blast radius was five tools, not one. analyze_trend, analyze_forecast, analyze_ab_test, analyze_pareto and analyze_segmentation all take an `agg` and all five rejected `avg` -- only forecast happened to be exercised. analyze_aggregate failed differently and more quietly: validator check 8c rejected the plan, burning a retry before execution. Aliasing rath…0f0af11 -
[NOTICKET] fix(query): align date filters with date32 columns (A23 Bug 1)
Rifqi Hafizuddin +1: Claude Opus 5Every date filter against an uploaded FILE was broken, and had been all along. A Parquet DATE column (arrow date32) reads back as *object* dtype holding datetime.date; the IR carries its literal as a string. _apply_filters compared them raw, so `>=` raised '>=' not supported between instances of 'datetime.date' and 'str' inside the executor -- BEFORE the planner's tools ever ran, which is why the dry-run caveat named no tool at all. That killed 2 of 7 turns (trend, anomaly). in/not_in were worse: .isin() raises nothing and matches NOTHING, so that filter shape returned a confidently empty table. A wrong answer, not an error, and nothing in the pipeline could have flagged it. _align_temporal() now normalises both sides to datetime64/Timestamp. Three choices worth stating, because each is a place this could have gone wrong: - Gated on the SERIES being temporal, never on value_type alone. The planner writes value_type; a mislabelled text column must not get dragged into da…a6444fb -
[NOTICKET] planner: group-labelled recipes + the feasibility gate the eval caught (A18)
Rifqi Hafizuddin +1: Claude Opus 5Recipes, part 2 of A18. The table goes R1-R6 -> 18 rows, each labelled with its GROUP so the tool list (organised by group, A17) and the rules (organised by recipe) stop being two overlapping schemes the planner has to reconcile. New: R7 data-quality, R8 forecast, R9 importance, R10 pareto, R11 segmentation, R12 clustering, R13 A/B test, R14 anomaly, R15 method-infeasible. Plus a "telling the close pairs apart" block for the six intra-group confusions the group hint cannot separate -- trend vs forecast, correlation vs importance, importance vs pareto, segmentation vs clustering, A/B vs aggregate, anomaly vs descriptive. R15 is not a softer R6: R6 is "the column does not exist", R15 is "the column exists but no tool does that". Numbering note -- the pre-implementation spec called the method-infeasible recipe R12; the implemented table gives R12 to clustering and R15 to method-infeasible. Stale references in DEV_PLAN, examples.py and groups.py corrected rather than renumbering the live …b56fb4c -
[NOTICKET] planner: ten exemplars, one per tool that had none (A18, part 1)
Rifqi Hafizuddin +1: Claude Opus 5After A25 scoped few-shots to the visible tool set, three groups rendered ZERO examples demonstrating their own tools -- relationship, grouping and evaluate each kept 7, none of which called a tool from that group. forecast, data_quality and correlation had no exemplar either, correlation never having had one. Those turns planned from DESCRIPTION alone. Ten new examples, N through W, one per undemonstrated tool, following the spec agreed in DEV_PLAN before implementation: N forecast pairs against B (trend): same retrieve, past vs project O correlation two NAMED columns P importance same retrieve as O -- so the contrast is the question alone: "how do X and Y relate" -> O, "what most affects X" -> P Q pareto arithmetic share, vs P's statistical explanation R segmentation rule GIVEN -> explicit bins + labels S cluster no rule -> discovered, vs R T ab_test two named groups + a significance verdict U anomaly …57f8cee -
[NOTICKET] planner: unblock the ML tools A15 just activated (A24, A25)
Rifqi Hafizuddin +1: Claude Opus 5A15 turned on all 18 tools. Two things would have kept the four ML tools effectively unusable, both found while drafting the A15 handoff, both mine. A24 -- modeling was still forbidden in CODE. prompt.py injected, on every planner call: - modeling_allowed: False (no modeling tools exist in v1 -- do not emit modeling tasks) Rule 5 in planner.md was rewritten to permit modeling, but this line sits in the constraints block, is more specific, and flatly asserts the tools do not exist. The planner would have been told to ignore forecast/anomaly/importance/cluster the moment they went live -- four of the demo's payoffs. Constraints.modeling_allowed now defaults True, and the rendered text follows the flag rather than asserting a fixed world. Kept as a knob, not deleted: a caller can still force a strictly descriptive run. A25 -- few-shots were not scoped with the registry, an A17 gap I introduced. render_examples() took no arguments, so all 13 rendered regardless of the group hint. A…3258aa2 -
[NOTICKET] feat(tools): activate all analytical tools (A15)
A0 decision = all 18 tools active. Moved the 8 new/renamed specs DEFERRED -> ACTIVE in registry.py; DEFERRED_ANALYTICS_TOOLS = [] (mechanism kept for future dark-shipping). ACTIVE = 14 analytics + 4 data-access; every category maps to an A17 group (verified). DEV_PLAN §0.9 A15 -> done. NOTE: pairs with A18 (Rifqi) — the 8 newly-active tools have no planner few-shots yet (examples.py). They are visible via grouping + their DESCRIPTION, but few-shots should land before the demo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>65ada2d -
[NOTICKET] planner: group-first tool selection (A17)
Rifqi Hafizuddin +1: Claude Opus 5A0 activated every analytics tool, which puts the planner-visible set at 18 -- inside the measured 15-20 band where tool-selection accuracy degrades, and this registry has exactly the semantic near-duplicates that drive it (trend/forecast, correlation/importance, segmentation/cluster). A0 chose to pay that cost with grouping rather than by turning tools off, so nothing goes dark. Three layers, and which does what is the whole design: router (LLM, already runs) -> emits group_hint on its existing structured output -> planner/groups.py (deterministic, 0 LLM) -> filters the registry -> planner (LLM) -> plans against the slice, unchanged The filter is deliberately NOT the planner's job: to choose a group it would have to be shown every group, which is the thing being avoided, and a second LLM call would break the two-call invariant for a benefit that only pays above ~60 tools. It is not the router's job either -- the router classifies, it does not filter. Groups are DERIVED…6371d85 -
[NOTICKET] fix(tools): analyze_ab_test quality — KeyError on the too-few-rows path
The quality block (added in 2082eba1) read test['p_value'] directly, but _welch_test's too-few-rows branch returns no 'p_value' key -> KeyError, so analyze_ab_test crashed on a valid 'too few rows for a t-test' case (became an error envelope via the never-throw seam instead of a graceful result). Fix: test.get('p_value'). The too-few case now returns significant=None + quality {value: None, weak: True}. Caught by a local ab_test test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>8400a9e -
[NOTICKET] tools: quality blocks for forecast + ab_test, forecast backtest, CK7 fix
Rifqi Hafizuddin +1: Claude Opus 5Extends the reserved `quality` convention to the tools where it means something, and fixes a CK7 message that was wrong for half its inputs. analyze_forecast -- the substantial one. It reported NO accuracy at all, so "how accurate is it?" had no answer. Now backtests: hold out the last h periods (bounded by a quarter of the series and by the fitting floor), refit, and score with MASE -- mean absolute error over the naive "next = last" error on the training span. MASE is scale-free (Hyndman & Koehler 2006) and, unlike MAPE, neither explodes near zero nor punishes over-prediction asymmetrically. Both fits run in ONE run_fit call, so a forecast still costs one thread hop and one timeout. A series too short to hold anything out, or one that never moves, ships UNVALIDATED -- also weak, with the two causes reported as distinct reasons rather than one wrong one. analyze_ab_test -- free and exact: the statistical verdict IS the quality verdict. A gap the test cannot separate from noise is we…2082eba
28 days ago
-
[NOTICKET] planner: rule 5 forbade the modeling tools we just shipped
Rifqi Hafizuddin +1: Claude Opus 5planner.md rule 5 read "No modeling in v1. There are no modeling tools... The product is descriptive/diagnostic only -- no predictions." Four ML tools have since landed (forecast, anomaly, importance, cluster), so on every planner call the prompt was instructing the model not to use tools the registry was about to offer it. The CRISP-DM bullet said "(Never `modeling`.)" for a stage the schema has always declared legal and the validator never restricted. Rule 5 now: - is phrased CONDITIONALLY ("when the Available tools list includes a modeling tool"), so it is true both today -- all 8 ML tools are DEFERRED, none visible -- and after A0 activates some. The tool list is injected per call; rule 3 already governs what exists. - keeps both refusals the 2026-09-15 discussion made: no AutoML, and no model-building as an end in itself ("membuat model, bukan melakukan analisis"). - extends the Example-K feasibility discipline to modeling: an explicit ask to forecast or…855e406 -
[NOTICKET] feat(tools): register analyze_importance + analyze_cluster (DEFERRED) — A8/A10
Wires Rifqi's inert analytics/importance.py + analytics/cluster.py into the dispatch + registry (Sofhia owns invoker.py/registry.py): both async, output_kind 'stats', DEFERRED via spec_from_model (importance -> analytics.relationship, cluster -> analytics.clustering). Smoke-tested invokable — both emit their quality block + chart. DEV_PLAN §0.9 A8/A10 -> done (registered). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>c793aa4 -
[NOTICKET] feat(tools): analyze_ab_test — rename analyze_comparison + Welch t-test (A11)
Renames analyze_comparison/comparison.py -> analyze_ab_test/ab_test.py. Keeps the two-group aggregate gap; adds a Welch two-sample t-test (t, p-value, significant at alpha) on the raw values and a bar chart of the two group means with 95% error bars, via the reserved 'charts' key. Registered DEFERRED via spec_from_model. Old name removed — 5 cross-ref DESCRIPTIONs + contracts.py comment repointed. DEV_PLAN §0.9 A11 -> done. Heads-up (Rifqi): analyze_comparison renamed to analyze_ab_test. Local tool tests under tests/unit/tools (gitignored) referencing it break on pull — rename to analyze_ab_test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>901b03f -
[NOTICKET] feat(tools): analyze_importance + analyze_cluster (unregistered) + CK7
Rifqi Hafizuddin +1: Claude Opus 5A8 and A10. Both files are INERT on purpose: not in _DISPATCH, not in either registry list. registry.py and invoker.py are the two files every tool addition touches, so they stay single-owner until Sofhia makes the call -- the DEFERRED mechanism means an unregistered tool costs nothing. analyze_importance -- rank what explains a target. Standardised coefficients, NOT permutation importance. For a linear model the coefficient IS the importance and comes free with the fit; permutation re-scores once per feature, which is the E4 latency tall pole, and buys nothing here. Ridge for a numeric target, Logistic for a categorical one, auto-detected. Leakage-safe: the scaler is fitted on the TRAIN split only, so no test statistic reaches the fit -- the thing R-3 and the survey literature both single out. analyze_cluster -- discover groups nobody named yet. KMeans with k chosen by silhouette over 2..8, every candidate score returned. Scatter plots two REAL columns rather than PC…9a29ca3 -
[NOTICKET] feat(tools): analyze_segmentation — activate/rename analyze_segment + chart (A13)
Activates + renames analyze_segment -> analyze_segmentation (file segmentation.py unchanged). Keeps edges/quantile binning; adds a per-segment bar chart (count, or the aggregate when value_column given) via the reserved 'charts' key. Registered DEFERRED via spec_from_model. DEV_PLAN §0.9 A13 -> done. Heads-up (Rifqi): analyze_segment renamed to analyze_segmentation. Local tool tests under tests/unit/tools (gitignored) referencing it break on pull — rename to analyze_segmentation. Also a stale '# analyze_segment' comment remains in src/agents/planner/validator.py:152 (your lane) — repoint when convenient. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>8e0301e -
[NOTICKET] feat(tools): analyze_data_quality — rename analyze_profile, activate + chart (A14)
Renames analyze_profile/quality.py -> analyze_data_quality/data_quality.py. Keeps the per-column profile; adds an issues summary (constant + high-null columns) and a missing-rate bar chart via the reserved 'charts' key. Registered DEFERRED via spec_from_model. Old name removed — no eval/agent refs; descriptive cross-ref repointed. DEV_PLAN §0.9 A14 -> done. Heads-up (Rifqi): analyze_profile is gone. Local tool tests under tests/unit/tools (gitignored) that reference it will break on pull — rename to analyze_data_quality. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>6be9611 -
[NOTICKET] feat(tools): analyze_pareto — rename analyze_contribution, Pareto 80/20 (A12)
Renames analyze_contribution/decomposition.py -> analyze_pareto/pareto.py. Keeps the share + cumulative compute; reframes to Pareto 80/20: adds categories_for_80pct and ships the pareto_chart (bars + cumulative-% line) via the reserved 'charts' key. Registered DEFERRED via spec_from_model. Old name fully removed — no eval/agent refs; 3 cross-ref DESCRIPTIONs (aggregation/comparison/segmentation) repointed. DEV_PLAN §0.9 A12 -> done. Heads-up (Rifqi): analyze_contribution is gone. Local tool tests under tests/unit/tools (gitignored) that reference it will break on pull — rename those refs to analyze_pareto. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>cb6a92e -
[NOTICKET] feat(report): embed analyze_* auto-charts, share one chart extractor (A21)
Rifqi Hafizuddin +1: Claude Opus 5Closes the gap Sofhia flagged at handoff: an analyze_* tool's auto-chart appeared under the chat answer and silently vanished from the downloaded report, because report/generator._collect_charts only ever looked at render_chart's chart-kind output and never at ToolOutput.meta["charts"]. Rather than copy the meta-charts logic into a second place, the extraction moved to one definition -- src/charts/extract.py::chart_envelopes -- now used by BOTH consumers: the per-turn ChartStore persistence and the report's ## EDA section. That is the actual fix; the missing charts were a symptom of having two implementations of "what charts does this output carry", only one of which was updated in A4. Also tightened, and this is a small behaviour change worth naming: an envelope must carry a `plotly` object to be emitted. The report path already required that; the persistence path accepted any dict, so a malformed envelope could be written as a message_charts row the frontend could not draw. Both pa…1c36d9a -
[NOTICKET] feat(slow-path): per-step SSE progress, bilingual (A22)
Rifqi Hafizuddin +1: Claude Opus 5A structured_flow turn took ~12s behind three stage pings -- "Planning", "Running N steps", "Composing". Past Miller's ~10s attention threshold what a waiting user needs is evidence that something is happening, not a faster answer, so the TaskRunner now announces each wave BEFORE running it. A 4-task plan emits ~6 status events, not 3. New src/agents/slow_path/progress.py holds the labels: - BILINGUAL. The three existing stage pings were English-only on a product whose traffic and eval set are majority Indonesian; they follow reply_language now, the same "id" if reply_language == "Indonesian" test refusals.py uses. - A bounded activity vocabulary keyed by tool ("retrieving data" / "mengambil data"), NOT the task's objective. Plan internals are deliberately English, so using objective as status text would show English prose to an Indonesian user. - Unknown tools fall back to "analysing" / "menganalisis", so the six tools still to land (DEV_PLAN 0.9) produce s…06aecc8 -
[NOTICKET] feat(tools): analyze_anomaly (Isolation Forest) — second ML tool, DEFERRED
analyze_anomaly: scores rows across numeric column_ids with a scikit-learn IsolationForest (fit offloaded via run_fit, fixed random_state), flags outliers, returns summary + flagged rows + a scatter with anomaly points highlighted (reserved 'charts' key). output_kind 'stats' (dict), mirroring analyze_forecast's shape. Registered DEFERRED via spec_from_model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>9f3267d
29 days ago
-
[NOTICKET] feat(tools): analyze_forecast (ETS) — first ML tool, DEFERRED
analyze_forecast: statsmodels ETS (Holt-Winters, additive trend, no AutoML), reusing analyze_trend's date parsing + resampling. Auto-chart (actual-vs-forecast line) via the reserved 'charts' key; deterministic output. Registered in _DISPATCH + registry (spec derived from ForecastInput via spec_builder) as DEFERRED — activation (ACTIVE + examples.py few-shot) follows once A0 scoping locks P1. Wires the ML executor for the first time: the fit is offloaded via run_fit, and the invoker now awaits coroutine compute fns (inspect.isawaitable — opt-in; sync tools unchanged). This resolves the conventions §4/§6 contradiction toward 'sync unless it fits a model'; doc §6 wording needs a follow-up tweak. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>dae102f -
[NOTICKET] planner: rebalance few-shot exemplars EN/ID, demo the reply-language rule
Rifqi Hafizuddin +1: Claude Opus 5A19. The exemplar set was 13/13 English while the eval dataset -- and real traffic -- are majority Indonesian (17 ID / 14 EN). In-context learning over-predicts whatever the examples show most (majority-label bias, Zhao et al. ICML 2021), so an all-English set teaches the planner that Indonesian questions are out-of-distribution. Converted four questions to Indonesian (now 9 EN / 4 ID), chosen so each conversion also breaks a near-duplicate: D breaks the 3x check_data -> retrieve_data -> analyze_aggregate stack (A/D/E) M stops reading as a near-duplicate of J's English bar-chart ask L R2b "berapa banyak ..." is the recipe most often asked in Indonesian H the infeasible example -- see below H now demonstrates the reply-language rule: an Indonesian question gets an Indonesian infeasible_reason while every other field stays English. prompt.py:151 has always mandated this ("If (and only if) you return an infeasible plan, write infeasible_reason in {reply_language}") but NO e…fb30aec -
feat(tools,spine): wire tool auto-charts to chart persistence
invoker: a compute fn may attach dataeyond.chart.v1 envelopes under a reserved 'charts' key; move them to ToolOutput.meta['charts'] (value stays clean). chat_handler: extend the chart save loop to persist meta['charts'] alongside render_chart's chart-kind output. NOTE: chat_handler.py is outside the tools area (CLAUDE.md) — needs principal review before merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>522e582 -
feat(tools): shared chart kit + auto-chart builders
Extract the house-style kit (colorway, layout preset, JSON-safe coercion, axis chrome) into analytics/charts.py as the single source of truth; render_chart now imports it (behaviour unchanged). Add dataeyond.chart.v1 builders: box, pareto, cluster_scatter, anomaly-highlight scatter. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>535f173 -
feat(tools): analytical-tools foundation — spec_builder + ml executor
spec_builder: derive a ToolSpec from a tool's pydantic InputModel (functional grain, no base class) — input_schema from model_json_schema(), 3-part house-style description helper. ml/executor: bounded, timed offload (run_fit) for sync sklearn/statsmodels fits, mirroring the db.py F-5 pool pattern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>ee1a76a
30 days ago
-
/fix parsing and term extract (#21)
- [NOTICKET] fix: retire the fuzzy clustering pass; add E1c post-cluster recall (2cc21c92688d662c5cfdfeeb6784bb85b98575ca) - [NOTICKET] feat: parsing schema 0.4.0 — asset pointers for figures and tables (e662ac089b741c90e19b9f51f4933fb97a1225bc) - [NOTICKET] docs: sync REPO_STATUS / DEV_PLAN / contracts for pipeline v2 (c3c184d83bec4a4d42e9ad3f7265560802817e5b) - [NOTICKET] docs: close K15 with the paid v1-vs-v2 comparison (62782fed9292f67d7b376783920a07197d6776e9) - [NOTICKET] fix: test the caption hypothesis; ship the signal at weight 0.0 (36a166e1844d186d6cfb8fff9ea070f3dc69b220) - [NOTICKET] fix: cap key-parameter pre-emption in the review queue (Step 2) (4e3a6aefef7024db3ea4b7946fd7771a6c269601) - [NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5) (f1d64b113db0289601806635a9b154201a6127fc) - [NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6) (cdb5fe82cc5f2893ccbb5e28282e8dc46a27e061) - [NOTICKET] fix: retire escalation…f07443e
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f
-
/fix planner count and report (#19)
- /fix planner count question (11de970ff65be9b4f81f773cfb355d3dc57358d7) - [noticket] fix report bug where some answered analysis isnt included in report summary answers (8dec1825cf803ad28f4ff0400b9c373250a7d90a) - [NOTICKET] fix: split report floor from report body; scope analysis reads by user_id (fd4865b20dfc46782706be9aab318f909b9903c3) - [NOTICKET] docs: sync report floor/body + security posture; add review remediation tracker (fc5200c994441ec505ea7a82469da577afe170a5) - [NOTICKET] test: planner eval harness (rule-compliance, --selfcheck/--rescore) (4292522aa3fa3a7af8f933307334cb7e5eb0b377) - [NOTICKET] eval: repair readiness harness; commit intent/help/readiness baselines (08a0a4ad5eacc6c90df22fab5be69f2d33579f29) - [NOTICKET] fix: compiler parity — F-17 fabricated column, F-18 LIKE nulls, F-25 empty in/not_in (8f736f90f1958960a83847d332b45b13fa346168) - [NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams (0cb7d53cb6f7f7e7b687dd6d9c27…f282b15
-
/fix report not working (#18)
- [NOTICKET] fix: write reports.user_id (Neon NOT NULL column broke report persistence) (13f74e11df6058800f71e539fe135eeba7576f24) - [NOTICKET] docs: correct 13f74e1 root cause — Go migrations are non-convergent (86d0805130637d66082c5a75b8da71fae36002ad)9070d67
-
/fix change model to gpt 5.4 mini (#17)
- [NOTICKET] chore: rename LLM env quad __4o -> __54m (GPT-5.4-mini) (790ef0c7aaac6e76eb01b0ef9a3398ff4249416d)d65c41d -
fix/ check and new chart tool (#16)
- check: table-level column drill + drop misleading "name a table" hint (ef0c7b1c5aa49a244b02f1b5eea15cc77dcf2bcb) - [KM-715] charts: render_chart tool + message_charts store + GET /api/v1/charts + planner viz slice (87cfcf884d923ee4bcd722e0e4ef4d90d7eaedca) - [KM-715] checkpoint: S1a deterministic quality checkpoint between runner and assembler (f3c8de34dc19a573031b67fd4f48dcfbadcdfed8) - [KM-715] docs: sync REPO_STATUS/DEV_PLAN + land SPINE_V2_PLAN (W1/W2 done) (434eacbf2ac8324188ebe5e32775dbc9d2a3f49a) - planner: value-handoff (${tN.col} → list) for tabular lookup/anti-join (54070ac1407b751000e49dfe7f102a2be5ab1ff8) - [KM-715] charts: GET /charts by message_id only + tri-state status marker (b8d15761addf4d499c9631df64e7693aa1d61fa2) - [KM-715] report: embed charts as plotly EDA fences; render_chart counts toward the floor (89db8be6b6f11e86ad0776d2ee07f2306e3b6079) - [KM-715] docs: contract + status deltas for the charts lead-review round (96347ee7c55a4eb39a906409ec214463e25c5e35) - …5a60e93
-
/fix traceability (#15)
- [KM-691] Traceability: add resolved data_used layer for the FE (a646735e33f6ac725d0756a88956b0aec15fe495) - check: don't leak user-scope catalog when analysis catalog missing (c0ba82938383169b41059c4125765db5275885e7) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>32abc41 -
/fix help and tools analyze (#14)
- [KM-711] Retire /report slash command; report is Generate-button only (f7f7b172da0dc16adac0a7555f6d34a704cc2050) - fix(analyze_trend): handle value_column == date_column (count events per period) (55e3327f7b2e21a954b317e143f46bcea8b5ada7) - [KM-711] Docs: drop /report from list_skills catalog examples (0772863b296e2ed3e2d559df05e98540764e58b0) - fix(planner): reject inlined table-ref as analyze_* data (Pattern A) (776fae3c7c6e3d3a07c22bef10898f6b1c33c701) - fix(analyze_aggregate): reject non-string group_by (CASE/binning) keys (b40fa76e2fa68572dad0d8532e07076207095d6b) - fix(planner): reply-language for infeasible_reason + answer-only-asked rule (4d89d7d93938e41892979cd20e5ccf0c97441dac) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>17e31c6
-
/fix planner and report (#13)
- [KM-644] planner: infeasible plan outcome + entity-ranking few-shots (089b69d9b6111e813307c6afbd3e7613bb823f9d) - [KM-644] harden query path: grouped-select rule, int-date trends, sample decode, trace guard (74ad5261736bda0bd72bccdb41565ea67313993a) - [KM-703] add analyze_merge tool: two-table combine (analytics.combine) (8abf635532366d670367557a3ffa03d42371d831) - [KM-567] planner support for analyze_merge: data_right guard + two-retrieve merge few-shot (a029a843247e6ea4d3c79a4e2c1b7571ba4cda69) - [KM-644] report v2: BQ answers, unresolved/excluded sections, evidence tables + curation/readiness endpoints (4ceb058d85c5a2a289443b9411d2b345027fa226) - [NOTICKET] refactor: mint message_id as UUID to mirror analyses_messages id shape (4253733b6b8db0449a82442a915c732ddd55f528) - [KM-644] report: drop Notes/Unresolved/Method sections from rendered markdown (compact report) (eef6de811a139402c79fd74ad4adfc00cf7aff01) - /edit md file (8198cc188ff05a2798ae2056b90b6e56e1bea87b) Co-authored-by…49b0848
-
fix/ check (#12)
- [NOTICKET] check: scope to analysis catalog + list database tables (db2a10f61e1f6ff9579deaaeafed8fdf0cdf5bb7) - feat(planner/help): graceful degradation for gated analytics capabilities (efc0af7c05308acb564bffdd7d8ab8abd2dd2e81) - Scope structured_flow + report to analysis catalog; drop dead data_sources binding (c333ec84a96b171484af7a87bdf07fd1fb0e9c69) - update md file (c5e2e1b1adcba46b56eaf6bbb85d9b8bb0a0d277) - [NOTICKET] check: read documents from analysis-scope catalog (6977394de88d7944dd2504beeed3e00eafe7857a) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>b9dfc76
-
/feat traceability (#11)
- [NOTICKET] Fix check skill: reply language + tabular inventory routing (1367f41269cf9ef42fd91f311b24d4edf5885c7b) - [KM-691] Develop traceability (23cc2079989fef931a6e146980fe6bf528ee6e92) - [NOTICKET] Humanize check output: hide UUIDs, add lead-in, list format (2bdda8fc3fc4b4e17b4df7fb53aee5abede45add) - [NOTICKET] check: schema-detail view for column/type questions (9e248d9cc5688d6ee622be523a6bedddf5010995) - [KM-691] Make sources trace-authoritative; drop stream sources (b43ecc77f26c1537cbbfddb6800ec4ff62d16441) - [KM-691] Bound traceability payload + reject contradictory IR filter ranges (db6db73d6ab0c2175447946742038443d67b2f78) - /fix planner can now plan quarter comparison (839e32c8d6d5895029072074ab7aba8bb4fcdb6e) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>f873f92
-
/fix validator and report (#10)
- /fix report: remove record_id hash citations, EDA (for future tool) change to data source, transform to mdx-safe (b1b101730ccf5628e43e80f2ad654f78fbed5207) - [NOTICKET] fix(planner): repair near-miss catalog ids in QueryIR before validation (aacd4d13def258a9815761aecbd7b246a8dae9fc) - /fix report: group findings by analysis, dedup, code-span-safe MDX escaping (029b48894e9f79af6c0d81c00a2d811a982f87ae) - [NOTICKET] fix(tools): log swallowed analytics tool errors in AnalyticsToolInvoker (93d9381b2a5f5efe058c913fe0b793ce994a9097) - fix(planner): harden plan validation to reduce retry thrash on structured questions (1619dd3c0fa74825010c7af6d4b375e3b9404847) - [NOTICKET] fix(tools): coerce numeric-string columns before analyze_* compute (6bda333866a1a8ccf1b3fe6fcee7a8e1832f28d7) - fix(planner): correct top-N guidance — in-query ranking IS supported (99aa4141aed8241a625d290fb170f74192a6d831) - feat(guardrails): add input guard, out_of_scope intent, and refusal handling (0bbc27bc40ba20e7a8e…0e5fdb5
-
/fix analysis + report_inputs and storage update (#9)
- fix(chat): drop ENABLE_SLOW_PATH flag + make chat persistence read-only (b8b76c4fb8db21de183cc6f80c0acba0d35bdad1) - [NOTICKET] feat(storage): read tabular Parquet from Supabase S3 via provider toggle (3f48226adaf2ed107a4eb2c0373cbb73fc62f552) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>079782d -
/fix check and help tool (#8)
- /fix data_catalog model + CatalogStore to dedorch schema (data → catalog_payload) (09857fc52486354f53e752e3ea329b064a9e4c4b) - [NOTICKET] fix(help): lock reply language to user's language on button path (cb7f5ac484ad4a510e68faf5291091958b6d2f42) - /fix default SKIP_INIT_DB=true for dedorch cutover (skip init_db on Go-owned schema) (781b93c6b8744209a1101629b5170175e9212592) - /fix surface catalog foreign keys to the planner (FK-backed joins) (446c57524e932fcdbb0ddecc049189c443b6ae97) - [NOTICKET] feat(eval): add help-skill eval harness + goal-based language fallback (cbdbefb7ab0ffa36949713f18422213db480a959) - [NOTICKET] fix(eval): update readiness eval to post-KM-652 floor (drop problem_validated) (6f46e81bdeb1eb987f9dac8af0e22a975474cbb1) - /fix infer FKs for FK-less dedorch catalog + log repr(e) in db executor (3669290474583ae011f5ed5e50b94dad146fbc77) - Merge eval/help + readiness eval fix into PR #8 (5f4818a8e38f93988ab5756f3a06c94645826b15) - /docs REPO_STATUS: FK inference (FK-…3743cfe