feat(planner/help): graceful degradation for gated analytics capabilities

When a question needs a gated/absent capability (e.g. a significance test →
analyze_comparison), the system used to set the user up to fail: the Help skill
would suggest the question, the planner would fall back to an unsupported
aggregation (std) and fail at runtime, and the Assembler would surrender with a
leaked "technical failure" message. Defense in depth across four layers:

- help.md v5: add a Capability boundary — Help only suggests analyses the live
tools can deliver (descriptive, group-by, correlation, trend); never
significance/forecasting/causal/clustering/share-of-total.
- aggregation.py: analyze_aggregate description bans std/var and adds an on-axis
graceful-degrade rule (degrade to grouped mean; never fake with std; never
jump to an unrelated tool).
- planner/validator.py: new check 8c validates aggregation FUNCTION values
against SUPPORTED_AGGS, so an unsupported func (std) is rejected at validation
with a corrective hint — the retry self-corrects before runtime instead of
failing silently.
- assembler.md rule #2: lead with successful results, describe limits in
business language, never leak the internal failure cause.

Verified per-component (validator rejects std/var, passes mean). Not yet tested
E2E live (needs Azure). Follow-up: make Help capability-aware from the registry
instead of the static boundary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

src/agents/planner/validator.py

@@ -19,6 +19,7 @@ from ...catalog.models import Catalog
 from ...query.ir.models import QueryIR
 from ...query.ir.repair import IRRepairer
 from ...query.ir.validator import IRValidationError, IRValidator
+from ...tools.analytics.aggregation import SUPPORTED_AGGS
 from .contracts import ToolRegistry
 from .errors import PlannerValidationError
 from .inputs import Constraints
@@ -105,6 +106,33 @@ class PlannerValidator:
                         f"{sorted(unknown)} (allowed: {sorted(allowed)})"
                     )
 
+                # Check 8c — analyze_aggregate: every aggregation FUNCTION must be one
+                # the tool supports. Check 8a only validates arg *names* (`aggregations`
+                # is allowed); it never looks at the function *values* inside the dict,
+                # so an unsupported func like `std` otherwise passes validation and only
+                # fails at execution — too late for a corrective retry, so the task
+                # reaches the Assembler as a silent failure. Catch it here so the planner
+                # is re-prompted to degrade to a supported function (e.g. `mean`).
+                if call.tool == "analyze_aggregate":
+                    aggs = call.args.get("aggregations")
+                    if isinstance(aggs, dict):
+                        bad = sorted(
+                            {
+                                f
+                                for funcs in aggs.values()
+                                for f in ([funcs] if isinstance(funcs, str) else funcs or [])
+                                if f not in SUPPORTED_AGGS
+                            }
+                        )
+                        if bad:
+                            raise PlannerValidationError(
+                                f"task {task.id}: analyze_aggregate has unsupported "
+                                f"aggregation function(s) {bad} (supported: "
+                                f"{sorted(SUPPORTED_AGGS)}). Use a supported function "
+                                "(e.g. mean/median); for the spread of a whole column "
+                                "use analyze_descriptive instead."
+                            )
+
                 # Check 3 — concrete source_id args must exist in the catalog.
                 src = call.args.get("source_id")
                 if isinstance(src, str) and not _is_placeholder(src):

src/config/prompts/assembler.md

@@ -25,9 +25,17 @@ You produce two things in one structured object:
    and values present in the task results. **Never invent, estimate, or extrapolate
    a number** that is not in the results. If the data does not answer part of the
    question, say so.
-2. **Report what failed.** Some tasks may have `status: partial` or `failure`. Do
-   not pretend they succeeded. Briefly state what could not be completed and how it
-   limits the answer; put unresolved items in `open_questions`.
+2. **Report what failed — in plain terms, and still answer.** Some tasks may have
+   `status: partial` or `failure`. Do not pretend they succeeded — but do not lead with
+   the failure either. **First** give the most useful answer the SUCCESSFUL tasks
+   support; **then** state, in business language, what could not be determined and how
+   it limits the answer; put unresolved items in `open_questions`. **Never expose the
+   internal cause** of a failure — no "the tool failed", "could not compute", "technical
+   error", task ids, or function/tool names. Describe the limit by what it *means for the
+   reader*, not by what broke internally. E.g. write "a formal significance test was not
+   run, so this shows the difference in averages but not whether it is statistically
+   significant" — NOT "the calculation of the score distribution failed". A narrower,
+   honest answer beats an apology.
 3. **Render, don't recompute.** Build markdown tables from the structured task
    outputs as they are. Do not do your own arithmetic beyond trivially restating a
    value already computed.

src/config/prompts/help.md

@@ -1,4 +1,9 @@
-<!-- help.md · v4 · Help skill prompt.
+<!-- help.md · v5 · Help skill prompt.
+     v5 (2026-07-07): added the "Capability boundary" section — Help now only suggests
+     analyses the live tools can actually deliver (descriptive, group-by, correlation, trend)
+     and must NOT suggest significance tests, forecasting/modeling, causal claims, clustering/
+     segmentation, or share-of-total. Fixes Help recommending a statistical-significance
+     question the system has no tool for (the user copy-pasted the suggestion → dead end).
      v4 (2026-07-03): reply language relaxed from hard-"only" to DEFAULT + explicit-request
      exception — an explicit user request ("jawab dalam bahasa Inggris") now overrides the
      detected [Reply language]; anti-drift default (incl. synthetic-trigger protection) is
@@ -75,6 +80,35 @@ Do not over-promise the report's depth.
 > chat skill to fix it; gently suggest they set the objective + business questions in the New
 > Analysis form.
 
+## Capability boundary — only suggest analyses the tools can actually do
+
+Every question you propose must be answerable by the system's live analysis capabilities.
+Suggesting an analysis the tools cannot perform sets the user up to fail: they copy your
+suggestion, ask it, and hit a dead end. Stay strictly inside this list.
+
+**You MAY suggest** (supported):
+- Descriptive summaries of a column — average, median, spread, min/max, distribution.
+- Group-by breakdowns — a total, average, or count of a metric **per category**. This is also
+  how to compare groups (e.g. "the average retention for online vs offline").
+- Correlation / relationship between numeric columns ("which factors relate to exam score?").
+- Trends over time.
+- Inventory — what data / tables / documents exist.
+
+**You must NOT suggest** (no tool exists — do not propose these even when they fit the goal):
+- **Statistical significance / hypothesis tests** — never use "significant", "statistically
+  significant", "significant difference", t-test, ANOVA, or p-value. To compare groups, suggest
+  a group-by average that shows the gap (e.g. "compare the average retention of online vs
+  offline"), NOT "is the difference significant".
+- Predictive modeling, forecasting, or regression models.
+- Causal claims — avoid "cause", "impact of", "effect of" framed as causation; keep it to
+  relationship / correlation.
+- Clustering or segmentation into discovered groups.
+- Share-of-total / contribution breakdowns.
+
+When a goal naturally invites a forbidden analysis (e.g. the user wants to know if a gap is
+"real"), degrade the suggestion to the nearest supported one — the group-by average that shows
+the gap — rather than promising the unsupported analysis.
+
 ## How-to phrasing (degrade gracefully)
 
 - **Via chat / skills** — write these **accurately and specifically**; they are stable (e.g. "type your question in the chat", "run `/report`").
@@ -102,6 +136,7 @@ spam, no overselling. A few sentences is usually enough.
 - Never suggest an action that the signals say isn't available or isn't ready.
 - One step at a time — give the next step, not the whole roadmap.
 - When you suggest questions, **dedupe against `chat_history`** — only propose analyses not yet run that move the goal forward; a question that already has an answer adds no fresh evidence.
+- **Stay inside the Capability boundary above** — never propose significance tests, forecasting, modeling, causal claims, clustering/segmentation, or share-of-total; there is no tool for them, so suggesting them sends the user into a dead end.
 - No markdown headers or code fences in your reply; short prose (and an inline `/command` or a tiny bullet list) is fine.
 
 ## Examples

src/tools/analytics/aggregation.py

@@ -44,8 +44,12 @@ def _clean(value: object) -> object:
 # Final destination is ToolSpec.description once the wrapper layer is built.
 DESCRIPTION = """\
 Summary: Group-by aggregation. Splits rows by one or more key columns and \
-computes aggregates per group (sum, mean, count, min, max, median, nunique). \
-Returns one row per group.
+computes aggregates per group. Returns one row per group.
+
+SUPPORTED FUNCTIONS — use ONLY these: sum, mean, count, min, max, median, \
+nunique. Standard deviation / variance are NOT available here: for the spread of \
+a whole column use analyze_descriptive (whole-column only — spread PER GROUP is \
+not available in v1). NEVER pass std / var / stdev to this tool; it will fail.
 
 USE WHEN the question groups a metric by a category — the tell-tale sign is \
 "per"/"each"/"by" a dimension. Trigger words: "per/each" (per/tiap), "by" \
@@ -57,6 +61,15 @@ DON'T USE WHEN:
   - it splits a single total into shares -> analyze_contribution
   - the grouping is over time periods -> analyze_trend
 
+GRACEFUL DEGRADE — if a tool named above (e.g. analyze_comparison) is NOT present \
+in the "Available tools" list, do NOT fake it with an unsupported aggregation \
+(like std) and do NOT abandon the task. Degrade to an answer that stays on the \
+SAME question: group the measure by the group column with `mean` (optionally \
+min/max/median) so each group's level and their difference are still shown. Do \
+NOT switch to an unrelated tool just to produce output. The Assembler will note \
+that a significance test / per-group spread was not computed — a narrower, true \
+answer beats a failed one.
+
 Example questions:
   - "total revenue per region"
   - "average order value by customer segment"