[KM-652] refactor: remove problem_statement skill + problem_validated gate

2026-06-24 pivot: the analysis goal is now two user-entered fields (objective + business_questions) captured at onboarding with no agent validation, so the problem_statement skill and the problem_validated gate are removed (comment, don't delete).

- Router: drop problem_statement from the Intent literal + intent_router.md (6->5 intents); comment the dispatch branch in chat_handler; remove /problem-statement from list_tools. problem_statement.py kept intact but unwired.
- Gate: gate() neutered to a no-op; call site commented in chat_handler; enable_gate marked deprecated. report_floor drops the problem_validated precondition -> floor is now >=1 completed analysis only.
- Docs: REPO_STATUS.md + DEV_PLAN.md updated.
- Tests (local, gitignored): test_gate / test_chat_handler / test_readiness updated. Full suite 284 passed, 7 skipped; ruff clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DEV_PLAN.md

@@ -82,8 +82,8 @@ Status legend: ⬜ not started · 🔄 in progress · ✅ done · ⛔ blocked ·
 
 | # | Task | Owner | Status | Note |
 |---|---|---|---|---|
-| 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | ⬜ | Do with #2 — 4 places name it |
-| 2 | Drop `problem_validated`: gate → identity/presence-check (or remove); `is_report_ready` → presence(`objective` + ≥1 `business_question`) + ≥1 analysis, no-LLM | Rifqi | ⬜ | Merges Sofhia 2+3. Make reads tolerant (getattr) |
+| 1 | Comment out `problem_statement` skill **+ `Intent` literal + router prompt + gate redirect target**; remove `/problem-statement` from `list_tools` | Rifqi | ✅ | Done 2026-06-25 (one commit w/ #2). Unwired in `orchestration.py`, `intent_router.md`, `chat_handler.py`, `tools.py`; `problem_statement.py` kept intact |
+| 2 | Drop `problem_validated`: gate neutered; `is_report_ready`/`report_floor` → **≥1 completed analysis** only, no-LLM | Rifqi | ✅ | Done 2026-06-25. `gate.py` no-op, gate call site commented in `chat_handler.py`, `report_floor` drops the goal check. Tests updated (`test_gate`/`test_chat_handler`/`test_readiness`). Suite: **284 passed, 7 skipped**; ruff clean |
 | 3 | dedorch `analysis` migration: drop `problem_statement`/`problem_validated`, add `objective` + `business_questions`, rename `owner_id`→`user_id` | Harry | ⬜ | Blocks #4 |
 | 4 | Update Python `analysis` model + `state_store` + `analysis.py` to match #3; `owner_id`→`user_id` | Rifqi/Sofhia | ⛔ after #3 | getattr-tolerant so deploy order isn't a hard blocker |
 | 5 | Report generator → `objective`+`business_questions`, "generated by {user}", formal outline | Sofhia | 🔄 pushed (KM-674) | Goal-shape done. Open: author name source (#19); background/EDA/insights mapping |
@@ -101,6 +101,7 @@ Status legend: ⬜ not started · 🔄 in progress · ✅ done · ⛔ blocked ·
 | 17 | Reconcile report contract with records-based: remove/flag `chat_history` | Sofhia/Rifqi | ⬜ new | Resolves #5 ↔ #6 contradiction |
 | 18 | Confirm Go owns ALL analysis-state writes + both creation gates; make Python `state_store.ensure` read-only | Rifqi + Harry | ⬜ new | Extends #7 to the full decision #4 (G5) |
 | 19 | Decide report author display-name source (`users.fullname` vs Go-passed name) | Sofhia | ⬜ new | Raw `user_id` in a formal report looks unprofessional |
+| 20 | **Help handoff:** update `handlers/help.py` + `help.md` — drop the `problem_validated` tier + `define_problem_statement` action (the skill it points at is gone as of #1) | Sofhia | ⬜ new | Created by #1/#2 (2026-06-25). Help currently still steers users to a removed skill |
 
 ## 5. Critical path & sequencing
 

REPO_STATUS.md

@@ -1,8 +1,8 @@
 # Data Eyond — Python Agentic Service: Current Status
 
 **Audience:** teammates onboarding onto the Python repo (`Agentic-Service-Data-Eyond-Catalog`).
-**Scope:** what the code does **right now** (branch `pr/3`, ticket KM-652). Describes current state only — no roadmap or to-dos.
-**Snapshot date:** 2026-06-24.
+**Scope:** what the code does **right now** (branch `pr/4`, ticket KM-652). Describes current state only — no roadmap or to-dos.
+**Snapshot date:** 2026-06-25.
 
 > This file is grounded in the source, not the older design docs. Where the two
 > disagree, the code wins — see [§11 Doc-vs-code](#11-where-the-older-docs-are-stale).
@@ -68,19 +68,21 @@ POST /chat/stream { user_id, room_id, message }
   └─ ChatHandler.handle:
        1. classify → RouterDecision               [1 GPT-4o call]
        2. ensure analysis-state row (get-or-create, idempotent)
-       3. gate (if ENABLE_GATE): structured_flow + not validated → problem_statement
-       4. emit `intent` (internal; gates caching), then dispatch:
+       3. emit `intent` (internal; gates caching), then dispatch:
             chat              → ChatbotAgent → SSE
             help              → HelpAgent (state + history + readiness) → SSE
             check             → check_data/check_knowledge tool → rendered table  [no LLM]
-            problem_statement → PS draft + validate → write state
             unstructured_flow → DocumentRetriever (PGVector RAG) → ChatbotAgent → SSE
             structured_flow   → CatalogReader → (slow path | QueryService) → SSE
-       5. SSE events: intent (internal), sources, chunk, status, done | error
+       4. SSE events: intent (internal), sources, chunk, status, done | error
 ```
 
 Only the `chat` intent is cached (stateless). Messages persist on `done`.
 
+> The router emits **5 intents** now. The `problem_statement` skill and the `problem_validated`
+> gate were removed 2026-06-25 (KM-652) — the analysis goal is two user-entered fields
+> (`objective` + `business_questions`) captured at onboarding, with no agent validation.
+
 ---
 
 ## 5. Report lifecycle
@@ -90,7 +92,7 @@ The report is a **dedicated API, not a chat route** (`src/api/v1/report.py`):
 ```
 POST /report?analysis_id&user_id
   ├─ load analysis state; enforce the report FLOOR
-  │     (validated goal + ≥1 substantive analyze_* success) → else 409
+  │     (≥1 substantive analyze_* success) → else 409
   ├─ ReportGenerator.generate (src/agents/report/generator.py):
   │     read persisted AnalysisRecords (list_for_analysis)
   │     deterministically assemble findings / caveats / open-questions /
@@ -115,9 +117,8 @@ Two facts to internalise:
 
 ## 6. Feature list (what's built)
 
-- **6-intent handler router** with history-aware query rewriting (EN/ID).
-- **Deterministic gate** — `problem_validated` precondition for analytical queries.
-- **Skills:** `help` (LLM, state-aware next-step guidance), `check` (no-LLM data/document inventory), `problem_statement` (LLM drafter + completeness validator).
+- **5-intent handler router** (`chat`/`help`/`check`/`unstructured_flow`/`structured_flow`) with history-aware query rewriting (EN/ID).
+- **Skills:** `help` (LLM, state-aware next-step guidance), `check` (no-LLM data/document inventory). *(The `problem_statement` skill and the `problem_validated` gate were removed 2026-06-25 — KM-652; `gate.py` kept as a no-op seam, `problem_statement.py` kept but unwired.)*
 - **Slow analytical path:** Planner → TaskRunner → Assembler (static plan, degrade-and-continue, 3 LLM calls fixed).
 - **Structured query engine:** catalog-driven JSON IR → deterministic SQL/pandas compiler → read-only executor, with **single-level FK joins** (DB sources only).
 - **Unstructured RAG** over PGVector.
@@ -132,7 +133,7 @@ Two facts to internalise:
 
 | Endpoint | Purpose | Caller |
 |---|---|---|
-| `POST /chat/stream` | Main chat SSE (router → gate → dispatch) | FE → Go → Python (the only FE→Python call today) |
+| `POST /chat/stream` | Main chat SSE (router → dispatch) | FE → Go → Python (the only FE→Python call today) |
 | `DELETE /chat/cache` · `/chat/cache/room/{id}` · `/retrieval/cache/{user_id}` | Cache management | internal / ops |
 | `POST /analysis/create` · `GET /analysis` · `GET /analysis/{id}` | Analysis-session CRUD (state + room + bindings created atomically) | intended FE → Go |
 | `POST /report` · `GET /report/{id}` · `GET /report/{id}/{ver}` | Report generate / list / fetch | FE → Go (report button) |
@@ -171,16 +172,18 @@ Joins are single-level equi-joins to a related table **in the same source**, FK-
 
 ### Router — `src/agents/orchestration.py`
 One GPT-4o structured-output call → `RouterDecision{intent, rewritten_query, confidence}`,
-`intent ∈ {chat, help, problem_statement, check, unstructured_flow, structured_flow}`. It's a
+`intent ∈ {chat, help, check, unstructured_flow, structured_flow}` (`problem_statement` removed
+2026-06-25). It's a
 *handler* classifier: `structured_flow` = slow path, `unstructured_flow` = fast RAG; the
 data-modality mix on the slow path is the Planner's job. Prompt: `src/config/prompts/intent_router.md`.
 
 ### Gate — `src/agents/gate.py`
-Pure function, no I/O. One rule: `structured_flow` + `not problem_validated` → redirect to
-`problem_statement`; everything else passes. Fails **closed** (not-validated stub) on a missing
-row / read error. `AnalysisState` is the locked contract (id, analysis_title, problem_statement,
-problem_validated, owner_id, report_id, created_at, updated_at). Applied in `chat_handler.handle`
-only when `ENABLE_GATE` and `analysis_id` are present.
+**Neutered 2026-06-25 (KM-652):** `gate()` now passes every intent through unchanged — the
+`problem_validated` redirect was removed (the goal is user-entered, no agent validation). The
+function + `AnalysisState` contract are kept as a no-op seam; the call site in
+`chat_handler.handle` is commented out. `AnalysisState` still carries (id, analysis_title,
+problem_statement, problem_validated, owner_id, report_id, created_at, updated_at) until the
+dedorch state migration (#3/#4) renames it.
 
 ### Skills — `src/agents/handlers/`
 - `help.py` — LLM (streamed). A consistency guard derives the *allowed* actions from state
@@ -188,9 +191,8 @@ only when `ENABLE_GATE` and `analysis_id` are present.
   isn't validated or there's nothing to report. Consumes a deterministic readiness signal.
 - `check.py` — **no LLM.** Keyword cues route to `check_data`, `check_knowledge`, or both
   (helicopter view, concurrent). Renders tool tables to markdown.
-- `problem_statement.py` — LLM drafts `{problem_statement, objective, metric, missing[], feedback}`;
-  valid iff a statement exists and `missing` is empty (a bare data question stays incomplete). On a
-  valid draft, writes `problem_statement` + `problem_validated=True`.
+- `problem_statement.py` — **unwired 2026-06-25** (no longer routed to; file kept intact). Was an
+  LLM drafter that validated a goal and wrote `problem_validated`.
 
 ### Slow path — `src/agents/slow_path/` + `src/agents/planner/`
 - **Planner** (`planner/service.py`) — 1 LLM call → `TaskList` (DAG of tool-call chains). 8-check
@@ -231,8 +233,9 @@ real `analytics_registry()`.
 ### Report — `src/agents/report/`
 `generator.py` reads records, deterministically assembles structured fields, 1 LLM call for the
 executive summary; `store.py` versions under an advisory lock and persists markdown to dedorch
-`reports`; `readiness.py` defines the **report floor** (validated goal + ≥1 successful `analyze_*`)
-shared by the report API and the Help readiness signal so the two can't disagree.
+`reports`; `readiness.py` defines the **report floor** (≥1 successful `analyze_*`; the
+`problem_validated` precondition was dropped 2026-06-25) shared by the report API and the Help
+readiness signal so the two can't disagree.
 
 ### Observability — Langfuse
 The endpoint's `ChatHandler` runs with `enable_tracing=True`. One trace per request groups
@@ -247,7 +250,7 @@ only.
 | Flag | Where | Default | Effect |
 |---|---|---|---|
 | `ENABLE_SLOW_PATH` | `settings.enable_slow_path` | **off** | Route `structured_flow` through Planner/TaskRunner/Assembler (vs single-query `QueryService`). Records persist only on the slow path → reports require this on. |
-| `ENABLE_GATE` | `settings.enable_gate` | **off** | Apply the `problem_validated` redirect. Off because legacy rooms have no state row. |
+| `ENABLE_GATE` | `settings.enable_gate` | **off** | **Deprecated 2026-06-25** — gate neutered; the flag has no effect. Kept to avoid `.env` churn. |
 | `SKIP_INIT_DB` | env, `main.py` | off | Skip `create_all` on startup — the dedorch cutover switch (Go owns dedorch migrations). |
 | `enable_tracing` | hardcoded `True` in `chat.py` | on (endpoint) | Langfuse tracing. |
 
@@ -260,7 +263,7 @@ copies disagree with the current code on:
 
 | Topic | Old docs | Current code |
 |---|---|---|
-| Router | 3-way `source_hint` (chat/unstructured/structured) | Flat **6-intent** `RouterDecision` |
+| Router | 3-way `source_hint` (chat/unstructured/structured) | Flat **5-intent** `RouterDecision` (was 6; `problem_statement` removed 2026-06-25) |
 | Joins in IR | "single-table only; deferred" | **Single-level FK-backed joins** (DB sources only) |
 | Analysis / report / gate / slow path | "Phase 2 spine only" | All built and present |
 | `analysis_id` | open question | resolved: **`analysis_id == room_id`** |

src/agents/chat_handler.py

@@ -9,8 +9,10 @@ End-to-end flow per user message:
        - `unstructured_flow` → DocumentRetriever (RAG over PGVector) →
                                list[DocumentChunk].
        - `check`             → check_data / check_knowledge tool → rendered table.
-       - `problem_statement` → PS skill: draft + validate → write analysis state.
        - `help`              → Help skill: analysis state + history → streamed guidance.
+
+  (`problem_statement` was removed 2026-06-24 — the goal is now user-entered
+  `objective` + `business_questions` captured at onboarding, with no agent skill.)
   3. `ChatbotAgent.astream` → yield text tokens.
   4. Wrap each step into an SSE-style event dict so the API endpoint can
      stream them as Server-Sent Events.
@@ -39,7 +41,9 @@ from src.retrieval.base import RetrievalResult
 from .chatbot import ChatbotAgent, DocumentChunk
 from .handlers.check import run_check
 from .handlers.help import HelpAgent
-from .handlers.problem_statement import ProblemStatementAgent, run_problem_statement
+# `run_problem_statement` unwired 2026-06-24 (problem_statement removed from the router).
+# `ProblemStatementAgent` kept — still referenced by the constructor + _get_ps_agent.
+from .handlers.problem_statement import ProblemStatementAgent
 from .orchestration import OrchestratorAgent
 
 if TYPE_CHECKING:
@@ -114,8 +118,8 @@ class ChatHandler:
         # `#10` data-source binding: scopes structured_flow's catalog to the sources
         # the analysis is bound to. Injectable for tests; fail-open when absent.
         self._binding_store = binding_store
-        # Deterministic gate: redirect structured_flow -> problem_statement until the
-        # analysis is validated. OFF by default (legacy rooms have no state row).
+        # Deterministic gate — DEPRECATED 2026-06-24 (problem_validated gate removed).
+        # Unused flag; the gate call site in handle() is commented out.
         self._enable_gate = enable_gate
 
     # ------------------------------------------------------------------
@@ -244,9 +248,8 @@ class ChatHandler:
 
         intent = decision.intent
         # ---- 1a. Ensure session state row (T-A) ----------------------
-        # Rooms created via /room/create have no `analysis_states` row. Without one
-        # the gate redirect-loops and problem_statement / report_id writes silently
-        # no-op. Lazily get-or-create it (idempotent) so any session is gate-ready.
+        # Rooms created via /room/create have no `analysis` row. Without one, Help and
+        # the report_id write-back silently no-op. Lazily get-or-create it (idempotent).
         analysis_state: AnalysisState | None = None
         if analysis_id:
             try:
@@ -256,18 +259,20 @@ class ChatHandler:
                     "analysis state ensure failed", analysis_id=analysis_id, error=str(e)
                 )
 
-        # ---- 1b. Gate (deterministic, post-router) -------------------
-        # Redirect structured_flow -> problem_statement until the analysis is
-        # validated. Fails closed (not-validated) when the state row is unavailable.
-        if self._enable_gate and analysis_id:
-            from .gate import gate, stub_analysis_state
-
-            intent = gate(
-                intent,
-                analysis_state
-                if analysis_state is not None
-                else stub_analysis_state(problem_validated=False),
-            )
+        # ---- 1b. Gate (REMOVED 2026-06-24) ---------------------------
+        # The problem_validated gate was dropped: structured_flow is no longer
+        # redirected to problem_statement (the goal is now user-entered objective +
+        # business_questions, no agent validation). `gate()` is neutered to a no-op; the
+        # call site is left commented for restorability.
+        # if self._enable_gate and analysis_id:
+        #     from .gate import gate, stub_analysis_state
+        #
+        #     intent = gate(
+        #         intent,
+        #         analysis_state
+        #         if analysis_state is not None
+        #         else stub_analysis_state(problem_validated=False),
+        #     )
 
         # The `intent` event is consumed by the endpoint (it gates response caching
         # on the effective intent) and is NOT forwarded to the frontend. We emit the
@@ -337,22 +342,24 @@ class ChatHandler:
             yield {"event": "chunk", "data": text}
             yield {"event": "done", "data": ""}
             return
-        elif intent == "problem_statement":
-            try:
-                text = await run_problem_statement(
-                    message,
-                    analysis_id,
-                    agent=self._get_ps_agent(),
-                    store=self._get_state_store(),
-                    history=history,
-                )
-            except Exception as e:
-                logger.error("problem_statement route failed", user_id=user_id, error=str(e))
-                yield {"event": "error", "data": f"Problem statement failed: {e}"}
-                return
-            yield {"event": "chunk", "data": text}
-            yield {"event": "done", "data": ""}
-            return
+        # problem_statement dispatch removed 2026-06-24 (skill unwired; intent no longer
+        # emitted by the router). Branch kept commented for restorability.
+        # elif intent == "problem_statement":
+        #     try:
+        #         text = await run_problem_statement(
+        #             message,
+        #             analysis_id,
+        #             agent=self._get_ps_agent(),
+        #             store=self._get_state_store(),
+        #             history=history,
+        #         )
+        #     except Exception as e:
+        #         logger.error("problem_statement route failed", user_id=user_id, error=str(e))
+        #         yield {"event": "error", "data": f"Problem statement failed: {e}"}
+        #         return
+        #     yield {"event": "chunk", "data": text}
+        #     yield {"event": "done", "data": ""}
+        #     return
         elif intent == "help":
             try:
                 state = analysis_state or await self._load_analysis_state(analysis_id)

src/agents/gate.py

@@ -47,19 +47,22 @@ class AnalysisState(BaseModel):
 
 
 def gate(intent: Intent, state: AnalysisState) -> Intent:
-    """Return the effective intent after applying the deterministic gate policy.
+    """Return the effective intent (NEUTERED 2026-06-24 — passes everything through).
 
-    `structured_flow` requires `problem_validated is True`; otherwise redirect to
-    `problem_statement`. All other intents pass through unchanged.
+    The `problem_validated` gate was removed: analysis is no longer gated on a validated
+    problem statement (the goal is now two user-entered fields, `objective` +
+    `business_questions`, captured at onboarding with no agent validation). Kept as a
+    no-op seam so gating can be restored without re-threading call sites.
     """
-    if intent == "structured_flow" and not state.problem_validated:
-        logger.info(
-            "gate redirect",
-            requested=intent,
-            effective="problem_statement",
-            reason="problem_not_validated",
-        )
-        return "problem_statement"
+    # Pre-2026-06-24 policy: redirect analytical requests until the goal was validated.
+    # if intent == "structured_flow" and not state.problem_validated:
+    #     logger.info(
+    #         "gate redirect",
+    #         requested=intent,
+    #         effective="problem_statement",
+    #         reason="problem_not_validated",
+    #     )
+    #     return "problem_statement"
     return intent
 
 

src/agents/handlers/problem_statement.py

@@ -1,3 +1,7 @@
+# UNWIRED 2026-06-24: the problem_statement skill is no longer routed to — it was removed
+# from the 6-intent router and the gate (the goal is now user-entered objective +
+# business_questions, no agent validation). File kept intact (comment, don't delete) so
+# the skill can be restored if needed. See DEV_PLAN.md #1.
 """Problem Statement skill — guide the user to a usable problem statement.
 
 Routed by the orchestrator (intent `problem_statement`) and callable as a skill.

src/agents/orchestration.py

@@ -32,7 +32,9 @@ logger = get_logger("orchestrator")
 Intent = Literal[
     "chat",
     "help",
-    "problem_statement",
+    # "problem_statement",  # removed 2026-06-24 — the analysis goal is now two
+    #                        # user-entered fields (objective + business_questions),
+    #                        # captured at onboarding with no agent validation.
     "check",
     "unstructured_flow",
     "structured_flow",
@@ -53,10 +55,10 @@ class RouterDecision(BaseModel):
         ...,
         description=(
             "Handler route for this message: 'chat' (conversational, no data), "
-            "'help' (what-to-do-next guidance), 'problem_statement' (define or "
-            "refine the analysis goal), 'check' (inventory: what data/documents "
-            "exist), 'unstructured_flow' (answer from documents, fast RAG), or "
-            "'structured_flow' (analytical question over data, slow Planner path)."
+            "'help' (what-to-do-next guidance), 'check' (inventory: what "
+            "data/documents exist), 'unstructured_flow' (answer from documents, fast "
+            "RAG), or 'structured_flow' (analytical question over data, slow Planner "
+            "path)."
         ),
     )
     rewritten_query: str | None = Field(

src/agents/report/readiness.py

@@ -7,8 +7,8 @@ not a judgement.
 
 The rule mirrors what makes a real report non-empty and worth generating, so Help can
 never suggest an action that would 409 or produce a duplicate:
-  1. `problem_validated` — the gate's own precondition (no validated goal, no
-     analysis worth reporting). Same rule `gate.gate` applies to `structured_flow`.
+  1. (removed 2026-06-24) a validated problem statement — the report no longer gates on
+     the goal (now user-entered `objective` + `business_questions`, no agent validation).
   2. at least one **substantive** persisted `AnalysisRecord` — a record whose
      *analysis* task succeeded. A failed run still persists a record WITH findings
      (they narrate the failure), and data-access tasks (check_/retrieve_) succeed even
@@ -45,7 +45,7 @@ if TYPE_CHECKING:
 logger = get_logger("report_readiness")
 
 # Human-readable gaps surfaced to the user via Help (kept stable for the prompt).
-_MISSING_PROBLEM = "a validated problem statement"
+# _MISSING_PROBLEM retired 2026-06-24 — the report no longer gates on a validated goal.
 _MISSING_ANALYSIS = "at least one completed analysis"
 _MISSING_DELTA = "a new analysis since the last report"
 
@@ -91,18 +91,22 @@ async def report_floor(
     *,
     record_store=None,
 ) -> tuple[list[str], list]:
-    """The report **floor**: a validated goal + ≥1 substantive analysis.
+    """The report **floor**: ≥1 substantive analysis.
 
     Returns `(missing, substantive_records)`. This is the shared gate both the Help
     readiness signal AND the report API enforce, so the button and Help can't drift
-    (T-D / T11). It deliberately excludes the delta-since-report check — that is
-    advisory and lives only in `is_report_ready`; the report button is always allowed
-    to cut a new version (decision 4A). Fails closed (counts as missing analysis) on
-    a record-store read error. `record_store` is injectable for tests.
+    (T-D / T11).
+
+    CHANGED 2026-06-24: the `problem_validated` precondition was dropped — analysis is no
+    longer gated on a validated goal (now user-entered `objective` + `business_questions`,
+    no agent validation), so the only floor is "is there anything worth reporting". The
+    delta-since-report check stays advisory and lives only in `is_report_ready`; the
+    report button is always allowed to cut a new version (decision 4A). Fails closed
+    (counts as missing analysis) on a record-store read error. `record_store` is
+    injectable for tests. `state` stays in the signature (callers + the `is_report_ready`
+    delta check use it).
     """
     missing: list[str] = []
-    if not state.problem_validated:
-        missing.append(_MISSING_PROBLEM)
 
     substantive: list = []
     if analysis_id:
@@ -116,7 +120,7 @@ async def report_floor(
                 analysis_id=analysis_id,
                 error=str(exc),
             )
-            return [*missing, _MISSING_ANALYSIS], []
+            return [_MISSING_ANALYSIS], []
 
     if not substantive:
         missing.append(_MISSING_ANALYSIS)

src/api/v1/tools.py

@@ -56,13 +56,16 @@ _COMMAND_CATALOG: list[CommandResponse] = [
         type="skill",
         description="Show what the assistant can do and guide your next step.",
     ),
-    CommandResponse(
-        command="/problem-statement",
-        name="problem_statement",
-        type="skill",
-        description="Define and validate your analysis goal (objective + metric) "
-        "before exploring data.",
-    ),
+    # /problem-statement removed 2026-06-24 — replaced by user-entered objective +
+    # business_questions captured at onboarding (no agent skill). Kept commented for
+    # restorability.
+    # CommandResponse(
+    #     command="/problem-statement",
+    #     name="problem_statement",
+    #     type="skill",
+    #     description="Define and validate your analysis goal (objective + metric) "
+    #     "before exploring data.",
+    # ),
     CommandResponse(
         command="/analyze-descriptive",
         name="analyze_descriptive",

src/config/prompts/intent_router.md

@@ -7,7 +7,7 @@ Return three fields:
 - **`intent`** — exactly one of:
   - `chat` — conversational, no data needed: greetings, farewells, thanks, "how are you", "what can you do", small talk.
   - `help` — the user wants to know **what to do next** or how the process works ("what's the next step?", "how do I start?", "what should I do now?").
-  - `problem_statement` — the user wants to **define or refine the analysis goal**: the business problem, objectives, what to increase/decrease, targets/success metrics — or is answering questions about the goal.
+  <!-- `problem_statement` intent removed 2026-06-24 — the analysis goal is now two user-entered fields (objective + business_questions) captured at onboarding, with no agent validation. -->
   - `check` — the user wants an **inventory** of what they have: "what data do I have?", "what columns are in this table?", "what documents did I upload?", "describe my dataset". This is metadata/listing, not analysis.
   - `unstructured_flow` — the user asks about a **topic, concept, feature, explanation, or factual knowledge** that may live in uploaded documents (PDF/DOCX/TXT). Pure document Q&A. The user need not mention a document.
   - `structured_flow` — the user asks an **analytical question over their data**: counts, sums, top-N, filters, comparisons, trends, correlations, segments, share-of-total, joins across structured sources. This routes to the slow analytical path.
@@ -18,16 +18,14 @@ Return three fields:
 
 1. Pure greeting / farewell / thanks / "what can you do" / compliment with no task → `chat`.
 2. "What do I do next / how do I proceed / where do I start" → `help`.
-3. The user states or refines a goal, objective, target, or success metric, or answers a goal-defining question → `problem_statement`.
-4. "What data / columns / tables / documents do I have", "describe my data", inventory or metadata requests → `check`.
-5. A question answerable from document prose — a topic, concept, feature, explanation, summary, or factual knowledge, even without naming a document → `unstructured_flow`.
-6. An analytical question answerable by computing over tabular/DB data (counts, sums, top-N, filters, comparisons, trends, correlations, segments) → `structured_flow`.
+3. "What data / columns / tables / documents do I have", "describe my data", inventory or metadata requests → `check`.
+4. A question answerable from document prose — a topic, concept, feature, explanation, summary, or factual knowledge, even without naming a document → `unstructured_flow`.
+5. An analytical question answerable by computing over tabular/DB data (counts, sums, top-N, filters, comparisons, trends, correlations, segments) → `structured_flow`.
 
 ## Disambiguation (the boundaries that matter)
 
 - **`check` vs `structured_flow`** — "what do I have / describe it" → `check`; "analyze / compute / trend / correlate / compare it" → `structured_flow`.
 - **`unstructured_flow` vs `structured_flow`** — pure document/concept Q&A → `unstructured_flow`; anything needing computation over tabular/DB data → `structured_flow`. **When in doubt between "analytical AND also needs document context" → `structured_flow`** (the analytical path can pull document context itself). Only choose `unstructured_flow` for *pure* document questions with no computation.
-- **`help` vs `problem_statement`** — "what's next?" → `help`; "here is my goal / let's define the objective" → `problem_statement`.
 - **`chat` vs everything else** — only use `chat` when there is no task and no data question at all.
 
 ## Rewriting follow-ups
@@ -58,16 +56,6 @@ User: "Okay I uploaded my data, what do I do next?"
 User: "How does this work? Where should I start?"
 → intent="help", rewritten_query=null, confidence=0.9
 
-User: "I want to reduce customer churn next quarter, target under 5%."
-→ intent="problem_statement",
-  rewritten_query="Define the analysis goal: reduce customer churn next quarter to under 5%.",
-  confidence=0.9
-
-User: "My goal is to grow revenue in the north region."
-→ intent="problem_statement",
-  rewritten_query="Define the analysis goal: grow revenue in the north region.",
-  confidence=0.88
-
 User: "What data do I have?"
 → intent="check", rewritten_query="What data sources do I have?", confidence=0.95
 
@@ -113,7 +101,7 @@ User: "And in March?"
 
 ## Constraints
 
-- Pick exactly one `intent`. Do not invent values outside the six listed.
+- Pick exactly one `intent`. Do not invent values outside the five listed.
 - Prefer `unstructured_flow` over `structured_flow` only for pure knowledge/document questions; prefer `structured_flow` whenever computation over data is involved.
 - Do not refuse — refusal happens later in guardrails. Just classify.
 - One JSON object as output; no prose, no markdown.

src/config/settings.py

@@ -24,11 +24,10 @@ class Settings(BaseSettings):
     # real source lands, so this stays opt-in.
     enable_slow_path: bool = Field(alias="enable_slow_path", default=False)
 
-    # Apply the deterministic gate (problem_validated) before dispatch: redirect
-    # `structured_flow` to `problem_statement` until the analysis is validated. Off
-    # by default — legacy `rooms` have no `analysis_states` row, so it would gate
-    # everything. Flip ENABLE_GATE=true once the frontend creates analyses via
-    # /analysis/create.
+    # DEPRECATED 2026-06-24: the problem_validated gate was removed (the goal is now
+    # user-entered objective + business_questions, no agent validation). This flag no
+    # longer has any effect — the gate call site in ChatHandler is commented out. Kept
+    # to avoid .env churn; remove once no environment references it.
     enable_gate: bool = Field(alias="enable_gate", default=False)
 
     # Database