fix(fact-find): KI-012/013/014 — flow corrections from user testing

KI-012 (P0) — Bot stuck in fact_find_complete readback loop
After fact-find completed, the bot kept emitting the same readback
summary instead of answering user's policy questions. Audit transcript
showed P002 wasted 19 of 30 turns echoing the profile back.
Fix: set session.free_form_session=True on completion + flush.

KI-013 (P0) — Bot recommends senior-citizen policy to non-seniors
Real user testing: vague opener → bot retrieved + pitched "Care
Senior" (senior-only policy) without ANY fact-find. Intent classifier
routed to recommendation, bypassing fact-find.
Fix: orchestrator forces fact-find when profile is empty (no age,
dependents, or income captured yet), regardless of intent.

KI-014 (P1) — "family" auto-mapped to self+spouse+kids
Real user testing: user said "family" → bot assumed self+spouse+kids.
User actually meant parents + siblings (joint family).
Fix: VAGUE_TERMS list in keyword normalizer returns None for unclear
terms ("family", "everyone", "whole family", "joint family") forcing
LLM normalizer or re-ask clarification. Disambiguated phrases like
"family — me and my wife" still parse correctly.

KI-015 (P1) — Age in readback mismatched user's stated age
Real user testing: user said 31, readback said 30. Documented in
known-issues.md for follow-up. Fix plan involves adding an explicit
profile-confirmation step before transitioning to recommendations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/fact_find_normalizer.py

@@ -144,6 +144,16 @@ def _keyword_normalize(question_id: str, raw_text: str) -> Any:
     s = raw_text.lower()
 
     if question_id == "dependents":
+        # KI-014 — vague terms like "family" / "everyone" must NOT auto-map.
+        # User testing surfaced: user said "family" → bot assumed self+spouse+kids
+        # → wrong (user may have meant parents, siblings, joint-family etc.).
+        # Returning None here forces the LLM normalizer (or re-ask) to clarify.
+        VAGUE_TERMS = ["family", "everyone", "all of us", "everybody", "whole family", "joint family"]
+        if any(v in s for v in VAGUE_TERMS) and not any(
+            k in s for k in ["spouse", "wife", "husband", "kid", "child", "parent"]
+        ):
+            return None  # force clarification
+
         if any(k in s for k in ["spouse", "wife", "husband"]) and "kid" in s and "parent" in s:
             return "self+spouse+kids+parents"
         if any(k in s for k in ["spouse", "wife", "husband"]) and "kid" in s:

backend/orchestrator.py

@@ -165,7 +165,22 @@ async def handle_turn(
     session = get_session(session_id or "anonymous")
 
     in_fact_find_continuation = bool(session.awaiting_question_id) and not session.free_form_session
-    treat_as_fact_find = (intent == "fact_find" and not session.free_form_session) or in_fact_find_continuation
+    # KI-013 — if the user has NO profile fields yet, FORCE fact-find regardless
+    # of intent classifier. Real user testing surfaced: a vague opener
+    # ("I want health insurance") got classified as "recommendation" and the
+    # bot retrieved "Care Senior" (a senior-citizen-only policy) and pitched it.
+    # Bot should never recommend without knowing the user's age / dependents /
+    # conditions / budget. Force fact-find until ≥1 profile field is set.
+    profile_is_empty = (
+        session.profile.age is None
+        and session.profile.dependents is None
+        and session.profile.income_band is None
+    )
+    treat_as_fact_find = (
+        (intent == "fact_find" and not session.free_form_session)
+        or in_fact_find_continuation
+        or (profile_is_empty and not session.free_form_session)
+    )
 
     if treat_as_fact_find:
         # If we were awaiting an answer, normalize + record it before picking next Q.

docs/40-evaluation/known-issues.md

@@ -254,3 +254,55 @@ hits this.
 `session.free_form_session = True` and flush to disk. Subsequent turns
 skip the fact-find branch entirely and go through retrieval + brain
 as intended.
+
+### KI-013 — Bot recommends policies on vague openers without fact-find — **FIXED in next commit**
+
+**Severity:** P0
+**Source:** `backend/orchestrator.py` intent → fact_find gating
+**Discovered:** Real user testing 2026-05-14
+
+User gave a vague opening ("I want a health policy") and bot
+immediately retrieved + pitched "Care Senior" — a senior-citizen-only
+policy. User is not a senior. The intent classifier routed the message
+to "recommendation" / "qa", not to fact_find. The orchestrator then
+went straight to retrieval + brain → bot recommended whatever scored
+highest, regardless of user demographics.
+
+**Fix:** Force fact-find whenever profile is empty (no age, no
+dependents, no income_band). Regardless of what the intent classifier
+says. Bot will now always start with "First, your age?" before any
+recommendation.
+
+### KI-014 — Vague dependents term "family" auto-mapped to self+spouse+kids — **FIXED in next commit**
+
+**Severity:** P1
+**Source:** `backend/fact_find_normalizer.py` keyword fast-path
+**Discovered:** Real user testing 2026-05-14
+
+User said "family" as their dependents answer. Bot assumed
+"self+spouse+kids". User had intended their joint family (parents +
+siblings). All subsequent recommendations were wrong.
+
+**Fix:** Add VAGUE_TERMS list (`family`, `everyone`, `joint family`,
+etc.) that explicitly returns None from the keyword fast-path,
+forcing either the LLM normalizer (which is more nuanced) or a re-ask
+clarifier. Phrases like "family — me and my wife" still parse
+correctly because the disambiguating words come through.
+
+### KI-015 — Age in readback summary doesn't match user's stated age
+
+**Severity:** P1
+**Source:** Possibly `backend/needs_finder.py::record_answer` for age,
+or LLM readback hallucination
+**Discovered:** Real user testing 2026-05-14
+
+User said "31" but bot's readback summary said "30". Possible causes:
+(a) User's earlier answer contained "30" that the int parser caught
+first; (b) The bot is using the LLM to generate the readback and the
+LLM is hallucinating numeric values.
+
+**Fix plan:** Add a CONFIRMATION step before the bot transitions to
+free-form recommendations. After fact-find readback, the bot should
+ask "Does this all look right? Reply 'yes' or correct anything that's
+off." Then proceed only if user confirms. Also: log the raw fact-find
+inputs vs the captured profile so we can debug mismatches.