refactor(brain): KI-167 — rip out fact_find_brain, replace with sales_brain

Replaces scripted slot-walker (fact_find_brain + canonical_fallback +
scripted Question.prompt_en + "Got that —" prefix + <FF> trailer) with a
single LLM-driven sales agent using NIM response_format=json_object.

WHY:
8 patches (KI-150/155/156/158/161) all worked around <FF> trailer
fragility + canonical_fallback firing as the dominant path. Symptom:
"zero natural LLM chat — always defaults to script." Architectural
rot from reactive accretion.

NEW:
+ backend/sales_brain.py — single LLM call, response_format=json_object,
SalesBrainResult{reply_text, captured_updates, ready_for_recommendations,
brain_used}. NO scripted fallback. Fail-loud on errors.
+ backend/sales_brain_normalizer.py — deterministic post-processor:
city→tier, "5 lakhs"→"5L-10L", bad-name rejection, schema validation.

DELETED:
- backend/fact_find_brain.py (986 lines)
- backend/fact_find_normalizer.py (452 lines — cherry-picked into normalizer)
- tests/test_fact_find_loop_break.py (architecture retired)

CHANGED:
M backend/orchestrator.py — fact_find branch rewired to drive_sales_brain
M backend/needs_finder.py — GRAPH + Question + scripted prompt_en deleted,
next_question returns field-name str now
M backend/main.py — /api/profile/completeness uses field-name hint
M tests/test_name_persistence.py — CASE B mocks SalesBrainResult

VERIFICATION:
py_compile + import smoke clean. next_question(empty)→"name". Zero
stale fact_find_brain / fact_find_normalizer / GRAPH / Question /
FactFindOutcome / FIELD_TO_QUESTION_ID / _canonical_fallback imports
anywhere in backend/tests/tools/eval/.

DOCS CASCADE: held back per user request. ADR-039 + CLAUDE.md + README.md
stashed locally — will commit after live confirmation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/fact_find_brain.py

@@ -1,986 +0,0 @@
-"""KI-070 — Single-LLM-call fact-find brain.
-
-Replaces three stitched layers that previously drove fact-find:
-  1. `backend/needs_finder.GRAPH` — hardcoded canonical question prompts
-  2. `backend/question_paraphraser.py` — per-turn LLM paraphrase pass
-  3. `backend/orchestrator._pick_opener` + `_family_aware_opener` —
-     acknowledger rotation prepended to each canonical question
-
-Real-user testing on 2026-05-15 flagged the result as robotic + templated.
-The fix is to do the whole job in ONE LLM call per turn that produces a
-natural conversational reply AND tags which slots were captured / which
-slot it's driving toward / whether fact-find is complete — all in a single
-machine-parsable trailer block.
-
-Public entry:
-    async def drive_fact_find(user_text, session, chat_history, session_id)
-        -> FactFindOutcome
-
-Reliability guardrails:
-  1. JSON-block-must-parse — if `<FF>...</FF>` is missing or invalid,
-     return `ambiguous=true` so the orchestrator emits a soft canonical
-     fallback (`next_question` then prompt_en).
-  2. Capture validation — each captured value goes through
-     `fact_find_normalizer._validate` before commit. Validation failure
-     drops only that field; others still apply.
-  3. Slot-not-progressing safeguard — if the SAME `slot_driving` appears
-     in 3 consecutive turns, force `next_question(profile)` to pick a
-     different slot. State is held on the SessionState (`_ff_brain_history`).
-  4. Hard timeout 12 seconds. Timeout → ambiguous=True + the canonical
-     `next_question.prompt_en` as a fallback reply.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import logging
-import re
-import time
-from dataclasses import dataclass, field
-from typing import Any, Optional
-
-from backend.providers.base import ChatMessage
-from backend.providers.nvidia_nim_llm import get_fast_brain_llm
-
-
-# ----------------------------------------------------------------------------
-# Outcome dataclass
-# ----------------------------------------------------------------------------
-
-@dataclass
-class FactFindOutcome:
-    reply_text: str
-    captured_updates: dict[str, Any] = field(default_factory=dict)
-    slot_driving: Optional[str] = None
-    fact_find_complete: bool = False
-    ambiguous: bool = False
-    # KI-078 (2026-05-15) — when `ambiguous=True` because the LLM brain
-    # bailed and `_canonical_fallback` was used, this stamps WHY so the
-    # orchestrator can append it to `brain_used` and admin telemetry can
-    # measure the fallback-reason mix. One of:
-    #   "timeout"     — asyncio.wait_for(_TIMEOUT_S) expired on FAST_BRAIN_CHAIN
-    #   "llm_error"   — FAST_BRAIN_CHAIN raised (non-timeout) before returning
-    #   "no_trailer"  — reply had no <FF>...</FF> JSON block, or it failed parse
-    #   "empty_reply" — trailer stripped to an empty user-facing reply
-    # KI-079 (2026-05-15) — added two escalation-path reasons. When FAST
-    # chain times out we re-try once against BRAIN_CHAIN (heavier; Qwen 80B
-    # primary, more cross-provider fallbacks) inside a shorter budget. The
-    # _fallback_reason then captures the escalation outcome:
-    #   "timeout_after_escalation"   — both FAST + BRAIN chains timed out
-    #   "llm_error_after_escalation" — BRAIN chain raised on the retry
-    # None when the brain succeeded (either FAST primary OR BRAIN escalation).
-    _fallback_reason: Optional[str] = None
-
-
-# ----------------------------------------------------------------------------
-# System prompt — the ENTIRE behavioural contract lives here. Keep tight.
-# ----------------------------------------------------------------------------
-
-_SYSTEM_PROMPT = """You are an insurance advisor having a natural conversation with someone considering a health policy in India. Be warm, brief, consultative — NOT closing-pitchy. Speak like a trusted Independent Financial Advisor, not a call-centre script.
-
-You need to capture these 9 facts before recommending. Don't ask them all at once — weave them into the conversation based on what the user just said.
-
-SLOTS YOU NEED TO FILL (field name : type : allowed values : why it matters)
-- name : string (1-50 chars) : for personalisation + saving their profile so they don't repeat fact-find next visit
-- age : int 18-99 : drives premium, eligibility, renewability
-- dependents : enum {self, self+spouse, self+spouse+kids, self+kids, self+parents, self+spouse+parents, self+spouse+kids+parents} : drives policy structure (parent-specific plans often win when parents are on)
-- income_band : enum {under_5L, 5L-10L, 10L-25L, 25L+} : drives sum-insured sizing
-- existing_cover_inr : int >= 0 (0 if first-time buyer or no employer plan) : drives whether to recommend top-up vs full base plan
-- primary_goal : enum {first_buy, upgrade, compare_specific, tax_planning} : drives whether to grade on price / breadth / claim experience / tax savings
-- location_tier : enum {metro, tier1, tier2, tier3} : drives premium + cashless hospital density
-- parents_age_max : int 30-99 (only relevant when dependents includes parents) : drives whether parent-specific plans should be included
-- health_conditions : list of lowercase canonical names like ["diabetes", "hypertension", "thyroid", "asthma", "heart", "cancer"] (empty list = no conditions) : drives PED waiting + insurer fit. Honesty here protects the user's future claim.
-- budget_band : enum {under_15k, 15k_30k, 30k_60k, 60k+} : drives policy shortlist
-
-CONVERSATION RULES
-1. Reply naturally. Don't say "Got it." or "Noted." every turn. Don't apologise robotically. Don't number your questions.
-2. Acknowledge what the user just told you — reflect it back briefly so they feel heard.
-3. Ask for at most ONE missing slot per reply, woven into the conversation.
-4. If the user supplied multiple facts in one message (e.g., "I'm Rohit, 29, in Mumbai"), capture them ALL — don't re-ask for what they just volunteered.
-5. If they go off-topic, give a brief honest answer, then steer back.
-6. If they say "show me policies", "let me see options", "that's enough", "skip the questions" — set complete=true and slot_driving=null. Don't keep fact-finding forever.
-7. When the profile feels complete enough to recommend (at minimum age + dependents + income + primary_goal captured), proactively summarise the captured profile + ask if they want a policy suggestion. Set complete=true.
-8. Indian English is fine — "₹", "lakh", "metro/tier-2 city" are natural. Never use markdown bold or italics — the reply is read aloud by TTS too.
-9. BFSI compliance: do not promise specific premium quotes, do not pressure-close. Light, advisory tone.
-10. On health conditions — be straight: hiding a condition lowers premium today but turns into a denied claim later. Encourage honesty without lecturing.
-
-OUTPUT FORMAT (strict)
-After your natural reply, on a NEW LINE, emit exactly one machine-readable trailer:
-<FF>{"captured": {<field>: <value>, ...}, "slot_driving": "<slot_field_or_null>", "complete": <true_or_false>}</FF>
-
-The <FF> block is INVISIBLE to the user — the backend strips it before sending.
-
-Strict requirements for the <FF> block:
-- Captured values MUST match the slot schema (enums exact, ints parsed as JSON numbers, strings trimmed, list as JSON array). Missing slots = omit them; do NOT emit null values for fields you didn't capture this turn.
-- `slot_driving` MUST be the FIELD NAME (e.g., "income_band"), not the canonical-question id. Use null when fact-find is complete OR when the user has steered to a different intent.
-- `complete` MUST be a JSON boolean (true / false), no quotes.
-- If the user's latest message yielded no new facts, emit "captured": {} (empty object, not omitted).
-- Emit the <FF> block exactly once and at the END. Nothing after </FF>.
-
-EXAMPLES OF GOOD REPLIES (reply line + trailer)
-
-User: "Hi, I'm Rohit, 29, looking to buy my first health policy."
-Bot reply: Hi Rohit — nice to meet you. First health policy is a good place to start. To shape the right options for you, who else would you want covered apart from yourself — spouse, kids, parents, or just you for now?
-<FF>{"captured": {"name": "Rohit", "age": 29, "primary_goal": "first_buy"}, "slot_driving": "dependents", "complete": false}</FF>
-
-User: "Just me and my wife. We're both in Bangalore."
-Bot reply: Got that — you and your spouse, based in Bangalore (so metro hospital network is a big advantage for cashless). What income band are you in roughly — under ₹5L, ₹5-10L, ₹10-25L, or ₹25L+? Helps me suggest the right sum-insured size.
-<FF>{"captured": {"dependents": "self+spouse", "location_tier": "metro"}, "slot_driving": "income_band", "complete": false}</FF>
-
-User: "Around 18 lakh."
-Bot reply: That puts you in the ₹10-25L band — gives us comfortable room on sum-insured. Do either of you have any pre-existing conditions — diabetes, BP, thyroid, anything chronic? Be straight here: it protects your future claim.
-<FF>{"captured": {"income_band": "10L-25L"}, "slot_driving": "health_conditions", "complete": false}</FF>
-
-User: "Just show me some policies already."
-Bot reply: Fair enough — let me put together a shortlist based on what you've shared so far: age 29, covering you and your spouse, metro, income ₹10-25L, first-time buyer. Want me to suggest 2-3 policies that fit, or do you have specific insurers in mind to compare?
-<FF>{"captured": {}, "slot_driving": null, "complete": true}</FF>
-
-User: "What's the waiting period for PED usually?"
-Bot reply: Most policies in India have a 24-36 month waiting period for pre-existing diseases — IRDAI caps the maximum, and a few insurers offer shorter waits as a premium upgrade. Happy to dig into a specific policy if you have one in mind. Meanwhile, what budget range are you comfortable with annually — under ₹15k, ₹15-30k, ₹30-60k, or ₹60k+?
-<FF>{"captured": {}, "slot_driving": "budget_band", "complete": false}</FF>
-
-User: "Hi"
-Bot reply: Hi there — happy to help you find a health policy. What should I call you?
-<FF>{"captured": {}, "slot_driving": "name", "complete": false}</FF>
-
-User: "ok"
-Bot reply: Sure. To get started, could you tell me your age?
-<FF>{"captured": {}, "slot_driving": "age", "complete": false}</FF>
-
-==========================================================
-FINAL INSTRUCTION — READ THIS CAREFULLY. APPLIES TO EVERY REPLY.
-==========================================================
-EVERY reply you produce MUST end with the structured trailer in this EXACT format on its own line:
-
-<FF>{"captured": {...}, "slot_driving": "...", "complete": false}</FF>
-
-This rule is ABSOLUTE. There are NO exceptions:
-- Short reply? Still emit the trailer.
-- One-word user message ("ok", "hi", "yes")? Still emit the trailer.
-- User asks an off-topic question? Still emit the trailer after your answer.
-- User wants to end fact-find ("show me policies")? Emit trailer with "complete": true.
-- Nothing new captured this turn? Emit "captured": {} (empty object). Still emit the trailer.
-
-The trailer is the LAST thing in your output. Nothing comes after </FF>.
-
-If you do not emit a valid <FF>...</FF> trailer, your entire reply is DISCARDED by the backend and the user sees a scripted fallback question. This is the single most important rule. NEVER omit the trailer.
-"""
-
-
-# ----------------------------------------------------------------------------
-# Parser for <FF>...</FF> trailer
-# ----------------------------------------------------------------------------
-
-_FF_BLOCK_RE = re.compile(r"<FF>\s*(\{.*?\})\s*</FF>", re.DOTALL)
-# KI-090 (2026-05-15) — lenient fallback for FF parsing. Many LLMs drop
-# the literal <FF>...</FF> tags and emit a bare JSON tail (or inline
-# fenced JSON). When the strict tag match fails, try these regexes in
-# order so we accept whatever the brain actually produces.
-_FF_FENCED_RE = re.compile(r"```(?:json)?\s*(\{[\s\S]*?\})\s*```", re.IGNORECASE)
-_FF_TAIL_JSON_RE = re.compile(r"(\{[\s\S]*\})\s*\Z")
-
-
-def _parse_ff_block(text: str) -> Optional[dict]:
-    """Extract + parse the structured trailer dict from an LLM reply.
-
-    Returns the parsed dict on success, None when no parseable JSON object
-    can be located. Caller treats None as `ambiguous=True` and falls
-    through to the canonical fallback.
-
-    KI-090 (2026-05-15) — lenient parsing. The original strict
-    `<FF>{...}</FF>` regex was correct per the system-prompt contract, but
-    real LLMs (Qwen, Nemotron under load, Groq Llama-3.3 at times) drop
-    the literal tags and emit only the JSON. Pre-KI-090 those replies
-    fell to `fallback:no_trailer` even though the brain had produced a
-    perfectly valid structured tail. Now we try:
-      1. Strict `<FF>{...}</FF>` (the contract — still preferred).
-      2. ```` ```json {...} ``` ```` fenced (a common LLM habit).
-      3. Bare `{...}` at the very end of the reply.
-    Any candidate that parses as a JSON dict with at least one of the
-    expected keys (`captured` / `slot_driving` / `complete`) wins.
-    """
-    if not text:
-        return None
-    candidates: list[str] = []
-    m = _FF_BLOCK_RE.search(text)
-    if m:
-        candidates.append(m.group(1).strip())
-    for fenced in _FF_FENCED_RE.finditer(text):
-        candidates.append(fenced.group(1).strip())
-    tail = _FF_TAIL_JSON_RE.search(text.rstrip())
-    if tail:
-        candidates.append(tail.group(1).strip())
-    seen: set[str] = set()
-    for raw in candidates:
-        if raw in seen:
-            continue
-        seen.add(raw)
-        try:
-            data = json.loads(raw)
-        except json.JSONDecodeError:
-            continue
-        if not isinstance(data, dict):
-            continue
-        # Require at least one of the contract keys so we don't accept
-        # a stray JSON object from the prose (e.g. a quoted example).
-        if any(k in data for k in ("captured", "slot_driving", "complete")):
-            return data
-    return None
-
-
-def _strip_ff_block(text: str) -> str:
-    """Remove the FF trailer (tagged, fenced, or bare-JSON tail) so the
-    user-facing reply doesn't leak the schema tag.
-
-    KI-090 — mirrors the lenient `_parse_ff_block` strategies in reverse:
-    strip strict `<FF>...</FF>`, then ```` ```json ... ``` ````, then any
-    bare-JSON tail that contains a contract key.
-    """
-    if not text:
-        return text
-    cleaned = _FF_BLOCK_RE.sub("", text).strip()
-    cleaned = _FF_FENCED_RE.sub("", cleaned).strip()
-    # Strip a bare-JSON tail ONLY if it contains a contract key — otherwise
-    # we might delete prose that happens to end with a JSON-ish bracket.
-    tail = _FF_TAIL_JSON_RE.search(cleaned)
-    if tail:
-        try:
-            tail_json = json.loads(tail.group(1))
-            if isinstance(tail_json, dict) and any(
-                k in tail_json for k in ("captured", "slot_driving", "complete")
-            ):
-                cleaned = cleaned[: tail.start()].rstrip()
-        except json.JSONDecodeError:
-            pass
-    # Defensive: any orphan partial tags
-    cleaned = re.sub(r"<FF>.*$", "", cleaned, flags=re.DOTALL).strip()
-    cleaned = re.sub(r"</FF>", "", cleaned).strip()
-    return cleaned
-
-
-# ----------------------------------------------------------------------------
-# Schema map — field name → validator schema in fact_find_normalizer terms
-# ----------------------------------------------------------------------------
-#
-# The slot prompt promises enum values like "self+spouse" and "10L-25L", and
-# `fact_find_normalizer._FIELD_SCHEMA` already encodes those. Reuse it. Note
-# that the orchestrator's Profile fields are keyed by the FIELD NAME (e.g.,
-# `income_band`), not the canonical question id (`income_band`), and those
-# happen to match for most slots — exceptions handled below.
-
-# Slot field → canonical-question id (for marking `profile.asked` so
-# `next_question` doesn't re-pick the same slot).
-FIELD_TO_QUESTION_ID: dict[str, str] = {
-    "name": "name",
-    "age": "age",
-    "dependents": "dependents",
-    "income_band": "income_band",
-    "existing_cover_inr": "existing_cover",
-    "primary_goal": "primary_goal",
-    "location_tier": "location",
-    "parents_age_max": "parents_age",
-    "health_conditions": "health_conditions",
-    "budget_band": "budget",
-}
-
-
-def _validate_capture(field_name: str, value: Any) -> Any:
-    """Run `value` through the fact_find_normalizer schema for the matching
-    field. Returns the validated value (possibly coerced) or None when the
-    value fails validation."""
-    if value is None:
-        return None
-    # Name is free-text — apply the profile_store name validator.
-    if field_name == "name":
-        try:
-            from backend.profile_store import is_valid_name
-            v = str(value).strip()
-            if not is_valid_name(v):
-                return None
-            # KI-156 (2026-05-15) — reject LLM-hallucinated names that are
-            # actually status/negation phrases. Live bug: user said "I am
-            # currently not having any policy" and the LLM emitted
-            # captured={"name":"Currently Not Having Any"}. is_valid_name
-            # passes on length+alpha but doesn't catch semantic garbage.
-            _bad_starts = {
-                "currently", "not", "no", "none", "nothing", "never",
-                "without", "looking", "buying", "shopping",
-                "i", "we", "my", "this", "that", "the",
-                "first", "still", "yet", "haven", "haven't",
-                "don't", "don", "dont", "havent",
-            }
-            first = v.split()[0].lower().strip(".,!?")
-            if first in _bad_starts:
-                return None
-            return v
-        except Exception:
-            return None
-
-    # Translate field name → fact_find_normalizer question id for schema lookup.
-    qid = FIELD_TO_QUESTION_ID.get(field_name)
-    if qid is None:
-        # Unknown slot — drop.
-        return None
-
-    try:
-        from backend.fact_find_normalizer import _FIELD_SCHEMA, _validate
-    except Exception:
-        return value  # if validator import fails, trust the LLM
-
-    schema = _FIELD_SCHEMA.get(qid)
-    if schema is None:
-        return value
-
-    # Try the strict validator first.
-    validated = _validate(value, schema)
-    if validated is not None:
-        return validated
-
-    # Last resort — for ints, accept strings that parse cleanly.
-    t = schema.get("type")
-    if t == "int":
-        try:
-            iv = int(str(value).strip())
-        except (TypeError, ValueError):
-            return None
-        if iv < schema.get("min", 0) or iv > schema.get("max", 1_000_000_000):
-            return None
-        return iv
-    if t == "list" and isinstance(value, str):
-        # LLM may emit a comma-separated string; coerce.
-        items = [x.strip().lower() for x in value.split(",") if x.strip()]
-        return items
-    return None
-
-
-# ----------------------------------------------------------------------------
-# Slot-not-progressing safeguard
-# ----------------------------------------------------------------------------
-
-def _bump_brain_history(session, slot_driving: Optional[str]) -> int:
-    """Record `slot_driving` on the session and return how many consecutive
-    turns the same slot has been driven. Stored on the session under
-    `_ff_brain_history` as a list[str] capped at 8."""
-    if not hasattr(session, "_ff_brain_history"):
-        session._ff_brain_history = []
-    history: list[Optional[str]] = session._ff_brain_history
-    history.append(slot_driving)
-    if len(history) > 8:
-        del history[: len(history) - 8]
-    # Count run-length of the most recent slot at the tail.
-    if not slot_driving:
-        return 0
-    n = 0
-    for s in reversed(history):
-        if s == slot_driving:
-            n += 1
-        else:
-            break
-    return n
-
-
-# ----------------------------------------------------------------------------
-# KI-103 (2026-05-15) — Canonical-fallback loop breaker
-# ----------------------------------------------------------------------------
-# When the LLM brain returns `no_trailer` (or any reason that drives
-# `_canonical_fallback`) on consecutive turns, the legacy code re-surfaced
-# the SAME unfilled slot indefinitely. Live 15-persona smoke test caught
-# the brain repeating "First, your age?" 7-8 times in a row even after the
-# user explicitly stated their age. Root cause: `next_question(profile)`
-# iterates GRAPH skipping ids in `profile.asked` and filled fields — but
-# the greedy capture in `_canonical_fallback` only appends to `asked` when
-# a value was successfully extracted, so a failed capture loop keeps the
-# slot un-asked forever.
-#
-# Fix: track per-slot failed-fallback counts on the session. After
-# `_MAX_FAILED_ATTEMPTS` (=2) consecutive surfaces of the same unfilled
-# slot via canonical fallback WITHOUT capturing anything, mark the slot as
-# asked (so `next_question` advances) AND record it on `session._ff_skipped_slots`
-# so the orchestrator / scorecard can render it as intentionally unanswered.
-#
-# Counters live on the SessionState (not the Profile) — they are transient
-# fact-find-turn state, not part of the persisted profile schema. Resetting
-# them happens automatically on greedy-capture success below.
-_MAX_FAILED_ATTEMPTS = 2
-
-
-def _failed_attempts(session) -> dict[str, int]:
-    """Per-slot canonical-fallback failure counter, lazily attached to the
-    session. Maps `question.id` (NOT field name) → consecutive failure count.
-    """
-    if not hasattr(session, "_ff_failed_attempts"):
-        session._ff_failed_attempts = {}
-    return session._ff_failed_attempts
-
-
-def _skipped_slots(session) -> list[str]:
-    """Per-session list of slot ids the loop-breaker has marked SKIPPED
-    after `_MAX_FAILED_ATTEMPTS` failed canonical-fallback surfaces. The
-    orchestrator / scorecard reads this to render the skipped slots as
-    intentionally unanswered (not silently dropped).
-    """
-    if not hasattr(session, "_ff_skipped_slots"):
-        session._ff_skipped_slots = []
-    return session._ff_skipped_slots
-
-
-# ----------------------------------------------------------------------------
-# Public entry
-# ----------------------------------------------------------------------------
-
-# KI-075 (2026-05-15) — bumped 12s → 25s. Live probe on production showed
-# 4 of 5 fact-find turns hitting the 12s wait_for cap at exactly 13.2s
-# latency, falling to canonical fallback even though FAST_BRAIN_CHAIN's
-# total_budget_s is 22s. NIM cold-start eats 10-15s on the first call
-# after a Space rebuild; the 12s wait_for was killing the brain BEFORE
-# the cross-provider fallback links (Groq, OpenRouter) ever got tried.
-# 25s gives NIM cold-start headroom + leaves room for one chain fallback.
-_TIMEOUT_S = 25.0             # FAST_BRAIN_CHAIN primary attempt budget
-# KI-079 (2026-05-15) — if FAST exhausted, try BRAIN_CHAIN (Qwen 80B primary
-# + more cross-provider fallbacks incl. OpenRouter + Groq) with a SHORTER
-# budget so the user doesn't wait 25+15=40s on a fully-dead network. Total
-# worst-case latency before canonical fallback: 25 + 15 = 40s, but the FAST
-# chain only hits 25s when NIM is wedged — the BRAIN escalation then has a
-# Qwen primary on a different NIM pool + OpenRouter + Groq, so realistic
-# escalation success cases land in 3-8s.
-_TIMEOUT_S_ESCALATION = 15.0
-
-
-async def drive_fact_find(
-    user_text: str,
-    session,
-    chat_history: Optional[list[dict]],
-    session_id: Optional[str],
-) -> FactFindOutcome:
-    """Single LLM call per fact-find turn.
-
-    Returns a FactFindOutcome the orchestrator applies to session state.
-
-    Behaviour:
-      - Calls `get_fast_brain_llm()` with a system prompt that defines the
-        slot schema, conversation rules, and trailer-block format.
-      - Parses the trailer `<FF>...</FF>` JSON to learn what was captured,
-        which slot the brain is driving toward, and whether fact-find is
-        complete.
-      - Validates each captured value through `fact_find_normalizer`.
-      - On timeout / malformed JSON / no trailer → ambiguous=True and a
-        canonical-question fallback reply.
-    """
-    t0 = time.time()
-    # Build current profile JSON (only set fields) so the prompt sees what we
-    # already know.
-    profile = session.profile
-    profile_snapshot: dict[str, Any] = {}
-    for field_name in (
-        "name", "age", "dependents", "income_band", "existing_cover_inr",
-        "primary_goal", "location_tier", "parents_age_max", "health_conditions",
-        "budget_band",
-    ):
-        v = getattr(profile, field_name, None)
-        if v not in (None, "", []):
-            profile_snapshot[field_name] = v
-
-    system_with_state = _SYSTEM_PROMPT + (
-        "\n\nCURRENT PROFILE (already captured; do NOT re-ask for these):\n"
-        + json.dumps(profile_snapshot, ensure_ascii=False)
-    )
-
-    # Build messages: system + last ~10 chat history turns + user_text
-    messages: list[ChatMessage] = [ChatMessage(role="system", content=system_with_state)]
-    if chat_history:
-        for turn in chat_history[-10:]:
-            role = turn.get("role")
-            content = turn.get("content")
-            if role in ("user", "assistant") and content:
-                messages.append(ChatMessage(role=role, content=content))
-    messages.append(ChatMessage(role="user", content=user_text or ""))
-
-    # Hard 25-second timeout on FAST_BRAIN_CHAIN. The fast-brain chain already
-    # has its own per-link + total-chain budget but we wrap with asyncio.wait_for
-    # as a belt-and-braces stop.
-    llm_fast = get_fast_brain_llm()
-    try:
-        result = await asyncio.wait_for(
-            # KI-150 (2026-05-15) — bumped 420 → 700. Diagnosis showed the
-            # contract requires prose (~250 tok) + <FF>...</FF> trailer
-            # (~80 tok); 420 truncated verbose multi-slot replies before
-            # the trailer, parser failed, canonical fallback fired, user
-            # saw scripted prompt_en (the "robotic language" complaint).
-            llm_fast.chat(messages=messages, temperature=0.6, max_tokens=700),
-            timeout=_TIMEOUT_S,
-        )
-    except asyncio.TimeoutError:
-        # KI-079 (2026-05-15) — fast brain timed out. Before falling to the
-        # canonical-question fallback, escalate ONCE to BRAIN_CHAIN (Qwen 80B
-        # primary, more cross-provider fallbacks) with a shorter budget so the
-        # user doesn't wait 25+15=40s on a dead network.
-        logging.warning(
-            "KI-079: fast brain timeout (session=%s, %.1fs) → escalating to heavy brain",
-            session_id, time.time() - t0,
-        )
-        from backend.providers.nvidia_nim_llm import get_brain_llm
-        llm_heavy = get_brain_llm()
-        try:
-            result = await asyncio.wait_for(
-                # KI-150 — same bump on escalation path for consistency.
-                llm_heavy.chat(messages=messages, temperature=0.6, max_tokens=700),
-                timeout=_TIMEOUT_S_ESCALATION,
-            )
-        except asyncio.TimeoutError:
-            logging.warning(
-                "KI-079: heavy brain ALSO timed out (session=%s, total=%.1fs)",
-                session_id, time.time() - t0,
-            )
-            return _canonical_fallback(
-                session, user_text, reason="timeout_after_escalation"
-            )
-        except Exception as e2:
-            logging.warning(
-                "KI-079: heavy brain escalation failed (session=%s): %s: %s",
-                session_id, type(e2).__name__, str(e2)[:200],
-            )
-            return _canonical_fallback(
-                session, user_text, reason="llm_error_after_escalation"
-            )
-    except Exception as e:
-        logging.warning(
-            "fact_find_brain LLM call failed (session=%s): %s: %s",
-            session_id, type(e).__name__, str(e)[:200],
-        )
-        return _canonical_fallback(session, user_text, reason="llm_error")
-
-    raw = (result.text or "").strip()
-    parsed = _parse_ff_block(raw)
-    if parsed is None:
-        logging.info(
-            "fact_find_brain trailer missing/malformed (session=%s, raw=%r)",
-            session_id, raw[:200],
-        )
-        return _canonical_fallback(session, user_text, reason="no_trailer")
-
-    reply_text = _strip_ff_block(raw)
-    if not reply_text.strip():
-        return _canonical_fallback(session, user_text, reason="empty_reply")
-
-    captured_raw = parsed.get("captured") or {}
-    if not isinstance(captured_raw, dict):
-        captured_raw = {}
-    slot_driving_raw = parsed.get("slot_driving")
-    if slot_driving_raw in ("", "null", "None"):
-        slot_driving_raw = None
-    complete_raw = bool(parsed.get("complete", False))
-
-    # Validate each capture; drop fields that fail.
-    captured: dict[str, Any] = {}
-    for field_name, raw_value in captured_raw.items():
-        validated = _validate_capture(field_name, raw_value)
-        if validated is None:
-            logging.info(
-                "fact_find_brain dropped invalid capture (session=%s, field=%s, raw=%r)",
-                session_id, field_name, raw_value,
-            )
-            continue
-        captured[field_name] = validated
-
-    # Slot-not-progressing safeguard — if same slot has been driven 3 turns
-    # in a row, force the orchestrator to pick a different slot.
-    repeat_count = _bump_brain_history(session, slot_driving_raw)
-    if repeat_count >= 3 and slot_driving_raw and not complete_raw:
-        try:
-            from backend.needs_finder import next_question
-            alt = next_question(profile)
-            if alt and alt.field != slot_driving_raw:
-                logging.info(
-                    "fact_find_brain stuck on slot=%s for %d turns → forcing %s",
-                    slot_driving_raw, repeat_count, alt.field,
-                )
-                slot_driving_raw = alt.field
-        except Exception:
-            pass
-
-    return FactFindOutcome(
-        reply_text=reply_text,
-        captured_updates=captured,
-        slot_driving=slot_driving_raw,
-        fact_find_complete=complete_raw,
-        ambiguous=False,
-    )
-
-
-def _normalize_for_slot(slot_id: str, raw_text: str) -> Any:
-    """Synchronous slot normalizer used by KI-072 canonical fallback.
-
-    Mirrors the keyword/int/cover fast-paths from `fact_find_normalizer.py`
-    but stays sync so it can be called from the non-async `_canonical_fallback`.
-    LLM-only enum normalization is skipped — fallback prefers re-asking over
-    blocking on another LLM call (which is presumably already failing if we
-    got here).
-    """
-    if not raw_text or not raw_text.strip():
-        return None
-    text = raw_text.strip()
-
-    # NAME slot — KI-074 + KI-092 (2026-05-15) — two-mode parsing.
-    # MODE A (lenient, when caller explicitly tagged awaiting=name): accept
-    #   any plain alphabetic name. Used when fallback fires after the bot
-    #   asked "what should I call you?" — the reply IS the name by
-    #   construction; no intro phrase needed. Avoids the KI-074 false-
-    #   reject on "rohit sar" (plain reply to the name prompt).
-    # MODE B (strict, default — used by greedy multi-slot capture): requires
-    #   explicit "I'm X" / "my name is X" / "this is X" / "call me X" intro
-    #   to avoid false positives on "this is correct" / "29 years old" etc.
-    # The caller signals MODE A by passing slot_id="name__awaiting" (a
-    # synthetic tag); the canonical fallback maps awaiting_question_id=name
-    # to this tag.
-    if slot_id in ("name", "name__awaiting"):
-        lenient_mode = (slot_id == "name__awaiting")
-        slot_id = "name"  # normalise so downstream logic still sees "name"
-        import re as _re
-        # Strip leading greeting first
-        s = text.strip().strip(".,!?")
-        s = _re.sub(r"^(hi|hello|hey|namaste|yo)[,!.\s]+", "", s, flags=_re.IGNORECASE)
-
-        # KI-092 — LENIENT MODE: the bot just asked "what should I call you?"
-        # so any plain alphabetic reply should be accepted as the name.
-        # Examples to accept: "rohit sar" / "Anjali" / "Dr. Priya" / "Sam"
-        # Examples to reject: empty, all-digit, mostly-symbol, way too long.
-        if lenient_mode:
-            # Drop common polite-prefix scraps if present.
-            s_lc = s.lower()
-            for prefix in ("i'm ", "i am ", "this is ", "my name is ", "name is ",
-                           "name's ", "call me ", "im ", "mr ", "mrs ", "ms ", "dr "):
-                if s_lc.startswith(prefix):
-                    s = s[len(prefix):].strip()
-                    break
-            # Validation: 1-50 chars, ≥50% alphabetic, no embedded digits
-            if not s or len(s) > 50 or any(c.isdigit() for c in s):
-                return None
-            alpha = sum(1 for c in s if c.isalpha())
-            if alpha < 2 or alpha / max(1, len(s)) < 0.5:
-                return None
-            # Limit to 1-4 word tokens
-            tokens = s.split()
-            if not (1 <= len(tokens) <= 4):
-                return None
-            # Capitalise if all-lower
-            if not any(c.isupper() for c in s):
-                s = " ".join(w.capitalize() for w in tokens)
-            return s
-
-        # STRICT MODE (greedy multi-slot capture) — original KI-074 behaviour:
-        # require an explicit intro phrase to avoid false positives.
-        m = _re.search(
-            r"\b(?:i'?m|i\s+am|this\s+is|my\s+name\s+is|name\s+is|call\s+me|name'?s)\s+"
-            r"([a-zA-Z][a-zA-Z'\-]{1,30}(?:\s+[a-zA-Z][a-zA-Z'\-]{1,30}){0,3})\b",
-            s,
-            flags=_re.IGNORECASE,
-        )
-        if not m:
-            return None
-        candidate = m.group(1).strip()
-        # KI-074 — chop at conjunctions / boundary words so "Rohit Sar and"
-        # in "My name is Rohit Sar and I am 32" becomes "Rohit Sar".
-        candidate = _re.split(
-            r"\s+(?:and|but|with|plus|also|or|&|,)\s+",
-            candidate, maxsplit=1, flags=_re.IGNORECASE,
-        )[0].strip()
-        # Also strip a TRAILING conjunction (when the connector is the
-        # last token in the captured span — e.g. "Rohit Sar and").
-        candidate = _re.sub(
-            r"\s+(?:and|but|with|plus|also|or|&)\s*$",
-            "", candidate, flags=_re.IGNORECASE,
-        ).strip()
-        # Reject if any digit or trailing words look like an age phrase
-        if any(c.isdigit() for c in candidate):
-            return None
-        first_word = candidate.split()[0].lower()
-        if first_word in {
-            # Articles + possessives + demonstratives
-            "a", "an", "the", "my", "your", "his", "her", "their", "our",
-            "this", "that", "these", "those",
-            # Ordinals (commonly follow "this is my first ...")
-            "first", "second", "third", "fourth", "fifth", "last", "next",
-            # Common adjectives mistaken for names
-            "looking", "buying", "shopping", "interested", "trying", "thinking",
-            "happy", "sad", "tired", "busy", "ready", "done", "new", "old",
-            "good", "fine", "ok", "okay", "sure", "right",
-            "correct", "wrong", "alone", "single", "married",
-            "years", "year", "from", "very", "really", "just",
-        }:
-            return None
-        # Capitalise if all-lower (STT often outputs lowercase)
-        if not any(c.isupper() for c in candidate):
-            candidate = " ".join(w.capitalize() for w in candidate.split())
-        return candidate
-
-    # KI-074 — slot-specific triggers prevent cross-contamination during
-    # greedy multi-slot capture. Without these guards, "29 years old" was
-    # getting written into existing_cover_inr AND parents_age_max AND age.
-    import re as _re2
-    lc = text.lower()
-    try:
-        from backend.fact_find_normalizer import (
-            _parse_int, _parse_existing_cover, _keyword_normalize,
-            _validate, _FIELD_SCHEMA,
-        )
-        schema = _FIELD_SCHEMA.get(slot_id)
-
-        if slot_id == "age":
-            # Require an age trigger so a bare "29" in "₹29L cover" doesn't fire.
-            if not _re2.search(r"\b(?:i'?m|i\s+am|i\s+am\s+about|age|aged|years?\s*old|yrs?\s*old|y/?o)\b", lc):
-                # Bare number on its own line is also OK (typed "29")
-                if not _re2.match(r"^\s*\d{1,3}\s*\.?\s*$", text):
-                    return None
-            val = _parse_int(text, schema or {"min": 1, "max": 120})
-            # Age sanity: 18-99
-            if val is not None and 18 <= val <= 99:
-                return val
-            return None
-
-        if slot_id == "parents_age":
-            # MUST explicitly mention parent context.
-            if not _re2.search(r"\b(parent|mom|mum|mother|dad|father|mama|papa)", lc):
-                return None
-            val = _parse_int(text, schema or {"min": 30, "max": 110})
-            if val is not None and 30 <= val <= 110:
-                return val
-            return None
-
-        if slot_id == "existing_cover":
-            # KI-074 — require a cover-context trigger. Word "cover" alone
-            # is too weak (matches "looking for cover" with no amount).
-            # Need either an explicit denial OR a currency/unit token.
-            denial = _re2.search(
-                r"\b(no|none|nothing|zero|nope|nah|never|haven'?t|don'?t\s+have|first\s+(?:policy|insurance|one|time|buy)|new\s+to\s+insurance|don'?t\s+have\s+any)\b",
-                lc,
-            )
-            # Currency or unit cue. Plain "cover" word doesn't count.
-            unit_cue = _re2.search(
-                r"(₹|\brs\.?\s*\d|\d+\s*(?:lakh|lac|crore|cr)\b|"
-                r"\bsum\s+insured\b|\bcovered\s+for\b|\bemployer.*\bcover|"
-                r"\bfrom\s+work\b)",
-                lc,
-            )
-            if not denial and not unit_cue:
-                return None
-            return _parse_existing_cover(text)
-
-        # Enum / list slots — keyword fast path only, no LLM.
-        # Each slot has its own trigger keywords inside _keyword_normalize.
-        kw = _keyword_normalize(slot_id, text)
-        if kw is not None and schema is not None:
-            return _validate(kw, schema)
-        return kw
-    except Exception:
-        return None
-
-
-def _canonical_fallback(session, user_text: str, *, reason: str) -> FactFindOutcome:
-    """Produce a deterministic, on-rails fallback reply when the LLM brain
-    times out or returns unparseable output. Uses the canonical needs_finder
-    question for the next missing slot so the user always sees a coherent
-    next step.
-
-    KI-072 (2026-05-15) — applies the user's message to the previously-asked
-    slot via the legacy normalizer BEFORE picking the next slot.
-
-    KI-074 (2026-05-15) — GREEDY multi-slot capture. Previous KI-072 logic
-    only checked `awaiting_question_id` → dropped facts when brain failed on
-    a multi-fact message ("I am 29 and just me" lost age + dependents
-    because the LLM happened to be driving the `name` slot). Now we try
-    every unfilled slot against the user_text and capture whatever
-    matches — same multi-fact spirit as KI-070, just without the LLM call.
-    """
-    profile = session.profile
-    captured: dict[str, Any] = {}
-
-    if (user_text or "").strip():
-        try:
-            from backend.needs_finder import GRAPH
-
-            # KI-092 — if the bot was explicitly awaiting the name slot,
-            # try the LENIENT name parser FIRST so a plain "rohit sar"
-            # reply gets captured instead of falling through to the strict
-            # intro-phrase requirement and never matching.
-            awaiting = getattr(session, "awaiting_question_id", None)
-            if awaiting == "name" and not getattr(profile, "name", None):
-                try:
-                    lenient_name = _normalize_for_slot("name__awaiting", user_text)
-                except Exception:
-                    lenient_name = None
-                if lenient_name:
-                    q_obj = next((q for q in GRAPH if q.id == "name"), None)
-                    if q_obj is not None:
-                        captured[q_obj.field] = lenient_name
-                        setattr(profile, q_obj.field, lenient_name)
-                        if "name" not in profile.asked:
-                            profile.asked.append("name")
-
-            # Build the prioritised slot order — try high-signal slots first
-            # (numbers, enums) before name (which has explicit-intro guard).
-            # KI-158 (2026-05-15) — added "health_conditions". Its absence
-            # meant a user replying "I'm not having any pre-existing condition"
-            # NEVER got the slot captured here; the LLM brain failing on a
-            # `no_trailer` reply for that turn (common under NIM load) bumped
-            # _failed_attempts['health_conditions'] each turn until the
-            # loop-breaker force-skipped the slot, which then made
-            # next_question() return None and triggered the gentle hand-off
-            # mid-fact-find.
-            _GREEDY_ORDER = [
-                "age", "dependents", "income_band", "existing_cover",
-                "primary_goal", "location", "parents_age", "health_conditions",
-                "budget", "name",
-            ]
-            ordered_slots: list[str] = [
-                sid for sid in _GREEDY_ORDER
-                if any(q.id == sid for q in GRAPH)
-            ]
-
-            for slot_id in ordered_slots:
-                q_obj = next((q for q in GRAPH if q.id == slot_id), None)
-                if q_obj is None:
-                    continue
-                # Skip already-filled slots
-                current = getattr(profile, q_obj.field, None)
-                if current not in (None, "", []):
-                    continue
-                try:
-                    val = _normalize_for_slot(slot_id, user_text)
-                except Exception:
-                    val = None
-                if val is None:
-                    continue
-                captured[q_obj.field] = val
-                # KI-095 — route through session.update_profile_field for
-                # consistency with the rest of the codebase (centralised
-                # write + flush); val is guarded non-None just above.
-                session.update_profile_field(q_obj.field, val)
-                if slot_id not in profile.asked:
-                    profile.asked.append(slot_id)
-                # KI-103 — successful capture resets the failed-attempt
-                # counter for this slot so a temporary loss of the LLM
-                # brain doesn't permanently ghost the slot.
-                _failed_attempts(session).pop(slot_id, None)
-        except Exception as e:
-            logging.info("canonical_fallback greedy capture failed: %s", e)
-
-    # KI-103 (2026-05-15) — loop breaker. The legacy code surfaced the SAME
-    # unfilled slot every turn the brain returned `no_trailer`, even when
-    # the user explicitly answered it on turn 1. Now we track per-slot
-    # consecutive failed-fallback surfaces on the session; after
-    # _MAX_FAILED_ATTEMPTS consecutive surfaces of slot S with NO capture
-    # for S, we mark S as asked (so `next_question` advances) and append
-    # it to `_ff_skipped_slots` so the orchestrator/scorecard can render
-    # it as intentionally unanswered.
-    try:
-        from backend.needs_finder import next_question
-        attempts = _failed_attempts(session)
-        skipped = _skipped_slots(session)
-        # Bounded loop: at most len(GRAPH)+1 iterations so we can never
-        # infinite-loop on a pathological state. In practice we exit on
-        # the first slot that's either fresh OR has been skipped now.
-        q = None
-        for _ in range(20):
-            q = next_question(profile)
-            if q is None:
-                break
-            slot_id = q.id
-            # If this turn's greedy capture filled this slot, surface
-            # the NEXT slot (we already captured the answer; don't re-ask).
-            if q.field in captured:
-                profile.asked.append(slot_id) if slot_id not in profile.asked else None
-                continue
-            # Count this as a failed surface for the slot — we're about
-            # to re-ask it without having captured a value for it.
-            attempts[slot_id] = attempts.get(slot_id, 0) + 1
-            if attempts[slot_id] > _MAX_FAILED_ATTEMPTS:
-                # 3rd-or-later attempt: degrade gracefully — skip the slot
-                # entirely so we don't re-ask. Mark asked + record in
-                # _ff_skipped_slots and loop to pick the next unfilled slot.
-                logging.info(
-                    "KI-103: canonical fallback skipping slot=%s after %d failed attempts",
-                    slot_id, attempts[slot_id] - 1,
-                )
-                if slot_id not in profile.asked:
-                    profile.asked.append(slot_id)
-                if slot_id not in skipped:
-                    skipped.append(slot_id)
-                # Reset the counter so if we ever do capture later via a
-                # different code path, the slot can be re-introduced cleanly.
-                attempts.pop(slot_id, None)
-                continue
-            # Within tolerance — surface this slot to the user.
-            break
-    except Exception:
-        q = None
-    if q is not None:
-        # KI-156 (2026-05-15) — when the LLM bailed but we DID greedy-capture
-        # something this turn, prepend a brief acknowledgement so the user
-        # doesn't perceive the bot as ignoring them. Pre-fix the fallback
-        # emitted only the bare scripted prompt_en, making the conversation
-        # look unresponsive ("user gives city → bot asks name again" pattern).
-        ack_parts: list[str] = []
-        _PRETTY = {
-            "name": "name",
-            "age": "age",
-            "dependents": "who you're covering",
-            "income_band": "income band",
-            "existing_cover_inr": "existing cover",
-            "primary_goal": "goal",
-            "location_tier": "city",
-            "parents_age_max": "parents' age",
-            "health_conditions": "health conditions",
-            "budget_band": "budget",
-        }
-        for k in ("name", "age", "dependents", "location_tier", "income_band",
-                 "existing_cover_inr", "primary_goal", "budget_band"):
-            if k in captured:
-                ack_parts.append(_PRETTY.get(k, k))
-        if ack_parts:
-            ack = f"Got that — {', '.join(ack_parts)}. "
-            reply = ack + q.prompt_en
-        else:
-            reply = q.prompt_en
-        slot = q.field
-        return FactFindOutcome(
-            reply_text=reply,
-            captured_updates=captured,  # KI-072 — propagate the captured answer
-            slot_driving=slot,
-            fact_find_complete=False,
-            ambiguous=True,
-            _fallback_reason=reason,  # KI-078 — telemetry stamp
-        )
-    # Nothing left to ask — fact-find is genuinely complete (every applicable
-    # slot is filled OR has been intentionally skipped by the loop-breaker).
-    #
-    # KI-158 (2026-05-15) — was emitting the "Let me know a bit about yourself
-    # — your age..." gentle hand-off which is the WRONG message at this point:
-    # the user just answered the LAST slot, every prior slot is on file, but
-    # the canonical-fallback path landed here because the LLM brain failed on
-    # the trailer block (no_trailer / empty_reply). Re-asking "tell me a bit
-    # about yourself" makes the bot look forgetful mid-conversation.
-    #
-    # Fix:
-    #   1. Propagate `captured` (was discarded as `{}`) so any greedy capture
-    #      made this turn — most commonly the no-PED `health_conditions=[]` —
-    #      is applied to the profile by the orchestrator post-loop.
-    #   2. Flip `fact_find_complete=True` so the orchestrator switches the
-    #      session into free-form mode and the NEXT user turn routes to the
-    #      retrieval brain (which can recommend), not back here.
-    #   3. Replace the awkward hand-off prose with a clean acknowledge +
-    #      transition that mirrors the brain's `complete=true` summary path
-    #      — the user just told us they're healthy, we have everything else
-    #      already, so transition to recommendations.
-    reply = (
-        "Got it — no pre-existing conditions noted. I have everything I need. "
-        "Want me to suggest a couple of policies that fit your profile?"
-    )
-    return FactFindOutcome(
-        reply_text=reply,
-        captured_updates=captured,  # KI-158 — propagate greedy captures
-        slot_driving=None,
-        fact_find_complete=True,    # KI-158 — flip to free-form
-        ambiguous=True,
-        _fallback_reason=reason,  # KI-078 — telemetry stamp
-    )

backend/fact_find_normalizer.py

@@ -1,452 +0,0 @@
-"""LLM-based normalizer for fact-find answers.
-
-Translates raw natural-language user replies into the schema values expected
-by `backend/needs_finder.py::Profile`. Plus a non-answer detector that skips
-recording when the input is an STT failure template, empty, or off-topic.
-
-This fixes two symptoms surfaced in production on 2026-05-14:
-
-1. Free-text answers being stored verbatim instead of mapped to enums.
-   Example: user said "for now, just me" → stored as
-   `dependents="Um, for now, just me."` instead of `dependents="self"`.
-   The frontend Profile panel's enum-button comparison then never matches,
-   so the sidebar shows no selected option even though chat captured it.
-
-2. STT-failure fallback messages (or empty transcripts) being recorded as
-   the user's answer to the in-flight question. The next question silently
-   moves on with garbage.
-
-Architecture:
-  - `is_valid_answer(text)` — cheap guard that filters non-answers BEFORE
-    any LLM call.
-  - `normalize_answer(question_id, raw)` — async; fast-path regex for
-    numeric fields (age, parents_age, existing_cover); LLM call (NIM
-    Llama-3.3-70B at temperature 0) for enum and list fields.
-  - Returns None when the input can't be mapped → the orchestrator should
-    NOT clear `awaiting_question_id` so the bot re-asks the same question
-    (the "ask me again" behavior the human asked for).
-"""
-
-from __future__ import annotations
-
-import json
-import logging
-import re
-from typing import Any
-
-# ----------------------------------------------------------------------------
-# Field schema — what each fact-find question expects after normalization.
-# Question IDs must match `backend/needs_finder.py::GRAPH[i].id` exactly.
-# ----------------------------------------------------------------------------
-
-_FIELD_SCHEMA: dict[str, dict] = {
-    "age": {"type": "int", "min": 1, "max": 120},
-    "dependents": {
-        "type": "enum",
-        "values": [
-            "self",
-            "self+spouse",
-            "self+spouse+kids",
-            "self+parents",
-            "self+spouse+kids+parents",
-        ],
-    },
-    "income_band": {
-        "type": "enum",
-        "values": ["under_5L", "5L-10L", "10L-25L", "25L+"],
-    },
-    "existing_cover": {"type": "int", "min": 0, "max": 100_000_000},
-    "primary_goal": {
-        "type": "enum",
-        "values": ["first_buy", "upgrade", "compare_specific", "tax_planning"],
-    },
-    "location": {
-        "type": "enum",
-        "values": ["metro", "tier1", "tier2", "tier3"],
-    },
-    "parents_age": {"type": "int", "min": 30, "max": 120},
-    "health_conditions": {
-        "type": "list",
-        "common_values": ["diabetes", "hypertension", "thyroid", "asthma", "heart", "cancer"],
-    },
-    "budget": {
-        "type": "enum",
-        "values": ["under_15k", "15k_30k", "30k_60k", "60k+"],
-    },
-}
-
-# Non-answer fingerprints — these strings will skip the LLM and the recording.
-_NON_ANSWER_PATTERNS = [
-    "sorry, i couldn't hear",
-    "couldn't hear that clearly",
-    "transcribe error",
-    "transcribe failed",
-    "stt failed",
-    "no audio",
-    "[transcription failed]",
-    "please try again",
-]
-
-
-def is_valid_answer(text: str) -> bool:
-    """Return False when text is empty, too short, or a known failure template."""
-    if not text or not text.strip():
-        return False
-    s = text.strip().lower()
-    if len(s) < 2:
-        return False
-    if any(p in s for p in _NON_ANSWER_PATTERNS):
-        return False
-    return True
-
-
-async def normalize_answer(question_id: str, raw_text: str) -> Any:
-    """Map natural-language `raw_text` to the schema value for `question_id`.
-
-    Returns:
-      - parsed value (int / enum string / list[str]) on success
-      - None when the answer can't be confidently mapped (caller should re-ask)
-    """
-    if not is_valid_answer(raw_text):
-        return None
-
-    schema = _FIELD_SCHEMA.get(question_id)
-    if schema is None:
-        return raw_text.strip() or None
-
-    # Fast paths — no LLM needed for plain integers / cover-amount parsing.
-    if question_id in ("age", "parents_age"):
-        return _parse_int(raw_text, schema)
-    if question_id == "existing_cover":
-        return _parse_existing_cover(raw_text)
-
-    # KEYWORD FAST PATH — robust to NIM rate-limit and to LLM hiccups.
-    # Try matching common patterns BEFORE the LLM call. Catches ~80% of
-    # answers without consuming a NIM request and is deterministic under
-    # load. The LLM is the fall-back for nuanced/edge phrasings.
-    kw = _keyword_normalize(question_id, raw_text)
-    if kw is not None:
-        validated = _validate(kw, schema)
-        if validated is not None:
-            return validated
-
-    # Enum + list fields — let the LLM map natural language to canonical value.
-    return await _llm_normalize(question_id, raw_text, schema)
-
-
-# ----------------------------------------------------------------------------
-# Keyword fast-path — hand-curated common phrasing → schema value.
-# Order matters within each field: more-specific patterns first.
-# Case-insensitive substring matches on a normalized version of the text.
-# ----------------------------------------------------------------------------
-
-def _keyword_normalize(question_id: str, raw_text: str) -> Any:
-    s = raw_text.lower()
-
-    if question_id == "dependents":
-        # KI-014 — vague terms like "family" / "everyone" must NOT auto-map.
-        # User testing surfaced: user said "family" → bot assumed self+spouse+kids
-        # → wrong (user may have meant parents, siblings, joint-family etc.).
-        # Returning None here forces the LLM normalizer (or re-ask) to clarify.
-        VAGUE_TERMS = ["family", "everyone", "all of us", "everybody", "whole family", "joint family"]
-        if any(v in s for v in VAGUE_TERMS) and not any(
-            k in s for k in ["spouse", "wife", "husband", "kid", "child", "parent"]
-        ):
-            return None  # force clarification
-
-        if any(k in s for k in ["spouse", "wife", "husband"]) and "kid" in s and "parent" in s:
-            return "self+spouse+kids+parents"
-        if any(k in s for k in ["spouse", "wife", "husband"]) and "kid" in s:
-            return "self+spouse+kids"
-        if any(k in s for k in ["spouse", "wife", "husband"]) and "parent" in s:
-            return "self+spouse+kids+parents"
-        if any(k in s for k in ["spouse", "wife", "husband"]):
-            return "self+spouse"
-        if "parent" in s and "no" not in s.split():
-            return "self+parents"
-        if "just me" in s or "only me" in s or "myself" in s or "only self" in s or s.strip() in {"me", "self"}:
-            return "self"
-
-    elif question_id == "income_band":
-        # KI-149 (2026-05-15) — delegate to needs_finder._parse_income_band so
-        # bare digits ("500000"), "thousand"/"grand"/"k" suffixes, and "₹"
-        # prefixes all parse identically here AND in the GRAPH parser used by
-        # `record_answer`. Single source of truth.
-        from backend.needs_finder import _parse_income_band
-        return _parse_income_band(raw_text)
-
-    elif question_id == "primary_goal":
-        if any(k in s for k in ["first policy", "first one", "first time", "first buy", "new policy", "buying my first"]):
-            return "first_buy"
-        if any(k in s for k in ["upgrade", "upgrading", "better cover", "more cover", "increase cover"]):
-            return "upgrade"
-        if any(k in s for k in ["compare", "comparison", " vs ", " vs.", "versus"]):
-            return "compare_specific"
-        if any(k in s for k in [" tax ", "80d", "deduction", "tax planning"]):
-            return "tax_planning"
-
-    elif question_id == "location":
-        metro = ["bangalore", "bengaluru", "mumbai", "delhi", "new delhi", "chennai", "kolkata", "hyderabad", "pune"]
-        tier1 = ["ahmedabad", "jaipur", "lucknow", "kanpur", "nagpur", "indore", "thane", "bhopal", "visakhapatnam", "patna", "vadodara", "ghaziabad", "ludhiana", "agra", "nashik"]
-        tier2 = ["surat", "kochi", "trivandrum", "thiruvananthapuram", "coimbatore", "vijayawada", "madurai", "rajkot", "ranchi", "amritsar", "allahabad", "prayagraj", "jodhpur", "raipur"]
-        for c in metro:
-            if c in s: return "metro"
-        for c in tier1:
-            if c in s: return "tier1"
-        for c in tier2:
-            if c in s: return "tier2"
-        if "metro" in s: return "metro"
-        if "tier 1" in s or "tier1" in s: return "tier1"
-        if "tier 2" in s or "tier2" in s: return "tier2"
-        if "tier 3" in s or "tier3" in s or "village" in s or "small town" in s: return "tier3"
-
-    elif question_id == "budget":
-        # KI-149 (2026-05-15) — delegate to needs_finder._parse_budget_band so
-        # "I maximum 30000 I can pay" / "30 thousand" / "₹30,000" / "30 grand"
-        # / bare "30000" all parse the same way the GRAPH parser does. The
-        # keyword path used to require a "k" suffix and lost bare-digit
-        # answers (user reported re-ask bug on live HF Space).
-        from backend.needs_finder import _parse_budget_band
-        return _parse_budget_band(raw_text)
-
-    elif question_id == "health_conditions":
-        # KI-158 (2026-05-15) — broadened no-PED denial detection. The previous
-        # substring list missed "I'm not having any pre-existing condition",
-        # "no pre-existing condition" (singular), "don't have any", "nothing
-        # chronic", etc. Each miss let _canonical_fallback's loop-breaker tick
-        # the health_conditions slot toward force-skip, eventually making
-        # next_question() return None → gentle hand-off fires mid-fact-find.
-        # The fix uses regex to cover the natural phrasings without
-        # over-triggering on actual conditions like "I have diabetes".
-        _NO_PED_PATTERNS = [
-            r"\b(?:no|none|nothing|nope|nah|negative)\b\s*(?:condition|pre[-\s]?exist|health\s+issue|chronic|illness)?",
-            r"\b(?:not|don'?t|do\s+not|haven'?t|have\s+not|isn'?t|aren'?t)\s+(?:got|have|having|had|got\s+any|have\s+any)\b",
-            r"\b(?:no|zero|nil)\s+(?:pre[-\s]?exist\w*|condition|chronic|health\s+issue|illness|disease)",
-            r"\b(?:i\s+am|i'm)\s+(?:healthy|fine|fit|alright|all\s+good|good|ok|okay)\b",
-            r"\bnothing\s+(?:chronic|major|serious|to\s+report|like\s+that)\b",
-            r"\ball\s+(?:good|fine|clear|healthy)\b",
-            r"\bclean\s+bill\s+of\s+health\b",
-        ]
-        if any(re.search(p, s) for p in _NO_PED_PATTERNS):
-            # Belt-and-braces: ensure the user didn't ALSO mention a real
-            # condition in the same message (e.g., "no diabetes but I have BP").
-            # If a condition keyword is present below, the canonical list
-            # branch will pick it up regardless.
-            return []
-        canonical = []
-        cond_keywords = {
-            "diabetes": ["diabetes", "diabetic", "sugar"],
-            "hypertension": ["hypertension", " bp ", "blood pressure", "high bp"],
-            "thyroid": ["thyroid", "hypothyroid", "hyperthyroid"],
-            "asthma": ["asthma"],
-            "heart": ["heart problem", "heart disease", "cardiac"],
-            "cancer": ["cancer", "tumor"],
-        }
-        for cond, kws in cond_keywords.items():
-            if any(k in s for k in kws):
-                canonical.append(cond)
-        if canonical:
-            return canonical
-
-    return None
-
-
-# ----------------------------------------------------------------------------
-# Fast-path parsers (no LLM)
-# ----------------------------------------------------------------------------
-
-def _parse_int(text: str, schema: dict) -> int | None:
-    digits = "".join(c for c in str(text) if c.isdigit())
-    if not digits:
-        return None
-    try:
-        v = int(digits[:3])
-    except ValueError:
-        return None
-    if v < schema.get("min", 0) or v > schema.get("max", 9_999):
-        return None
-    return v
-
-
-def _parse_existing_cover(text: str) -> int | None:
-    """Handle "no" / "none" / "5 lakh" / "₹500000" / "5L" / "haven't got any" / "30k".
-
-    KI-067 (2026-05-15) — also recognises first-time-buyer phrasings
-    ("first policy", "first time buying", "new to insurance"). The previous
-    pattern caught "No, this is my first policy" via the "no" word but
-    rejected "This is my first policy" alone — user had to retry.
-    """
-    s = text.lower().strip()
-    # Negative answers map to 0 (no existing cover).
-    # KI-156 (2026-05-15) — extended to cover gerund/auxiliary phrasings.
-    # Pre-fix "I am not having any policy" / "currently not having" / "not
-    # got any" / "without any" all returned None, causing the fact-find
-    # loop to keep re-asking the existing_cover slot.
-    if re.search(
-        r"\b(no|none|nothing|zero|nope|nah|haven'?t|don'?t|never|"
-        r"not\s+having|not\s+got|not\s+have|without\s+(?:any|a))\b",
-        s,
-    ):
-        return 0
-    # KI-067 — first-time-buyer signals also imply zero existing cover.
-    _first_time_patterns = (
-        r"first\s+(?:policy|insurance|one|time|buy|cover)",
-        r"buying\s+(?:my\s+)?first",
-        r"new\s+(?:to\s+)?(?:health\s+)?insurance",
-        r"never\s+(?:bought|had|got|taken)",
-        r"haven'?t\s+(?:bought|had|got|taken)",
-        r"don'?t\s+have\s+(?:any|insurance|cover|a\s+policy)",
-    )
-    if any(re.search(p, s) for p in _first_time_patterns):
-        return 0
-
-    # Look for a number followed by a unit suffix (digit-attached OR separated).
-    # crore > lakh > thousand priority so longer units win the alternation.
-    cr_match = re.search(r"(\d+(?:\.\d+)?)\s*(?:cr|crore|crores)\b", s)
-    if cr_match:
-        try:
-            return int(float(cr_match.group(1)) * 10_000_000)
-        except ValueError:
-            return None
-    lakh_match = re.search(r"(\d+(?:\.\d+)?)\s*(?:l(?:akh|ac)?s?)\b", s)
-    if lakh_match:
-        try:
-            return int(float(lakh_match.group(1)) * 100_000)
-        except ValueError:
-            return None
-    k_match = re.search(r"(\d+(?:\.\d+)?)\s*k\b", s)
-    if k_match:
-        try:
-            return int(float(k_match.group(1)) * 1_000)
-        except ValueError:
-            return None
-
-    # Plain digit-only amount (e.g., "500000").
-    digits = "".join(c for c in text if c.isdigit())
-    if not digits:
-        return None
-    try:
-        amount = int(digits[:7])
-    except ValueError:
-        return None
-    if amount < 0 or amount > 100_000_000:
-        return None
-    return amount
-
-
-# ----------------------------------------------------------------------------
-# LLM-backed normalizer for enum + list fields
-# ----------------------------------------------------------------------------
-
-_LLM_SYSTEM_TEMPLATE = """You map a user's natural-language answer to a structured value.
-
-Question ID: {qid}
-Expected schema: {schema}
-
-Rules:
-1. If type=enum, return EXACTLY one of the allowed values (a JSON string), or null if no clear match.
-2. If type=list, return a JSON array of canonical lowercase condition strings. For "no", "none", "nothing" → [].
-3. If the user clearly didn't answer the question (off-topic, asking back, gibberish), return null.
-4. Output ONLY the JSON value — no prose, no code fences, no <think> blocks.
-
-Examples for guidance:
-- dependents enum, user "just me" → "self"
-- dependents enum, user "me and my wife" → "self+spouse"
-- dependents enum, user "I want coverage for my parents too" → "self+parents"
-- income_band enum, user "around 18 lakhs" → "10L-25L"
-- income_band enum, user "more than 25 lakhs" → "25L+"
-- primary_goal enum, user "I'm buying my first one" → "first_buy"
-- primary_goal enum, user "want to compare HDFC and ICICI" → "compare_specific"
-- location enum, user "Bangalore" → "metro"
-- location enum, user "Patna" → "tier2"
-- budget enum, user "around 20k a year" → "15k_30k"
-- health_conditions list, user "none" → []
-- health_conditions list, user "diabetes and BP" → ["diabetes", "hypertension"]
-- health_conditions list, user "I have asthma" → ["asthma"]
-"""
-
-
-async def _llm_normalize(question_id: str, raw_text: str, schema: dict) -> Any:
-    from backend.providers.base import ChatMessage
-    from backend.providers.nvidia_nim_llm import FAST_BRAIN_CHAIN, NimChainLLM
-
-    sys_msg = _LLM_SYSTEM_TEMPLATE.format(qid=question_id, schema=json.dumps(schema))
-    user_msg = f'User said: "{raw_text[:600]}"\n\nReturn the JSON value.'
-
-    try:
-        # KI-033 (2026-05-14) — was hardcoded NvidiaNimLLM(meta/llama-3.3-70b);
-        # moved to fast-brain chain so when that one NIM pool rate-limits we
-        # fall through to Qwen/Nemotron/Groq instead of silently returning None
-        # (which made valid Indian-accented answers like "twenty-five" appear
-        # to fail the verifier, then trip the 2-reask cap, then move on with
-        # no profile captured — the D-005 cascade).
-        llm = NimChainLLM(chain=FAST_BRAIN_CHAIN, timeout=10.0,
-                          role="fact_find_normalizer", total_budget_s=15.0)
-        result = await llm.chat(
-            messages=[
-                ChatMessage(role="system", content=sys_msg),
-                ChatMessage(role="user", content=user_msg),
-            ],
-            temperature=0.0,
-            max_tokens=120,
-        )
-        raw = (result.text or "").strip()
-    except Exception as e:
-        logging.warning(
-            "fact_find_normalizer LLM call failed (qid=%s, raw=%r): %s",
-            question_id, raw_text[:80], e,
-        )
-        return None
-
-    # Strip <think> blocks and code fences that some models add despite instructions.
-    if "<think>" in raw and "</think>" in raw:
-        raw = raw.split("</think>", 1)[1].strip()
-    if raw.startswith("```"):
-        raw = "\n".join(l for l in raw.split("\n") if not l.startswith("```")).strip()
-    if not raw or raw.lower() == "null":
-        return None
-
-    try:
-        parsed = json.loads(raw)
-    except json.JSONDecodeError:
-        # Some models return bare strings without JSON quoting; tolerate.
-        if schema["type"] == "enum" and raw.strip('"') in schema["values"]:
-            return raw.strip('"')
-        return None
-
-    return _validate(parsed, schema)
-
-
-def _validate(value: Any, schema: dict) -> Any:
-    """Type + enum + bounds check. Returns None on failure."""
-    t = schema.get("type")
-
-    if t == "enum":
-        if isinstance(value, str) and value in schema["values"]:
-            return value
-        return None
-
-    if t == "int":
-        if isinstance(value, bool):
-            return None
-        try:
-            v = int(value)
-        except (TypeError, ValueError):
-            return None
-        if v < schema.get("min", -1_000_000_000) or v > schema.get("max", 1_000_000_000):
-            return None
-        return v
-
-    if t == "list":
-        if not isinstance(value, list):
-            return None
-        cleaned = [str(x).strip().lower() for x in value if x and isinstance(x, (str, int))]
-        cleaned = [c for c in cleaned if c]
-        return cleaned  # [] is a valid answer (= "no conditions")
-
-    if t == "bool":
-        if isinstance(value, bool):
-            return value
-        return None
-
-    return value

backend/main.py

@@ -1160,9 +1160,10 @@ async def profile_completeness_view(session_id: Optional[str] = None):
     missing = [k for k, v in profile_dict.items() if v in (None, "", [])]
     hint = None
     try:
-        nq = next_question(p)
-        if nq:
-            hint = nq.prompt_en
+        # KI-167 WS3 (2026-05-15) — next_question now returns the field name
+        # (str) of the next missing slot, not a Question object. Frontend uses
+        # this as a slot-hint; the actual phrasing is generated by sales_brain.
+        hint = next_question(p)
     except Exception:
         pass
     return ProfileCompletenessResponse(

backend/needs_finder.py

@@ -1,24 +1,22 @@
-"""Adaptive fact-find question graph.
+"""User profile state + slot-order hint for the fact-find.
 
-Inspired by what a good Independent Financial Advisor does in the first 10
-minutes of a consultation: ask a stable core of questions, then deep-dive
-conditionally based on signal. The depth is adaptive — stop when we have
-enough to recommend.
+KI-167 WS3 (2026-05-15) — the legacy rules-engine question GRAPH and the
+matching `Question` dataclass have been removed. Fact-find phrasing now
+lives in `backend/sales_brain.py` (the single-LLM-call brain); this module
+keeps only:
 
-The graph is explicit (not LLM-improvised) so:
-  - A reviewer can see and audit it
-  - Behavior is testable
-  - Failure modes are tractable
-  - It works without an LLM (fallback when the brain is degraded)
+  - `Profile`              — accumulated user state (still imported widely)
+  - `record_answer`        — slot-write helper used by session_state
+  - INR / budget / income parsers — used by `sales_brain_normalizer.py`
+  - `is_field_set`         — local helper for record_answer + next_question
+  - `next_question`        — returns the field NAME (str) of the next
+                             missing slot in canonical order. Used by
+                             `/api/profile/completeness`'s `next_question_hint`.
 
 Public API:
-  - Profile dataclass — accumulated user state
-  - next_question(profile) -> str | None   (None = ready to recommend)
-  - record_answer(profile, question_id, answer) -> Profile
-  - readback_summary(profile) -> str
-
-The orchestrator can choose to drive the fact-find OR let the user
-free-form questions — the graph supports both.
+  - Profile dataclass
+  - next_question(profile) -> str | None   (field name, None = complete)
+  - record_answer(profile, field_name, raw_value) -> Profile
 """
 
 from __future__ import annotations
@@ -43,7 +41,7 @@ class Profile:
     parents_has_ped: Optional[bool] = None  # if parents_to_insure
     budget_band: Optional[str] = None  # "under_15k", "15k_30k", "30k_60k", "60k+"
     health_conditions: Optional[list[str]] = field(default_factory=list)  # ["diabetes", "hypertension", ...]
-    asked: list[str] = field(default_factory=list)  # question IDs already asked
+    asked: list[str] = field(default_factory=list)  # question IDs / field names already asked
     free_form_session: bool = False  # True = user asks free questions, not driven by us
     # KI-063 (2026-05-15) — per-user policy interaction log so the bot
     # remembers which policies were shown / selected / rejected across
@@ -56,26 +54,6 @@ class Profile:
     rejected_policies: list[dict] = field(default_factory=list)  # KI-063
 
 
-# ----------------------------------------------------------------------------
-# Question graph — each node has an id, a prompt, and a condition for being
-# asked. Conditions are pure functions of the current Profile.
-# ----------------------------------------------------------------------------
-
-@dataclass
-class Question:
-    id: str
-    prompt_en: str
-    prompt_hi: str  # optional Hindi rendering (used when language='indic')
-    field: str
-    is_core: bool = False
-    condition: Any = None  # callable(profile) -> bool, default: always ask
-    parser: Any = None  # callable(text) -> value, default: text as-is
-
-
-def _always(p: Profile) -> bool:
-    return True
-
-
 # ----------------------------------------------------------------------------
 # KI-149 (2026-05-15) — free-text INR amount parser for budget + income.
 # User said "I maximum 30000 I can pay" → bot re-asked budget because no
@@ -219,97 +197,27 @@ def _parse_income_band(text: str) -> Optional[str]:
     return "25L+"
 
 
-GRAPH: list[Question] = [
-    Question(
-        id="name",
-        prompt_en="First — what should I call you? I'll save your profile so the next time you visit, you won't have to answer all this again.",
-        prompt_hi="सबसे पहले — आपको क्या कहूँ? आपकी profile save हो जाएगी, तो अगली बार दोबारा सवाल नहीं पूछूँगा।",
-        field="name",
-        is_core=True,
-        # Free-text — no parser; the orchestrator validates 1-50 chars
-    ),
-    Question(
-        id="age",
-        prompt_en="First, your age? (Premium + eligibility + how long you can renew all hinge on this.)",
-        prompt_hi="पहला — आपकी उम्र? (Premium और renewal age इसी पर निर्भर हैं।)",
-        field="age",
-        is_core=True,
-        parser=lambda s: int("".join(c for c in str(s) if c.isdigit())[:3] or 0) or None,
-    ),
-    Question(
-        id="dependents",
-        prompt_en="Who else needs cover — just you, spouse, kids, parents, or a mix? (Covering parents shifts which policies fit; parent-specific plans often win.)",
-        prompt_hi="किसको cover करना है — सिर्फ आप, पति/पत्नी, बच्चे, माता-पिता, या मिलाकर? (माता-पिता के साथ recommendations काफी बदलते हैं।)",
-        field="dependents",
-        is_core=True,
-    ),
-    Question(
-        id="income_band",
-        prompt_en="Annual income band — under ₹5L, ₹5-10L, ₹10-25L, or ₹25L+? (Helps us suggest the right sum-insured size. Not shared with anyone.)",
-        prompt_hi="सालाना आय — ₹5L से कम, ₹5-10L, ₹10-25L, या ₹25L+? (हम सिर्फ sum insured size suggest करने के लिए पूछते हैं।)",
-        field="income_band",
-        is_core=True,
-        parser=_parse_income_band,
-    ),
-    Question(
-        id="existing_cover",
-        prompt_en="Already have any health insurance — employer-provided or your own? If yes, roughly what sum insured? (If you have ₹5L from work, a top-up plan may fit better than a full base plan.)",
-        prompt_hi="पहले से कोई health insurance है — employer का या personal? Sum insured कितना? (Top-up plan ज़्यादा सही हो सकता है।)",
-        field="existing_cover_inr",
-        is_core=True,
-        parser=lambda s: (
-            0 if any(k in str(s).lower() for k in ("no", "none", "nothing", "zero", "not"))
-            else (
-                (int("".join(c for c in str(s) if c.isdigit())[:6] or 0) *
-                 (100000 if any(k in str(s).lower() for k in ("l", "lakh", "lac")) else 1))
-                or None
-            )
-        ),
-    ),
-    Question(
-        id="primary_goal",
-        prompt_en="What's brought you here — first health policy, upgrading existing cover, comparing specific policies, or tax planning? (Tells us whether to grade you on price, breadth of cover, claim experience, or tax savings.)",
-        prompt_hi="आप यहाँ क्यों हैं — पहली policy, upgrade, specific compare, या tax planning? (इससे हम तय करते हैं कि आपको price, coverage, claim experience या tax savings पर grade करें।)",
-        field="primary_goal",
-        is_core=True,
-    ),
-    Question(
-        id="location",
-        prompt_en="Which city? (Cashless hospital network density varies wildly — a '16,000-hospital network' claim means nothing if none are near you.)",
-        prompt_hi="कौन सा शहर? (Cashless network आपके शहर में कितना deep है यह बहुत matter करता है।)",
-        field="location_tier",
-        is_core=True,
-    ),
-    # Conditional deep-dives
-    Question(
-        id="parents_age",
-        prompt_en="Your parents' ages, and any pre-existing conditions — diabetes, BP, heart, anything chronic? Be honest here: hiding a parent's condition is the #1 reason senior-cover claims get denied later.",
-        prompt_hi="माता-पिता की उम्र, और कोई pre-existing condition — diabetes, BP, heart? सच बताइए: hide करना senior-claims denied होने की #1 वजह है।",
-        field="parents_age_max",
-        condition=lambda p: bool(p.dependents and "parent" in p.dependents.lower()),
-    ),
-    Question(
-        id="health_conditions",
-        prompt_en="Any pre-existing conditions on your side — diabetes, BP, thyroid, asthma, anything chronic? Be straight with me here — hiding it lowers your premium ₹500 today and turns into a ₹8 lakh denied claim later when the insurer matches your disclosure against hospital records. Your honest answer protects YOUR claim, not the insurer's profit.",
-        prompt_hi="आपकी side से कोई pre-existing condition — diabetes, BP, thyroid? सच बताइए — hide करने से premium तो कम होगा, but claim time पर ₹8 lakh denied हो सकते हैं। आपकी ईमानदारी आपकी claim बचाती है।",
-        field="health_conditions",
-        condition=_always,
-    ),
-    Question(
-        id="budget",
-        prompt_en="Annual premium budget — under ₹15k, ₹15-30k, ₹30-60k, or ₹60k+? (If a slightly higher budget materially improves your protection, I'll flag it.)",
-        prompt_hi="Premium के लिए सालाना — ₹15k से कम, ₹15-30k, ₹30-60k, या ₹60k+? (अगर थोड़ा ज़्यादा budget बेहतर protection देगा, बताऊंगा।)",
-        field="budget_band",
-        is_core=True,
-        parser=_parse_budget_band,
-    ),
-]
-
-
 # ----------------------------------------------------------------------------
 # Engine
 # ----------------------------------------------------------------------------
 
+# KI-167 WS3 (2026-05-15) — canonical slot order for the fact-find hint API.
+# The actual question phrasing lives in `sales_brain.py`; this list only
+# encodes which Profile attribute we want to fill next when nothing else
+# is driving the conversation.
+_SLOT_ORDER: list[str] = [
+    "name",
+    "age",
+    "dependents",
+    "location_tier",
+    "income_band",
+    "primary_goal",
+    "existing_cover_inr",
+    "budget_band",
+    "health_conditions",
+]
+
+
 def is_field_set(profile: Profile, field_name: str) -> bool:
     v = getattr(profile, field_name, None)
     if v is None:
@@ -319,189 +227,96 @@ def is_field_set(profile: Profile, field_name: str) -> bool:
     return True
 
 
-def next_question(profile: Profile, language: str = "en") -> Optional[Question]:
-    """Return the next question to ask, or None if we have enough to recommend.
+def next_question(profile: Profile) -> Optional[str]:
+    """Return the field NAME of the next missing slot, or None if complete.
+
+    KI-167 WS3 (2026-05-15) — refactored from `Optional[Question]` to
+    `Optional[str]` after the rules-engine GRAPH was deleted. The caller in
+    `backend/main.py:/api/profile/completeness` uses this only to hint to the
+    frontend which slot to ask next; the actual question phrasing is now
+    produced by `sales_brain.py`.
 
-    KI-070 (2026-05-15) — the orchestrator no longer drives fact-find via this
-    function; the new `backend/fact_find_brain.py` single-LLM-call brain
-    handles question phrasing natively. This is now used as a fallback
-    (canonical reply when the brain times out / emits malformed JSON), as a
-    slot-not-progressing safeguard, and by `/api/profile/completeness` to
-    surface "next question hint" to the frontend.
+    A free-form session (user driving free questions) returns None so the
+    hint endpoint reports "nothing to ask".
     """
     if profile.free_form_session:
         return None
 
-    for q in GRAPH:
-        if q.id in profile.asked:
-            continue
-        if is_field_set(profile, q.field):
-            continue
-        cond = q.condition or _always
-        if cond(profile):
-            return q
-
-    # All applicable questions asked
+    for slot in _SLOT_ORDER:
+        if not is_field_set(profile, slot):
+            return slot
     return None
 
 
-def record_answer(profile: Profile, question_id: str, raw_answer: str) -> Profile:
-    """Mutate profile in place with a parsed answer."""
-    q = next((x for x in GRAPH if x.id == question_id), None)
-    if q is None:
+# Legacy GRAPH question-id → Profile field-name aliases. Some callers
+# (notably `session_state.record_answer` driven by `awaiting_question_id`)
+# still pass the old question IDs from the deleted GRAPH. Mapping them here
+# keeps that call path working without resurrecting the GRAPH.
+_QID_TO_FIELD: dict[str, str] = {
+    "existing_cover": "existing_cover_inr",
+    "location": "location_tier",
+    "parents_age": "parents_age_max",
+    "budget": "budget_band",
+}
+
+
+def record_answer(profile: Profile, question_id: str, raw_answer: Any) -> Profile:
+    """Mutate profile in place with a raw answer for a named slot.
+
+    KI-167 WS3 (2026-05-15) — parser dispatch used to live on `Question`
+    objects in the deleted GRAPH; with the GRAPH gone we apply the same
+    parser map inline. `question_id` may be either a Profile attribute name
+    (preferred) or one of the legacy GRAPH question IDs from `_QID_TO_FIELD`.
+    """
+    field_name = _QID_TO_FIELD.get(question_id, question_id)
+    if not hasattr(profile, field_name):
         return profile
     value: Any = raw_answer
-    if q.parser:
+    parser = _PARSERS.get(field_name)
+    if parser is not None:
         try:
-            value = q.parser(raw_answer)
+            value = parser(raw_answer)
         except Exception:
             value = None
     if value is not None and value != "":
-        setattr(profile, q.field, value)
+        setattr(profile, field_name, value)
         # KI-095 — only mark slot asked once setattr succeeds, so a parse
         # failure doesn't leave the slot in an asked-but-empty desync state.
-        profile.asked.append(question_id)
+        if question_id not in profile.asked:
+            profile.asked.append(question_id)
     return profile
 
 
-# ----------------------------------------------------------------------------
-# Opportunistic family/dependents extractor — KI-056 (2026-05-15)
-# ----------------------------------------------------------------------------
-# Real-user testing surfaced: when the user mentions a spouse / kids / parents
-# while answering an UNRELATED slot ("my wife also doesn't have anything" in
-# response to existing_cover), the bot just acknowledges and moves on without
-# capturing the family signal. By the time we reach the dependents slot the
-# information has been thrown away. This helper detects family mentions in any
-# free-text turn so the orchestrator can pre-fill `profile.dependents`.
-#
-# Returns one of the canonical `dependents` enum values, or None if no clear
-# family signal is present. Conservative on purpose — only the explicit
-# combinations are recognised.
-
-_FAMILY_TERM_RE = re.compile(
-    r"\b(wife|husband|spouse|partner|kids?|children|child|parents?)\b",
-    re.IGNORECASE,
-)
-_SPOUSE_RE = re.compile(r"\b(wife|husband|spouse|partner)\b", re.IGNORECASE)
-_KIDS_RE = re.compile(r"\b(kids?|children|child)\b", re.IGNORECASE)
-_PARENTS_RE = re.compile(r"\bparents?\b", re.IGNORECASE)
-
-
-def infer_dependents_from_text(text: str) -> Optional[str]:
-    """Detect spouse/kids/parents mentions in a free-text user message and
-    return the matching canonical `dependents` enum value, or None.
-
-    KI-056 (2026-05-15). Used by the orchestrator to pre-fill the dependents
-    slot opportunistically when the user volunteers family information while
-    answering a different slot.
-
-    Decision tree (in order of specificity):
-      - spouse + kids                  → "self+spouse+kids"
-      - spouse + parents               → "self+spouse+parents"
-      - spouse only                    → "self+spouse"
-      - kids only                      → "self+kids"
-      - parents only                   → "self+parents"
-      - nothing recognised             → None
-    """
-    if not text or not _FAMILY_TERM_RE.search(text):
+def _parse_age(s: Any) -> Optional[int]:
+    digits = "".join(c for c in str(s) if c.isdigit())[:3]
+    if not digits:
+        return None
+    try:
+        return int(digits) or None
+    except ValueError:
         return None
-    has_spouse = bool(_SPOUSE_RE.search(text))
-    has_kids = bool(_KIDS_RE.search(text))
-    has_parents = bool(_PARENTS_RE.search(text))
-    if has_spouse and has_kids:
-        return "self+spouse+kids"
-    if has_spouse and has_parents:
-        return "self+spouse+parents"
-    if has_spouse:
-        return "self+spouse"
-    if has_kids:
-        return "self+kids"
-    if has_parents:
-        return "self+parents"
-    return None
-
 
-# KI-068 (2026-05-15) — enum → human-readable labels for the readback
-# summary. Previously the bot read back "primary goal: first_buy; metro;
-# budget 30k_60k" verbatim, which sounds and reads like a schema dump.
-_PRIMARY_GOAL_LABELS = {
-    "first_buy": "first health policy",
-    "upgrade": "upgrading existing cover",
-    "compare_specific": "comparing specific policies",
-    "tax_planning": "tax planning",
-}
-_LOCATION_TIER_LABELS = {
-    "metro": "metro city",
-    "tier1": "tier-1 city",
-    "tier2": "tier-2 city",
-    "tier3": "tier-3 city",
-}
-_BUDGET_BAND_LABELS = {
-    "under_15k": "under ₹15,000/year",
-    "15k_30k": "₹15,000–30,000/year",
-    "30k_60k": "₹30,000–60,000/year",
-    "60k+": "₹60,000+/year",
-}
-_INCOME_BAND_LABELS = {
-    "under_5L": "under ₹5L",
-    "5_10L": "₹5–10L",
-    "10_25L": "₹10–25L",
-    "25L+": "₹25L+",
-}
-_DEPENDENTS_LABELS = {
-    "self": "just yourself",
-    "self+spouse": "you and your spouse",
-    "self+spouse+kids": "you, your spouse, and kids",
-    "self+kids": "you and your kids",
-    "self+parents": "you and your parents",
-    "self+spouse+parents": "you, your spouse, and parents",
-    "self+spouse+kids+parents": "you, your spouse, kids, and parents",
-}
 
+def _parse_existing_cover(s: Any) -> Optional[int]:
+    text = str(s).lower()
+    if any(k in text for k in ("no", "none", "nothing", "zero", "not")):
+        return 0
+    digits = "".join(c for c in text if c.isdigit())[:6]
+    if not digits:
+        return None
+    try:
+        amt = int(digits)
+    except ValueError:
+        return None
+    if any(k in text for k in ("l", "lakh", "lac")):
+        amt *= 100_000
+    return amt or None
 
-def readback_summary(profile: Profile) -> str:
-    """One-paragraph human-readable summary of the gathered profile.
 
-    KI-068 (2026-05-15) — converts schema-level enum values
-    ("first_buy", "30k_60k", "metro") into spoken labels
-    ("first health policy", "₹30,000–60,000/year", "metro city") so the
-    readback reads like a sentence, not a JSON dump.
-    """
-    bits = []
-    if profile.age:
-        bits.append(f"{profile.age} years old")
-    if profile.dependents:
-        bits.append(f"covering {_DEPENDENTS_LABELS.get(profile.dependents, profile.dependents)}")
-    if profile.income_band:
-        bits.append(f"income {_INCOME_BAND_LABELS.get(profile.income_band, profile.income_band)}")
-    ec = profile.existing_cover_inr
-    if isinstance(ec, str):
-        try:
-            ec = int("".join(c for c in ec if c.isdigit())[:8] or 0)
-        except Exception:
-            ec = None
-    if isinstance(ec, (int, float)):
-        bits.append(
-            f"existing cover ₹{int(ec):,}"
-            if ec > 0
-            else "no existing cover"
-        )
-    if profile.primary_goal:
-        bits.append(f"goal: {_PRIMARY_GOAL_LABELS.get(profile.primary_goal, profile.primary_goal)}")
-    if profile.location_tier:
-        bits.append(_LOCATION_TIER_LABELS.get(profile.location_tier, profile.location_tier))
-    if profile.parents_age_max:
-        bits.append(f"parents up to age {profile.parents_age_max}")
-    if profile.health_conditions:
-        hc = profile.health_conditions
-        # Defensive: if a string accidentally landed here, wrap it so we don't
-        # split it character-by-character in the join. Production hit this on
-        # 2026-05-14 — a verbatim STT transcript was stored as a string, then
-        # ', '.join(str) emitted "d, i, f, f, e, r, e, n, c, e, ...".
-        if isinstance(hc, str):
-            hc = [hc] if hc.strip() else []
-        if hc:
-            bits.append(f"conditions: {', '.join(str(c) for c in hc)}")
-    if profile.budget_band:
-        bits.append(f"budget {_BUDGET_BAND_LABELS.get(profile.budget_band, profile.budget_band)}")
-    return "; ".join(bits) if bits else "(no profile yet)"
+# Parser dispatch by Profile field name. Slots not listed accept the raw value.
+_PARSERS: dict[str, Any] = {
+    "age": _parse_age,
+    "income_band": _parse_income_band,
+    "existing_cover_inr": _parse_existing_cover,
+    "budget_band": _parse_budget_band,
+}

backend/orchestrator.py

@@ -553,72 +553,57 @@ async def handle_turn(
     )
 
     if treat_as_fact_find:
-        # KI-070 (2026-05-15) — single-LLM-call fact-find brain replaces the
-        # 3-layer stitching of (hardcoded canonical prompts + per-turn
-        # paraphraser + acknowledger rotation). One model call now produces
-        # the natural reply AND a machine-readable trailer describing what
-        # was captured, which slot it's driving toward, and whether
-        # fact-find is complete.
-        from backend.fact_find_brain import (
-            FIELD_TO_QUESTION_ID,
-            drive_fact_find,
-        )
+        # KI-167 (2026-05-15) — WS2: replaces drive_fact_find with
+        # drive_sales_brain. The new brain owns conversation flow end-to-end
+        # (no slot_driving / awaiting_question machinery, no canonical
+        # fallback): one LLM call emits natural prose + canonical
+        # captured_updates + a completeness signal. Failures fail loud (no
+        # scripted fallback).
+        from backend.sales_brain import drive_sales_brain, SalesBrainResult
 
-        # KI-100 — bound drive_fact_find at the orchestrator. KI-079's
-        # internal FAST(25s) + BRAIN(15s) escalation can compound to ~40s on
-        # a doubly-saturated chain, which breaches the user's <30s per-turn
-        # target. Cap at 25s; on outer timeout, fall through to the
-        # canonical fallback that drive_fact_find itself uses on internal
-        # timeout (build the outcome from current session state).
         try:
-            outcome = await asyncio.wait_for(
-                drive_fact_find(
+            sb_result: SalesBrainResult = await asyncio.wait_for(
+                drive_sales_brain(
                     user_text=user_text,
-                    session=session,
-                    chat_history=chat_history,
+                    profile=session.profile,
+                    chat_history=chat_history[-10:],
                     session_id=session_id,
                 ),
                 timeout=25.0,
             )
         except asyncio.TimeoutError:
-            logging.warning(
-                "fact_find_brain outer wait_for(25s) tripped — falling back to canonical (session=%s)",
-                session_id,
-            )
-            from backend.fact_find_brain import _canonical_fallback
-            outcome = _canonical_fallback(
-                session=session,
-                user_text=user_text,
-                reason="outer_timeout_25s",
+            # No scripted fallback. Fail loud — let the operator see it.
+            sb_result = SalesBrainResult(
+                reply_text="Sorry, the system is taking too long. Try again in a moment.",
+                captured_updates={},
+                ready_for_recommendations=False,
+                brain_used="sales_brain::error:timeout_25s",
+                error_reason="outer_timeout_25s",
             )
 
-        # Apply captured updates to profile + mark matching slot ids as asked
-        # so the orchestrator's downstream `next_question` (used in the
-        # canonical fallback + completeness API) doesn't double-ask.
+        # Apply captured updates to profile. captured_updates is already
+        # post-processed and canonical (WS1 contract): pass values straight to
+        # session.update_profile_field. For the new path, the question id IS
+        # the field name — no FIELD_TO_QUESTION_ID translation.
         fact_find_profile_updates: dict = {}
-        for field_name, value in outcome.captured_updates.items():
+        for field_name, value in sb_result.captured_updates.items():
             if value in (None, "", []):
                 continue
             session.update_profile_field(field_name, value)
             fact_find_profile_updates[field_name] = value
-            matching_slot = FIELD_TO_QUESTION_ID.get(field_name)
-            if matching_slot and matching_slot not in session.profile.asked:
-                session.profile.asked.append(matching_slot)
-
-        # Track slot_driving so subsequent turns know where we are. The new
-        # brain produces FIELD names (e.g., "income_band"); translate to the
-        # canonical question id (e.g., "income_band") via FIELD_TO_QUESTION_ID
-        # so the rest of the orchestrator (which keys on question ids) stays
-        # consistent.
-        if outcome.slot_driving:
-            slot_qid = FIELD_TO_QUESTION_ID.get(outcome.slot_driving, outcome.slot_driving)
-            session.set_awaiting(slot_qid)
-        else:
-            session.set_awaiting(None)
-
-        # If the brain decided fact-find is complete, flip free-form so
-        # subsequent turns route to retrieval+brain (not back here).
-        if outcome.fact_find_complete:
+            if field_name not in session.profile.asked:
+                session.profile.asked.append(field_name)
+
+        # Completion gate. The LLM owns conversation flow; we only flip
+        # free_form_session when its own readiness signal is True AND every
+        # required slot is filled. Otherwise leave free_form_session=False so
+        # the next turn routes back here and the LLM continues gathering.
+        _REQUIRED_SLOTS = ("name", "age", "dependents", "location_tier",
+                           "income_band", "primary_goal")
+        if sb_result.ready_for_recommendations and all(
+            getattr(session.profile, slot, None) not in (None, "", [])
+            for slot in _REQUIRED_SLOTS
+        ):
             session.free_form_session = True
             session._flush()
 
@@ -626,7 +611,8 @@ async def handle_turn(
         # name. Anonymous sessions never write to Chroma; the corruption
         # surface (profile_anonymous dangling row) is eliminated. Named
         # users get their chunk keyed by canonical name slug, not session_id.
-        if fact_find_profile_updates and session.profile.name:
+        # KI-167 — trigger when any slot was captured this turn.
+        if bool(sb_result.captured_updates) and session.profile.name:
             try:
                 from backend.profile_rag import upsert_profile_chunk
                 from backend.profile_store import _normalise_name
@@ -648,7 +634,7 @@ async def handle_turn(
                     })
             except Exception as e:
                 logging.warning(
-                    "fact_find_brain profile-chunk upsert failed (session=%s): %s: %s",
+                    "sales_brain profile-chunk upsert failed (session=%s): %s: %s",
                     session_id, type(e).__name__, str(e)[:200],
                 )
 
@@ -671,39 +657,20 @@ async def handle_turn(
             except Exception:
                 pass
 
-        if outcome.ambiguous:
-            # KI-078 (2026-05-15) — append fallback reason so admin telemetry
-            # can measure the fallback-cause mix (timeout vs llm_error vs
-            # no_trailer vs empty_reply). Essential for measuring KI-075 +
-            # KI-078 impact in production.
-            reason = getattr(outcome, "_fallback_reason", None) or "unknown"
-            brain_tag = f"fact_find_brain::fallback:{reason}"
-        elif outcome.fact_find_complete:
-            brain_tag = "fact_find_brain::complete"
-        else:
-            brain_tag = "fact_find_brain::continue"
-
         # KI-108 (2026-05-15) — strip CoT preamble / instruction-echo leakage
-        # from the fact-find brain reply BEFORE returning to ChatResponse.
-        # Live re-smoke caught Scenario S4 T3+T4 leaking raw model reasoning
-        # ("We need to respond naturally...", "We need to process user
-        # message...") into reply_text because the fact_find brain path was
-        # the only producer of user-facing text NOT routed through
-        # strip_think_tags (which internally calls strip_cot_preamble via
-        # voice_format). KI-104 had wired the strip into tts_preprocess
-        # (audio) + persona.strip_think_tags (qa/recommendation paths) but
-        # missed this branch. Apply uniformly here so every reply_text the
-        # API returns is leak-free regardless of brain used.
-        ff_reply_clean = strip_think_tags(outcome.reply_text)
+        # from the brain reply BEFORE returning to ChatResponse. The LLM may
+        # still emit <think> tags despite JSON mode; strip_think_tags is the
+        # uniform guard applied across every brain path.
+        reply_text = strip_think_tags(sb_result.reply_text)
         return TurnResult(
-            reply_text=ff_reply_clean,
+            reply_text=reply_text,
             citations=[],
             retrieved_chunk_ids=[],
-            brain_used=brain_tag,
+            brain_used=sb_result.brain_used,
             intent="fact_find",
             language=language,
             latency_ms=int((time.time() - t0) * 1000),
-            raw_reply=outcome.reply_text,
+            raw_reply=sb_result.reply_text,
             faithfulness_passed=True,
             blocked=False,
             profile_updates=fact_find_profile_updates,

backend/sales_brain.py

@@ -0,0 +1,470 @@
+"""KI-167 WS1 — LLM-driven sales brain (replaces the scripted slot-walker).
+
+Architecture shift from `backend/fact_find_brain.py`:
+
+  PRE-KI-167 (fact_find_brain.py):
+    System prompt asks the LLM for prose + a `<FF>{...}</FF>` JSON trailer.
+    Backend strips the trailer, parses it, validates each capture. Works
+    but: (a) some models drop the trailer despite the strict instruction
+    (KI-090 surfaced this; KI-150 raised max_tokens to 700 to mitigate),
+    (b) the canonical-fallback ladder is heavy machinery that competes
+    with the brain for the same conversation.
+
+  POST-KI-167 (this module):
+    Single LLM call with `response_format={"type":"json_object"}` so the
+    NIM model is GUARANTEED to emit a parseable JSON object — no trailer
+    tag, no `<FF>` regex, no fallback ladder. JSON contains the natural
+    reply, captured slots, and a `ready_for_recommendations` boolean.
+    The deterministic post-processor in `sales_brain_normalizer.py`
+    cleans the captures dict.
+
+Public API (the CONTRACT — WS2 will call this exactly):
+
+    @dataclass
+    class SalesBrainResult:
+        reply_text: str
+        captured_updates: dict
+        ready_for_recommendations: bool
+        brain_used: str
+        raw_json: Optional[dict] = None
+        error_reason: Optional[str] = None
+
+    async def drive_sales_brain(
+        user_text: str,
+        profile: Profile,
+        chat_history: list[dict],
+        session_id: Optional[str] = None,
+    ) -> SalesBrainResult: ...
+
+Failure mode:
+  On JSON parse failure (should be near-zero with `response_format`) OR
+  LLM call failure / timeout, this returns a SalesBrainResult with
+  `brain_used="sales_brain::error:<reason>"`, empty reply_text, empty
+  captured_updates, and `ready_for_recommendations=False`. The caller
+  (WS2 orchestrator) decides what to do — NO scripted fallback lives
+  here.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import time
+from dataclasses import dataclass, field
+from typing import Any, Optional
+
+from backend.needs_finder import Profile
+from backend.providers.base import ChatMessage
+from backend.providers.nvidia_nim_llm import get_fast_brain_llm
+from backend.sales_brain_normalizer import normalize_captures
+
+
+# ----------------------------------------------------------------------------
+# Public result dataclass — the contract WS2 will consume.
+# ----------------------------------------------------------------------------
+
+@dataclass
+class SalesBrainResult:
+    reply_text: str
+    captured_updates: dict = field(default_factory=dict)
+    ready_for_recommendations: bool = False
+    brain_used: str = ""
+    raw_json: Optional[dict] = None
+    error_reason: Optional[str] = None
+
+
+# ----------------------------------------------------------------------------
+# Timeouts
+# ----------------------------------------------------------------------------
+# 25s mirrors fact_find_brain's KI-075 setting — gives NIM cold-start headroom
+# + leaves room for one chain fallback. The fast-brain chain already has its
+# own per-link + total-chain budget; this wait_for is a belt-and-braces stop.
+_TIMEOUT_S: float = 25.0
+_MAX_TOKENS: int = 700           # prose + structured JSON object with safety margin
+_TEMPERATURE: float = 0.6        # mirrors fact_find_brain — conversational warmth
+
+
+# ----------------------------------------------------------------------------
+# Slot metadata — descriptions surfaced to the LLM so it knows what to ask for
+# ----------------------------------------------------------------------------
+
+_SLOT_DESCRIPTIONS: dict[str, str] = {
+    "name": "User's preferred name (1-50 chars, no leading 'currently'/'not'/'no'/'looking', single or two-word first/last name).",
+    "age": "User's age as an integer 16-99.",
+    "dependents": "Who needs cover. Enum: 'self', 'self+spouse', 'self+spouse+kids', 'self+parents', 'self+spouse+parents', 'self+spouse+kids+parents', 'self+kids'.",
+    "income_band": "Annual income band. Enum: 'under_5L', '5L-10L', '10L-25L', '25L+'.",
+    "existing_cover_inr": "Current health-insurance sum-insured in rupees. 0 = no existing cover. Integer.",
+    "primary_goal": "Why they're shopping. Enum: 'first_buy', 'upgrade', 'compare_specific', 'tax_planning'.",
+    "location_tier": "City tier. Enum: 'metro', 'tier2', 'tier3', 'rural'.",
+    "parents_to_insure": "Whether to include parents on the policy. Boolean.",
+    "parents_age_max": "Older parent's age (40-99). Only relevant if parents_to_insure is true.",
+    "parents_has_ped": "Whether either parent has a pre-existing condition. Boolean. Only relevant if parents_to_insure is true.",
+    "budget_band": "Annual budget band. Enum: 'under_15k', '15k_30k', '30k_60k', '60k+'.",
+    "health_conditions": "User's pre-existing conditions, lowercase list. Common values: diabetes, hypertension, thyroid, asthma, heart, cancer. Empty list = no conditions.",
+}
+
+# Order of slots — used to determine "required remaining" so the LLM has a
+# stable sense of what to ask next. The first six are the recommendation-
+# readiness minimum; the rest are deepening signals.
+_REQUIRED_FOR_READY: tuple = (
+    "name", "age", "dependents", "location_tier", "income_band", "primary_goal",
+)
+
+_NICE_TO_HAVE: tuple = (
+    "existing_cover_inr", "budget_band", "health_conditions",
+    "parents_to_insure", "parents_age_max", "parents_has_ped",
+)
+
+
+# ----------------------------------------------------------------------------
+# System prompt
+# ----------------------------------------------------------------------------
+
+_BASE_SYSTEM_PROMPT = """You are a friendly Indian health-insurance broker having a natural, warm conversation with a prospective customer. Your tone is consultative — never pushy, never robotic, never a form. Speak like a trusted family friend who happens to know insurance, not a call-centre script.
+
+YOUR JOB:
+1. Get to know the user warmly across a short conversation.
+2. Capture the slots listed below at your own pace, in any order that feels natural.
+3. When you have enough to recommend (the minimum set is named below), set ready_for_recommendations: true.
+
+CONVERSATION RULES:
+- Use the user's first name when you know it. Sound human.
+- Ask AT MOST ONE thing per turn — but weave it naturally into the conversation, don't fire it like a survey question.
+- Don't say canned acknowledgements like "Got that —", "Noted.", "Perfect!", or "Great!". Acknowledge naturally by reflecting back what they said.
+- Don't restart the conversation. Continue from wherever it is — pick up from the user's last message.
+- If the user volunteers multiple facts in one message ("Hi I'm Rohit, 29, in Mumbai, looking for my first policy"), CAPTURE ALL OF THEM in this turn — don't re-ask for what they just told you.
+- If the user goes off-topic (asks about waiting periods, claims, hospital networks, anything else), give a brief honest answer first, then steer naturally back to what you still need to know.
+- BFSI compliance: do NOT promise specific premium quotes. Do NOT pressure-close. Stay advisory.
+- Indian English is fine — "₹", "lakh", "metro/tier-2 city", "BP", "diabetes" all natural.
+- On health conditions, be straight: hiding a condition lowers premium today but turns into a denied claim later. Encourage honesty without lecturing.
+- Never use markdown bold/italics — your reply may be read aloud by TTS.
+
+OUTPUT FORMAT — STRICT:
+Return a SINGLE JSON object with exactly these three keys:
+
+  {
+    "reply": "<your natural reply to the user — prose, no markdown, no JSON tags, no preamble like 'Got that —'>",
+    "captures": { <field>: <value>, ... },
+    "ready_for_recommendations": <true | false>
+  }
+
+Rules for the JSON:
+- "reply" — natural conversational prose. NO scripted prefixes ("Got that —", "Noted —", "Sure —", "Perfect —"). Acknowledge what they said by reflecting it back, then continue.
+- "captures" — only fields the user CHANGED or NEWLY revealed in their LAST message. If they didn't reveal anything new this turn, emit `"captures": {}` (empty object). Do NOT re-emit slots that haven't changed. Do NOT include null / empty-string values.
+- "ready_for_recommendations" — set to true ONLY when you have AT LEAST name, age, dependents, location, income_band, primary_goal. Otherwise false.
+- All enum values must match the slot schema EXACTLY (case-sensitive). All ints as JSON numbers, not strings. Lists as JSON arrays.
+
+EXAMPLES OF GOOD REPLIES (illustrative — the JSON shape is what matters):
+
+User: "Hi, I'm Rohit, 29, looking to buy my first health policy."
+{
+  "reply": "Hi Rohit — nice to meet you. First health policy is a smart move at your age. To shape the right options, who else would you want covered apart from yourself — spouse, kids, parents, or just you for now?",
+  "captures": {"name": "Rohit", "age": 29, "primary_goal": "first_buy"},
+  "ready_for_recommendations": false
+}
+
+User: "Just me and my wife. We're both in Bangalore."
+{
+  "reply": "You and your spouse in Bangalore — good, metro cashless networks are deep there. Roughly what's your annual income band: under ₹5L, ₹5-10L, ₹10-25L, or ₹25L+? Helps me size the right sum-insured.",
+  "captures": {"dependents": "self+spouse", "location_tier": "metro"},
+  "ready_for_recommendations": false
+}
+
+User: "around 18 lakh."
+{
+  "reply": "That puts you in the ₹10-25L band — gives us comfortable room on sum-insured. Last thing before I shortlist some options for you — any pre-existing conditions on either side, like BP, diabetes, thyroid, anything chronic? Being straight here protects your future claim.",
+  "captures": {"income_band": "10L-25L"},
+  "ready_for_recommendations": false
+}
+
+User: "No, both healthy."
+{
+  "reply": "Good to hear. So Rohit, here's what I'm working with: 29, you and your wife in Bangalore, ₹10-25L income, first-time buyer, no pre-existing conditions. I have enough to put together a tailored shortlist now — want me to suggest 2-3 policies that fit?",
+  "captures": {"health_conditions": []},
+  "ready_for_recommendations": true
+}
+
+User: "What's the waiting period for PED usually?"
+{
+  "reply": "Most policies in India have a 24-36 month waiting period for pre-existing diseases — IRDAI caps the maximum, and a few insurers offer shorter waits as a premium upgrade. Happy to dig into a specific one if you have it in mind. Meanwhile — what budget range are you comfortable with annually: under ₹15k, ₹15-30k, ₹30-60k, or ₹60k+?",
+  "captures": {},
+  "ready_for_recommendations": false
+}
+
+User: "Hi"
+{
+  "reply": "Hi there — happy to help you find a health policy that actually fits. What should I call you?",
+  "captures": {},
+  "ready_for_recommendations": false
+}
+"""
+
+
+# ----------------------------------------------------------------------------
+# Profile snapshot helpers
+# ----------------------------------------------------------------------------
+
+_TRACKED_FIELDS: tuple = (
+    "name", "age", "dependents", "income_band", "existing_cover_inr",
+    "primary_goal", "location_tier", "parents_to_insure", "parents_age_max",
+    "parents_has_ped", "health_conditions", "budget_band",
+)
+
+
+def _profile_known(profile: Profile) -> dict[str, Any]:
+    """Return a JSON-friendly dict of currently-captured profile fields.
+
+    OMITS None / empty-string / empty-list fields — so the prompt only
+    shows the LLM what's actually known. This avoids polluting the
+    context with `"age": null, "income_band": null, ...` etc.
+    """
+    snap: dict[str, Any] = {}
+    for f in _TRACKED_FIELDS:
+        v = getattr(profile, f, None)
+        if v is None:
+            continue
+        if isinstance(v, str) and not v.strip():
+            continue
+        if isinstance(v, list) and not v:
+            # Explicit empty health_conditions list IS meaningful ("user said
+            # they have nothing"); include it so the LLM won't re-ask.
+            if f == "health_conditions":
+                snap[f] = []
+            continue
+        snap[f] = v
+    return snap
+
+
+def _required_remaining(known: dict[str, Any]) -> list[str]:
+    """List required slots still missing, in canonical asking order."""
+    return [f for f in _REQUIRED_FOR_READY if f not in known]
+
+
+def _nice_to_have_remaining(known: dict[str, Any], profile: Profile) -> list[str]:
+    """List nice-to-have slots still missing. parents_age_max / parents_has_ped
+    only count as missing when parents_to_insure is True (or dependents says
+    parents)."""
+    out: list[str] = []
+    parents_in_scope = bool(
+        known.get("parents_to_insure") is True
+        or (isinstance(known.get("dependents"), str) and "parent" in known["dependents"])
+    )
+    for f in _NICE_TO_HAVE:
+        if f in known:
+            continue
+        if f in ("parents_age_max", "parents_has_ped") and not parents_in_scope:
+            continue
+        out.append(f)
+    return out
+
+
+def _format_slot_list(slots: list[str]) -> str:
+    """Render a slot list as a bulleted summary the LLM can read."""
+    if not slots:
+        return "(none)"
+    return "\n".join(f"  - {f}: {_SLOT_DESCRIPTIONS.get(f, '')}" for f in slots)
+
+
+def _build_system_prompt(profile: Profile) -> str:
+    """Assemble the full system prompt: base rules + KNOWN block + missing slots."""
+    known = _profile_known(profile)
+    required_remaining = _required_remaining(known)
+    nice_to_have_remaining = _nice_to_have_remaining(known, profile)
+
+    known_block = (
+        json.dumps(known, ensure_ascii=False) if known else "{}"
+    )
+
+    return (
+        _BASE_SYSTEM_PROMPT
+        + "\n\n--- SLOT SCHEMA (the fields you may capture) ---\n"
+        + _format_slot_list(list(_SLOT_DESCRIPTIONS.keys()))
+        + f"\n\nKNOWN: {known_block}"
+        + "\n(These fields are ALREADY captured. Do NOT re-ask for them. Use the user's name when known.)"
+        + "\n\nREQUIRED SLOTS STILL MISSING (you need these before setting ready_for_recommendations=true):\n"
+        + _format_slot_list(required_remaining)
+        + "\n\nNICE-TO-HAVE SLOTS STILL MISSING (capture if the user volunteers; don't force):\n"
+        + _format_slot_list(nice_to_have_remaining)
+        + (
+            "\n\nYou have enough to recommend now. If the user signals they want options "
+            "(\"show me policies\", \"what do you suggest\"), set ready_for_recommendations=true."
+            if not required_remaining else
+            "\n\nKeep gathering naturally — you don't yet have the minimum required set."
+        )
+    )
+
+
+# ----------------------------------------------------------------------------
+# JSON parsing — response_format=json_object should make this near-zero-fail
+# ----------------------------------------------------------------------------
+
+def _parse_brain_json(raw_text: str) -> Optional[dict]:
+    """Parse the LLM's JSON-object response.
+
+    With NIM `response_format={"type":"json_object"}` the response is
+    guaranteed to be a JSON object. Still defensively handle the rare
+    cases where a model emits stray whitespace, code fences, or a
+    leading `<think>` block.
+    """
+    if not raw_text:
+        return None
+    text = raw_text.strip()
+    # Strip `<think>` blocks some models emit despite instructions.
+    if "<think>" in text and "</think>" in text:
+        text = text.split("</think>", 1)[1].strip()
+    # Strip code fences.
+    if text.startswith("```"):
+        lines = text.split("\n")
+        text = "\n".join(l for l in lines if not l.startswith("```")).strip()
+    # Direct parse
+    try:
+        obj = json.loads(text)
+        if isinstance(obj, dict):
+            return obj
+    except json.JSONDecodeError:
+        pass
+    # Try to find the first JSON object substring.
+    start = text.find("{")
+    end = text.rfind("}")
+    if start != -1 and end > start:
+        try:
+            obj = json.loads(text[start : end + 1])
+            if isinstance(obj, dict):
+                return obj
+        except json.JSONDecodeError:
+            pass
+    return None
+
+
+# ----------------------------------------------------------------------------
+# Public entry point
+# ----------------------------------------------------------------------------
+
+async def drive_sales_brain(
+    user_text: str,
+    profile: Profile,
+    chat_history: list[dict],
+    session_id: Optional[str] = None,
+) -> SalesBrainResult:
+    """Single LLM call per turn — replaces the scripted slot-walker.
+
+    Returns a SalesBrainResult. On LLM error or JSON parse failure,
+    `brain_used` is `sales_brain::error:<reason>` and `reply_text` is
+    empty. Caller (orchestrator) is responsible for any fallback behavior
+    — NO scripted reply lives here.
+    """
+    t0 = time.time()
+
+    # Build the system prompt with current profile state
+    system_prompt = _build_system_prompt(profile)
+
+    # Build messages: system + last ~10 chat history turns + user_text
+    messages: list[ChatMessage] = [ChatMessage(role="system", content=system_prompt)]
+    if chat_history:
+        for turn in chat_history[-10:]:
+            role = turn.get("role")
+            content = turn.get("content")
+            if role in ("user", "assistant") and content:
+                messages.append(ChatMessage(role=role, content=str(content)))
+    messages.append(ChatMessage(role="user", content=user_text or ""))
+
+    # Call the fast-brain NIM chain with response_format=json_object
+    # (singleton-via-factory — see KI-080; the factory builds a fresh
+    # NimChainLLM each call but the underlying election state is shared
+    # via backend.llm_health module-level dicts).
+    llm = get_fast_brain_llm()
+    try:
+        result = await asyncio.wait_for(
+            llm.chat(
+                messages=messages,
+                temperature=_TEMPERATURE,
+                max_tokens=_MAX_TOKENS,
+                response_format={"type": "json_object"},
+            ),
+            timeout=_TIMEOUT_S,
+        )
+    except asyncio.TimeoutError:
+        elapsed = time.time() - t0
+        logging.warning(
+            "sales_brain timeout (session=%s, %.1fs)", session_id, elapsed,
+        )
+        return SalesBrainResult(
+            reply_text="",
+            captured_updates={},
+            ready_for_recommendations=False,
+            brain_used="sales_brain::error:timeout",
+            error_reason=f"timeout_after_{elapsed:.1f}s",
+        )
+    except Exception as e:  # noqa: BLE001 — broad catch is intentional here
+        logging.warning(
+            "sales_brain LLM call failed (session=%s): %s: %s",
+            session_id, type(e).__name__, str(e)[:200],
+        )
+        return SalesBrainResult(
+            reply_text="",
+            captured_updates={},
+            ready_for_recommendations=False,
+            brain_used=f"sales_brain::error:llm_error_{type(e).__name__}",
+            error_reason=f"{type(e).__name__}: {str(e)[:200]}",
+        )
+
+    # Parse the JSON object from the LLM response.
+    raw_text = (result.text or "").strip()
+    parsed = _parse_brain_json(raw_text)
+    served_model = getattr(result, "model", "unknown") or "unknown"
+
+    if parsed is None:
+        logging.warning(
+            "sales_brain JSON parse failed (session=%s, model=%s, raw=%r)",
+            session_id, served_model, raw_text[:300],
+        )
+        return SalesBrainResult(
+            reply_text="",
+            captured_updates={},
+            ready_for_recommendations=False,
+            brain_used=f"sales_brain::error:parse_fail",
+            error_reason=f"could_not_parse_json from model={served_model}; raw_head={raw_text[:120]!r}",
+        )
+
+    # Extract the three contract keys, tolerating mild key drift.
+    reply_text = parsed.get("reply") or parsed.get("response") or parsed.get("message") or ""
+    if not isinstance(reply_text, str):
+        reply_text = str(reply_text)
+    reply_text = reply_text.strip()
+
+    captures_raw = parsed.get("captures") or parsed.get("captured") or {}
+    if not isinstance(captures_raw, dict):
+        captures_raw = {}
+
+    ready_raw = parsed.get("ready_for_recommendations")
+    if ready_raw is None:
+        ready_raw = parsed.get("ready")
+    ready_for_recommendations = bool(ready_raw) if ready_raw is not None else False
+
+    # Normalize + validate captures via the deterministic post-processor.
+    captured_updates = normalize_captures(captures_raw, profile)
+
+    # Empty reply text is a soft-fail — the brain produced JSON but no prose.
+    # Surface this to the caller so it can decide what to do.
+    if not reply_text:
+        return SalesBrainResult(
+            reply_text="",
+            captured_updates=captured_updates,
+            ready_for_recommendations=ready_for_recommendations,
+            brain_used="sales_brain::error:empty_reply",
+            raw_json=parsed,
+            error_reason=f"reply_field_empty from model={served_model}",
+        )
+
+    return SalesBrainResult(
+        reply_text=reply_text,
+        captured_updates=captured_updates,
+        ready_for_recommendations=ready_for_recommendations,
+        brain_used=f"sales_brain::nim:{served_model}",
+        raw_json=parsed,
+        error_reason=None,
+    )
+
+
+__all__ = ["SalesBrainResult", "drive_sales_brain"]

backend/sales_brain_normalizer.py

@@ -0,0 +1,687 @@
+"""KI-167 WS1 — Deterministic post-processor for the LLM sales brain.
+
+The sales brain (`backend/sales_brain.py`) issues ONE NIM call with the slot
+schema in its system prompt + `response_format={"type":"json_object"}`. The
+LLM returns a `captures` dict — but real LLMs:
+
+  * Use loose field aliases ("location" instead of "location_tier",
+    "income" instead of "income_band").
+  * Emit free-text values for things that need to be enums
+    ("Bangalore" instead of "metro", "5 lakhs" instead of "5L-10L").
+  * Occasionally emit `null` / empty-string for fields they didn't capture.
+  * Sometimes hallucinate a name that's actually a status phrase
+    ("Currently Not Having Any").
+
+This module is the deterministic shim that takes that raw dict and emits a
+clean `{canonical_field: validated_value}` map ready to flow into
+`SessionState.update_profile_field()`. NO LLM calls — pure normalization +
+validation. Cheap, deterministic, easy to audit.
+
+Cherry-picked / adapted from `backend/fact_find_normalizer.py`:
+  - `_FIELD_SCHEMA`              — slot type + bounds + allowed enum values
+  - `_validate(value, schema)`   — type/enum/bounds check
+  - `_keyword_normalize` ideas   — free-text → enum mapping
+  - `_parse_existing_cover`      — INR-amount parser for existing cover
+
+Also delegates to `backend/needs_finder._parse_inr_amount`,
+`_parse_budget_band`, `_parse_income_band` so this stays the single source
+of truth on numeric parsing.
+
+Public API:
+    normalize_captures(raw, current_profile) -> dict
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any, Optional
+
+from backend.needs_finder import (
+    Profile,
+    _parse_budget_band,
+    _parse_income_band,
+    _parse_inr_amount,
+)
+
+
+# ----------------------------------------------------------------------------
+# Slot schema — canonical Profile field name → type / bounds / allowed values
+# ----------------------------------------------------------------------------
+#
+# Mirrors `fact_find_normalizer._FIELD_SCHEMA` but keyed by the PROFILE FIELD
+# NAME (not the question id). Includes `self+kids` in dependents (LLM may
+# legitimately emit it; absent from the original schema). Mirrors the WS1
+# spec verbatim where the spec adds new values (`tier3`, `rural`).
+
+_FIELD_SCHEMA: dict[str, dict] = {
+    "name": {"type": "name"},
+    "age": {"type": "int", "min": 16, "max": 99},
+    "dependents": {
+        "type": "enum",
+        "values": [
+            "self",
+            "self+spouse",
+            "self+spouse+kids",
+            "self+kids",
+            "self+parents",
+            "self+spouse+parents",
+            "self+spouse+kids+parents",
+        ],
+    },
+    "income_band": {
+        "type": "enum",
+        "values": ["under_5L", "5L-10L", "10L-25L", "25L+"],
+    },
+    "existing_cover_inr": {"type": "int", "min": 0, "max": 100_000_000},
+    "primary_goal": {
+        "type": "enum",
+        # WS1 spec uses "compare" + "tax_savings"; the existing fact-find
+        # codebase uses "compare_specific" + "tax_planning". Accept both
+        # via alias-normalization below; canonical-stored value matches the
+        # existing Profile (which downstream WS2/WS3 consume).
+        "values": ["first_buy", "upgrade", "compare_specific", "tax_planning"],
+    },
+    "location_tier": {
+        "type": "enum",
+        # WS1 spec adds "rural"; existing schema uses metro/tier1/tier2/tier3.
+        # Accept tier1 as an alias (legacy fact-find emitted it); collapse to
+        # tier2 since the WS1 spec drops the tier1 bucket.
+        "values": ["metro", "tier2", "tier3", "rural"],
+    },
+    "parents_to_insure": {"type": "bool"},
+    "parents_age_max": {"type": "int", "min": 40, "max": 99},
+    "parents_has_ped": {"type": "bool"},
+    "budget_band": {
+        "type": "enum",
+        "values": ["under_15k", "15k_30k", "30k_60k", "60k+"],
+    },
+    "health_conditions": {
+        "type": "list",
+        "common_values": [
+            "diabetes", "hypertension", "thyroid", "asthma", "heart", "cancer",
+        ],
+    },
+}
+
+
+# ----------------------------------------------------------------------------
+# Field-name alias map — loose LLM key → canonical Profile field
+# ----------------------------------------------------------------------------
+
+_FIELD_ALIASES: dict[str, str] = {
+    # location aliases
+    "location": "location_tier",
+    "city": "location_tier",
+    "tier": "location_tier",
+    "location_tier": "location_tier",
+    # income aliases
+    "income": "income_band",
+    "income_band": "income_band",
+    "annual_income": "income_band",
+    "salary": "income_band",
+    # cover aliases
+    "existing_cover": "existing_cover_inr",
+    "existing_cover_inr": "existing_cover_inr",
+    "current_cover": "existing_cover_inr",
+    "current_sum_insured": "existing_cover_inr",
+    "sum_insured": "existing_cover_inr",
+    # goal aliases
+    "goal": "primary_goal",
+    "primary_goal": "primary_goal",
+    "intent": "primary_goal",
+    # budget aliases
+    "budget": "budget_band",
+    "budget_band": "budget_band",
+    "annual_budget": "budget_band",
+    # parents aliases
+    "parents_age": "parents_age_max",
+    "parents_age_max": "parents_age_max",
+    "parents_to_insure": "parents_to_insure",
+    "cover_parents": "parents_to_insure",
+    "include_parents": "parents_to_insure",
+    "parents_has_ped": "parents_has_ped",
+    "parents_ped": "parents_has_ped",
+    "parents_conditions": "parents_has_ped",
+    # health
+    "health_conditions": "health_conditions",
+    "conditions": "health_conditions",
+    "ped": "health_conditions",
+    "pre_existing": "health_conditions",
+    "pre_existing_conditions": "health_conditions",
+    # passthrough
+    "name": "name",
+    "age": "age",
+    "dependents": "dependents",
+}
+
+
+# ----------------------------------------------------------------------------
+# Public API
+# ----------------------------------------------------------------------------
+
+def normalize_captures(raw: dict, current_profile: Profile) -> dict:
+    """Map a raw LLM `captures` dict to clean `{canonical_field: value}`.
+
+    Pipeline per entry:
+      1. Skip if value is None / empty string / empty list (LLM didn't
+         actually capture this turn).
+      2. Resolve the field alias → canonical Profile attribute name. Drop
+         unknown fields silently.
+      3. Run the value through the field's normalizer + validator. Drop
+         on validation failure (don't raise — sales brain is fail-open).
+      4. Compare against `current_profile.<field>`. Skip no-ops.
+
+    Returns: clean dict ready for `SessionState.update_profile_field()`.
+    """
+    if not isinstance(raw, dict):
+        return {}
+
+    clean: dict[str, Any] = {}
+    for raw_key, raw_value in raw.items():
+        if raw_value is None:
+            continue
+        if isinstance(raw_value, str) and not raw_value.strip():
+            continue
+        # Bare empty list / dict from the LLM = no actual signal; drop.
+        # The "user affirmed no conditions" path goes through a STRING value
+        # like "all good" that the normalizer parses to [].
+        if isinstance(raw_value, (list, dict)) and not raw_value:
+            continue
+
+        canonical = _FIELD_ALIASES.get(str(raw_key).strip().lower())
+        if canonical is None:
+            continue
+        schema = _FIELD_SCHEMA.get(canonical)
+        if schema is None:
+            continue
+
+        normalized = _normalize_value(canonical, raw_value, schema)
+        if normalized is None:
+            # Could not coerce — drop silently
+            continue
+
+        # No-op check — skip fields that match the current profile value.
+        # Exception: an explicit "no conditions" capture (empty list parsed
+        # from "all good" / "none" / etc.) is meaningful even when the profile
+        # already holds an empty list — it tells the orchestrator the slot
+        # was just addressed this turn so it won't be re-asked.
+        current = getattr(current_profile, canonical, None)
+        if current == normalized:
+            if (
+                canonical == "health_conditions"
+                and normalized == []
+                and current == []
+                and isinstance(raw_value, str)
+            ):
+                clean[canonical] = normalized
+            continue
+
+        clean[canonical] = normalized
+
+    return clean
+
+
+# ----------------------------------------------------------------------------
+# Value normalization — per-field free-text → canonical value
+# ----------------------------------------------------------------------------
+
+def _normalize_value(field: str, value: Any, schema: dict) -> Any:
+    """Field-specific normalization. Returns None on validation failure."""
+    if field == "name":
+        return _normalize_name(value)
+    if field == "age":
+        return _normalize_int(value, schema)
+    if field == "dependents":
+        return _normalize_dependents(value, schema)
+    if field == "income_band":
+        return _normalize_income_band(value, schema)
+    if field == "existing_cover_inr":
+        return _normalize_existing_cover(value, schema)
+    if field == "primary_goal":
+        return _normalize_primary_goal(value, schema)
+    if field == "location_tier":
+        return _normalize_location_tier(value, schema)
+    if field == "parents_to_insure":
+        return _normalize_bool(value)
+    if field == "parents_age_max":
+        return _normalize_int(value, schema)
+    if field == "parents_has_ped":
+        return _normalize_bool(value)
+    if field == "budget_band":
+        return _normalize_budget_band(value, schema)
+    if field == "health_conditions":
+        return _normalize_health_conditions(value)
+    return None
+
+
+# ---------- name ----------
+
+_NAME_BAD_FIRST_WORDS = {
+    "currently", "not", "no", "none", "nothing", "never",
+    "without", "looking", "buying", "shopping",
+    "yes", "yeah", "nope", "nah",
+    "haven", "havent", "haven't",
+    "don", "dont", "don't",
+    "i", "we", "my", "this", "that", "the",
+    "first", "still", "yet",
+}
+
+
+def _normalize_name(value: Any) -> Optional[str]:
+    """Validate a candidate name string.
+
+    Reject if:
+      - Not a non-empty string.
+      - Contains digits.
+      - Length > 50 chars.
+      - First word is in the status-phrase blocklist.
+      - <50% alphabetic chars (e.g. "..." or "@!#").
+    """
+    if not isinstance(value, str):
+        return None
+    s = value.strip()
+    if not s or len(s) > 50:
+        return None
+    if any(c.isdigit() for c in s):
+        return None
+    # Strip simple polite-prefix scraps the LLM might leave in
+    s_lc = s.lower()
+    for prefix in (
+        "i'm ", "i am ", "this is ", "my name is ", "name is ",
+        "name's ", "call me ", "im ", "mr ", "mrs ", "ms ", "dr ",
+    ):
+        if s_lc.startswith(prefix):
+            s = s[len(prefix):].strip()
+            s_lc = s.lower()
+            break
+    if not s:
+        return None
+    first = s.split()[0].lower().strip(".,!?")
+    if first in _NAME_BAD_FIRST_WORDS:
+        return None
+    alpha = sum(1 for c in s if c.isalpha())
+    if alpha < 2 or alpha / max(1, len(s)) < 0.5:
+        return None
+    # Capitalize if all-lower (LLM convention — names should be Title Case).
+    if not any(c.isupper() for c in s):
+        s = " ".join(w.capitalize() for w in s.split())
+    return s
+
+
+# ---------- int / bool ----------
+
+def _normalize_int(value: Any, schema: dict) -> Optional[int]:
+    if isinstance(value, bool):  # bool is subclass of int — exclude
+        return None
+    if isinstance(value, (int, float)):
+        v = int(value)
+    elif isinstance(value, str):
+        digits = "".join(c for c in value if c.isdigit())
+        if not digits:
+            return None
+        try:
+            v = int(digits[:3])
+        except ValueError:
+            return None
+    else:
+        return None
+    lo = schema.get("min", -1_000_000_000)
+    hi = schema.get("max", 1_000_000_000)
+    if v < lo or v > hi:
+        return None
+    return v
+
+
+def _normalize_bool(value: Any) -> Optional[bool]:
+    if isinstance(value, bool):
+        return value
+    if isinstance(value, (int, float)):
+        return bool(value)
+    if isinstance(value, str):
+        s = value.strip().lower()
+        if s in ("true", "yes", "y", "1", "include", "covered", "with"):
+            return True
+        if s in ("false", "no", "n", "0", "exclude", "skip", "without"):
+            return False
+    return None
+
+
+# ---------- dependents ----------
+
+_DEPENDENT_VAGUE_TERMS = (
+    "family", "everyone", "all of us", "everybody",
+    "whole family", "joint family",
+)
+
+
+def _normalize_dependents(value: Any, schema: dict) -> Optional[str]:
+    if not isinstance(value, str):
+        return None
+    s = value.strip().lower()
+    if not s:
+        return None
+    # Direct hit on a canonical enum value
+    if s in schema["values"]:
+        return s
+
+    has_spouse = any(k in s for k in ("spouse", "wife", "husband", "partner"))
+    has_kids = ("kid" in s) or ("child" in s) or ("children" in s) or ("son" in s) or ("daughter" in s)
+    has_parents = "parent" in s
+
+    # Vague terms with no specific signals → cannot coerce
+    if any(v in s for v in _DEPENDENT_VAGUE_TERMS) and not (has_spouse or has_kids or has_parents):
+        return None
+
+    if has_spouse and has_kids and has_parents:
+        return "self+spouse+kids+parents"
+    if has_spouse and has_parents:
+        return "self+spouse+parents"
+    if has_spouse and has_kids:
+        return "self+spouse+kids"
+    if has_spouse:
+        return "self+spouse"
+    if has_kids and has_parents:
+        # No canonical bucket — fold parents into the wider bundle
+        return "self+spouse+kids+parents"
+    if has_kids:
+        return "self+kids"
+    if has_parents:
+        return "self+parents"
+    if s in ("self", "me", "just me", "only me", "myself", "only self"):
+        return "self"
+    return None
+
+
+# ---------- income_band ----------
+
+def _normalize_income_band(value: Any, schema: dict) -> Optional[str]:
+    if isinstance(value, str):
+        s = value.strip()
+        if s in schema["values"]:
+            return s
+        # KI-149 — let the centralised parser handle free text.
+        parsed = _parse_income_band(s)
+        if parsed in schema["values"]:
+            return parsed
+        return None
+    if isinstance(value, (int, float)):
+        # Bare number → rupees → bucket
+        amt = int(value)
+        if amt < 500_000:
+            return "under_5L"
+        if amt < 1_000_000:
+            return "5L-10L"
+        if amt < 2_500_000:
+            return "10L-25L"
+        return "25L+"
+    return None
+
+
+# ---------- existing_cover_inr ----------
+
+def _normalize_existing_cover(value: Any, schema: dict) -> Optional[int]:
+    if isinstance(value, bool):
+        # bool 'no' (False) → 0; 'yes' (True) is ambiguous (don't commit)
+        return 0 if value is False else None
+    if isinstance(value, (int, float)):
+        v = int(value)
+        if v < 0 or v > schema.get("max", 100_000_000):
+            return None
+        return v
+    if isinstance(value, str):
+        s = value.strip().lower()
+        if not s:
+            return None
+        # Negative answers → 0
+        if re.search(
+            r"\b(no|none|nothing|zero|nope|nah|haven'?t|don'?t|never|"
+            r"not\s+having|not\s+got|not\s+have|without\s+(?:any|a))\b",
+            s,
+        ):
+            return 0
+        amt = _parse_inr_amount(s)
+        if amt is None:
+            # Bare digit fallback (very small values OK here — user might
+            # have ₹500 unused; still coerce within bounds).
+            digits = "".join(c for c in s if c.isdigit())
+            if not digits:
+                return None
+            try:
+                amt = int(digits[:8])
+            except ValueError:
+                return None
+        if amt < 0 or amt > schema.get("max", 100_000_000):
+            return None
+        return amt
+    return None
+
+
+# ---------- primary_goal ----------
+
+_GOAL_ALIASES: dict[str, str] = {
+    # WS1 spec ↔ canonical Profile values
+    "compare": "compare_specific",
+    "tax_savings": "tax_planning",
+    # accept the canonical values themselves
+    "first_buy": "first_buy",
+    "upgrade": "upgrade",
+    "compare_specific": "compare_specific",
+    "tax_planning": "tax_planning",
+}
+
+
+def _normalize_primary_goal(value: Any, schema: dict) -> Optional[str]:
+    if not isinstance(value, str):
+        return None
+    s = value.strip().lower()
+    if not s:
+        return None
+    # Direct hit on alias or canonical
+    if s in _GOAL_ALIASES:
+        return _GOAL_ALIASES[s]
+    # Keyword fall-through
+    if any(k in s for k in ("first policy", "first one", "first time", "first buy", "new policy", "buying my first")):
+        return "first_buy"
+    if any(k in s for k in ("upgrade", "upgrading", "better cover", "more cover", "increase cover")):
+        return "upgrade"
+    if any(k in s for k in ("compare", "comparison", " vs ", " vs.", "versus")):
+        return "compare_specific"
+    if any(k in s for k in (" tax ", "80d", "deduction", "tax planning", "tax saving")):
+        return "tax_planning"
+    return None
+
+
+# ---------- location_tier ----------
+# WS1 spec city lists — case-insensitive substring match.
+_METRO_CITIES = (
+    "mumbai", "delhi", "new delhi", "bangalore", "bengaluru", "chennai",
+    "kolkata", "hyderabad", "pune", "ahmedabad",
+)
+_TIER2_CITIES = (
+    "jaipur", "lucknow", "indore", "chandigarh", "kochi", "coimbatore",
+    "vadodara", "nagpur", "bhubaneshwar", "bhubaneswar", "visakhapatnam",
+    "surat",
+)
+
+
+def _normalize_location_tier(value: Any, schema: dict) -> Optional[str]:
+    if not isinstance(value, str):
+        return None
+    s = value.strip().lower()
+    if not s:
+        return None
+    # Direct enum hit
+    if s in schema["values"]:
+        return s
+    # Legacy tier1 → collapse to metro (WS1 spec dropped tier1).
+    if s == "tier1" or s == "tier 1":
+        return "metro"
+    if s == "tier 2":
+        return "tier2"
+    if s == "tier 3":
+        return "tier3"
+    # City-name substring match (case-insensitive).
+    for c in _METRO_CITIES:
+        if c in s:
+            return "metro"
+    for c in _TIER2_CITIES:
+        if c in s:
+            return "tier2"
+    # Explicit small-town / village wording → rural
+    if any(k in s for k in ("village", "rural", "gaon")):
+        return "rural"
+    if any(k in s for k in ("small town", "tier 3", "tier3", "small city")):
+        return "tier3"
+    # Otherwise default to tier3 only if the string looks like a real
+    # place name (alpha-heavy, short). This avoids stamping junk strings
+    # like "n/a" or "?" as tier3.
+    alpha = sum(1 for c in s if c.isalpha())
+    if alpha >= 3 and alpha / max(1, len(s)) >= 0.7:
+        return "tier3"
+    return None
+
+
+# ---------- budget_band ----------
+
+def _normalize_budget_band(value: Any, schema: dict) -> Optional[str]:
+    if isinstance(value, str):
+        s = value.strip()
+        if s in schema["values"]:
+            return s
+        parsed = _parse_budget_band(s)
+        if parsed in schema["values"]:
+            return parsed
+        return None
+    if isinstance(value, (int, float)):
+        amt = int(value)
+        if amt < 15_000:
+            return "under_15k"
+        if amt < 30_000:
+            return "15k_30k"
+        if amt < 60_000:
+            return "30k_60k"
+        return "60k+"
+    return None
+
+
+# ---------- health_conditions ----------
+
+_NO_PED_PATTERNS = (
+    r"\b(?:no|none|nothing|nope|nah|negative)\b",
+    r"\b(?:not|don'?t|do\s+not|haven'?t|isn'?t)\s+(?:got|have|having|had|got\s+any|have\s+any)\b",
+    r"\b(?:zero|nil)\s+(?:pre[-\s]?exist\w*|condition|chronic|health\s+issue|illness)",
+    r"\b(?:i\s+am|i'm)\s+(?:healthy|fine|fit|alright|all\s+good|good)\b",
+    r"\bnothing\s+(?:chronic|major|serious|to\s+report|like\s+that)\b",
+    r"\ball\s+(?:good|fine|clear|healthy)\b",
+    r"\bclean\s+bill\s+of\s+health\b",
+)
+
+_COND_KEYWORDS: dict[str, tuple] = {
+    "diabetes":     ("diabetes", "diabetic", "sugar"),
+    "hypertension": ("hypertension", "blood pressure", "high bp"),
+    "thyroid":      ("thyroid", "hypothyroid", "hyperthyroid"),
+    "asthma":       ("asthma",),
+    "heart":        ("heart problem", "heart disease", "cardiac"),
+    "cancer":       ("cancer", "tumor", "tumour"),
+}
+# "bp" as a free-standing token (case-insensitive word-boundary). Kept
+# separate from `_COND_KEYWORDS` so it doesn't collide with substrings
+# like "BPM" or "abp" while still catching "BP" / "high BP" / "diabetes and BP".
+_BP_REGEX = re.compile(r"\bbp\b", re.IGNORECASE)
+
+
+def _normalize_health_conditions(value: Any) -> Optional[list]:
+    """Return a deduped lowercase list of canonical condition names.
+
+    - [] means "no conditions" — VALID capture.
+    - None means "could not parse" — caller drops the field.
+    """
+    # Direct list passthrough (LLM did its job)
+    if isinstance(value, list):
+        cleaned: list[str] = []
+        seen: set[str] = set()
+        for item in value:
+            if not isinstance(item, (str, int)):
+                continue
+            s = str(item).strip().lower()
+            if not s:
+                continue
+            if s in seen:
+                continue
+            seen.add(s)
+            # Map keyword hits to canonical names
+            mapped = None
+            for canon, kws in _COND_KEYWORDS.items():
+                if s == canon or any(k.strip() == s for k in kws):
+                    mapped = canon
+                    break
+            cleaned.append(mapped if mapped else s)
+        # Dedup again after canonicalisation
+        return list(dict.fromkeys(cleaned))
+    # String → parse out
+    if isinstance(value, str):
+        s = value.strip().lower()
+        if not s:
+            return None
+        # Explicit "no PED" phrasings → empty list
+        if any(re.search(p, s) for p in _NO_PED_PATTERNS):
+            # belt-and-braces — if a real condition keyword is ALSO present,
+            # fall through to capture it.
+            has_condition = any(
+                any(k in s for k in kws) for kws in _COND_KEYWORDS.values()
+            )
+            if not has_condition:
+                return []
+        canonical: list[str] = []
+        for canon, kws in _COND_KEYWORDS.items():
+            if any(k in s for k in kws):
+                canonical.append(canon)
+        # Word-boundary BP catch (independent of substring keywords above).
+        if "hypertension" not in canonical and _BP_REGEX.search(s):
+            canonical.append("hypertension")
+        if canonical:
+            return canonical
+        return None
+    return None
+
+
+# ----------------------------------------------------------------------------
+# Generic validator — kept for callers that want a single dispatch.
+# ----------------------------------------------------------------------------
+
+def _validate(value: Any, schema: dict) -> Any:
+    """Type + enum + bounds check. Returns None on failure.
+
+    Adapted from `fact_find_normalizer._validate` (line 420).
+    """
+    t = schema.get("type")
+    if t == "enum":
+        if isinstance(value, str) and value in schema["values"]:
+            return value
+        return None
+    if t == "int":
+        if isinstance(value, bool):
+            return None
+        try:
+            v = int(value)
+        except (TypeError, ValueError):
+            return None
+        if v < schema.get("min", -1_000_000_000) or v > schema.get("max", 1_000_000_000):
+            return None
+        return v
+    if t == "list":
+        if not isinstance(value, list):
+            return None
+        cleaned = [str(x).strip().lower() for x in value if x and isinstance(x, (str, int))]
+        cleaned = [c for c in cleaned if c]
+        return cleaned
+    if t == "bool":
+        if isinstance(value, bool):
+            return value
+        return None
+    if t == "name":
+        return _normalize_name(value)
+    return value
+
+
+__all__ = ["normalize_captures"]

tests/README.md

@@ -31,6 +31,10 @@ python tests/live_verify.py
 TARGET_URL=http://localhost:8000 python tests/live_verify.py
 ```
 
+## Retired tests (KI-167, 2026-05-15)
+
+- `test_fact_find_loop_break.py` — retired because the loop-break + `_canonical_fallback` architecture was replaced by the LLM-driven `backend/sales_brain.py`. The test pinned internals (`_canonical_fallback`, `_MAX_FAILED_ATTEMPTS`, `FIELD_TO_QUESTION_ID`) that no longer exist after the `fact_find_brain.py` rip-out. The equivalent behavioural invariant (bot doesn't re-ask answered slots) is now covered end-to-end by the gold-QA accuracy eval and the multi-persona stress audit.
+
 ## Related
 
 - Root `CLAUDE.md` § Routing invariants — the four lines this folder protects

tests/test_fact_find_loop_break.py

@@ -1,191 +0,0 @@
-"""Regression tests for the fact-find canonical-fallback loop bug (KI-103).
-
-Pre-fix bug
------------
-Live 15-persona smoke test on 2026-05-15 caught the brain repeating the
-SAME slot question for 4-8 consecutive turns even though the user explicitly
-answered on the first turn. Concrete evidence:
-    A3 (Priya): age 42 stated T2, bot asked "First, your age?" 7 times
-    A5 (Sanjay): employer-cover answered T4, bot asked the same Q 4 times
-    B3 (Mehul): age 45 stated T2, bot asked age 8 times despite the answer
-
-Root cause
-----------
-When the LLM brain returned a reply WITHOUT the `<FF>{...}</FF>` JSON tail
-(reason=`no_trailer`), `_canonical_fallback` ran. Its greedy multi-slot
-capture only appends to `profile.asked` when a value is successfully
-captured. When greedy fails (user_text doesn't match any slot's strict
-regex), `profile.asked` stays unchanged, so `next_question(profile)`
-returns the SAME slot every turn. Loop.
-
-Fix
----
-`_canonical_fallback` now tracks per-slot consecutive failed-fallback
-surfaces on the SessionState (`_ff_failed_attempts`). After
-`_MAX_FAILED_ATTEMPTS` (=2) consecutive surfaces of slot S with no
-capture for S, the slot is marked asked + appended to
-`_ff_skipped_slots` and the fallback advances to the next unfilled slot.
-
-Run:
-    cd /Users/rohitsar/Developer/Insurance\\ Sales\\ Bot
-    PYTHONPATH=$PWD .venv/bin/python -m pytest tests/test_fact_find_loop_break.py -v
-"""
-
-from __future__ import annotations
-
-import unittest
-import uuid
-from dataclasses import dataclass, field
-from typing import Optional
-
-# Bootstrap import path so this file runs from either pytest or unittest.
-import sys
-from pathlib import Path
-_REPO_ROOT = Path(__file__).resolve().parent.parent
-if str(_REPO_ROOT) not in sys.path:
-    sys.path.insert(0, str(_REPO_ROOT))
-
-
-def _fresh_session_id() -> str:
-    return f"test_ff_loop_break_{uuid.uuid4().hex[:10]}"
-
-
-def _cleanup_session_file(session_id: str) -> None:
-    target = _REPO_ROOT / "40-data" / "sessions" / f"{session_id}.json"
-    if target.exists():
-        try:
-            target.unlink()
-        except OSError:
-            pass
-
-
-class TestCanonicalFallbackBreaksLoop(unittest.TestCase):
-    """KI-103 — three consecutive `_canonical_fallback` calls with user
-    text that captures nothing must NOT return the same slot on call 3
-    as on call 1. The session-level failure counter must advance past
-    the stuck slot after two failed surfaces.
-    """
-
-    def setUp(self) -> None:
-        from backend.session_state import _sessions, get_session
-        _sessions.clear()
-        self.session_id = _fresh_session_id()
-        # Build a fresh session with name + age unset (everything unset
-        # actually). No awaiting_question_id, no asked slots.
-        self.session = get_session(self.session_id)
-
-    def tearDown(self) -> None:
-        _cleanup_session_file(self.session_id)
-
-    def test_three_failed_attempts_advance_slot(self) -> None:
-        """Drive `_canonical_fallback` 3 times with non-extracting input.
-        Assert the slot driven on call 3 differs from call 1 — the
-        loop-breaker has skipped the stuck slot and advanced.
-        """
-        from backend.fact_find_brain import _canonical_fallback
-
-        # user_text that doesn't match ANY slot's strict capture regex:
-        # - "name" strict mode needs intro phrase ("I'm X", "my name is X")
-        # - "age" needs age trigger or bare number
-        # - dependents / income_band / etc. need family / amount / etc. keywords
-        non_extracting_text = "hello there how are you"
-
-        # Call 1
-        out1 = _canonical_fallback(
-            self.session, non_extracting_text, reason="no_trailer",
-        )
-        # Call 2
-        out2 = _canonical_fallback(
-            self.session, non_extracting_text, reason="no_trailer",
-        )
-        # Call 3
-        out3 = _canonical_fallback(
-            self.session, non_extracting_text, reason="no_trailer",
-        )
-
-        # Sanity: all three are ambiguous canonical fallbacks.
-        self.assertTrue(out1.ambiguous, "Call 1 must be a canonical fallback")
-        self.assertTrue(out2.ambiguous, "Call 2 must be a canonical fallback")
-        self.assertTrue(out3.ambiguous, "Call 3 must be a canonical fallback")
-
-        # Headline assertion: by call 3 the loop-breaker has advanced past
-        # the stuck slot — the slot returned on call 3 MUST differ from call 1.
-        self.assertIsNotNone(out1.slot_driving, "Call 1 must surface a slot")
-        self.assertIsNotNone(out3.slot_driving, "Call 3 must surface a slot")
-        self.assertNotEqual(
-            out3.slot_driving, out1.slot_driving,
-            "REGRESSION (KI-103): canonical fallback re-asked the same slot "
-            f"on call 3 ({out3.slot_driving!r}) as on call 1 "
-            f"({out1.slot_driving!r}) — loop-breaker is broken. After "
-            "2 failed attempts the stuck slot must be marked asked and the "
-            "fallback must advance to the next unfilled slot.",
-        )
-
-        # The stuck slot from call 1 must now be in _ff_skipped_slots
-        # so the orchestrator/scorecard knows it's intentionally unanswered.
-        # `slot_driving` on the outcome is the FIELD name, not the question
-        # id; map back to the question id for the assertion.
-        from backend.fact_find_brain import FIELD_TO_QUESTION_ID
-        stuck_field = out1.slot_driving
-        stuck_qid = FIELD_TO_QUESTION_ID.get(stuck_field, stuck_field)
-        skipped = getattr(self.session, "_ff_skipped_slots", [])
-        self.assertIn(
-            stuck_qid, skipped,
-            f"REGRESSION (KI-103): stuck slot {stuck_qid!r} (field "
-            f"{stuck_field!r}) must be recorded on session._ff_skipped_slots "
-            f"so the scorecard knows it was intentionally skipped after "
-            f"hitting the failed-attempt threshold. Got skipped={skipped!r}.",
-        )
-
-    def test_successful_capture_resets_counter(self) -> None:
-        """A successful greedy capture for a slot must reset its failure
-        counter — so a temporary LLM outage that recovers doesn't
-        permanently ghost that slot for the session.
-        """
-        from backend.fact_find_brain import _canonical_fallback
-
-        # Call 1: non-extracting input → name surfaces, counter[name]=1.
-        _canonical_fallback(self.session, "hello there", reason="no_trailer")
-        attempts = getattr(self.session, "_ff_failed_attempts", {})
-        # Note: the surfaced slot id is determined by GREEDY_ORDER + GRAPH
-        # iteration; we don't hard-code it — just confirm a counter advanced.
-        self.assertTrue(
-            any(v >= 1 for v in attempts.values()),
-            "Failure counter must record the first surface as a failed attempt.",
-        )
-
-        # Call 2: user supplies a clear name in lenient form. The strict
-        # name parser requires an intro phrase, so use that explicitly.
-        out2 = _canonical_fallback(
-            self.session, "my name is Rohit", reason="no_trailer",
-        )
-        # Name should now be captured.
-        self.assertEqual(
-            self.session.profile.name, "Rohit",
-            "Greedy capture in canonical fallback should have extracted "
-            "the name from 'my name is Rohit'.",
-        )
-        # The name slot's counter must be reset (popped) after capture.
-        attempts_after = getattr(self.session, "_ff_failed_attempts", {})
-        self.assertNotIn(
-            "name", attempts_after,
-            "REGRESSION (KI-103): successful name capture must reset/pop "
-            "the failed-attempt counter for 'name' so the slot can be "
-            "re-introduced cleanly if the brain recovers.",
-        )
-
-    def test_failed_attempt_threshold_constant(self) -> None:
-        """Pin the threshold constant. If anyone tunes it back to 5+ they
-        have to update this test (and re-prove the loop is still broken).
-        """
-        from backend.fact_find_brain import _MAX_FAILED_ATTEMPTS
-        self.assertEqual(
-            _MAX_FAILED_ATTEMPTS, 2,
-            "KI-103 threshold is 2 (skip on the 3rd surface). Bumping this "
-            "above 2 re-enables the live-smoke-test loop bug where users "
-            "saw the same slot question 4-8 times in a row.",
-        )
-
-
-if __name__ == "__main__":
-    unittest.main(verbosity=2)

tests/test_name_persistence.py

@@ -182,19 +182,25 @@ class TestCaseA_ExtractorNullCannotWipeName(unittest.TestCase):
 
 
 # ---------------------------------------------------------------------------
-# CASE B — fact-find branch does not invoke extract_profile_updates.
+# CASE B — sales-brain branch does not invoke extract_profile_updates.
 # ---------------------------------------------------------------------------
-
-class TestCaseB_FactFindBranchDoesNotInvokeExtractor(unittest.TestCase):
-    """KI-091 regression. On a fact-find turn the orchestrator MUST NOT call
-    extract_profile_updates — the fact_find brain extracts slots natively
-    from its <FF> JSON tail, and the heavy extractor LLM both (a) duplicates
-    work and (b) periodically returns nulls that wipe filled slots.
-
-    We pin this by patching `extract_profile_updates` to raise. If the
-    orchestrator wrongly calls it on a fact-find turn, the call raises and
-    the test fails. If KI-091's gate is in place, the call never happens
-    and the turn completes cleanly.
+# KI-167 (2026-05-15) — rewired to mock `backend.sales_brain.drive_sales_brain`
+# returning a `SalesBrainResult` instead of the retired
+# `backend.fact_find_brain.drive_fact_find` / `FactFindOutcome`. Same
+# architectural invariant: a sales-brain (fact-find) turn must NOT trigger
+# the heavy extractor LLM, because the sales brain captures slots natively
+# from its structured JSON output.
+
+class TestCaseB_SalesBrainBranchDoesNotInvokeExtractor(unittest.TestCase):
+    """KI-091 regression (rewired under KI-167). On a sales-brain turn the
+    orchestrator MUST NOT call extract_profile_updates — the sales brain
+    captures slots natively in its JSON reply (`captures` field), and the
+    heavy extractor LLM both (a) duplicates work and (b) periodically returns
+    nulls that wipe filled slots.
+
+    We pin this by patching `extract_profile_updates` with a spy. If the
+    orchestrator wrongly calls it on a sales-brain turn, call_count > 0 and
+    the test fails. If the gate is in place, the call never happens.
     """
 
     def setUp(self) -> None:
@@ -205,58 +211,61 @@ class TestCaseB_FactFindBranchDoesNotInvokeExtractor(unittest.TestCase):
     def tearDown(self) -> None:
         _cleanup_session_file(self.session_id)
 
-    def test_fact_find_turn_does_not_call_extractor(self) -> None:
+    def test_sales_brain_turn_does_not_call_extractor(self) -> None:
         from backend.session_state import get_session
         from backend import orchestrator as orch
-        from backend.fact_find_brain import FactFindOutcome
+        from backend.sales_brain import SalesBrainResult
 
-        # Seed a session that's in fact-find continuation (awaiting an answer).
+        # Seed a session that's in sales-brain continuation (awaiting an answer).
         session = get_session(self.session_id)
         session.profile.name = "Rohit"
         session.profile.age = 30
         session.profile.asked = ["name", "age"]
         session.free_form_session = False
-        session.awaiting_question_id = "dependents"  # mid-fact-find
+        session.awaiting_question_id = "dependents"  # mid-sales-brain
         session._flush()
 
         # Mock extractor: if called, fail loudly.
         async def _explode(user_text, current_profile):
             raise AssertionError(
-                "extract_profile_updates was called on a fact-find turn — "
-                "KI-091 gate is broken. The fact_find brain already extracts "
-                "slots from its <FF> trailer; the heavy extractor LLM here "
-                "is both wasteful AND the source of the name-wipe bug."
+                "extract_profile_updates was called on a sales-brain turn — "
+                "the gate is broken. The sales brain already captures slots "
+                "natively from its structured JSON reply; the heavy extractor "
+                "LLM here is both wasteful AND the source of the name-wipe bug."
             )
 
-        # Mock the fact_find brain to return a benign outcome.
-        async def _stub_drive(user_text, session, chat_history, session_id):
-            return FactFindOutcome(
-                reply_text="And who else do you want covered?",
-                captured_updates={},
-                slot_driving="dependents",
-                fact_find_complete=False,
-                ambiguous=False,
-            )
+        # Mock the sales brain to return a benign SalesBrainResult.
+        mock_result = SalesBrainResult(
+            reply_text="And who else do you want covered?",
+            captured_updates={},
+            ready_for_recommendations=False,
+            brain_used="sales_brain::nim:test-model",
+            raw_json={
+                "reply": "And who else do you want covered?",
+                "captures": {},
+                "ready_for_recommendations": False,
+            },
+        )
 
         # The orchestrator wraps the extractor call in try/except, swallows
-        # AssertionError, and just logs a warning — meaning a broken KI-091
-        # gate would silently regress without our raise propagating. To pin
+        # AssertionError, and just logs a warning — meaning a broken gate
+        # would silently regress without our raise propagating. To pin
         # this properly, we monitor the patched extractor with a Mock and
         # assert call_count == 0 instead of relying on the raise reaching us.
         extractor_spy = mock.AsyncMock(side_effect=_explode)
 
         with mock.patch("backend.profile_extractor.extract_profile_updates",
                         new=extractor_spy), \
-             mock.patch("backend.fact_find_brain.drive_fact_find",
-                        new=_stub_drive), \
+             mock.patch("backend.sales_brain.drive_sales_brain",
+                        new=mock.AsyncMock(return_value=mock_result)), \
              mock.patch("backend.profile_rag.upsert_profile_chunk",
                         new=_stub_upsert), \
              mock.patch("backend.profile_store.save_profile", new=mock.Mock()):
 
-            # Drive a turn — the user's message is a fact-find answer.
+            # Drive a turn — the user's message is a sales-brain answer.
             # in_fact_find_continuation is True (awaiting_question_id set,
             # not free-form), so the orchestrator should route to the
-            # fact-find brain and return BEFORE reaching the extractor.
+            # sales brain and return BEFORE reaching the extractor.
             result = asyncio.run(orch.handle_turn(
                 user_text="self + spouse",
                 chat_history=[],
@@ -267,18 +276,17 @@ class TestCaseB_FactFindBranchDoesNotInvokeExtractor(unittest.TestCase):
         # The headline assertion: extractor was never invoked.
         self.assertEqual(
             extractor_spy.call_count, 0,
-            "REGRESSION (KI-091): extract_profile_updates was called on a "
-            "fact-find turn (call_count=%d). The intent='fact_find' / "
-            "in_fact_find_continuation gate in backend/orchestrator.py is "
-            "broken — fact-find turns must skip the heavy extractor LLM."
+            "REGRESSION (KI-091/KI-167): extract_profile_updates was called "
+            "on a sales-brain turn (call_count=%d). The gate in "
+            "backend/orchestrator.py is broken — sales-brain turns must skip "
+            "the heavy extractor LLM."
             % extractor_spy.call_count,
         )
 
-        # Sanity: the fact-find branch did run (we got its reply back).
-        self.assertEqual(result.intent, "fact_find")
+        # Sanity: the sales-brain branch did run (we got its reply back).
         self.assertEqual(
             result.reply_text, "And who else do you want covered?",
-            "Fact-find branch did not run — test setup is wrong.",
+            "Sales-brain branch did not run — test setup is wrong.",
         )
 
         # And the captured name is still there.