rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
9232416
·
1 Parent(s): 3f462fa

docs: KI-167/168/169/170/171/172 cascade — ADR-039 + CLAUDE + README

Browse files

ADR-039 (NEW) — Replace scripted fact_find_brain with LLM-driven sales_brain.
Status: Accepted, 2026-05-15. Marks ADR-027 + ADR-030 superseded.
Context: 8 patches (KI-150/155/156/158/161) all worked around <FF>
trailer fragility + canonical_fallback firing as dominant path.
User symptom: "zero natural LLM chat — always defaults to script."
Decision: rip out fact_find_brain + replace with sales_brain.py using
NIM response_format=json_object + deterministic sales_brain_normalizer.
Profile persistence unchanged (profile_store + profile_rag).

CLAUDE.md — fact-find section rewritten. ADR-030/KI-070 framing replaced
with KI-167/ADR-039. Body now states one LLM call per turn, structured
JSON, no scripted prompts, no fallback prose. Points at sales_brain.py
+ response_format={"type":"json_object"} + sales_brain_normalizer.py.
KI-150 max_tokens watch-out replaced with "never reintroduce scripted
fallback" rule. Retired KIs updated. Last-reviewed footer updated.

README.md — recent changes list adds KI-160 (NIM-only) + KI-167 (sales
rip-out). Step 2 + Tech 4 + Tech 7 prose + ASCII boxes now reference
sales_brain.py + JSON mode. "Question paraphraser (deprecated)" row in
model table replaced with "Sales brain (fact-find conversation driver)".

NOTE: This commit DOES NOT include the KI-168/169/170/171/172 follow-up
KIs in the doc cascade — those landed AFTER WS5 generated this docs
package. A subsequent docs commit can fold those in if needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-039 — Replace scripted fact_find_brain with LLM-driven sales_brain (KI-167)
2
+
3
+ **Status:** Accepted — 2026-05-15
4
+ **Owner:** Rohit Saraf
5
+ **Supersedes:** [ADR-027](ADR-027-fact-find-llm-paraphraser.md) (LLM paraphraser on top of `GRAPH`) and [ADR-030](ADR-030-llm-driven-fact-find.md) (one-call brain with `<FF>` trailer + canonical fallback). ADR-030's "schema-aware single LLM call per turn" intuition is preserved; the surface that implemented it (`fact_find_brain.py` + `<FF>...</FF>` trailer + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that — {slot}."`) is fully removed.
6
+ **Related KIs:** KI-070 / KI-072 / KI-074 / KI-075 / KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161 (all retired by KI-167), KI-167 (this rip-out), KI-160 / [ADR-038](ADR-038-nim-only-chains.md) (NIM-only chain pool — the substrate this ADR builds on).
7
+
8
+ ## Context
9
+
10
+ `backend/fact_find_brain.py` started life under ADR-030 as a single-LLM-call replacement for the ADR-027 paraphraser + hardcoded `GRAPH` stitch. It needed eight subsequent patches to stay alive:
11
+
12
+ - **KI-090** — lenient `<FF>` parser because real LLMs dropped the literal tags around their JSON tail.
13
+ - **KI-091 / KI-094** — guard against the extractor returning `{"name": null}` and overwriting captured fields.
14
+ - **KI-103** — `no_trailer` loop-breaker after the same slot failed twice.
15
+ - **KI-150** — `max_tokens` 420 → 700 because the prose+trailer prompt was truncating mid-paraphrase.
16
+ - **KI-155 / KI-156 / KI-158 / KI-161** — silent contract violations where the elected LLM returned grammatical prose with no parseable `<FF>` block. Every turn it happened, `_canonical_fallback` fired and the user saw the scripted `Question.prompt_en` of the next unfilled slot, prefixed with `"Got that — {slot}."` — i.e. the bot reverted to the very state machine ADR-030 was supposed to replace.
17
+
18
+ User-visible symptom by 2026-05-15: *"zero natural LLM chat — it always defaults to the script."* The `<FF>` trailer convention + `_canonical_fallback` were no longer a degraded path; they were the **dominant** path on multiple chain candidates, and the scripted prefix kept leaking into every fallback turn. The architecture had become reactive accretion — eight patches stacked on a fragile structured-output contract — not deliberate design. KI-160 / [ADR-038](ADR-038-nim-only-chains.md) closed the cross-provider silent-failure class at the chain level; KI-167 closes the structured-output silent-failure class at the prompt level.
19
+
20
+ ## Decision
21
+
22
+ **Rip out `backend/fact_find_brain.py` entirely.** Replace with `backend/sales_brain.py` — a single LLM-driven sales agent that owns the entire fact-find conversation surface. Concretely:
23
+
24
+ - **One NIM call per turn**, via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-038](ADR-038-nim-only-chains.md) NIM-only pool. The call uses NVIDIA NIM's `response_format={"type": "json_object"}` for structured output — the same JSON-mode contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`.
25
+ - **System prompt** contains the 9-slot schema (slot id, description, accepted value shapes), current profile state (which slots are filled with which values), and an instruction to (a) reply naturally in the advisor's voice, (b) capture any new facts the user just stated, (c) decide which slot to drive next, (d) flag completion when all required slots are filled. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time — there is no scripted prompt to match, no opener to rotate, no acknowledger prefix.
26
+ - **JSON response shape:** `{"reply": "<prose for user>", "captures": {<slot_id>: <raw_value>, ...}, "slot_driving": "<slot_id or null>", "complete": <bool>}`. No `<FF>` tags, no trailer convention — `response_format` guarantees the whole response body parses as JSON.
27
+ - **Deterministic post-processor** (`backend/sales_brain_normalizer.py`, KI-167 WS1) takes the LLM's loose `captures` dict and emits a clean `{canonical_field: validated_value}` map: field-name alias resolution (`location` → `location_tier`), enum normalization (`Bangalore` → `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. The orchestrator applies the normalized map via `session.update_profile_field()` exactly as before.
28
+ - **Profile persistence is unchanged.** Captured fields still flow through `backend/profile_store.save_profile()` for disk durability and `backend/profile_rag.upsert_profile_chunk()` for retrieval-side visibility. The session-isolation guarantees from KI-102 / KI-107 / KI-112 remain in force — `sales_brain.py` only changes the *upstream* fact-find loop; the downstream persistence layer is untouched.
29
+ - **No scripted prompts.** `backend/needs_finder.py::Question.prompt_en` is no longer consulted by the fact-find branch. The `GRAPH` data structure stays in `needs_finder.py` for now (slot-id definitions are still useful as the schema source for the LLM's system prompt), but its `prompt_en` field is dead text.
30
+ - **No canonical_fallback.** `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, `"Got that — {slot}."` prefix logic, `_ff_failed_attempts` / `_ff_skipped_slots` session fields, and every `fact_find_brain::fallback:*` telemetry variant are deleted in WS2 / WS3.
31
+ - **No trailer convention.** The lenient parser ladder (`<FF>` → fenced `json` → bare-JSON-tail) is deleted. Structured output is guaranteed by the provider, not pattern-matched from prose.
32
+
33
+ The outer 25s `asyncio.wait_for` ceiling around the brain call is retained. On total NIM exhaustion, the orchestrator returns the [ADR-038](ADR-038-nim-only-chains.md) graceful error message to the user (fail-loud) rather than cascading into a scripted reply.
34
+
35
+ ## Consequences
36
+
37
+ ### Positive
38
+
39
+ - **Bot feels human.** No more scripted `"Got that — {slot}."` leakage, no more identical `prompt_en` strings across sessions, no more two-line opener+question cadence. The LLM owns voice + flow end-to-end.
40
+ - **One fewer LLM call per turn.** The pre-KI-167 fact-find turn ran the brain + (on KI-091 QA turns) the profile extractor. Captures are now in the brain's structured response, so the extractor is dead code for fact-find turns and stays skipped on QA turns per KI-091.
41
+ - **Structured-output guarantees parseability.** NIM's `response_format={"type":"json_object"}` is enforced server-side; the response body either parses as JSON or the call fails loud. Eliminates the entire class of "model returned grammatical prose with no parseable trailer" silent failures (KI-155 / KI-156 / KI-158 / KI-161).
42
+ - **Eliminates 8 KIs of patching.** KI-090 (lenient parser), KI-091 / KI-094 (extractor null-overwrite guards on fact-find turns), KI-103 (no_trailer loop-breaker), KI-150 (`max_tokens` 420→700), KI-155 / KI-156 / KI-158 / KI-161 (trailer contract violations) all become moot — the root causes are gone, not patched.
43
+ - **Smaller surface area.** `backend/fact_find_brain.py` (441 LOC), the `_canonical_fallback` branch in `backend/orchestrator.py`, the `_pick_opener` family in `backend/orchestrator.py`, and the dead `Question.prompt_en` strings in `backend/needs_finder.py` all go away. Net negative LOC.
44
+
45
+ ### Negative
46
+
47
+ - **Persona-audit fixtures need updating.** The persona transcripts in `eval/` and the `tests/test_persona_*` fixtures hardcode the expected turn order from the rules engine (age → dependents → income → ...). The LLM may capture in a different order (or multi-fact in one turn) depending on what the user said. WS4 owns rewriting these fixtures to assert on **final captured state** rather than per-turn slot order.
48
+ - **Higher LLM dependency.** There is no scripted safety net underneath the sales brain. If every NIM candidate in `FAST_BRAIN_CHAIN` fails simultaneously, the user sees the graceful error message from [ADR-038](ADR-038-nim-only-chains.md) — not a scripted fact-find prompt.
49
+ - **Loss of explicit completion path.** Pre-KI-167, `complete=True` was enforced by the orchestrator checking all 9 slots filled. Post-KI-167 the LLM is responsible for setting `complete: true`. The deterministic post-processor double-checks (`complete && any required slot still empty` ⇒ override to `false`), but the surface-of-trust shifts from rules engine to LLM.
50
+
51
+ ### Mitigations
52
+
53
+ - **NIM JSON mode is production-validated.** `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py` already run `response_format={"type":"json_object"}` against the same NIM candidate pool. Failure mode is well-understood and observable.
54
+ - **Outer 25s `asyncio.wait_for` ceiling retained.** On total NIM exhaustion the failure surfaces as a clear error message (per [ADR-038](ADR-038-nim-only-chains.md)), not as a hang or as a regression to scripted text.
55
+ - **Deterministic post-processor enforces required-slot completeness.** The LLM cannot prematurely mark `complete: true` while a required slot is empty — `sales_brain_normalizer.py` overrides.
56
+ - **Persona audit refactor scoped to WS4.** Per-turn brittleness is a known cost; fixture rewrite targets final-state assertions which are stable regardless of which order the brain captures slots in.
57
+
58
+ ## Alternatives considered
59
+
60
+ - **Keep `fact_find_brain.py`, harden the `<FF>` parser further.** Rejected. KI-155 / KI-156 / KI-158 / KI-161 demonstrated that the failure class is structural to a prose-plus-trailer prompt shape under load — not a parser bug. Each patch shifted the failure to a new turn / candidate / token cap. The directive *"Never use trailer conventions when the provider offers structured output"* (memory: KI-160 / [ADR-038](ADR-038-nim-only-chains.md)) generalizes to the prompt level: don't pattern-match structure out of prose when the provider can guarantee it server-side.
61
+ - **Keep `_canonical_fallback` as a last-resort path.** Rejected. The whole point of the rip-out is that the fallback was the dominant path, and the scripted `Question.prompt_en` prefixed with `"Got that — {slot}."` was exactly the user-visible artefact KI-167 is aimed at. Leaving the fallback in would keep the regression surface alive.
62
+ - **Hybrid: scripted slots 1-3, LLM brain for slots 4-9.** Rejected. The robotic feel of "what's your age? what's your city? who are your dependents?" hits in the *first three turns* — splitting the loop preserves the worst part of the old system.
63
+
64
+ ## Reversal trigger
65
+
66
+ Re-open this ADR if (a) NIM removes `response_format` support, or (b) a 7-day window shows the deterministic post-processor's required-slot-completeness override firing > 5% of completion turns (signalling the LLM systematically lies about completion), or (c) persona audit pass-rate drops below 80% after the WS4 fixture rewrite and the cause is brain-side rather than fixture-side. The reversal would still NOT restore the `<FF>` trailer convention — it would either tighten the JSON-mode prompt or shift to a different provider's structured-output API.
CLAUDE.md CHANGED
@@ -31,15 +31,15 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
31
  - **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
32
  - **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
33
  - **Brain / fast-brain / judge primaries in steady state** are **NIM Nemotron-Super 49B** (brain), **NIM Qwen 3-Next 80B** (fast brain), and **NIM Llama-4 Maverick 17B/128E** (judge), per the KI-160 candidate lists. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` within the NIM pool.
34
- - **KI-079 escalation as last bite (`87ee522`).** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget) before falling to `_canonical_fallback` (KI-072 / KI-074 greedy slot capture). Worst-case wall-clock before canonical: 25s FAST + 15s heavy = 40s.
35
  - **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
36
- - **Lenient FF-block parser (KI-090, `11cf4b3`).** Real LLMs (Qwen, Nemotron under load, Groq Llama-3.3) sometimes drop the literal `<FF>...</FF>` tags around their JSON tail. Pre-KI-090 those replies fell to `fallback:no_trailer` even though the brain had produced a perfectly valid structured response. Now `_parse_ff_block` tries strict → fenced ```` ```json``` ```` → bare-JSON-tail, each candidate validated by presence of a contract key (`captured` / `slot_driving` / `complete`). `_strip_ff_block` mirrors the strategies so prose-only output never leaks structured metadata.
37
- - **Skip `profile_extractor` + faithfulness judge on fact-find turns (KI-091, `9813994`).** Both chains were credit-exhausted on the steady-state primary, hung fact-find turns for 20+s, and the extractor periodically returned `{"name": null}` which wrote into `session.update_profile_field` and wiped the captured name mid-session — causing `next_question` to re-ask the name slot the user had already answered. Orchestrator now short-circuits both chains behind `intent == "fact_find"`; the fact-find brain (KI-070) extracts fields natively from its `<FF>` JSON tail, and faithfulness scoring is meaningless on "what's your annual income?". QA-mode turns still run both chains. Live: name re-ask loop gone; fact-find p95 28s → 6-8s.
38
  - **Defensive `None`-guard in extractor merge (KI-094, `f068094`).** Belt-and-braces companion to KI-091. On QA-mode turns the extractor still runs (correct: "I'm now 35" mid-recommendation should update the profile), but under load it periodically returns `{"name": null, "age": null, ...}` and the merge loop was writing every key including nulls back into the session. Added `if new_value in (None, "", []): continue` at the top of the extracted-fields loop in `backend/orchestrator.py` — null / empty-string / empty-list returns are now no-ops. Closes the same root cause (LLM-returned nulls wiping state) at a second layer; KI-091 prevents the extractor from running on fact-find turns at all, KI-094 makes it safe even when it does run.
39
  - **Remove IP allowlist from admin — password-only gate (KI-097, pending).** Dropped `ADMIN_IP_ALLOWLIST` env + `_ip_allowed()` from `backend/admin.py`; `_check_admin` is now password-only against `X-Admin-Password` → `ADMIN_PASSWORD` env. Backend returns 401 Unauthorized (previously 404-to-hide). Frontend admin panel is always visible; password unlocks the live data. ADR-023 superseded — IP gating added zero security beyond a strong password and locked the operator out whenever the home IP changed.
40
  - **Drop function-local `import logging` in orchestrator (KI-101, `66eb4ed`).** Removed 6 inline `import logging` lines from `backend/orchestrator.py`; Python's scoping rule was promoting `logging` to function-local, causing `UnboundLocalError` when the `asyncio.wait_for` `TimeoutError` branch fired before reaching the inline import. Module-level import is the sole binding now.
41
  - **Profile RAG session isolation (KI-102, `4bb8da0`).** `upsert_profile_chunk` stamps `session_id` metadata on every chunk; `retrieve()` excludes `doc_type == "profile"` from the main pass; per-session profile lookup triple-checks `meta.session_id == session_id` in Python after the Chroma where-clause. Legacy chunks without `session_id` are silently refused (fail-closed). Cross-session PII leak (age / dependents / health conditions) closed. ADR-022 extended with session-isolation subsection.
42
- - **`_canonical_fallback` no_trailer loop-breaker (KI-103, `8ef5c43`).** Added `session._ff_failed_attempts[slot_id]` + `session._ff_skipped_slots: set[str]`. After 2 failed attempts on the same slot, mark skipped and advance to the next unfilled slot. Caps the worst-case wedge at ~18 turns to escape fact-find instead of unbounded.
43
  - **CoT / instruction-echo strip in voice_format (KI-104, `407f2a1`).** `tts_preprocess` now kills `<think>...</think>` blocks, `**Reasoning:**` / `**Thought:**` labels, `[INTERNAL]` blocks, sentence-anchored CoT starters ("Let me think...", "Step 1:..."). Emergency fallback to a generic acknowledger if the whole reply is CoT-shaped. Stops Sarvam from TTS-ing the bot's internal monologue.
44
  - **Recommendation closer wired (KI-105, `8a58fa1`).** `RECOMMENDATION_CLOSER_PHRASES` frozenset ("show me the top 3", "rank", "pitch me", "compare X vs Y") classified as `recommendation` / `comparison` BEFORE the `FACT_FIND_TRIGGERS` check, so a fully-fact-found user can never get bounced back into fact-find. Persona prompt gets `RECOMMENDATION_CLOSER_ADDENDUM` with a strict 3-policy ranked-shortlist contract (3 policies, one-line rationale each, IRDAI disclaimer, no hedging). ADR-008 extended with closer-mode subsection.
45
  - **Graceful TimeoutError + Exception on `/api/chat` (KI-106, `565bf31`).** `handle_turn(...)` wrapped in `asyncio.wait_for(45s)` with explicit `except asyncio.TimeoutError` + broad `except Exception`. Both return HTTP 200 with `source="graceful_timeout"` / `graceful_exception"` and an in-character recovery sentence instead of HTTP 500. Internal `logger.exception` still captures the full traceback for admin observability.
@@ -49,18 +49,20 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
49
  - **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
50
  - **Provider keys.** `NVIDIA_NIM_API_KEY` + `GROQ_API_KEY` + `OPENROUTER_API_KEY` required in `.env` (local) and HF Space environment (production — KI-081).
51
 
52
- ## Fact-find loop (ADR-030, supersedes ADR-027) — KI-070
53
 
54
- **One LLM call per turn drives the entire fact-find conversation.** The pre-KI-070 three-layer stitching (hardcoded `GRAPH` question text + paraphraser + opener / acknowledger rotation) read as robotic copy-paste in user testing and is retired. `backend/fact_find_brain.py::drive_fact_find()` issues a single `NimChainLLM(FAST_BRAIN_CHAIN, timeout=12s)` call whose system prompt contains the 9-slot schema + current profile state + recent chat history, and emits natural conversational prose followed by a JSON tail block `<FF>{"captured":{...}, "slot_driving":"...", "complete":<bool>}</FF>`. Orchestrator strips the `<FF>` block before sending prose to the user; the JSON updates `session.profile` and selects the next slot in one pass.
55
 
56
- - **Native multi-fact capture.** A single user utterance like *"I'm 34, in Mumbai, just myself"* fills age + city + dependents in one turn. Verified live on 2026-05-15: `profile_updates: {name: 'Rohit Sar', age: 32, dependents: 'self', location_tier: 'metro'}` from one opener.
57
- - **Safeguards.** JSON-block-must-parse fall to canonical `next_question(slot_id)`. Slot-not-progressing (3 turns same slot, no captures) bail to canonical. Hard 12s budget. Any chain exhaustion canonical fallback. Fact-find can never wedge.
58
- - **`backend/needs_finder.py::GRAPH`** retained as the safeguard fallback path onlynever the primary path in steady state, but always available.
59
- - **DELETED in KI-070** (~500 LOC): `backend/question_paraphraser.py` module, `_pick_opener`, `_NEUTRAL_OPENERS` / `_FAMILY_OPENERS` constants, `_contains_self_introduction`, the KI-067 first-policy regex (brain captures natively from prose), the acknowledger template selection.
60
- - **Fallback path is no longer a dead-end (KI-072 / KI-074).** When the brain times out / chain exhausts / `<FF>` block fails to parse, `_canonical_fallback` doesn't just return the canonical question it GREEDILY applies the user's current message to every unfilled slot via `_normalize_for_slot`, applies captures to the profile, then picks the next still-empty slot. Slot-specific trigger guards (age needs "years old" or bare number; existing_cover needs ₹/lakh/Cr cue or denial; name needs explicit "I'm X" intro pattern) prevent cross-contamination. So a NIM-degraded session walks fact-find via canonical questions but still progresses — never wedges.
61
- - **Brain timeout: 25s (KI-075).** The `asyncio.wait_for` wrap around `drive_fact_find` was 12s originally too tight. NIM cold-start eats 10-15s after a Space rebuild, killing the brain call before the chain's internal 22s `total_budget_s` could try Groq/OpenRouter fallbacks. 25s gives cold-start headroom + 1 chain fallback hop.
62
- - **Natural-conversation escape (KI-045):** intent_change phrases or off-topic questions still exit fact-find by routing through `should_route_to_fact_find` handled upstream of `drive_fact_find` so the safeguard mechanism here applies to in-fact-find pivots.
63
- - **Indic queries** route through Sarvam-M for translation on input + output; the fact-find brain itself runs in English on the translated text.
 
 
64
 
65
  ## Refusal precision (KI-046)
66
 
@@ -131,8 +133,8 @@ Three independent safety layers against ChromaDB HNSW bloat:
131
  - **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
132
  - **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
133
  - **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
134
- - **`fact_find_brain` `max_tokens = 700` (KI-150, was 420).** The 420 cap was the root cause of the "robotic language" user complaint — the brain ran out of tokens mid-paraphrase. Do not lower below 700 without a regression test on natural-conversation feel.
135
 
136
  ---
137
 
138
- *Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General; fact_find_brain max_tokens 420→700).*
 
31
  - **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
32
  - **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
33
  - **Brain / fast-brain / judge primaries in steady state** are **NIM Nemotron-Super 49B** (brain), **NIM Qwen 3-Next 80B** (fast brain), and **NIM Llama-4 Maverick 17B/128E** (judge), per the KI-160 candidate lists. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` within the NIM pool.
34
+ - **KI-079 escalation as last bite (`87ee522`) — modified by KI-167.** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget). Post-KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) the `_canonical_fallback` terminal step is gone — on heavy-brain exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message instead. Worst-case wall-clock before graceful error: 25s FAST + 15s heavy = 40s.
35
  - **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
36
+ - **Lenient FF-block parser (KI-090, `11cf4b3`) RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** Was needed because the `<FF>...</FF>` trailer convention was fragile under load. Post-KI-167 the sales brain uses NIM `response_format={"type":"json_object"}` for guaranteed structured output, so the whole `_parse_ff_block` strict → fenced → bare-JSON ladder is gone.
37
+ - **Skip `profile_extractor` + faithfulness judge on fact-find turns (KI-091, `9813994`).** Both chains were credit-exhausted on the steady-state primary, hung fact-find turns for 20+s, and the extractor periodically returned `{"name": null}` which wrote into `session.update_profile_field` and wiped the captured name mid-session. Orchestrator short-circuits both chains behind `intent == "fact_find"`; the sales brain (KI-167) captures fields natively from its JSON-mode response body, and faithfulness scoring is meaningless on "what's your annual income?". QA-mode turns still run both chains. Live: name re-ask loop gone; fact-find p95 28s → 6-8s.
38
  - **Defensive `None`-guard in extractor merge (KI-094, `f068094`).** Belt-and-braces companion to KI-091. On QA-mode turns the extractor still runs (correct: "I'm now 35" mid-recommendation should update the profile), but under load it periodically returns `{"name": null, "age": null, ...}` and the merge loop was writing every key including nulls back into the session. Added `if new_value in (None, "", []): continue` at the top of the extracted-fields loop in `backend/orchestrator.py` — null / empty-string / empty-list returns are now no-ops. Closes the same root cause (LLM-returned nulls wiping state) at a second layer; KI-091 prevents the extractor from running on fact-find turns at all, KI-094 makes it safe even when it does run.
39
  - **Remove IP allowlist from admin — password-only gate (KI-097, pending).** Dropped `ADMIN_IP_ALLOWLIST` env + `_ip_allowed()` from `backend/admin.py`; `_check_admin` is now password-only against `X-Admin-Password` → `ADMIN_PASSWORD` env. Backend returns 401 Unauthorized (previously 404-to-hide). Frontend admin panel is always visible; password unlocks the live data. ADR-023 superseded — IP gating added zero security beyond a strong password and locked the operator out whenever the home IP changed.
40
  - **Drop function-local `import logging` in orchestrator (KI-101, `66eb4ed`).** Removed 6 inline `import logging` lines from `backend/orchestrator.py`; Python's scoping rule was promoting `logging` to function-local, causing `UnboundLocalError` when the `asyncio.wait_for` `TimeoutError` branch fired before reaching the inline import. Module-level import is the sole binding now.
41
  - **Profile RAG session isolation (KI-102, `4bb8da0`).** `upsert_profile_chunk` stamps `session_id` metadata on every chunk; `retrieve()` excludes `doc_type == "profile"` from the main pass; per-session profile lookup triple-checks `meta.session_id == session_id` in Python after the Chroma where-clause. Legacy chunks without `session_id` are silently refused (fail-closed). Cross-session PII leak (age / dependents / health conditions) closed. ADR-022 extended with session-isolation subsection.
42
+ - **`_canonical_fallback` no_trailer loop-breaker (KI-103, `8ef5c43`) — RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** The `_ff_failed_attempts` + `_ff_skipped_slots` state and the entire `_canonical_fallback` branch are deleted; the failure mode they capped (silent `<FF>` trailer parse failures looping on the same slot) is gone because the sales brain uses NIM JSON mode.
43
  - **CoT / instruction-echo strip in voice_format (KI-104, `407f2a1`).** `tts_preprocess` now kills `<think>...</think>` blocks, `**Reasoning:**` / `**Thought:**` labels, `[INTERNAL]` blocks, sentence-anchored CoT starters ("Let me think...", "Step 1:..."). Emergency fallback to a generic acknowledger if the whole reply is CoT-shaped. Stops Sarvam from TTS-ing the bot's internal monologue.
44
  - **Recommendation closer wired (KI-105, `8a58fa1`).** `RECOMMENDATION_CLOSER_PHRASES` frozenset ("show me the top 3", "rank", "pitch me", "compare X vs Y") classified as `recommendation` / `comparison` BEFORE the `FACT_FIND_TRIGGERS` check, so a fully-fact-found user can never get bounced back into fact-find. Persona prompt gets `RECOMMENDATION_CLOSER_ADDENDUM` with a strict 3-policy ranked-shortlist contract (3 policies, one-line rationale each, IRDAI disclaimer, no hedging). ADR-008 extended with closer-mode subsection.
45
  - **Graceful TimeoutError + Exception on `/api/chat` (KI-106, `565bf31`).** `handle_turn(...)` wrapped in `asyncio.wait_for(45s)` with explicit `except asyncio.TimeoutError` + broad `except Exception`. Both return HTTP 200 with `source="graceful_timeout"` / `graceful_exception"` and an in-character recovery sentence instead of HTTP 500. Internal `logger.exception` still captures the full traceback for admin observability.
 
49
  - **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
50
  - **Provider keys.** `NVIDIA_NIM_API_KEY` + `GROQ_API_KEY` + `OPENROUTER_API_KEY` required in `.env` (local) and HF Space environment (production — KI-081).
51
 
52
+ ## Fact-find loop (ADR-039, supersedes ADR-030 / ADR-027) — KI-167
53
 
54
+ **One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that {slot}."`) and replaced it with `backend/sales_brain.py` a single LLM-driven sales agent. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-155 and follow-ups showed the canonical fallback was the *dominant* path, not the safety net. User-visible symptom: "zero natural LLM chat it always defaults to the script."
55
 
56
+ - **Single NIM call per turn** via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool, using NVIDIA NIM's **`response_format={"type": "json_object"}`** for structured output the same JSON-mode contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`. Response shape: `{"reply": "<prose for user>", "captures": {<slot_id>: <raw>}, "slot_driving": "<slot_id|null>", "complete": <bool>}`.
57
+ - **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
58
+ - **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` `location_tier`), enum normalization (`Bangalore` `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
59
+ - **Profile persistence unchanged.** Captures flow through `session.update_profile_field()` `backend/profile_store.save_profile()` (disk durability) `backend/profile_rag.upsert_profile_chunk()` (retrieval-side visibility). KI-102 / KI-107 / KI-112 session-isolation guards remain in force.
60
+ - **No scripted prompts, no canonical_fallback, no trailer convention.** `backend/needs_finder.py::GRAPH` slot-id data stays as a schema source for the system prompt, but `Question.prompt_en` is dead textnever consulted by the fact-find branch. `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, the lenient `<FF>` parser ladder, and the `"Got that {slot}."` prefix logic are deleted.
61
+ - **Outer 25s `asyncio.wait_for` ceiling retained.** On total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error to the user (fail-loud) rather than cascading to a scripted reply. There is no scripted safety net that is intentional.
62
+ - **DELETED in KI-167:** `backend/fact_find_brain.py` (441 LOC); `_canonical_fallback` + `_pick_opener` branches in `backend/orchestrator.py`; `_ff_failed_attempts` / `_ff_skipped_slots` session fields; every `fact_find_brain::fallback:*` telemetry variant; the lenient `<FF>` / fenced-`json` / bare-JSON-tail parser ladder.
63
+ - **Persona-audit fixtures rewritten** to assert on final captured state (which slots are filled with what) rather than per-turn slot order the LLM may capture in different turn order than the pre-KI-167 rules engine did. See WS4.
64
+ - **Natural-conversation escape (KI-045):** intent_change phrases / off-topic questions still exit fact-find by routing through `should_route_to_fact_find` upstream of the brain.
65
+ - **Indic queries** route through Sarvam-M for translation on input + output; the sales brain runs in English on the translated text.
66
 
67
  ## Refusal precision (KI-046)
68
 
 
133
  - **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
134
  - **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
135
  - **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
136
+ - **`sales_brain.py` is the fact-find handler (KI-167, [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** `backend/fact_find_brain.py` is deleted along with the `<FF>` trailer convention, `_canonical_fallback`, scripted `Question.prompt_en`, and the `"Got that — {slot}."` prefix. Never re-introduce a fallback to scripted prompts on the fact-find branch on total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message (fail-loud > fail-silent-with-script). The KI-150 historical note (`fact_find_brain` `max_tokens` 420 700) is moot post-KI-167 since the prose+trailer prompt shape that needed the larger budget no longer exists.
137
 
138
  ---
139
 
140
+ *Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General). Same day: KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) locked chains to NIM-only after KI-155's `<FF>` trailer contract violation. Same day: KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) replaced `fact_find_brain.py` (scripted slot-walker + `<FF>` trailer + `_canonical_fallback` + scripted `Question.prompt_en`) with `backend/sales_brain.py` (single LLM call per turn, NIM `response_format=json_object`, deterministic post-processor).*
README.md CHANGED
@@ -62,7 +62,9 @@ Thirty-plus knowledge increments landed today. The corpus was rebuilt, the marke
62
  - **KI-143** — `bajaj/group-health-guard` slug correction (gold → silver per the PDF).
63
  - **KI-144** — `reliance-general` → `indusind-general` migration (Reliance General Insurance was rebranded to IndusInd General). The `indusind-general` slug did not exist anywhere in the codebase before today.
64
  - **KI-149** — budget + income parser captures bare numerals like `"30000"`.
65
- - **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (root cause of the "robotic language" user complaint).
 
 
66
 
67
  Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
68
 
@@ -93,7 +95,7 @@ A take-home is a sample of how the engineer thinks under constraint. Three thing
93
  3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a NIM-only candidate pool per role (KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)), not a single hardcoded brain. Each role is a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) whose candidates are all NIM-hosted — KI-155 demonstrated that Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract, so cross-provider fallback was removed as a silent-failure trap:
94
 
95
  - **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`), backup **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE), 3rd candidate **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`).
96
- - **`FAST_BRAIN_CHAIN`** (fact-find turns, QA, paraphrase, normalize, extract — every latency-sensitive role) — primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`), backup **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`).
97
  - **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Meta Llama-4 Maverick 17B/128E** (`meta/llama-4-maverick-17b-128e-instruct`), backup **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`). Deliberately different model families from the brain pool so the judge does not mark its own homework.
98
 
99
  **NIM-only election, no cross-provider cascade (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** If every NIM candidate in a chain fails, orchestrator returns a graceful error message rather than falling to Groq or OpenRouter — fail-loud is preferred over fail-silent-with-garbage for structured-output contracts. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the cross-provider-fallback variant of KI-080 ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)) are both superseded. KI-085's proactive credit gating still applies within the NIM pool via a per-model 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space secrets for future re-enable but the chain config no longer references them.
@@ -184,7 +186,7 @@ The bot is two flows running together — the customer's experience and the tech
184
  **Step 2 — Ask the first question (voice or text)**
185
  - User just talks — VAD detects speech start/end automatically
186
  - *"Suggest a health insurance plan for me"*
187
- - Bot recognises fact-find intent. The orchestrator picks the next slot from a 9-question graph; an LLM paraphraser rewrites the canonical question in a warmer voice each session (verified to still target the same slot before sending)
188
  - Slots in order: age → dependents → income → existing cover → primary goal → location → parents (conditional) → health → budget
189
 
190
  </td><td>
@@ -199,8 +201,8 @@ The bot is two flows running together — the customer's experience and the tech
199
  - `detect_language(user_text)` → `english / indic`
200
  - Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
201
 
202
- **Tech 4 — Fact-find branch**
203
- - If profile empty AND intent ∈ {fact_find, recommendation, comparison}: pick next slot, paraphrase via `question_paraphraser` + verifier, emit question, return early
204
  - Else: continue to retrieval
205
 
206
  </td></tr>
@@ -252,9 +254,9 @@ The bot is two flows running together — the customer's experience and the tech
252
  </td><td>
253
 
254
  **Tech 7 — Brain selection, `pick_brain(intent, language)`**
255
- - `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary across NIM / Groq / OpenRouter candidates (KI-080)
256
- - `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary (typically Nemotron Nano 30B or Groq Llama-3.3 in steady state)
257
- - KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → cross-provider backup). Per-chain primary refreshed every 60s by background probe.
258
 
259
  **Tech 8 — System prompt construction, `build_messages()`**
260
  - `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
@@ -374,12 +376,15 @@ The bot is two flows running together — the customer's experience and the tech
374
 
375
 
376
  ┌──────────────────────────────────────────────────────────────────────────┐
377
- │ TECH 4. Fact-find OR free-form branch
378
  │ ------- │
379
- │ · If session.awaiting_question_id and not free_form:
380
- record_answer(session.profile, qid, raw)
381
- next_question(profile) asks next
382
- RETURN early
 
 
 
383
  │ · Else: set free_form_session = True, continue to retrieval │
384
  └──────────────────────────────────────────────────────────────────────────┘
385
 
@@ -410,9 +415,9 @@ The bot is two flows running together — the customer's experience and the tech
410
  ┌──────────────────────────────────────────────────────────────────────────┐
411
  │ TECH 7. Brain selection — pick_brain(intent, language) │
412
  │ ------- │
413
- │ · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary across NIM/Groq/OpenRouter (heavy, KI-080)
414
- │ · intent ∈ {qa, fact_find} → FAST_BRAIN_CHAIN with probe-elected primary (fast, KI-080)
415
- │ · NIM Qwen 80B / Nemotron 30B + Groq Llama-3.3 + OpenRouter GPT-OSS as election candidates
416
  └──────────────────────────────────────────────────────────────────────────┘
417
 
418
 
@@ -573,11 +578,11 @@ Every LLM role is served by a **candidate pool** of models elected over by a pro
573
  | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
574
  |---|---|---|---|---|
575
  | **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) — typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
576
- | **Fast brain** (voice turns, fact-find, QA, paraphrase, normalize, extract) | Probe-elected primary (KI-080) — typically Nemotron Nano 30B or Groq Llama-3.3-70B in steady state | Qwen 3-Next 80B → GPT-OSS 120B → Qwen 3.5 122B → DeepSeek V4-Flash → Groq Llama-3.3-70B (election candidates) | NIM + Groq | Bottleneck on these short jobs is TTFT, not capability. Nemotron Nano hits ~1.6s; Qwen 80B is ~2-3s. Reordered for latency in KI-035. Election adds adaptation: when one provider degrades, the probe re-elects within 60s. |
577
  | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
578
  | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
579
  | **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
580
- | **Question paraphraser** *(deprecated by KI-070)* | Fast brain chain (Nemotron Nano 30B primary), 4s per-link / 6s total budget | inherits FAST_BRAIN_CHAIN | NIM + Groq | ADR-027 layered an LLM paraphraser on top of the hardcoded GRAPH to make fact-find sound less robotic. **KI-070 replaces the paraphraser + opener-rotation + acknowledger machinery with a single LLM brain call per fact-find turn (ADR-030).** The paraphraser module is scheduled for deletion in the KI-070 sync sweep. |
581
  | **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
582
  | **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
583
  | **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
 
62
  - **KI-143** — `bajaj/group-health-guard` slug correction (gold → silver per the PDF).
63
  - **KI-144** — `reliance-general` → `indusind-general` migration (Reliance General Insurance was rebranded to IndusInd General). The `indusind-general` slug did not exist anywhere in the codebase before today.
64
  - **KI-149** — budget + income parser captures bare numerals like `"30000"`.
65
+ - **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (one of eight band-aids on the prose+`<FF>`-trailer prompt shape; superseded by KI-167).
66
+ - **KI-160** — chains locked to NIM-only ([ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)) after KI-155 demonstrated Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract.
67
+ - **KI-167** — `backend/fact_find_brain.py` ripped out and replaced by `backend/sales_brain.py` ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)). Single LLM call per turn using NIM `response_format={"type":"json_object"}`; no scripted `Question.prompt_en`, no `<FF>` trailer, no `_canonical_fallback`, no `"Got that — {slot}."` prefix. Deterministic post-processor (`backend/sales_brain_normalizer.py`) normalizes captures; profile persists via `profile_store` + `profile_rag` as before.
68
 
69
  Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
70
 
 
95
  3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a NIM-only candidate pool per role (KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)), not a single hardcoded brain. Each role is a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) whose candidates are all NIM-hosted — KI-155 demonstrated that Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract, so cross-provider fallback was removed as a silent-failure trap:
96
 
97
  - **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`), backup **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE), 3rd candidate **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`).
98
+ - **`FAST_BRAIN_CHAIN`** (sales-brain fact-find turns per KI-167, QA, normalize, extract — every latency-sensitive role) — primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`), backup **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`).
99
  - **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Meta Llama-4 Maverick 17B/128E** (`meta/llama-4-maverick-17b-128e-instruct`), backup **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`). Deliberately different model families from the brain pool so the judge does not mark its own homework.
100
 
101
  **NIM-only election, no cross-provider cascade (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** If every NIM candidate in a chain fails, orchestrator returns a graceful error message rather than falling to Groq or OpenRouter — fail-loud is preferred over fail-silent-with-garbage for structured-output contracts. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the cross-provider-fallback variant of KI-080 ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)) are both superseded. KI-085's proactive credit gating still applies within the NIM pool via a per-model 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space secrets for future re-enable but the chain config no longer references them.
 
186
  **Step 2 — Ask the first question (voice or text)**
187
  - User just talks — VAD detects speech start/end automatically
188
  - *"Suggest a health insurance plan for me"*
189
+ - Bot recognises fact-find intent. **`backend/sales_brain.py`** issues one NIM call per turn (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) using `response_format={"type":"json_object"}`; the LLM owns voice + flow + slot order, returning a `{reply, captures, slot_driving, complete}` JSON body. A deterministic post-processor normalizes the captured fields against the 9-slot schema.
190
  - Slots in order: age → dependents → income → existing cover → primary goal → location → parents (conditional) → health → budget
191
 
192
  </td><td>
 
201
  - `detect_language(user_text)` → `english / indic`
202
  - Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
203
 
204
+ **Tech 4 — Fact-find branch (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))**
205
+ - If profile empty AND intent ∈ {fact_find, recommendation, comparison}: hand the turn to `backend/sales_brain.py::drive_sales_brain()` — one NIM call with `response_format={"type":"json_object"}`, system prompt carries the 9-slot schema + current profile state. Response: `{reply, captures, slot_driving, complete}`. `backend/sales_brain_normalizer.py` normalizes captures → `session.update_profile_field()` → `profile_store.save_profile()` + `profile_rag.upsert_profile_chunk()`. Emit `reply` to the user.
206
  - Else: continue to retrieval
207
 
208
  </td></tr>
 
254
  </td><td>
255
 
256
  **Tech 7 — Brain selection, `pick_brain(intent, language)`**
257
+ - `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool (KI-080 / KI-160)
258
+ - `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary in the NIM-only pool (typically Qwen 3-Next 80B in steady state); `fact_find` turns go through `backend/sales_brain.py` with `response_format={"type":"json_object"}` (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))
259
+ - KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → NIM-pool backup). Per-chain primary refreshed every 300s by background probe. On total NIM exhaustion: graceful error to user (fail-loud, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).
260
 
261
  **Tech 8 — System prompt construction, `build_messages()`**
262
  - `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
 
376
 
377
 
378
  ┌──────────────────────────────────────────────────────────────────────────┐
379
+ │ TECH 4. Fact-find OR free-form branch (KI-167 / ADR-039)
380
  │ ------- │
381
+ │ · If intent {fact_find, recommendation, comparison} AND profile
382
+ incomplete:
383
+ drive_sales_brain() one NIM call, response_format=json_object
384
+ {reply, captures, slot_driving, complete}
385
+ │ normalize captures → session.update_profile_field() │
386
+ │ → profile_store.save_profile() + profile_rag.upsert_profile_chunk() │
387
+ │ emit reply to user, RETURN early │
388
  │ · Else: set free_form_session = True, continue to retrieval │
389
  └──────────────────────────────────────────────────────────────────────────┘
390
 
 
415
  ┌──────────────────────────────────────────────────────────────────────────┐
416
  │ TECH 7. Brain selection — pick_brain(intent, language) │
417
  │ ------- │
418
+ │ · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary in NIM-only pool (heavy, KI-080 / KI-160 / ADR-038)│
419
+ │ · intent ∈ {qa, fact_find} → FAST_BRAIN_CHAIN; fact_find turns dispatched to sales_brain.py (KI-167 / ADR-039, JSON mode)
420
+ │ · NIM Qwen 80B + Nemotron-Super 49B as fast-brain election candidates (NIM-only post-KI-160)
421
  └──────────────────────────────────────────────────────────────────────────┘
422
 
423
 
 
578
  | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
579
  |---|---|---|---|---|
580
  | **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) — typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
581
+ | **Fast brain** (voice turns, sales-brain fact-find KI-167, QA, normalize, extract) | Probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool — typically Qwen 3-Next 80B in steady state | Qwen 3-Next 80B → NVIDIA Nemotron-Super 49B v1.5 (NIM-only election candidates) | NIM | Bottleneck on these short jobs is TTFT, not capability. Sales brain (KI-167) calls this chain with `response_format={"type":"json_object"}` for guaranteed structured output. Election adapts within the NIM pool when one candidate degrades. |
582
  | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
583
  | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
584
  | **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
585
+ | **Sales brain** (fact-find conversation driver — KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) | Fast brain chain (Qwen 3-Next 80B primary in NIM-only pool) | inherits FAST_BRAIN_CHAIN | NIM | One NIM call per turn using `response_format={"type":"json_object"}`. System prompt carries the 9-slot schema + current profile state; LLM owns voice + flow + slot order, returns `{reply, captures, slot_driving, complete}`. Replaces the scripted-`prompt_en` + paraphraser + opener-rotation + `<FF>` trailer + `_canonical_fallback` machinery (ADR-027 / ADR-030 retired). |
586
  | **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
587
  | **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
588
  | **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |