Spaces:
Sleeping
docs: KI-167/168/169/170/171/172 cascade — ADR-039 + CLAUDE + README
Browse filesADR-039 (NEW) — Replace scripted fact_find_brain with LLM-driven sales_brain.
Status: Accepted, 2026-05-15. Marks ADR-027 + ADR-030 superseded.
Context: 8 patches (KI-150/155/156/158/161) all worked around <FF>
trailer fragility + canonical_fallback firing as dominant path.
User symptom: "zero natural LLM chat — always defaults to script."
Decision: rip out fact_find_brain + replace with sales_brain.py using
NIM response_format=json_object + deterministic sales_brain_normalizer.
Profile persistence unchanged (profile_store + profile_rag).
CLAUDE.md — fact-find section rewritten. ADR-030/KI-070 framing replaced
with KI-167/ADR-039. Body now states one LLM call per turn, structured
JSON, no scripted prompts, no fallback prose. Points at sales_brain.py
+ response_format={"type":"json_object"} + sales_brain_normalizer.py.
KI-150 max_tokens watch-out replaced with "never reintroduce scripted
fallback" rule. Retired KIs updated. Last-reviewed footer updated.
README.md — recent changes list adds KI-160 (NIM-only) + KI-167 (sales
rip-out). Step 2 + Tech 4 + Tech 7 prose + ASCII boxes now reference
sales_brain.py + JSON mode. "Question paraphraser (deprecated)" row in
model table replaced with "Sales brain (fact-find conversation driver)".
NOTE: This commit DOES NOT include the KI-168/169/170/171/172 follow-up
KIs in the doc cascade — those landed AFTER WS5 generated this docs
package. A subsequent docs commit can fold those in if needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md +66 -0
- CLAUDE.md +18 -16
- README.md +23 -18
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ADR-039 — Replace scripted fact_find_brain with LLM-driven sales_brain (KI-167)
|
| 2 |
+
|
| 3 |
+
**Status:** Accepted — 2026-05-15
|
| 4 |
+
**Owner:** Rohit Saraf
|
| 5 |
+
**Supersedes:** [ADR-027](ADR-027-fact-find-llm-paraphraser.md) (LLM paraphraser on top of `GRAPH`) and [ADR-030](ADR-030-llm-driven-fact-find.md) (one-call brain with `<FF>` trailer + canonical fallback). ADR-030's "schema-aware single LLM call per turn" intuition is preserved; the surface that implemented it (`fact_find_brain.py` + `<FF>...</FF>` trailer + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that — {slot}."`) is fully removed.
|
| 6 |
+
**Related KIs:** KI-070 / KI-072 / KI-074 / KI-075 / KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161 (all retired by KI-167), KI-167 (this rip-out), KI-160 / [ADR-038](ADR-038-nim-only-chains.md) (NIM-only chain pool — the substrate this ADR builds on).
|
| 7 |
+
|
| 8 |
+
## Context
|
| 9 |
+
|
| 10 |
+
`backend/fact_find_brain.py` started life under ADR-030 as a single-LLM-call replacement for the ADR-027 paraphraser + hardcoded `GRAPH` stitch. It needed eight subsequent patches to stay alive:
|
| 11 |
+
|
| 12 |
+
- **KI-090** — lenient `<FF>` parser because real LLMs dropped the literal tags around their JSON tail.
|
| 13 |
+
- **KI-091 / KI-094** — guard against the extractor returning `{"name": null}` and overwriting captured fields.
|
| 14 |
+
- **KI-103** — `no_trailer` loop-breaker after the same slot failed twice.
|
| 15 |
+
- **KI-150** — `max_tokens` 420 → 700 because the prose+trailer prompt was truncating mid-paraphrase.
|
| 16 |
+
- **KI-155 / KI-156 / KI-158 / KI-161** — silent contract violations where the elected LLM returned grammatical prose with no parseable `<FF>` block. Every turn it happened, `_canonical_fallback` fired and the user saw the scripted `Question.prompt_en` of the next unfilled slot, prefixed with `"Got that — {slot}."` — i.e. the bot reverted to the very state machine ADR-030 was supposed to replace.
|
| 17 |
+
|
| 18 |
+
User-visible symptom by 2026-05-15: *"zero natural LLM chat — it always defaults to the script."* The `<FF>` trailer convention + `_canonical_fallback` were no longer a degraded path; they were the **dominant** path on multiple chain candidates, and the scripted prefix kept leaking into every fallback turn. The architecture had become reactive accretion — eight patches stacked on a fragile structured-output contract — not deliberate design. KI-160 / [ADR-038](ADR-038-nim-only-chains.md) closed the cross-provider silent-failure class at the chain level; KI-167 closes the structured-output silent-failure class at the prompt level.
|
| 19 |
+
|
| 20 |
+
## Decision
|
| 21 |
+
|
| 22 |
+
**Rip out `backend/fact_find_brain.py` entirely.** Replace with `backend/sales_brain.py` — a single LLM-driven sales agent that owns the entire fact-find conversation surface. Concretely:
|
| 23 |
+
|
| 24 |
+
- **One NIM call per turn**, via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-038](ADR-038-nim-only-chains.md) NIM-only pool. The call uses NVIDIA NIM's `response_format={"type": "json_object"}` for structured output — the same JSON-mode contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`.
|
| 25 |
+
- **System prompt** contains the 9-slot schema (slot id, description, accepted value shapes), current profile state (which slots are filled with which values), and an instruction to (a) reply naturally in the advisor's voice, (b) capture any new facts the user just stated, (c) decide which slot to drive next, (d) flag completion when all required slots are filled. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time — there is no scripted prompt to match, no opener to rotate, no acknowledger prefix.
|
| 26 |
+
- **JSON response shape:** `{"reply": "<prose for user>", "captures": {<slot_id>: <raw_value>, ...}, "slot_driving": "<slot_id or null>", "complete": <bool>}`. No `<FF>` tags, no trailer convention — `response_format` guarantees the whole response body parses as JSON.
|
| 27 |
+
- **Deterministic post-processor** (`backend/sales_brain_normalizer.py`, KI-167 WS1) takes the LLM's loose `captures` dict and emits a clean `{canonical_field: validated_value}` map: field-name alias resolution (`location` → `location_tier`), enum normalization (`Bangalore` → `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. The orchestrator applies the normalized map via `session.update_profile_field()` exactly as before.
|
| 28 |
+
- **Profile persistence is unchanged.** Captured fields still flow through `backend/profile_store.save_profile()` for disk durability and `backend/profile_rag.upsert_profile_chunk()` for retrieval-side visibility. The session-isolation guarantees from KI-102 / KI-107 / KI-112 remain in force — `sales_brain.py` only changes the *upstream* fact-find loop; the downstream persistence layer is untouched.
|
| 29 |
+
- **No scripted prompts.** `backend/needs_finder.py::Question.prompt_en` is no longer consulted by the fact-find branch. The `GRAPH` data structure stays in `needs_finder.py` for now (slot-id definitions are still useful as the schema source for the LLM's system prompt), but its `prompt_en` field is dead text.
|
| 30 |
+
- **No canonical_fallback.** `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, `"Got that — {slot}."` prefix logic, `_ff_failed_attempts` / `_ff_skipped_slots` session fields, and every `fact_find_brain::fallback:*` telemetry variant are deleted in WS2 / WS3.
|
| 31 |
+
- **No trailer convention.** The lenient parser ladder (`<FF>` → fenced `json` → bare-JSON-tail) is deleted. Structured output is guaranteed by the provider, not pattern-matched from prose.
|
| 32 |
+
|
| 33 |
+
The outer 25s `asyncio.wait_for` ceiling around the brain call is retained. On total NIM exhaustion, the orchestrator returns the [ADR-038](ADR-038-nim-only-chains.md) graceful error message to the user (fail-loud) rather than cascading into a scripted reply.
|
| 34 |
+
|
| 35 |
+
## Consequences
|
| 36 |
+
|
| 37 |
+
### Positive
|
| 38 |
+
|
| 39 |
+
- **Bot feels human.** No more scripted `"Got that — {slot}."` leakage, no more identical `prompt_en` strings across sessions, no more two-line opener+question cadence. The LLM owns voice + flow end-to-end.
|
| 40 |
+
- **One fewer LLM call per turn.** The pre-KI-167 fact-find turn ran the brain + (on KI-091 QA turns) the profile extractor. Captures are now in the brain's structured response, so the extractor is dead code for fact-find turns and stays skipped on QA turns per KI-091.
|
| 41 |
+
- **Structured-output guarantees parseability.** NIM's `response_format={"type":"json_object"}` is enforced server-side; the response body either parses as JSON or the call fails loud. Eliminates the entire class of "model returned grammatical prose with no parseable trailer" silent failures (KI-155 / KI-156 / KI-158 / KI-161).
|
| 42 |
+
- **Eliminates 8 KIs of patching.** KI-090 (lenient parser), KI-091 / KI-094 (extractor null-overwrite guards on fact-find turns), KI-103 (no_trailer loop-breaker), KI-150 (`max_tokens` 420→700), KI-155 / KI-156 / KI-158 / KI-161 (trailer contract violations) all become moot — the root causes are gone, not patched.
|
| 43 |
+
- **Smaller surface area.** `backend/fact_find_brain.py` (441 LOC), the `_canonical_fallback` branch in `backend/orchestrator.py`, the `_pick_opener` family in `backend/orchestrator.py`, and the dead `Question.prompt_en` strings in `backend/needs_finder.py` all go away. Net negative LOC.
|
| 44 |
+
|
| 45 |
+
### Negative
|
| 46 |
+
|
| 47 |
+
- **Persona-audit fixtures need updating.** The persona transcripts in `eval/` and the `tests/test_persona_*` fixtures hardcode the expected turn order from the rules engine (age → dependents → income → ...). The LLM may capture in a different order (or multi-fact in one turn) depending on what the user said. WS4 owns rewriting these fixtures to assert on **final captured state** rather than per-turn slot order.
|
| 48 |
+
- **Higher LLM dependency.** There is no scripted safety net underneath the sales brain. If every NIM candidate in `FAST_BRAIN_CHAIN` fails simultaneously, the user sees the graceful error message from [ADR-038](ADR-038-nim-only-chains.md) — not a scripted fact-find prompt.
|
| 49 |
+
- **Loss of explicit completion path.** Pre-KI-167, `complete=True` was enforced by the orchestrator checking all 9 slots filled. Post-KI-167 the LLM is responsible for setting `complete: true`. The deterministic post-processor double-checks (`complete && any required slot still empty` ⇒ override to `false`), but the surface-of-trust shifts from rules engine to LLM.
|
| 50 |
+
|
| 51 |
+
### Mitigations
|
| 52 |
+
|
| 53 |
+
- **NIM JSON mode is production-validated.** `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py` already run `response_format={"type":"json_object"}` against the same NIM candidate pool. Failure mode is well-understood and observable.
|
| 54 |
+
- **Outer 25s `asyncio.wait_for` ceiling retained.** On total NIM exhaustion the failure surfaces as a clear error message (per [ADR-038](ADR-038-nim-only-chains.md)), not as a hang or as a regression to scripted text.
|
| 55 |
+
- **Deterministic post-processor enforces required-slot completeness.** The LLM cannot prematurely mark `complete: true` while a required slot is empty — `sales_brain_normalizer.py` overrides.
|
| 56 |
+
- **Persona audit refactor scoped to WS4.** Per-turn brittleness is a known cost; fixture rewrite targets final-state assertions which are stable regardless of which order the brain captures slots in.
|
| 57 |
+
|
| 58 |
+
## Alternatives considered
|
| 59 |
+
|
| 60 |
+
- **Keep `fact_find_brain.py`, harden the `<FF>` parser further.** Rejected. KI-155 / KI-156 / KI-158 / KI-161 demonstrated that the failure class is structural to a prose-plus-trailer prompt shape under load — not a parser bug. Each patch shifted the failure to a new turn / candidate / token cap. The directive *"Never use trailer conventions when the provider offers structured output"* (memory: KI-160 / [ADR-038](ADR-038-nim-only-chains.md)) generalizes to the prompt level: don't pattern-match structure out of prose when the provider can guarantee it server-side.
|
| 61 |
+
- **Keep `_canonical_fallback` as a last-resort path.** Rejected. The whole point of the rip-out is that the fallback was the dominant path, and the scripted `Question.prompt_en` prefixed with `"Got that — {slot}."` was exactly the user-visible artefact KI-167 is aimed at. Leaving the fallback in would keep the regression surface alive.
|
| 62 |
+
- **Hybrid: scripted slots 1-3, LLM brain for slots 4-9.** Rejected. The robotic feel of "what's your age? what's your city? who are your dependents?" hits in the *first three turns* — splitting the loop preserves the worst part of the old system.
|
| 63 |
+
|
| 64 |
+
## Reversal trigger
|
| 65 |
+
|
| 66 |
+
Re-open this ADR if (a) NIM removes `response_format` support, or (b) a 7-day window shows the deterministic post-processor's required-slot-completeness override firing > 5% of completion turns (signalling the LLM systematically lies about completion), or (c) persona audit pass-rate drops below 80% after the WS4 fixture rewrite and the cause is brain-side rather than fixture-side. The reversal would still NOT restore the `<FF>` trailer convention — it would either tighten the JSON-mode prompt or shift to a different provider's structured-output API.
|
|
@@ -31,15 +31,15 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
|
|
| 31 |
- **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
|
| 32 |
- **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
|
| 33 |
- **Brain / fast-brain / judge primaries in steady state** are **NIM Nemotron-Super 49B** (brain), **NIM Qwen 3-Next 80B** (fast brain), and **NIM Llama-4 Maverick 17B/128E** (judge), per the KI-160 candidate lists. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` within the NIM pool.
|
| 34 |
-
- **KI-079 escalation as last bite (`87ee522`).** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget)
|
| 35 |
- **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
|
| 36 |
-
- **Lenient FF-block parser (KI-090, `11cf4b3`)
|
| 37 |
-
- **Skip `profile_extractor` + faithfulness judge on fact-find turns (KI-091, `9813994`).** Both chains were credit-exhausted on the steady-state primary, hung fact-find turns for 20+s, and the extractor periodically returned `{"name": null}` which wrote into `session.update_profile_field` and wiped the captured name mid-session
|
| 38 |
- **Defensive `None`-guard in extractor merge (KI-094, `f068094`).** Belt-and-braces companion to KI-091. On QA-mode turns the extractor still runs (correct: "I'm now 35" mid-recommendation should update the profile), but under load it periodically returns `{"name": null, "age": null, ...}` and the merge loop was writing every key including nulls back into the session. Added `if new_value in (None, "", []): continue` at the top of the extracted-fields loop in `backend/orchestrator.py` — null / empty-string / empty-list returns are now no-ops. Closes the same root cause (LLM-returned nulls wiping state) at a second layer; KI-091 prevents the extractor from running on fact-find turns at all, KI-094 makes it safe even when it does run.
|
| 39 |
- **Remove IP allowlist from admin — password-only gate (KI-097, pending).** Dropped `ADMIN_IP_ALLOWLIST` env + `_ip_allowed()` from `backend/admin.py`; `_check_admin` is now password-only against `X-Admin-Password` → `ADMIN_PASSWORD` env. Backend returns 401 Unauthorized (previously 404-to-hide). Frontend admin panel is always visible; password unlocks the live data. ADR-023 superseded — IP gating added zero security beyond a strong password and locked the operator out whenever the home IP changed.
|
| 40 |
- **Drop function-local `import logging` in orchestrator (KI-101, `66eb4ed`).** Removed 6 inline `import logging` lines from `backend/orchestrator.py`; Python's scoping rule was promoting `logging` to function-local, causing `UnboundLocalError` when the `asyncio.wait_for` `TimeoutError` branch fired before reaching the inline import. Module-level import is the sole binding now.
|
| 41 |
- **Profile RAG session isolation (KI-102, `4bb8da0`).** `upsert_profile_chunk` stamps `session_id` metadata on every chunk; `retrieve()` excludes `doc_type == "profile"` from the main pass; per-session profile lookup triple-checks `meta.session_id == session_id` in Python after the Chroma where-clause. Legacy chunks without `session_id` are silently refused (fail-closed). Cross-session PII leak (age / dependents / health conditions) closed. ADR-022 extended with session-isolation subsection.
|
| 42 |
-
- **`_canonical_fallback` no_trailer loop-breaker (KI-103, `8ef5c43`).**
|
| 43 |
- **CoT / instruction-echo strip in voice_format (KI-104, `407f2a1`).** `tts_preprocess` now kills `<think>...</think>` blocks, `**Reasoning:**` / `**Thought:**` labels, `[INTERNAL]` blocks, sentence-anchored CoT starters ("Let me think...", "Step 1:..."). Emergency fallback to a generic acknowledger if the whole reply is CoT-shaped. Stops Sarvam from TTS-ing the bot's internal monologue.
|
| 44 |
- **Recommendation closer wired (KI-105, `8a58fa1`).** `RECOMMENDATION_CLOSER_PHRASES` frozenset ("show me the top 3", "rank", "pitch me", "compare X vs Y") classified as `recommendation` / `comparison` BEFORE the `FACT_FIND_TRIGGERS` check, so a fully-fact-found user can never get bounced back into fact-find. Persona prompt gets `RECOMMENDATION_CLOSER_ADDENDUM` with a strict 3-policy ranked-shortlist contract (3 policies, one-line rationale each, IRDAI disclaimer, no hedging). ADR-008 extended with closer-mode subsection.
|
| 45 |
- **Graceful TimeoutError + Exception on `/api/chat` (KI-106, `565bf31`).** `handle_turn(...)` wrapped in `asyncio.wait_for(45s)` with explicit `except asyncio.TimeoutError` + broad `except Exception`. Both return HTTP 200 with `source="graceful_timeout"` / `graceful_exception"` and an in-character recovery sentence instead of HTTP 500. Internal `logger.exception` still captures the full traceback for admin observability.
|
|
@@ -49,18 +49,20 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
|
|
| 49 |
- **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
|
| 50 |
- **Provider keys.** `NVIDIA_NIM_API_KEY` + `GROQ_API_KEY` + `OPENROUTER_API_KEY` required in `.env` (local) and HF Space environment (production — KI-081).
|
| 51 |
|
| 52 |
-
## Fact-find loop (ADR-
|
| 53 |
|
| 54 |
-
**One LLM call per turn
|
| 55 |
|
| 56 |
-
- **
|
| 57 |
-
- **
|
| 58 |
-
- **`backend/
|
| 59 |
-
- **
|
| 60 |
-
- **
|
| 61 |
-
- **
|
| 62 |
-
- **
|
| 63 |
-
- **
|
|
|
|
|
|
|
| 64 |
|
| 65 |
## Refusal precision (KI-046)
|
| 66 |
|
|
@@ -131,8 +133,8 @@ Three independent safety layers against ChromaDB HNSW bloat:
|
|
| 131 |
- **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
|
| 132 |
- **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
|
| 133 |
- **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
|
| 134 |
-
- **`
|
| 135 |
|
| 136 |
---
|
| 137 |
|
| 138 |
-
*Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General
|
|
|
|
| 31 |
- **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
|
| 32 |
- **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
|
| 33 |
- **Brain / fast-brain / judge primaries in steady state** are **NIM Nemotron-Super 49B** (brain), **NIM Qwen 3-Next 80B** (fast brain), and **NIM Llama-4 Maverick 17B/128E** (judge), per the KI-160 candidate lists. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` within the NIM pool.
|
| 34 |
+
- **KI-079 escalation as last bite (`87ee522`) — modified by KI-167.** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget). Post-KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) the `_canonical_fallback` terminal step is gone — on heavy-brain exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message instead. Worst-case wall-clock before graceful error: 25s FAST + 15s heavy = 40s.
|
| 35 |
- **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
|
| 36 |
+
- **Lenient FF-block parser (KI-090, `11cf4b3`) — RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** Was needed because the `<FF>...</FF>` trailer convention was fragile under load. Post-KI-167 the sales brain uses NIM `response_format={"type":"json_object"}` for guaranteed structured output, so the whole `_parse_ff_block` strict → fenced → bare-JSON ladder is gone.
|
| 37 |
+
- **Skip `profile_extractor` + faithfulness judge on fact-find turns (KI-091, `9813994`).** Both chains were credit-exhausted on the steady-state primary, hung fact-find turns for 20+s, and the extractor periodically returned `{"name": null}` which wrote into `session.update_profile_field` and wiped the captured name mid-session. Orchestrator short-circuits both chains behind `intent == "fact_find"`; the sales brain (KI-167) captures fields natively from its JSON-mode response body, and faithfulness scoring is meaningless on "what's your annual income?". QA-mode turns still run both chains. Live: name re-ask loop gone; fact-find p95 28s → 6-8s.
|
| 38 |
- **Defensive `None`-guard in extractor merge (KI-094, `f068094`).** Belt-and-braces companion to KI-091. On QA-mode turns the extractor still runs (correct: "I'm now 35" mid-recommendation should update the profile), but under load it periodically returns `{"name": null, "age": null, ...}` and the merge loop was writing every key including nulls back into the session. Added `if new_value in (None, "", []): continue` at the top of the extracted-fields loop in `backend/orchestrator.py` — null / empty-string / empty-list returns are now no-ops. Closes the same root cause (LLM-returned nulls wiping state) at a second layer; KI-091 prevents the extractor from running on fact-find turns at all, KI-094 makes it safe even when it does run.
|
| 39 |
- **Remove IP allowlist from admin — password-only gate (KI-097, pending).** Dropped `ADMIN_IP_ALLOWLIST` env + `_ip_allowed()` from `backend/admin.py`; `_check_admin` is now password-only against `X-Admin-Password` → `ADMIN_PASSWORD` env. Backend returns 401 Unauthorized (previously 404-to-hide). Frontend admin panel is always visible; password unlocks the live data. ADR-023 superseded — IP gating added zero security beyond a strong password and locked the operator out whenever the home IP changed.
|
| 40 |
- **Drop function-local `import logging` in orchestrator (KI-101, `66eb4ed`).** Removed 6 inline `import logging` lines from `backend/orchestrator.py`; Python's scoping rule was promoting `logging` to function-local, causing `UnboundLocalError` when the `asyncio.wait_for` `TimeoutError` branch fired before reaching the inline import. Module-level import is the sole binding now.
|
| 41 |
- **Profile RAG session isolation (KI-102, `4bb8da0`).** `upsert_profile_chunk` stamps `session_id` metadata on every chunk; `retrieve()` excludes `doc_type == "profile"` from the main pass; per-session profile lookup triple-checks `meta.session_id == session_id` in Python after the Chroma where-clause. Legacy chunks without `session_id` are silently refused (fail-closed). Cross-session PII leak (age / dependents / health conditions) closed. ADR-022 extended with session-isolation subsection.
|
| 42 |
+
- **`_canonical_fallback` no_trailer loop-breaker (KI-103, `8ef5c43`) — RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** The `_ff_failed_attempts` + `_ff_skipped_slots` state and the entire `_canonical_fallback` branch are deleted; the failure mode they capped (silent `<FF>` trailer parse failures looping on the same slot) is gone because the sales brain uses NIM JSON mode.
|
| 43 |
- **CoT / instruction-echo strip in voice_format (KI-104, `407f2a1`).** `tts_preprocess` now kills `<think>...</think>` blocks, `**Reasoning:**` / `**Thought:**` labels, `[INTERNAL]` blocks, sentence-anchored CoT starters ("Let me think...", "Step 1:..."). Emergency fallback to a generic acknowledger if the whole reply is CoT-shaped. Stops Sarvam from TTS-ing the bot's internal monologue.
|
| 44 |
- **Recommendation closer wired (KI-105, `8a58fa1`).** `RECOMMENDATION_CLOSER_PHRASES` frozenset ("show me the top 3", "rank", "pitch me", "compare X vs Y") classified as `recommendation` / `comparison` BEFORE the `FACT_FIND_TRIGGERS` check, so a fully-fact-found user can never get bounced back into fact-find. Persona prompt gets `RECOMMENDATION_CLOSER_ADDENDUM` with a strict 3-policy ranked-shortlist contract (3 policies, one-line rationale each, IRDAI disclaimer, no hedging). ADR-008 extended with closer-mode subsection.
|
| 45 |
- **Graceful TimeoutError + Exception on `/api/chat` (KI-106, `565bf31`).** `handle_turn(...)` wrapped in `asyncio.wait_for(45s)` with explicit `except asyncio.TimeoutError` + broad `except Exception`. Both return HTTP 200 with `source="graceful_timeout"` / `graceful_exception"` and an in-character recovery sentence instead of HTTP 500. Internal `logger.exception` still captures the full traceback for admin observability.
|
|
|
|
| 49 |
- **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
|
| 50 |
- **Provider keys.** `NVIDIA_NIM_API_KEY` + `GROQ_API_KEY` + `OPENROUTER_API_KEY` required in `.env` (local) and HF Space environment (production — KI-081).
|
| 51 |
|
| 52 |
+
## Fact-find loop (ADR-039, supersedes ADR-030 / ADR-027) — KI-167
|
| 53 |
|
| 54 |
+
**One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that — {slot}."`) and replaced it with `backend/sales_brain.py` — a single LLM-driven sales agent. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-155 and follow-ups showed the canonical fallback was the *dominant* path, not the safety net. User-visible symptom: "zero natural LLM chat — it always defaults to the script."
|
| 55 |
|
| 56 |
+
- **Single NIM call per turn** via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool, using NVIDIA NIM's **`response_format={"type": "json_object"}`** for structured output — the same JSON-mode contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`. Response shape: `{"reply": "<prose for user>", "captures": {<slot_id>: <raw>}, "slot_driving": "<slot_id|null>", "complete": <bool>}`.
|
| 57 |
+
- **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
|
| 58 |
+
- **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` → `location_tier`), enum normalization (`Bangalore` → `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
|
| 59 |
+
- **Profile persistence unchanged.** Captures flow through `session.update_profile_field()` → `backend/profile_store.save_profile()` (disk durability) → `backend/profile_rag.upsert_profile_chunk()` (retrieval-side visibility). KI-102 / KI-107 / KI-112 session-isolation guards remain in force.
|
| 60 |
+
- **No scripted prompts, no canonical_fallback, no trailer convention.** `backend/needs_finder.py::GRAPH` slot-id data stays as a schema source for the system prompt, but `Question.prompt_en` is dead text — never consulted by the fact-find branch. `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, the lenient `<FF>` parser ladder, and the `"Got that — {slot}."` prefix logic are deleted.
|
| 61 |
+
- **Outer 25s `asyncio.wait_for` ceiling retained.** On total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error to the user (fail-loud) rather than cascading to a scripted reply. There is no scripted safety net — that is intentional.
|
| 62 |
+
- **DELETED in KI-167:** `backend/fact_find_brain.py` (441 LOC); `_canonical_fallback` + `_pick_opener` branches in `backend/orchestrator.py`; `_ff_failed_attempts` / `_ff_skipped_slots` session fields; every `fact_find_brain::fallback:*` telemetry variant; the lenient `<FF>` / fenced-`json` / bare-JSON-tail parser ladder.
|
| 63 |
+
- **Persona-audit fixtures rewritten** to assert on final captured state (which slots are filled with what) rather than per-turn slot order — the LLM may capture in different turn order than the pre-KI-167 rules engine did. See WS4.
|
| 64 |
+
- **Natural-conversation escape (KI-045):** intent_change phrases / off-topic questions still exit fact-find by routing through `should_route_to_fact_find` upstream of the brain.
|
| 65 |
+
- **Indic queries** route through Sarvam-M for translation on input + output; the sales brain runs in English on the translated text.
|
| 66 |
|
| 67 |
## Refusal precision (KI-046)
|
| 68 |
|
|
|
|
| 133 |
- **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
|
| 134 |
- **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
|
| 135 |
- **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
|
| 136 |
+
- **`sales_brain.py` is the fact-find handler (KI-167, [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** `backend/fact_find_brain.py` is deleted along with the `<FF>` trailer convention, `_canonical_fallback`, scripted `Question.prompt_en`, and the `"Got that — {slot}."` prefix. Never re-introduce a fallback to scripted prompts on the fact-find branch — on total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message (fail-loud > fail-silent-with-script). The KI-150 historical note (`fact_find_brain` `max_tokens` 420 → 700) is moot post-KI-167 since the prose+trailer prompt shape that needed the larger budget no longer exists.
|
| 137 |
|
| 138 |
---
|
| 139 |
|
| 140 |
+
*Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General). Same day: KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) locked chains to NIM-only after KI-155's `<FF>` trailer contract violation. Same day: KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) replaced `fact_find_brain.py` (scripted slot-walker + `<FF>` trailer + `_canonical_fallback` + scripted `Question.prompt_en`) with `backend/sales_brain.py` (single LLM call per turn, NIM `response_format=json_object`, deterministic post-processor).*
|
|
@@ -62,7 +62,9 @@ Thirty-plus knowledge increments landed today. The corpus was rebuilt, the marke
|
|
| 62 |
- **KI-143** — `bajaj/group-health-guard` slug correction (gold → silver per the PDF).
|
| 63 |
- **KI-144** — `reliance-general` → `indusind-general` migration (Reliance General Insurance was rebranded to IndusInd General). The `indusind-general` slug did not exist anywhere in the codebase before today.
|
| 64 |
- **KI-149** — budget + income parser captures bare numerals like `"30000"`.
|
| 65 |
-
- **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (
|
|
|
|
|
|
|
| 66 |
|
| 67 |
Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
|
| 68 |
|
|
@@ -93,7 +95,7 @@ A take-home is a sample of how the engineer thinks under constraint. Three thing
|
|
| 93 |
3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a NIM-only candidate pool per role (KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)), not a single hardcoded brain. Each role is a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) whose candidates are all NIM-hosted — KI-155 demonstrated that Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract, so cross-provider fallback was removed as a silent-failure trap:
|
| 94 |
|
| 95 |
- **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`), backup **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE), 3rd candidate **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`).
|
| 96 |
-
- **`FAST_BRAIN_CHAIN`** (fact-find turns, QA,
|
| 97 |
- **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Meta Llama-4 Maverick 17B/128E** (`meta/llama-4-maverick-17b-128e-instruct`), backup **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`). Deliberately different model families from the brain pool so the judge does not mark its own homework.
|
| 98 |
|
| 99 |
**NIM-only election, no cross-provider cascade (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** If every NIM candidate in a chain fails, orchestrator returns a graceful error message rather than falling to Groq or OpenRouter — fail-loud is preferred over fail-silent-with-garbage for structured-output contracts. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the cross-provider-fallback variant of KI-080 ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)) are both superseded. KI-085's proactive credit gating still applies within the NIM pool via a per-model 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space secrets for future re-enable but the chain config no longer references them.
|
|
@@ -184,7 +186,7 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 184 |
**Step 2 — Ask the first question (voice or text)**
|
| 185 |
- User just talks — VAD detects speech start/end automatically
|
| 186 |
- *"Suggest a health insurance plan for me"*
|
| 187 |
-
- Bot recognises fact-find intent.
|
| 188 |
- Slots in order: age → dependents → income → existing cover → primary goal → location → parents (conditional) → health → budget
|
| 189 |
|
| 190 |
</td><td>
|
|
@@ -199,8 +201,8 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 199 |
- `detect_language(user_text)` → `english / indic`
|
| 200 |
- Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
|
| 201 |
|
| 202 |
-
**Tech 4 — Fact-find branch**
|
| 203 |
-
- If profile empty AND intent ∈ {fact_find, recommendation, comparison}:
|
| 204 |
- Else: continue to retrieval
|
| 205 |
|
| 206 |
</td></tr>
|
|
@@ -252,9 +254,9 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 252 |
</td><td>
|
| 253 |
|
| 254 |
**Tech 7 — Brain selection, `pick_brain(intent, language)`**
|
| 255 |
-
- `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary
|
| 256 |
-
- `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary
|
| 257 |
-
- KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time →
|
| 258 |
|
| 259 |
**Tech 8 — System prompt construction, `build_messages()`**
|
| 260 |
- `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
|
|
@@ -374,12 +376,15 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 374 |
│
|
| 375 |
▼
|
| 376 |
┌──────────────────────────────────────────────────────────────────────────┐
|
| 377 |
-
│ TECH 4. Fact-find OR free-form branch
|
| 378 |
│ ------- │
|
| 379 |
-
│ · If
|
| 380 |
-
│
|
| 381 |
-
│
|
| 382 |
-
│
|
|
|
|
|
|
|
|
|
|
| 383 |
│ · Else: set free_form_session = True, continue to retrieval │
|
| 384 |
└──────────────────────────────────────────────────────────────────────────┘
|
| 385 |
│
|
|
@@ -410,9 +415,9 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 410 |
┌──────────────────────────────────────────────────────────────────────────┐
|
| 411 |
│ TECH 7. Brain selection — pick_brain(intent, language) │
|
| 412 |
│ ------- │
|
| 413 |
-
│ · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary
|
| 414 |
-
│ · intent ∈ {qa, fact_find} → FAST_BRAIN_CHAIN
|
| 415 |
-
│ · NIM Qwen 80B
|
| 416 |
└──────────────────────────────────────────────────────────────────────────┘
|
| 417 |
│
|
| 418 |
▼
|
|
@@ -573,11 +578,11 @@ Every LLM role is served by a **candidate pool** of models elected over by a pro
|
|
| 573 |
| Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
|
| 574 |
|---|---|---|---|---|
|
| 575 |
| **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) — typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
|
| 576 |
-
| **Fast brain** (voice turns, fact-find, QA,
|
| 577 |
| **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
|
| 578 |
| **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
|
| 579 |
| **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
|
| 580 |
-
| **
|
| 581 |
| **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
|
| 582 |
| **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
|
| 583 |
| **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
|
|
|
|
| 62 |
- **KI-143** — `bajaj/group-health-guard` slug correction (gold → silver per the PDF).
|
| 63 |
- **KI-144** — `reliance-general` → `indusind-general` migration (Reliance General Insurance was rebranded to IndusInd General). The `indusind-general` slug did not exist anywhere in the codebase before today.
|
| 64 |
- **KI-149** — budget + income parser captures bare numerals like `"30000"`.
|
| 65 |
+
- **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (one of eight band-aids on the prose+`<FF>`-trailer prompt shape; superseded by KI-167).
|
| 66 |
+
- **KI-160** — chains locked to NIM-only ([ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)) after KI-155 demonstrated Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract.
|
| 67 |
+
- **KI-167** — `backend/fact_find_brain.py` ripped out and replaced by `backend/sales_brain.py` ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)). Single LLM call per turn using NIM `response_format={"type":"json_object"}`; no scripted `Question.prompt_en`, no `<FF>` trailer, no `_canonical_fallback`, no `"Got that — {slot}."` prefix. Deterministic post-processor (`backend/sales_brain_normalizer.py`) normalizes captures; profile persists via `profile_store` + `profile_rag` as before.
|
| 68 |
|
| 69 |
Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
|
| 70 |
|
|
|
|
| 95 |
3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a NIM-only candidate pool per role (KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)), not a single hardcoded brain. Each role is a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) whose candidates are all NIM-hosted — KI-155 demonstrated that Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract, so cross-provider fallback was removed as a silent-failure trap:
|
| 96 |
|
| 97 |
- **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`), backup **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE), 3rd candidate **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`).
|
| 98 |
+
- **`FAST_BRAIN_CHAIN`** (sales-brain fact-find turns per KI-167, QA, normalize, extract — every latency-sensitive role) — primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`), backup **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`).
|
| 99 |
- **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Meta Llama-4 Maverick 17B/128E** (`meta/llama-4-maverick-17b-128e-instruct`), backup **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`). Deliberately different model families from the brain pool so the judge does not mark its own homework.
|
| 100 |
|
| 101 |
**NIM-only election, no cross-provider cascade (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** If every NIM candidate in a chain fails, orchestrator returns a graceful error message rather than falling to Groq or OpenRouter — fail-loud is preferred over fail-silent-with-garbage for structured-output contracts. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the cross-provider-fallback variant of KI-080 ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)) are both superseded. KI-085's proactive credit gating still applies within the NIM pool via a per-model 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space secrets for future re-enable but the chain config no longer references them.
|
|
|
|
| 186 |
**Step 2 — Ask the first question (voice or text)**
|
| 187 |
- User just talks — VAD detects speech start/end automatically
|
| 188 |
- *"Suggest a health insurance plan for me"*
|
| 189 |
+
- Bot recognises fact-find intent. **`backend/sales_brain.py`** issues one NIM call per turn (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) using `response_format={"type":"json_object"}`; the LLM owns voice + flow + slot order, returning a `{reply, captures, slot_driving, complete}` JSON body. A deterministic post-processor normalizes the captured fields against the 9-slot schema.
|
| 190 |
- Slots in order: age → dependents → income → existing cover → primary goal → location → parents (conditional) → health → budget
|
| 191 |
|
| 192 |
</td><td>
|
|
|
|
| 201 |
- `detect_language(user_text)` → `english / indic`
|
| 202 |
- Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
|
| 203 |
|
| 204 |
+
**Tech 4 — Fact-find branch (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))**
|
| 205 |
+
- If profile empty AND intent ∈ {fact_find, recommendation, comparison}: hand the turn to `backend/sales_brain.py::drive_sales_brain()` — one NIM call with `response_format={"type":"json_object"}`, system prompt carries the 9-slot schema + current profile state. Response: `{reply, captures, slot_driving, complete}`. `backend/sales_brain_normalizer.py` normalizes captures → `session.update_profile_field()` → `profile_store.save_profile()` + `profile_rag.upsert_profile_chunk()`. Emit `reply` to the user.
|
| 206 |
- Else: continue to retrieval
|
| 207 |
|
| 208 |
</td></tr>
|
|
|
|
| 254 |
</td><td>
|
| 255 |
|
| 256 |
**Tech 7 — Brain selection, `pick_brain(intent, language)`**
|
| 257 |
+
- `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool (KI-080 / KI-160)
|
| 258 |
+
- `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary in the NIM-only pool (typically Qwen 3-Next 80B in steady state); `fact_find` turns go through `backend/sales_brain.py` with `response_format={"type":"json_object"}` (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))
|
| 259 |
+
- KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → NIM-pool backup). Per-chain primary refreshed every 300s by background probe. On total NIM exhaustion: graceful error to user (fail-loud, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).
|
| 260 |
|
| 261 |
**Tech 8 — System prompt construction, `build_messages()`**
|
| 262 |
- `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
|
|
|
|
| 376 |
│
|
| 377 |
▼
|
| 378 |
┌──────────────────────────────────────────────────────────────────────────┐
|
| 379 |
+
│ TECH 4. Fact-find OR free-form branch (KI-167 / ADR-039) │
|
| 380 |
│ ------- │
|
| 381 |
+
│ · If intent ∈ {fact_find, recommendation, comparison} AND profile │
|
| 382 |
+
│ incomplete: │
|
| 383 |
+
│ drive_sales_brain() — one NIM call, response_format=json_object │
|
| 384 |
+
│ → {reply, captures, slot_driving, complete} │
|
| 385 |
+
│ normalize captures → session.update_profile_field() │
|
| 386 |
+
│ → profile_store.save_profile() + profile_rag.upsert_profile_chunk() │
|
| 387 |
+
│ emit reply to user, RETURN early │
|
| 388 |
│ · Else: set free_form_session = True, continue to retrieval │
|
| 389 |
└──────────────────────────────────────────────────────────────────────────┘
|
| 390 |
│
|
|
|
|
| 415 |
┌──────────────────────────────────────────────────────────────────────────┐
|
| 416 |
│ TECH 7. Brain selection — pick_brain(intent, language) │
|
| 417 |
│ ------- │
|
| 418 |
+
│ · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary in NIM-only pool (heavy, KI-080 / KI-160 / ADR-038)│
|
| 419 |
+
│ · intent ∈ {qa, fact_find} → FAST_BRAIN_CHAIN; fact_find turns dispatched to sales_brain.py (KI-167 / ADR-039, JSON mode) │
|
| 420 |
+
│ · NIM Qwen 80B + Nemotron-Super 49B as fast-brain election candidates (NIM-only post-KI-160) │
|
| 421 |
└──────────────────────────────────────────────────────────────────────────┘
|
| 422 |
│
|
| 423 |
▼
|
|
|
|
| 578 |
| Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
|
| 579 |
|---|---|---|---|---|
|
| 580 |
| **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) — typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
|
| 581 |
+
| **Fast brain** (voice turns, sales-brain fact-find KI-167, QA, normalize, extract) | Probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool — typically Qwen 3-Next 80B in steady state | Qwen 3-Next 80B → NVIDIA Nemotron-Super 49B v1.5 (NIM-only election candidates) | NIM | Bottleneck on these short jobs is TTFT, not capability. Sales brain (KI-167) calls this chain with `response_format={"type":"json_object"}` for guaranteed structured output. Election adapts within the NIM pool when one candidate degrades. |
|
| 582 |
| **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
|
| 583 |
| **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
|
| 584 |
| **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
|
| 585 |
+
| **Sales brain** (fact-find conversation driver — KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) | Fast brain chain (Qwen 3-Next 80B primary in NIM-only pool) | inherits FAST_BRAIN_CHAIN | NIM | One NIM call per turn using `response_format={"type":"json_object"}`. System prompt carries the 9-slot schema + current profile state; LLM owns voice + flow + slot order, returns `{reply, captures, slot_driving, complete}`. Replaces the scripted-`prompt_en` + paraphraser + opener-rotation + `<FF>` trailer + `_canonical_fallback` machinery (ADR-027 / ADR-030 retired). |
|
| 586 |
| **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
|
| 587 |
| **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
|
| 588 |
| **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
|