rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
132a829
·
1 Parent(s): dc57a46

feat(llm+docs): KI-177 + KI-179 + KI-183 + ADR-040 docs cascade

Browse files

THREE-TIER LLM CHAIN — Google Gemini PRIMARY, NIM Tier 1, OR Tier 2.

Live audit (KI-178) of OpenRouter's free-tier catalog killed my earlier
recommendation: gemini-2.0-flash isn't in OR's catalog, llama-3.3-70b-
:free and hermes-3-405b:free don't expose response_format. So OR's
free-with-JSON pool is only nemotron-3-super-120b / qwen3-next-80b /
gemma-4-31b. User obtained a Google AI Studio key, set up billing
($0 historical spend, $10 cap, free 1500/day quota active), confirmed
working live on gemini-2.5-flash + gemini-2.5-flash-lite.

NEW MODEL TIERS:

Brain Fast (sales_brain):
Tier 0 Google gemini-2.5-flash-lite (KI-183 — 2.0-flash retired)
Tier 1 NIM qwen3-next-80b → mistral-large-3 → llama-4-maverick
Tier 2 OR nemotron-3-super-120b:free → qwen3-next-80b:free → gemma-4-31b:free
Last NIM nemotron-super-49b (only when all else dies)

Brain Main (orchestrator):
Tier 0 Google gemini-2.5-flash
Tier 1+ same NIM/OR cascade

Judge (faithfulness):
NIM-only at top mistral-large-3 → llama-4-maverick → OR qwen-free → nemotron-49b
(cross-family from Gemini brain preserves grading invariant)

CODE (KI-179 + KI-177 + KI-183):
+ backend/providers/google_gemini_llm.py (NEW) — direct REST,
native responseMimeType=application/json, factory get_gemini_llm
+ backend/providers/tiered_brain_llm.py (NEW) — wrapper
walking Gemini → NIM → OR for orchestrator brain path
M backend/sales_brain.py — 3-tier cascade; brain_used now
sales_brain::{gemini,nim,or}:<model>; KI-169 think-strip retry stays
in-tier; KI-183 model defaults updated for retired gemini-2.0-flash
M backend/orchestrator.py — pick_brain returns get_tiered_brain_llm(role)
M backend/providers/openrouter_llm.py — supports models=[...] param
M frontend/public/admin/llm-control.html — KI-177 clearChildren null guard
M tools/set_hf_secrets.py — GOOGLE_API_KEY in SECRETS_TO_SET

DOCS CASCADE (KI-180 + KI-182):
+ 70-docs/60-decisions/ADR-040-google-gemini-primary.md (NEW)
M ARCHITECTURE.md, CLAUDE.md, README.md
M 70-docs/00-overview/{roadmap,problem-statement}.md
M 70-docs/10-architecture/{system-overview,stack-rationale,safety-architecture}.md
M 70-docs/30-engineering/{discovery-script,needs-analysis-flow}.md
M 70-docs/40-evaluation/{eval-methodology,known-issues,quality-sprint-2026-05-14}.md
M 70-docs/60-decisions/ADR-{014,016,019,026,030,031,032,038}.md
→ all marked Superseded by ADR-040 (2026-05-15)
M 80-audit/ENTERPRISE_AUDIT.md
M backend/README.md, frontend/README.md, rag/README.md, eval/README.md
M kb/AUDIT_TRAIL.md, tools/README.md

VERIFICATION:
py_compile all .py — clean
npx tsc --noEmit — clean
Live Gemini probe: HTTP 200 gemini-2.5-flash 943-1205ms; gemini-2.5-
flash-lite 739-1161ms; gemini-2.0-flash HTTP 404 (retired)
Tiered chain mock tests: gemini-up / gemini-down / gemini-nim-down /
all-down — all correct tag emission

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (36) hide show
  1. 70-docs/00-overview/problem-statement.md +2 -2
  2. 70-docs/00-overview/roadmap.md +11 -2
  3. 70-docs/10-architecture/safety-architecture.md +6 -6
  4. 70-docs/10-architecture/stack-rationale.md +9 -6
  5. 70-docs/10-architecture/system-overview.md +160 -99
  6. 70-docs/30-engineering/discovery-script.md +9 -5
  7. 70-docs/30-engineering/needs-analysis-flow.md +76 -59
  8. 70-docs/40-evaluation/eval-methodology.md +5 -5
  9. 70-docs/40-evaluation/known-issues.md +52 -0
  10. 70-docs/40-evaluation/quality-sprint-2026-05-14.md +42 -0
  11. 70-docs/60-decisions/ADR-014-groq-llama-grader.md +5 -1
  12. 70-docs/60-decisions/ADR-016-hybrid-brain-router.md +5 -1
  13. 70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md +1 -1
  14. 70-docs/60-decisions/ADR-026-provider-load-balancing.md +3 -2
  15. 70-docs/60-decisions/ADR-030-llm-driven-fact-find.md +8 -3
  16. 70-docs/60-decisions/ADR-031-sticky-primary-election.md +5 -1
  17. 70-docs/60-decisions/ADR-032-llm-chain-architecture.md +5 -5
  18. 70-docs/60-decisions/ADR-038-nim-only-chains.md +3 -1
  19. 70-docs/60-decisions/ADR-040-google-gemini-primary.md +90 -0
  20. 80-audit/ENTERPRISE_AUDIT.md +22 -4
  21. ARCHITECTURE.md +95 -50
  22. CLAUDE.md +17 -12
  23. README.md +57 -36
  24. backend/README.md +14 -4
  25. backend/orchestrator.py +12 -8
  26. backend/providers/google_gemini_llm.py +233 -0
  27. backend/providers/openrouter_llm.py +63 -1
  28. backend/providers/tiered_brain_llm.py +229 -0
  29. backend/sales_brain.py +154 -30
  30. eval/README.md +4 -3
  31. frontend/README.md +3 -0
  32. frontend/public/admin/llm-control.html +1 -0
  33. kb/AUDIT_TRAIL.md +34 -15
  34. rag/README.md +1 -1
  35. tools/README.md +3 -3
  36. tools/set_hf_secrets.py +1 -0
70-docs/00-overview/problem-statement.md CHANGED
@@ -219,9 +219,9 @@ To protect against scope creep — and to give the interviewer something to grad
219
 
220
  ### 8.4 Technical envelope
221
 
222
- - v1 UI: Streamlit. Business logic is decoupled into a separate `app/` Python module so a future FastAPI + React v2 swaps the UI layer only.
223
  - v1 corpus: 15–20 health policies across 5 leading Indian insurers (Star Health, HDFC ERGO, Niva Bupa, Care Health, ICICI Lombard — exact list pending availability check).
224
- - v1 deployment: local development + a single-instance deploy (Streamlit Community Cloud or equivalent) for the interviewer to demo.
225
 
226
  ---
227
 
 
219
 
220
  ### 8.4 Technical envelope
221
 
222
+ - v1 UI: Next.js 14 (App Router) + Tailwind v4 + shadcn/ui, calling a FastAPI backend. (The original draft scoped Streamlit; the migration to Next.js + FastAPI is captured in [ADR-005](../60-decisions/ADR-005-nextjs-fastapi-frontend.md).)
223
  - v1 corpus: 15–20 health policies across 5 leading Indian insurers (Star Health, HDFC ERGO, Niva Bupa, Care Health, ICICI Lombard — exact list pending availability check).
224
+ - v1 deployment: local development + a single-instance deploy on Hugging Face Spaces (Docker) for the interviewer to demo.
225
 
226
  ---
227
 
70-docs/00-overview/roadmap.md CHANGED
@@ -10,12 +10,21 @@
10
 
11
  v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first advisor. The architecture is built so v2 is a **data/config change, not a rebuild**. This document maps the path.
12
 
 
 
 
 
 
 
 
 
 
13
  ## 1. What v1 ships
14
 
15
  **Working product:**
16
  - Voice-first chat advisor over a curated corpus of Indian health insurance policies (~76 PDFs from 10 insurers, ingested into Chroma + DuckDB)
17
  - Multi-language: English + Hindi/Hinglish via Sarvam Saarika STT + Sarvam Bulbul TTS
18
- - Brain router (D-019, KI-080): BRAIN_CHAIN with probe-elected primary across NIM Qwen 80B / Groq Llama-3.3 / OpenRouter candidates (heavy intents) + FAST_BRAIN_CHAIN with probe-elected primary across Nemotron 30B / Qwen 80B / Groq (voice + fact-find), Mistral Large 3 675B judge primary. Sarvam-M scoped to Indic translation + voice only.
19
  - 4-gate hallucination defense + auditable refusal log
20
  - 62-field structured extraction per policy
21
  - Clean Next.js + Tailwind frontend
@@ -23,7 +32,7 @@ v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first
23
  - 8 design / decision documents totaling ~30 pages
24
 
25
  **Eval signal:**
26
- - Gold Q&A harness (~300 pairs targeted) + automated grader (the judge chain (Mistral Large 3 675B primary) — different family from DeepSeek brain)
27
  - `eval/results.md` versioned table per run
28
  - Live audit log `logs/hallucinations.jsonl` for every blocked claim
29
 
 
10
 
11
  v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first advisor. The architecture is built so v2 is a **data/config change, not a rebuild**. This document maps the path.
12
 
13
+ ## 0.1 Recently shipped (2026-05-15)
14
+
15
+ - **KI-167 / [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md)** — Ripped out `backend/fact_find_brain.py` + canonical_fallback + the `<FF>` trailer convention. Replaced with `backend/sales_brain.py` — one LLM call per turn using native provider JSON mode. No scripted prompts, multi-fact capture per turn, ~500 LOC deleted.
16
+ - **KI-168** — Hybrid voice capture: Web Speech API streams interim transcripts for live UX feel while MediaRecorder runs in parallel; Sarvam STT is the authoritative transcript posted on browser silence-detect.
17
+ - **KI-171** — Skip faithfulness judge on `fact_find` + `recommendation` intents (the judge has no retrieved context to grade against on those turns).
18
+ - **KI-173 / KI-174** — Voice heartbeat + `visibilitychange` / `focus` revival hooks keep the mic alive across tab and app switches.
19
+ - **KI-175 / KI-176** — NIM chain reorder (Nemotron 49B demoted to last resort), OpenRouter re-added as cross-provider diversity via `models: [...]` server-side fallback.
20
+ - **KI-178 / KI-179 / [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md)** — OpenRouter free-tier `response_format` audit; added Google AI Studio (Gemini 2.0 / 2.5 Flash, 1500 req/day free) as Tier 0 primary on Brain Fast + Brain Main.
21
+
22
  ## 1. What v1 ships
23
 
24
  **Working product:**
25
  - Voice-first chat advisor over a curated corpus of Indian health insurance policies (~76 PDFs from 10 insurers, ingested into Chroma + DuckDB)
26
  - Multi-language: English + Hindi/Hinglish via Sarvam Saarika STT + Sarvam Bulbul TTS
27
+ - 3-tier brain chain (ADR-040, KI-179): Brain Fast (`backend/sales_brain.py`, fact-find) leads with **Gemini 2.0 Flash** (Google AI Studio, 1500 req/day free, native JSON mode) → NIM Qwen 80B / Mistral Large 3 675B / Llama-4 Maverick → OpenRouter `:free` (Nemotron-3-Super 120B, Qwen 80B) NIM Nemotron 49B last resort. Brain Main (synthesis / comparison / recommendation) leads with **Gemini 2.5 Flash** NIM Mistral 675B / Maverick / Qwen 80B OR Nemotron-3-Super NIM Nemotron 49B. Judge primary stays on **NIM Mistral Large 3 675B** (different family from the Gemini brain — preserves the cross-family invariant). Sticky-primary election (KI-080) per chain. Sarvam scoped to Indic translation + voice only.
28
  - 4-gate hallucination defense + auditable refusal log
29
  - 62-field structured extraction per policy
30
  - Clean Next.js + Tailwind frontend
 
32
  - 8 design / decision documents totaling ~30 pages
33
 
34
  **Eval signal:**
35
+ - Gold Q&A harness (~300 pairs targeted) + automated grader (the JUDGE_CHAIN with NIM Mistral Large 3 675B primary — different family from the Gemini brain so grading stays non-circular)
36
  - `eval/results.md` versioned table per run
37
  - Live audit log `logs/hallucinations.jsonl` for every blocked claim
38
 
70-docs/10-architecture/safety-architecture.md CHANGED
@@ -19,7 +19,7 @@ In a regulated BFSI domain, the worst failure isn't "the bot looks slow" — it'
19
 
20
  **Description:** Bot claims a policy covers something it doesn't, or quotes a wrong waiting period / sub-limit.
21
  **Detection:**
22
- - Run-time: Faithfulness Gate 4 (LLM-judge the judge chain (Mistral Large 3 675B primary) different family from the brain (Qwen 80B primary); see D-019).
23
  - Run-time: Faithfulness Gate 3 (regex grounding — every ₹, %, day/month/year in the reply must appear in retrieved chunks).
24
  - Post-hoc: Gold Q&A eval (factual_accuracy metric).
25
  **Mitigation:** Block reply → return safe refusal → log to `logs/hallucinations.jsonl`. Audit log enables manual review and corpus improvement.
@@ -50,7 +50,7 @@ In a regulated BFSI domain, the worst failure isn't "the bot looks slow" — it'
50
 
51
  **Description:** Sarvam-M emits `<think>...</think>` reasoning. If `max_tokens` is exhausted before `</think>` is reached, the reply is unusable. This was a recurring issue when Sarvam-M was the primary brain.
52
  **Detection:** `strip_think_tags()` checks for `<think>` without matching `</think>`.
53
- **Resolution (2026-05-14, D-019; refined 2026-05-15, KI-080):** Sarvam-M moved out of the brain role entirely. The NIM brain chains (Qwen 80B / Nemotron 30B / Groq Llama-3.3 / OpenRouter GPT-OSS as probe-elected candidates) handle all reasoning; they emit direct responses without `<think>` preambles. Per-turn primary is elected from a 60s background probe (KI-080, [ADR-031](../60-decisions/ADR-031-sticky-primary-election.md)) — adapts to live provider degradation. Sarvam-M remains only for Indic translation (Hinglish ↔ English), where its `<think>` doesn't interfere because translation outputs are short. F-04 cannot fire on the current stack.
54
  **Owner:** `backend/orchestrator.py`
55
  **Status:** Resolved by architecture change.
56
 
@@ -95,9 +95,9 @@ In a regulated BFSI domain, the worst failure isn't "the bot looks slow" — it'
95
  **Description:** End-to-end (user speech end → bot speech start) p95 > 7s (Doc 01 C1).
96
  **Detection:** Per-turn latency logged in `logs/turns.jsonl`. Aggregate via eval harness.
97
  **Mitigation v1:**
98
- - Tiered brain routing (D-019): voice turns and fact-find go to the fast-brain chain (Nemotron Nano 30B primary) (~3-4s TTFT); only `comparison` and `recommendation` intents hit V4-Pro (slower but higher quality). Sarvam-M no longer in the brain hot path.
99
  - TTS happens server-side and is streamed via base64 in same response. Future: WebSocket for streaming TTS.
100
- **Status:** Mostly within budget for Llama brain (3-4s); Sarvam-M brain occasionally hits 15-25s due to reasoning.
101
 
102
  ### F-10 — Streamlit-Cloud-Free / Render-Free cold start
103
 
@@ -171,7 +171,7 @@ Run-time. Auditable. Tested.
171
 
172
  ### F-16 — Translation cascade introduces drift after faithfulness gate
173
 
174
- **Description:** Indic queries use the cascade: Sarvam translates Hinglish → English, DeepSeek reasons → English answer, faithfulness gates run on English answer, Sarvam translates English answer → Hinglish. **Faithfulness does NOT re-verify the final Hinglish output.** If Sarvam corrupts the translation (drops a citation, changes a number, invents a benefit), we wouldn't catch it.
175
  **Detection v1:** None automated. Manual spot-check of bilingual eval set.
176
  **Mitigation v1:** Sarvam translator system prompt explicitly forbids changing numbers/citations + caps at 60 words; preserves `[Source: ...]` tags.
177
  **Mitigation v2:** Back-translate Hinglish→English; compare against original English; block if cosine similarity < 0.85.
@@ -190,5 +190,5 @@ Run-time. Auditable. Tested.
190
  | M-07 | Nightly synthetic eval cron | Infra | v2 |
191
  | M-08 | Live-traffic spot grading | Eval | v2 |
192
  | M-09 | 3-judge consensus for grader | Eval | v2 |
193
- | M-10 | Sarvam-M no longer brain — V4-Flash/V4-Pro on NIM | Orchestrator (D-019) | Live ✅ |
194
  | M-11 | Faithfulness 4-gate verifier | Orchestrator | Live ✅ |
 
19
 
20
  **Description:** Bot claims a policy covers something it doesn't, or quotes a wrong waiting period / sub-limit.
21
  **Detection:**
22
+ - Run-time: Faithfulness Gate 4 (LLM-judge JUDGE_CHAIN with NIM Mistral Large 3 675B primary; different family from the Gemini brain so grading stays non-circular; see [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md)). KI-171 skips Gate 4 on `fact_find` + `recommendation` intents.
23
  - Run-time: Faithfulness Gate 3 (regex grounding — every ₹, %, day/month/year in the reply must appear in retrieved chunks).
24
  - Post-hoc: Gold Q&A eval (factual_accuracy metric).
25
  **Mitigation:** Block reply → return safe refusal → log to `logs/hallucinations.jsonl`. Audit log enables manual review and corpus improvement.
 
50
 
51
  **Description:** Sarvam-M emits `<think>...</think>` reasoning. If `max_tokens` is exhausted before `</think>` is reached, the reply is unusable. This was a recurring issue when Sarvam-M was the primary brain.
52
  **Detection:** `strip_think_tags()` checks for `<think>` without matching `</think>`.
53
+ **Resolution (2026-05-14, D-019 refined 2026-05-15 across KI-080 / KI-167 / KI-179):** Sarvam-M moved out of the brain role entirely. The current Brain Fast + Brain Main chains lead with Google AI Studio (Gemini 2.0 / 2.5 Flash) with NIM (Qwen 80B / Mistral Large 3 675B / Llama-4 Maverick) and OpenRouter `:free` (Nemotron-3-Super, Qwen 80B) as fallbacks; see [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md). All chain primaries emit direct responses without `<think>` preambles, and native JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM / OpenRouter) provides server-side parseable output. Per-turn primary is elected from a 300s background probe (KI-080, [ADR-031](../60-decisions/ADR-031-sticky-primary-election.md)) — adapts to live provider degradation. Sarvam-M remains only for Indic translation. F-04 cannot fire on the current stack.
54
  **Owner:** `backend/orchestrator.py`
55
  **Status:** Resolved by architecture change.
56
 
 
95
  **Description:** End-to-end (user speech end → bot speech start) p95 > 7s (Doc 01 C1).
96
  **Detection:** Per-turn latency logged in `logs/turns.jsonl`. Aggregate via eval harness.
97
  **Mitigation v1:**
98
+ - Tiered brain routing (ADR-040): voice turns and fact-find go to Brain Fast (Gemini 2.0 Flash primary, ~1-2s TTFT); `comparison` and `recommendation` intents hit Brain Main (Gemini 2.5 Flash primary, slightly higher latency but stronger synthesis). Sarvam-M no longer in the brain hot path.
99
  - TTS happens server-side and is streamed via base64 in same response. Future: WebSocket for streaming TTS.
100
+ **Status:** Brain Fast / Brain Main typically respond in 2-6s; long-tail latency comes from heavy fallback candidates (NIM Mistral 675B) only when Google quota is exhausted.
101
 
102
  ### F-10 — Streamlit-Cloud-Free / Render-Free cold start
103
 
 
171
 
172
  ### F-16 — Translation cascade introduces drift after faithfulness gate
173
 
174
+ **Description:** Indic queries use the cascade: Sarvam translates Hinglish → English, the elected Brain Main candidate (Gemini 2.5 Flash by default, NIM Mistral 675B / Maverick / Qwen as fallbacks) reasons → English answer, faithfulness gates run on English answer, Sarvam translates English answer → Hinglish. **Faithfulness does NOT re-verify the final Hinglish output.** If Sarvam corrupts the translation (drops a citation, changes a number, invents a benefit), we wouldn't catch it.
175
  **Detection v1:** None automated. Manual spot-check of bilingual eval set.
176
  **Mitigation v1:** Sarvam translator system prompt explicitly forbids changing numbers/citations + caps at 60 words; preserves `[Source: ...]` tags.
177
  **Mitigation v2:** Back-translate Hinglish→English; compare against original English; block if cosine similarity < 0.85.
 
190
  | M-07 | Nightly synthetic eval cron | Infra | v2 |
191
  | M-08 | Live-traffic spot grading | Eval | v2 |
192
  | M-09 | 3-judge consensus for grader | Eval | v2 |
193
+ | M-10 | Sarvam-M no longer brain — Gemini 2.0 / 2.5 Flash primary on Brain Fast / Brain Main, NIM fallback | Orchestrator (ADR-040) | Live ✅ |
194
  | M-11 | Faithfulness 4-gate verifier | Orchestrator | Live ✅ |
70-docs/10-architecture/stack-rationale.md CHANGED
@@ -62,14 +62,15 @@ The product is split into five layers. Each layer has independent technology cho
62
  | 8 | Backend | FE↔BE type safety | **`openapi-typescript` codegen** from FastAPI's auto-generated OpenAPI | Locked | Single source of truth; backend route changes propagate automatically | tRPC (Node-only), GraphQL, manual TS types |
63
  | 9 | Orchestration | STT | **Sarvam Saarika v2.5** | Locked | Sarvam-first; their newer Indic ASR model | Whisper-large-v3, Deepgram Nova-2, Sarvam Saaras v3 |
64
  | 10 | Orchestration | TTS | **Sarvam Bulbul** | Locked | Sarvam-first; Indic prosody first-class | ElevenLabs, OpenAI TTS, Coqui local |
65
- | 11 | Orchestration | LLM (brain) | **NVIDIA NIM tiered: Qwen 80B (current brain primary) (heavy) + Nemotron Nano 30B (current fast-brain primary) (fast)** | Locked (D-019) | V4-Pro (1.6T/49B MoE, frontier, beats Opus-4.6 + GPT-5.4 on SimpleQA) for comparison/recommendation; V4-Flash (284B/13B MoE, lower TTFT) for voice + fact-find. Single NIM key, free tier, no daily cap. Sarvam-M moved to Indic translation only its 2048 output cap was truncating advisor responses. | Sarvam-M as brain (rejected — truncation), OpenRouter `:free` (rejected — 50/day cap), Gemini 2.5 Flash (rejected — adds 2nd provider ecosystem), GPT-4o via GitHub Models (rejected same 50/day cap) |
66
- | 12 | Orchestration | LLM (grader / self-critique) | **NVIDIA the judge chain (Mistral Large 3 675B primary) (Meta MoE)** | Locked (D-019) | Different family from the brain (Qwen 80B primary) Meta MoE judging DeepSeek MoE, so the brain does not mark its own homework. Same NIM endpoint and key. 400B/17B MoE, frontier-tier. Replaces Groq Llama-3.3-70B (30 req/min cap blocked sweeps). | Groq Llama (rejected rate limit), Cerebras Qwen-235B (rejected — same family as another DeepSeek path), Sarvam-M held-out |
 
67
  | 13 | Orchestration | Embeddings | **BGE-small-en-v1.5 (local CPU, sentence-transformers)** | Locked | Free, runs offline, 384-dim, BEIR-strong on insurance/legal text. Voyage path kept for v2 swap (~3pp retrieval-quality delta in BEIR-style spot checks). | Voyage AI voyage-3 (rejected for v1 — 3 RPM free-tier blocked 104-PDF ingest), OpenAI text-embedding-3-small (rejected — no GPT), Cohere |
68
  | 14 | Storage | Structured DB | **DuckDB** (single file) | Locked | One file, columnar, no server; deploys via `git clone` | SQLite, Postgres |
69
  | 15 | Storage | Vector DB | **Chroma** (local persisted) | Locked | Embedded, no infra, supports metadata filtering | FAISS, Pinecone, Qdrant, Weaviate |
70
  | 16 | Ingest | PDF parsing | **pdfplumber** | Locked | Clean text + page numbers — critical for citation grammar | PyMuPDF, Unstructured.io |
71
  | 17 | Ingest | Chunking | **Custom 800-token chunks, 120-token overlap, page-aware** | Locked | Standard for technical docs; overlap protects clause boundaries | LangChain text splitter, fixed-size |
72
- | 18 | Ingest | Structured extraction | **Sarvam-M (Llama-3.3-70B / DeepSeek-V3 fallback) with Pydantic structured output + self-critique pass** | Locked | Reliable JSON-mode extraction; self-critique gives per-field confidence; fallback brain handles tables / complex clauses Sarvam-M misses | Pure regex, LangChain extraction, Instructor lib |
73
  | 19 | Pricing | Approach | **Hand-curated illustrative bands** from public PolicyBazaar quotes | Locked | Honest, defensible; "advisor-not-broker" positioning | Live scraping, actuarial model |
74
  | 20 | Cross-cutting | Auth | **None for v1** (single-tenant demo) | Locked | Out of scope per `01-requirements.md` §7 | Auth0, Clerk, NextAuth |
75
  | 21 | Cross-cutting | Observability | **JSONL turn log + cost tracker** | Locked | Lightweight; one log file, queryable post-hoc | Langfuse, Helicone, custom dashboard |
@@ -169,10 +170,12 @@ Every pick here corresponds to one or more entries in `decisions.md`:
169
  | Consultative persona | D-008 |
170
  | 10 insurers × all health policies | D-009 |
171
  | Secret handling | D-010 |
172
- | Embeddings provider (pending) | D-011 |
173
- | Render deployment over alternatives | D-012 |
174
  | Next.js + Tailwind + shadcn UI stack | D-013 |
175
- | GPT-4o-mini as grader to avoid circular eval | D-014 |
 
 
176
 
177
  ---
178
 
 
62
  | 8 | Backend | FE↔BE type safety | **`openapi-typescript` codegen** from FastAPI's auto-generated OpenAPI | Locked | Single source of truth; backend route changes propagate automatically | tRPC (Node-only), GraphQL, manual TS types |
63
  | 9 | Orchestration | STT | **Sarvam Saarika v2.5** | Locked | Sarvam-first; their newer Indic ASR model | Whisper-large-v3, Deepgram Nova-2, Sarvam Saaras v3 |
64
  | 10 | Orchestration | TTS | **Sarvam Bulbul** | Locked | Sarvam-first; Indic prosody first-class | ElevenLabs, OpenAI TTS, Coqui local |
65
+ | 11 | Orchestration | LLM — Brain Fast (sales_brain) | **Gemini 2.0 Flash** primary (Google AI Studio, 1500 req/day free, native JSON mode) NIM Qwen 80B NIM Mistral Large 3 675B NIM Llama-4 Maverick → OR `:free` (Nemotron-3-Super 120B, Qwen 80B) NIM Nemotron 49B last resort | Locked (ADR-040) | Frontier free-tier conversational quality on the fact-find surface; native JSON mode via `response_mime_type=application/json` matches NIM's `response_format` contract validated since [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) / KI-167. 6-level fallback survives Google quota exhaustion or 429. | NIM-only (rejected — strictly worse free-tier primary post-Gemini), OR Llama 3.3 70B (rejected — no `response_format` on OR free per KI-178), Sarvam-M as brain (rejected long ago for `<think>` truncation) |
66
+ | 11a | Orchestration | LLM — Brain Main (QA / comparison / recommendation) | **Gemini 2.5 Flash** primary → NIM Mistral Large 3 675B NIM Llama-4 Maverick NIM Qwen 80B OR Nemotron-3-Super 120B NIM Nemotron 49B last resort | Locked (ADR-040) | Higher synthesis quality than 2.0 Flash on long-context recommendation; shares the 1500 req/day Google quota. | (see row 11) |
67
+ | 12 | Orchestration | LLM — Judge (faithfulness Gate 4) | **NIM Mistral Large 3 675B** primary → NIM Llama-4 Maverick → OR Qwen 80B `:free` → NIM Nemotron 49B last resort | Locked (ADR-040) | Different family from the Gemini brain — preserves the brain ↔ judge family-diversity invariant ([ADR-014](../60-decisions/ADR-014-groq-llama-grader.md) lineage). KI-171 skips the judge on `fact_find` + `recommendation` intents (no retrieved context to grade). | Gemini as judge (rejected — same family as brain, circular grading), Groq Llama (rejected — rate cap), Sarvam-M held out |
68
  | 13 | Orchestration | Embeddings | **BGE-small-en-v1.5 (local CPU, sentence-transformers)** | Locked | Free, runs offline, 384-dim, BEIR-strong on insurance/legal text. Voyage path kept for v2 swap (~3pp retrieval-quality delta in BEIR-style spot checks). | Voyage AI voyage-3 (rejected for v1 — 3 RPM free-tier blocked 104-PDF ingest), OpenAI text-embedding-3-small (rejected — no GPT), Cohere |
69
  | 14 | Storage | Structured DB | **DuckDB** (single file) | Locked | One file, columnar, no server; deploys via `git clone` | SQLite, Postgres |
70
  | 15 | Storage | Vector DB | **Chroma** (local persisted) | Locked | Embedded, no infra, supports metadata filtering | FAISS, Pinecone, Qdrant, Weaviate |
71
  | 16 | Ingest | PDF parsing | **pdfplumber** | Locked | Clean text + page numbers — critical for citation grammar | PyMuPDF, Unstructured.io |
72
  | 17 | Ingest | Chunking | **Custom 800-token chunks, 120-token overlap, page-aware** | Locked | Standard for technical docs; overlap protects clause boundaries | LangChain text splitter, fixed-size |
73
+ | 18 | Ingest | Structured extraction | **`NimChainLLM(BRAIN_CHAIN)` with Pydantic structured output + self-critique pass** | Locked | Reliable JSON-mode extraction via native provider JSON mode; self-critique runs through `NimChainLLM(JUDGE_CHAIN)` for per-field confidence. | Pure regex, LangChain extraction, Instructor lib |
74
  | 19 | Pricing | Approach | **Hand-curated illustrative bands** from public PolicyBazaar quotes | Locked | Honest, defensible; "advisor-not-broker" positioning | Live scraping, actuarial model |
75
  | 20 | Cross-cutting | Auth | **None for v1** (single-tenant demo) | Locked | Out of scope per `01-requirements.md` §7 | Auth0, Clerk, NextAuth |
76
  | 21 | Cross-cutting | Observability | **JSONL turn log + cost tracker** | Locked | Lightweight; one log file, queryable post-hoc | Langfuse, Helicone, custom dashboard |
 
170
  | Consultative persona | D-008 |
171
  | 10 insurers × all health policies | D-009 |
172
  | Secret handling | D-010 |
173
+ | Embeddings provider (pending → BGE local) | D-011 / ADR-011 |
174
+ | Render deployment (superseded by HF Spaces) | D-012 / ADR-012 |
175
  | Next.js + Tailwind + shadcn UI stack | D-013 |
176
+ | Groq Llama grader (superseded NIM Mistral) | D-014 / ADR-014 / ADR-040 |
177
+ | LLM-driven sales brain (rip-out of `<FF>` trailer) | ADR-039 |
178
+ | Google Gemini Flash as Tier 0 primary | ADR-040 |
179
 
180
  ---
181
 
70-docs/10-architecture/system-overview.md CHANGED
@@ -3,10 +3,10 @@
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
- | Document version | 0.1 (draft) |
7
- | Date | 2026-05-13 |
8
- | Depends on | `01-requirements.md` |
9
- | Status | In review |
10
 
11
  ---
12
 
@@ -27,40 +27,64 @@ This document specifies the technical architecture: the stack, the data flow, th
27
  │ audio bytes / text / clicks
28
 
29
  ┌─────────────────────────────────────────────────────────────────────┐
30
- STREAMLIT UI (app/main.py)
31
- ─ chat pane · audio recorder · filter sidebar · comparison view
32
- session state · @st.cache_resource for heavy clients
 
33
  └─────────────────────────────────────────────────────────────────────┘
34
  │ │ │
35
- │ audio bytes │ user_query (text) │ filter / select
36
  ▼ ▼ ▼
37
  ┌──────────────────┐ ┌─────────────────────┐ ┌──────────────────────┐
38
- │ VOICE (app/voice)│ │ ORCHESTRATOR │ │ STRUCTURED STORE │
39
- │ ─ Sarvam Saaras │ ──────► │ (app/orchestrator) │ ◄ │ DuckDB │
40
- (STT) │ text │ ─ persona prompt │ │ ─ 1 row per policy │
41
- │ ─ Sarvam Bulbul │ ◄────── │ ─ fact-find graph │ │ ─ 40-50 fields
42
- (TTS) │ audio │ ─ tool routing │ │ ─ filter / compare │
43
- └──────────────────┘ │ ─ citation grammar │ └──────────────────────┘
44
- │ ▲
45
- │ │ extracted
46
  └──────────┬──────────┘ │ at ingest time
47
  │ │
48
  ▼ │
49
- ┌─────────────────────┐
50
- Sarvam-M (LLM)
51
- ─────────────────────
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  ▲ │
53
  │ retrieved chunks │
54
  │ │
55
  ┌──────────┴──────────┐ │
56
  │ VECTOR STORE │ ┌───────────┴──────────┐
57
  │ Chroma │ │ INGEST PIPELINE │
58
- │ ─ chunk + embedding │ ◄ │ (rag/ingest.py) │
59
  │ ─ metadata: policy, │ │ ─ download PDFs │
60
  │ page, clause │ │ ─ chunk + embed │
61
- ────────────────────┘ │ ─ LLM extract → DB │
62
- │ ─ per-insurer adapter│
63
- └──────────────────────┘
64
 
65
  │ raw PDFs
66
  ┌────────────┴─────────┐
@@ -71,21 +95,26 @@ This document specifies the technical architecture: the stack, the data flow, th
71
 
72
  ---
73
 
74
- ## 2. Stack picks (v1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
- | Layer | Pick | Why | Alternative considered | Decision in `decisions.md` |
77
- | --- | --- | --- | --- | --- |
78
- | STT | **Sarvam Saaras** | Sarvam-first per assignment context; strong Indic/Hinglish handling | Whisper-large-v3, Deepgram Nova | D-006 |
79
- | TTS | **Sarvam Bulbul** | First-party Indic prosody; Sarvam-first | ElevenLabs, OpenAI TTS | D-006 |
80
- | LLM | **Sarvam-M** (or current Sarvam flagship) | Sarvam-first; Indic-tuned reasoning | GPT-4o, Claude Sonnet | D-006 |
81
- | Embeddings | **Sarvam embedding API if available, else `text-embedding-3-small`** | Will benchmark; whichever scores higher on retrieval @5 against our gold set | OpenAI embeddings, BGE-large local | D-006 |
82
- | Vector DB | **Chroma** (local, persisted to disk) | Zero-config, embedded, works on Streamlit Cloud | FAISS (no metadata filtering), Qdrant (overkill) | D-004 |
83
- | Structured DB | **DuckDB** (single file, embedded) | One-file, columnar, no server, deployable to Streamlit Cloud | SQLite, Postgres | D-004 |
84
- | UI | **Streamlit** | Fastest v1; business logic decoupled so v2 swaps UI only | FastAPI + React (v2) | D-005 |
85
- | Audio capture | **`streamlit-mic-recorder`** or `streamlit-webrtc` (whichever proves stable in <1h spike) | Required for browser-side mic | Native HTML5 + WebSocket (too much glue) | — |
86
- | Deployment | **Streamlit Community Cloud** | Free, GitHub-connected, supports Streamlit out of box, secrets UI for API keys | Render, Fly.io | — |
87
 
88
- **Provider abstraction:** every provider is behind a thin interface (`app/providers/stt.py`, `app/providers/tts.py`, `app/providers/llm.py`) so a benchmark swap is a config flag. **This is non-negotiable** — it's c-readiness commitment #5.
89
 
90
  ---
91
 
@@ -94,31 +123,46 @@ This document specifies the technical architecture: the stack, the data flow, th
94
  ```
95
  user speaks
96
 
97
- (audio bytes)
98
- voice.transcribe() ── Sarvam Saaras ──► text
99
 
100
-
101
- orchestrator.handle_turn(text, session)
 
102
 
103
- ├── if fact-find phase: needs_finder.next_question(profile) → returns next Q
 
104
 
105
- ├── if filter/compare intent detected: structured_store.query(filters) → rows
 
 
 
 
 
 
106
 
107
- ── if Q&A intent:
108
- rag.retrieve(query, top_k=5)
 
 
109
  ─ returns chunks with (policy_id, page, clause, text, score)
110
- llm.generate(persona_prompt + chat_history + retrieved_chunks + user_query)
 
 
111
  ─ enforces citation grammar [Source: ...]
112
- enforces refusal when retrieval is weak
113
- returns text response
 
 
 
114
 
115
 
116
- voice.synthesize(text) ── Sarvam Bulbul ──► audio bytes
 
117
 
118
 
119
- ui.play_audio(audio_bytes)
120
- ui.append_to_chat(text + citations)
121
- log_turn({stt_confidence, retrieved_ids, latency_ms, cost_usd, refused})
122
  ```
123
 
124
  ---
@@ -151,18 +195,18 @@ Full Pydantic model lives in `rag/schema.py` (designed in parallel by a sub-agen
151
  for pdf in corpus:
152
  text_chunks = chunk(pdf, target_tokens=800, overlap=120)
153
  # chunks indexed in Chroma with metadata: policy_id, page, clause_path
154
-
155
  # one-shot LLM extraction with the schema as a structured-output target
156
- extracted = llm.extract_structured(
157
  full_pdf_text,
158
  schema=HealthPolicy,
159
  few_shot_examples=[hdfc_ergo_example, niva_bupa_example]
160
  )
161
-
162
- # self-critique pass: LLM checks every field's evidence in the source
163
- confidence_per_field = llm.self_critique(extracted, full_pdf_text)
164
  extracted.extraction_confidence_pct = aggregate(confidence_per_field)
165
-
166
  # write to DuckDB
167
  structured_store.upsert(extracted)
168
  ```
@@ -246,52 +290,68 @@ Every turn writes one JSON line to `logs/turns.jsonl`:
246
 
247
  ```
248
  insurance-sales-bot/
249
- ├── app/
250
- │ ├── __init__.py
251
- │ ├── main.py # Streamlit entrypoint
252
  │ ├── orchestrator.py # turn handler, persona prompt, intent routing
253
- │ ├── needs_finder.py # adaptive fact-find question graph
254
- │ ├── ui_chat.py
255
- │ ├── ui_filter.py # schema-driven filter UI
256
- │ ├── ui_compare.py # side-by-side comparison
 
 
 
 
 
 
 
 
257
  │ └── providers/
258
- │ ├── stt.py # Sarvam Saaras client behind interface
259
- │ ├── tts.py # Sarvam Bulbul client behind interface
260
- │ ├── llm.py # Sarvam-M client behind interface
261
- ── embeddings.py
 
 
 
 
 
 
 
262
  ├── rag/
263
- │ ├── corpus/ # raw PDFs (committed for reproducibility)
264
- │ ├── extracted/ # per-policy JSON (committed)
265
- │ ├── vectors/ # Chroma persistence (gitignored rebuilt)
266
- │ ├── policies.duckdb # structured store (committed)
267
  │ ├── schema.py # Pydantic HealthPolicy model
268
  │ ├── ingest.py # download → chunk → embed → extract → store
269
  │ ├── retrieve.py # query → top-k chunks
270
- ── extract.py # LLM-driven structured extraction
271
- │ └── adapters/
272
- ├── base.py
273
- ├── star_health.py
274
- ├── hdfc_ergo.py
275
- ── ... (10 adapters)
 
 
276
  ├── eval/
277
- │ ├── gold_qa.json # 50–100 gold Q&A pairs (10/policy for top policies)
278
- │ ├── grader.py # automated grader (LLM judge + regex)
279
- ── run.py # batch runner
280
- │ └── results.md # last run's accuracy table
281
- ├── data/
282
- │ ├── insurers.json # 10 insurers metadata
283
- │ └── corpus_urls.md # output of corpus-discovery sub-agent
284
  ├── 70-docs/
285
- │ ├── 01-requirements.md
286
- │ ├── 02-architecture.md # THIS FILE
287
- │ ├── 03-eval-plan.md
288
- │ ├── 04-failure-modes.md
289
- │ ├── 05-needs-analysis-flow.md
290
- ── decisions.md
291
- │ └── ROADMAP.md
 
 
 
292
  ├── tests/
293
- ── test_smoke.py
294
- ── streamlit_app.py # Streamlit Cloud entrypoint (thin wrapper)
 
295
  ├── requirements.txt
296
  ├── .env / .env.example
297
  ├── .gitignore
@@ -302,11 +362,12 @@ insurance-sales-bot/
302
 
303
  ## 10. Deployment
304
 
305
- - **Repo:** `github.com/rohitsar567/insurance-sales-bot` (public)
306
- - **Hosting:** Streamlit Community Cloud `share.streamlit.io` connects to the repo, deploys `streamlit_app.py` on push
307
- - **Secrets:** Sarvam API key set via Streamlit Cloud's secrets manager (mirrors `.env`)
308
- - **Resources:** ~1 GB ephemeral, persistent disk for `rag/` (corpus + DuckDB committed to repo so deploy is reproducible from `git clone`)
309
- - **Vector store rebuild:** on first deploy, app detects empty `rag/vectors/` and rebuilds from `rag/corpus/` (one-time ~5 min cold start; subsequent starts cached)
 
310
 
311
  ---
312
 
 
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
+ | Document version | 0.2 |
7
+ | Date | 2026-05-15 |
8
+ | Depends on | `problem-statement.md` |
9
+ | Status | Live |
10
 
11
  ---
12
 
 
27
  │ audio bytes / text / clicks
28
 
29
  ┌─────────────────────────────────────────────────────────────────────┐
30
+ NEXT.JS 14 UI (frontend/src/app/page.tsx)
31
+ ─ chat surface · Live + push-to-talk toggle · profile builder
32
+ Web Speech (interim) + MediaRecorder (authoritative) hybrid
33
+ │ ─ in-DOM <audio> for TTS playback (barge-in compatible) │
34
  └─────────────────────────────────────────────────────────────────────┘
35
  │ │ │
36
+ │ audio blob │ user_query (text) │ filter / select
37
  ▼ ▼ ▼
38
  ┌──────────────────┐ ┌─────────────────────┐ ┌──────────────────────┐
39
+ │ VOICE │ │ ORCHESTRATOR │ │ STRUCTURED STORE │
40
+ │ ─ Sarvam Saarika │ ──────► │ (backend/ │ ◄ │ DuckDB │
41
+ │ STT │ text │ orchestrator.py) │ │ ─ 1 row per policy │
42
+ │ ─ Sarvam Bulbul │ ◄────── │ ─ classify_intent │ │ ─ 62-field schema
43
+ │ TTS │ audio │ ─ profile RAG │ │ ─ filter / compare │
44
+ └──────────────────┘ │ ─ sales_brain / │ └──────────────────────┘
45
+ QA brain dispatch │ ▲
46
+ ─ faithfulness gates│ │ extracted
47
  └──────────┬──────────┘ │ at ingest time
48
  │ │
49
  ▼ │
50
+ ┌──────────────────────────────────
51
+ LLM CHAINS (3-tier, ADR-040)
52
+ │ ┌────────────────────────────┐
53
+ │ │ Brain Fast (sales_brain): │ │ │
54
+ │ │ Gemini 2.0 Flash (Tier 0) │ │ │
55
+ │ │ → NIM Qwen/Mistral/Llama4 │ │ │
56
+ │ │ → OR :free fallback │ │ │
57
+ │ │ → NIM Nemotron 49B (last) │ │ │
58
+ │ ├────────────────────────────┤ │ │
59
+ │ │ Brain Main (synthesis): │ │ │
60
+ │ │ Gemini 2.5 Flash (Tier 0) │ │ │
61
+ │ │ → NIM Mistral/Maverick/ │ │ │
62
+ │ │ Qwen │ │ │
63
+ │ │ → OR Nemotron-3-Super │ │ │
64
+ │ │ → NIM Nemotron 49B (last) │ │ │
65
+ │ ├────────────────────────────┤ │ │
66
+ │ │ Judge (cross-family): │ │ │
67
+ │ │ NIM Mistral Large 3 675B │ │ │
68
+ │ │ → NIM Llama-4 Maverick │ │ │
69
+ │ │ → OR Qwen 80B :free │ │ │
70
+ │ │ → NIM Nemotron 49B (last) │ │ │
71
+ │ └────────────────────────────┘ │ │
72
+ │ Sticky-primary election (KI-080)│ │
73
+ │ per chain via │ │
74
+ │ backend/llm_health.py probe loop│ │
75
+ └──────────────────────────────────┘ │
76
  ▲ │
77
  │ retrieved chunks │
78
  │ │
79
  ┌──────────┴──────────┐ │
80
  │ VECTOR STORE │ ┌───────────┴──────────┐
81
  │ Chroma │ │ INGEST PIPELINE │
82
+ │ ─ chunk + BGE-small │ ◄ │ (rag/ingest.py) │
83
  │ ─ metadata: policy, │ │ ─ download PDFs │
84
  │ page, clause │ │ ─ chunk + embed │
85
+ profile chunks │ │ ─ LLM extract → DB │
86
+ (session-scoped) │ │ ─ per-insurer adapter│
87
+ └─────────────────────┘ └─────────────────────
88
 
89
  │ raw PDFs
90
  ┌────────────┴─────────┐
 
95
 
96
  ---
97
 
98
+ ## 2. Stack picks (current)
99
+
100
+ | Layer | Pick | Why | Decision |
101
+ | --- | --- | --- | --- |
102
+ | STT | **Sarvam Saarika v2.5** | Best-in-class Indian-accent + Hinglish handling. | [ADR-006](../60-decisions/ADR-006-sarvam-first-stack.md) |
103
+ | TTS | **Sarvam Bulbul v2** | First-party Indic prosody; ₹ + lakh shorthand normalised pre-TTS by `voice_format.py`. | [ADR-006](../60-decisions/ADR-006-sarvam-first-stack.md) |
104
+ | LLM — Brain Fast (sales_brain) | **Gemini 2.0 Flash** primary → NIM Qwen 80B / Mistral Large 3 675B / Llama-4 Maverick → OR `:free` (Nemotron-3-Super, Qwen 80B) → NIM Nemotron 49B last resort. Native JSON mode via `response_mime_type=application/json` / `response_format={"type":"json_object"}`. | Frontier free-tier quality on Gemini (1500 req/day) with NIM diversity below it. Sticky-primary election (KI-080) picks one candidate per call. | [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md), [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) |
105
+ | LLM — Brain Main (QA / comparison / recommendation) | **Gemini 2.5 Flash** primary → NIM Mistral 675B / Maverick / Qwen 80B → OR Nemotron-3-Super → NIM Nemotron 49B last resort. | Higher synthesis quality than 2.0 on long-context recommendation; shares the Google quota with sales_brain. | [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md) |
106
+ | LLM �� Judge (faithfulness Gate 4) | **NIM Mistral Large 3 675B** primary → NIM Llama-4 Maverick → OR Qwen 80B `:free` → NIM Nemotron 49B last resort. | Different family from the Gemini brain — preserves brain ↔ judge family-diversity invariant ([ADR-014](../60-decisions/ADR-014-groq-llama-grader.md) lineage). | [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md) |
107
+ | Indic translation | **Sarvam-M** | Best-in-class Hinglish ↔ English translation. | [ADR-006](../60-decisions/ADR-006-sarvam-first-stack.md) |
108
+ | Embeddings | **BGE-small-en-v1.5 (local CPU, 384-d)** | Free, runs offline, no rate limits. | [ADR-011](../60-decisions/ADR-011-bge-local-embeddings.md) |
109
+ | Vector DB | **Chroma** (local persisted) | Embedded, supports metadata filtering, HNSW bloat guard ([ADR-029](../60-decisions/ADR-029-hnsw-bloat-tripwire.md)) | [ADR-004](../60-decisions/ADR-004-hybrid-structured-vector.md) |
110
+ | Structured DB | **DuckDB** (single file) | One-file, columnar, no server. | [ADR-004](../60-decisions/ADR-004-hybrid-structured-vector.md) |
111
+ | UI | **Next.js 14 (App Router) + Tailwind v4 + shadcn/ui** | Production-pattern, full UI flexibility. | [ADR-005](../60-decisions/ADR-005-nextjs-fastapi-frontend.md), [ADR-013](../60-decisions/ADR-013-tailwind-shadcn-ui.md) |
112
+ | Audio capture | **Web Speech API (interim text) + `MediaRecorder` (authoritative blob), hybrid (KI-168)** | Live UX cadence on the UI + Sarvam-grade STT for the actual transcript. | [ADR-028](../60-decisions/ADR-028-voice-ux-single-default-mode.md) |
113
+ | Deployment | **HF Spaces (Docker) + companion HF Dataset for corpus / vectors** | Free, GitHub-mirrored, snapshot_download for data hydration. | [ADR-012](../60-decisions/ADR-012-render-then-hf-space-deploy.md), [ADR-020](../60-decisions/ADR-020-code-data-split-hf-dataset.md) |
114
 
115
+ **Provider keys:** `GOOGLE_API_KEY` (Google AI Studio, 1500 req/day free), `NVIDIA_NIM_API_KEY` (free, 40 req/min), `OPENROUTER_API_KEY` ($10 unlocks 1000 req/day on `:free` models), `SARVAM_API_KEY` (Sarvam STT + TTS + translation).
 
 
 
 
 
 
 
 
 
 
116
 
117
+ **Provider abstraction:** every LLM provider lives under `backend/providers/` (`google_gemini_llm.py`, `nvidia_nim_llm.py`, `openrouter_llm.py`) behind a common `LLMProvider` interface. `NimChainLLM` is provider-agnostic chains may freely mix Google / NIM / OpenRouter candidates by URL. **This is non-negotiable** — it's c-readiness commitment #5.
118
 
119
  ---
120
 
 
123
  ```
124
  user speaks
125
 
126
+ │ Web Speech streams interim transcript to chat input (live UX)
127
+ MediaRecorder captures audio blob in parallel
128
 
129
+ (browser silence-detect → audio blob)
130
+ POST /api/transcribe ── Sarvam Saarika v2.5 ──► authoritative text
131
+ │ (fallback to Web Speech transcript if Sarvam errors)
132
 
133
+ (auto-submit)
134
+ POST /api/chat → orchestrator.handle_turn(text, session)
135
 
136
+ ├── if intent == fact_find:
137
+ │ sales_brain.drive_sales_turn(profile, history, user_text)
138
+ │ ─ ONE LLM call against FAST_BRAIN_CHAIN (Gemini 2.0 Flash primary)
139
+ │ ─ native JSON mode returns {reply, captures, slot_driving, complete}
140
+ │ ─ sales_brain_normalizer maps loose captures → canonical fields
141
+ │ ─ KI-094 None-guard ensures null captures don't wipe filled fields
142
+ │ ─ KI-171 skips faithfulness judge on fact-find turns
143
 
144
+ ── if intent == comparison: structured_store.query(filters) → rows
145
+
146
+ └── if intent ∈ {qa, recommendation}:
147
+ rag.retrieve(query, top_k=10)
148
  ─ returns chunks with (policy_id, page, clause, text, score)
149
+ profile_rag chunks filtered to current session_id (KI-102)
150
+ BRAIN_CHAIN.chat(persona_prompt + history + chunks + user_query)
151
+ ─ Gemini 2.5 Flash primary → NIM Mistral 675B fallback
152
  ─ enforces citation grammar [Source: ...]
153
+ ─ refusal returned when retrieval is weak
154
+ faithfulness gates (skipped on recommendation per KI-171):
155
+ Gate 1: retrieval floor Gate 2: citation integrity
156
+ Gate 3: regex numeric Gate 4: cross-family judge
157
+ (NIM Mistral Large 3 675B)
158
 
159
 
160
+ voice.synthesize(text) ── Sarvam Bulbul v2 ──► base64 audio
161
+ │ (voice_format.tts_preprocess strips CoT leaks + expands ₹ shorthand)
162
 
163
 
164
+ frontend in-DOM <audio> element plays the response (barge-in compatible)
165
+ log_turn({intent, brain_used, retrieved_ids, latency_ms, faithfulness, refused})
 
166
  ```
167
 
168
  ---
 
195
  for pdf in corpus:
196
  text_chunks = chunk(pdf, target_tokens=800, overlap=120)
197
  # chunks indexed in Chroma with metadata: policy_id, page, clause_path
198
+
199
  # one-shot LLM extraction with the schema as a structured-output target
200
+ extracted = NimChainLLM(BRAIN_CHAIN).extract_structured(
201
  full_pdf_text,
202
  schema=HealthPolicy,
203
  few_shot_examples=[hdfc_ergo_example, niva_bupa_example]
204
  )
205
+
206
+ # self-critique pass: judge LLM checks every field's evidence in the source
207
+ confidence_per_field = NimChainLLM(JUDGE_CHAIN).self_critique(extracted, full_pdf_text)
208
  extracted.extraction_confidence_pct = aggregate(confidence_per_field)
209
+
210
  # write to DuckDB
211
  structured_store.upsert(extracted)
212
  ```
 
290
 
291
  ```
292
  insurance-sales-bot/
293
+ ├── backend/
294
+ │ ├── main.py # FastAPI app; HTTP routes (/api/chat, /api/transcribe, /api/profile, admin)
 
295
  │ ├── orchestrator.py # turn handler, persona prompt, intent routing
296
+ │ ├── sales_brain.py # LLM-driven fact-find (one call/turn, native JSON mode)
297
+ │ ├── sales_brain_normalizer.py # deterministic post-processor over LLM captures
298
+ │ ├── needs_finder.py # 9-slot fact-find SCHEMA (data only; prompts retired)
299
+ │ ├── profile_extractor.py # conversational profile-update extractor
300
+ │ ├── profile_store.py # persistent named-profile JSON store (KI-040)
301
+ │ ├── profile_rag.py # session-scoped profile chunk in Chroma (KI-102)
302
+ │ ├── faithfulness.py # 4-gate hallucination guard
303
+ │ ├── persona.py # consultative-advisor system prompt
304
+ │ ├── voice_format.py # TTS pre-processing (CoT strip, ₹ shorthand)
305
+ │ ├── translator.py # Sarvam-M Indic ↔ English translation
306
+ │ ├── llm_health.py # background probe + sticky-primary election
307
+ │ ├── admin.py # /api/admin/* (LLM health, performance, profiles)
308
  │ └── providers/
309
+ │ ├── sarvam_stt.py # Sarvam Saarika v2.5 client
310
+ │ ├── sarvam_tts.py # Sarvam Bulbul v2 client
311
+ │ ├── google_gemini_llm.py # Google AI Studio (Tier 0 primary)
312
+ ── nvidia_nim_llm.py # NIM client + NimChainLLM dispatcher
313
+ │ ├── openrouter_llm.py # OpenRouter client (cross-provider diversity)
314
+ │ └── local_embeddings.py # BGE-small-en-v1.5 sentence-transformers
315
+ ├── frontend/
316
+ │ ├── src/app/page.tsx # chat surface + voice toolbar + tab switcher
317
+ │ ├── src/lib/api.ts # typed API client (openapi-typescript codegen)
318
+ │ ├── src/lib/useStreamingVoice.ts # hybrid Web Speech + MediaRecorder hook (KI-168)
319
+ │ └── src/lib/useLiveConversation.ts # Live-mode VAD hook
320
  ├── rag/
321
+ │ ├── corpus/ # raw PDFs (hydrated from HF dataset at Docker build)
322
+ │ ├── extracted/ # per-policy JSON (62-field HealthPolicy)
323
+ │ ├── vectors/ # Chroma persistence (HF dataset side)
324
+ │ ├── policies.duckdb # DuckDB rollup
325
  │ ├── schema.py # Pydantic HealthPolicy model
326
  │ ├── ingest.py # download → chunk → embed → extract → store
327
  │ ├── retrieve.py # query → top-k chunks
328
+ ── extract.py # LLM-driven structured extraction
329
+ ── 40-data/
330
+ ├── policy_facts/ # curated {value, source_quote} JSONs (256 variants)
331
+ ├── premiums/illustrative_premiums.json
332
+ ├── reviews/ # per-insurer IRDAI + aggregator + Reddit data
333
+ ── profiles/ # persistent named profiles (KI-040)
334
+ │ ├── llm_health.json # last probe snapshot
335
+ │ └── llm_usage.jsonl # per-call telemetry
336
  ├── eval/
337
+ │ ├── gold_qa.json # 96+ gold Q&A pairs
338
+ │ ├── run.py # in-process orchestrator runner + regex + judge
339
+ ── results.md / results.json
 
 
 
 
340
  ├── 70-docs/
341
+ │ ├── 00-overview/ # roadmap + problem-statement
342
+ │ ├── 10-architecture/ # this doc + stack-rationale + safety + scoring
343
+ │ ├── 20-data-pipeline/ # ingestion-policy + info-source-map
344
+ │ ├── 30-engineering/ # discovery-script + needs-analysis-flow
345
+ │ ├── 40-evaluation/ # eval-methodology + known-issues + quality sprints
346
+ ── 60-decisions/ # ADR-001 … ADR-040
347
+ ── 80-audit/
348
+ │ ├── ENTERPRISE_AUDIT.md # live defect register
349
+ │ └── <run_id>/ # 100-persona audit transcripts
350
+ ├── tools/ # operational scripts (corpus / KB / probes / cron)
351
  ├── tests/
352
+ ── test_routing_regression.py
353
+ │ └── test_persona_*.py
354
+ ├── Dockerfile # HF Space image (Python + snapshot_download)
355
  ├── requirements.txt
356
  ├── .env / .env.example
357
  ├── .gitignore
 
362
 
363
  ## 10. Deployment
364
 
365
+ - **Code repo:** `github.com/rohitsar567/insurance-sales-bot` (public; mirrored to HF Space `huggingface.co/spaces/rohitsar567/InsuranceBot`).
366
+ - **Data repo:** `huggingface.co/datasets/rohitsar567/insurance-bot-data` 539 files / ~498 MB (corpus PDFs + Chroma vectors + extracted JSONs). See [ADR-020](../60-decisions/ADR-020-code-data-split-hf-dataset.md).
367
+ - **Hosting:** HF Space Docker image. `Dockerfile` runs `huggingface_hub.snapshot_download` at build time to hydrate `rag/` from the data repo so the Space repo stays code-only (~3 MB).
368
+ - **Secrets (HF Space repository secrets):** `GOOGLE_API_KEY`, `NVIDIA_NIM_API_KEY`, `OPENROUTER_API_KEY`, `SARVAM_API_KEY`, admin password / IP allowlist. Mirrored locally in `.env` (chmod 600, gitignored).
369
+ - **Per-deploy verification:** `tests/live_verify.py` runs a small smoke against the live Space URL post-deploy.
370
+ - **Triple-mirror invariant ([ADR-024](../60-decisions/ADR-024-triple-mirror-code-and-data.md)):** every commit pushed to BOTH GitHub origin AND the HF Space remote; data side mirrors via `tools/upload_*_to_dataset.py`.
371
 
372
  ---
373
 
70-docs/30-engineering/discovery-script.md CHANGED
@@ -129,10 +129,14 @@ This is the customer-protection framing. It tells the user honesty is **self-pro
129
 
130
  ## Implementation notes
131
 
132
- The questions live in `backend/needs_finder.py::GRAPH`. To add a new question:
133
 
134
- 1. Add a `Question(...)` entry with `id`, `prompt_en` (plain language + WHY), `prompt_hi` (Hindi rendering), `field` (which Profile attribute it sets), `is_core` (boolean — counts toward completeness), optional `condition` callable, optional `parser`.
135
- 2. Add a row in `70-docs/scorecard-knowledge-graph.md` Part B showing how the new input shifts weights.
136
- 3. Wire the shift into `_profile_tuned_weights()` in `backend/scorecard.py`.
137
 
138
- Drift between these three places breaks the transparency promise. Keep them in sync.
 
 
 
 
 
 
 
129
 
130
  ## Implementation notes
131
 
132
+ The 9-slot fact-find SCHEMA still lives in `backend/needs_finder.py::GRAPH` used now as the schema source for the `sales_brain` LLM system prompt rather than as a scripted question list. Each entry's `id`, `field`, `is_core`, and `condition` are consumed by the LLM as a structured contract; the `prompt_en` / `prompt_hi` strings are no longer rendered to the user post-[ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) / KI-167 (the LLM owns voice + cadence end-to-end via its system prompt + the conversation so far).
133
 
134
+ To add a new question:
 
 
135
 
136
+ 1. Add a `Question(...)` entry with `id`, `field` (which Profile attribute it sets), `is_core` (boolean — counts toward completeness), optional `condition` callable, optional `parser`.
137
+ 2. Surface the new slot in the `sales_brain` system prompt's 9-slot schema (alongside accepted value shapes + examples) so the LLM knows to capture it. See `backend/sales_brain.py::_SYSTEM_PROMPT`.
138
+ 3. Wire any post-capture validation into `backend/sales_brain_normalizer.py` (enum coercion, INR parsing, bounds).
139
+ 4. Add a row in `70-docs/scorecard-knowledge-graph.md` Part B showing how the new input shifts weights.
140
+ 5. Wire the shift into `_profile_tuned_weights()` in `backend/scorecard.py`.
141
+
142
+ Drift between these places breaks the transparency promise. Keep them in sync.
70-docs/30-engineering/needs-analysis-flow.md CHANGED
@@ -1,24 +1,31 @@
1
- # 05 — Needs Analysis Flow (Fact-Find)
2
 
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
- | Version | 0.1 |
7
- | Date | 2026-05-13 |
8
- | Implementation | `backend/needs_finder.py` |
9
 
10
- ## 0. Why an explicit graph (not "let the LLM figure it out")
11
 
12
- A good Independent Financial Advisor opens with a stable, repeatable set of questions — and conditionally deep-dives based on the buyer's signal. We replicate this with an **explicit question graph**, not an LLM that improvises every session.
13
 
14
- Why:
15
 
16
- 1. **Auditable behavior.** A reviewer can see the graph, trace any session, and check why a question was (or wasn't) asked.
17
- 2. **Testable.** Pure functions of `Profile → Question`. Every branch can be unit-tested.
18
- 3. **Fail-soft.** Even if the LLM brain degrades, the question flow still works.
19
- 4. **Bilingual by construction.** Each node has English + Hindi prompts side by side.
20
 
21
- ## 1. The graph
 
 
 
 
 
 
 
 
 
 
22
 
23
  ```
24
  ┌─────────────────────────────┐
@@ -74,76 +81,86 @@ Why:
74
 
75
  ## 2. Termination criteria
76
 
77
- The graph emits `next_question = None` (i.e., ready to recommend) when all of:
78
-
79
- 1. All 6 core questions answered (age, dependents, income, existing_cover, primary_goal, location)
80
- 2. All applicable conditional questions answered (parents_age if dependents include parents)
81
- 3. health_conditions and budget_band answered
82
-
83
- Or when the user sets `profile.free_form_session = True` — the orchestrator skips fact-find and answers free questions directly.
84
 
85
- ## 3. Bilingual prompts
86
 
87
- Every node has both `prompt_en` and `prompt_hi`. The orchestrator picks based on the user's detected language. Example:
88
-
89
- | Node | English | Hindi |
90
- | --- | --- | --- |
91
- | age | "To start, what's your age?" | "शुरू करते हैं — आपकी उम्र क्या है?" |
92
- | dependents | "Who else do you want to cover…" | "आपके अलावा किस-किस को cover करना है…" |
93
- | budget | "What annual premium budget…" | "Premium के लिए सालाना कितना खर्च…" |
94
 
95
- This is what "Indic-native, not translated" (Doc 01 §5.3) looks like in practice the question is *authored* in both languages, not machine-translated at runtime.
96
 
97
  ## 4. Parsing user answers
98
 
99
- Each node has a `parser` callable that normalizes the user's natural-language answer into a typed Profile field:
100
 
101
- - `age`: `lambda s: int("".join(c for c in s if c.isdigit())[:3] or 0) or None`
102
- - `dependents`: free-form string (will be normalized via LLM in v1.1)
103
- - `income_band`: bucketed via keyword match ("5 lakh" → "under_5L")
 
 
104
 
105
- When the parser returns `None`, we treat the question as unanswered and re-ask (v1 simply leaves the field as None and moves on re-ask is v1.1).
106
 
107
  ## 5. How the orchestrator uses this
108
 
109
  ```python
110
  profile = Profile.from_session(session_id)
111
- q = next_question(profile, language=detected_language)
112
- if q is not None:
113
- # Still in fact-find phase ask the next question
114
- reply = q.prompt_hi if language == "indic" else q.prompt_en
115
- profile.asked.append(q.id)
 
 
 
 
 
 
 
 
116
  else:
117
- # Profile complete proceed to retrieval + recommendation
118
- chunks = retrieve(user_query, top_k=5)
119
- reply = brain.generate(...)
 
 
 
 
 
120
  ```
121
 
122
- The orchestrator can also let the user drive the conversation freely — if the user immediately asks a specific question like "compare Star and HDFC ERGO," we set `free_form_session = True` and skip the fact-find.
123
-
124
  ## 6. What the user sees
125
 
 
 
126
  ```
127
- Bot: "Hi, I'm your AI insurance advisor. To start, what's your age?"
128
- User: "I'm 32"
129
- Bot: "Got it. Who else do you want to cover — just yourself, spouse, kids, or parents?"
130
- User: "self + spouse, kids later"
131
- Bot: "Roughly what's your annual income under 5 lakh, 5-10, 10-25, or 25+ lakh?"
132
- User: "around 18 lakhs"
133
- ... [continues through the core 6] ...
134
- Bot: "Here's what I've understood: 32 years old; covering self+spouse; income 10L-25L;
135
- no existing cover; goal: first health policy; in a metro; budget 15K-30K.
136
- Did I get that right?"
137
- User: "yes"
138
- Bot: "Based on that, three policies stand out: [...] — let me explain why."
 
 
 
 
 
139
  ```
140
 
 
 
141
  ## 7. v2 enhancements
142
 
143
  | # | Enhancement | Why |
144
  | --- | --- | --- |
145
- | 1 | LLM-parser for free-form answers ("I'm in my early 30s with 2 kids") | Handles natural-language replies instead of rigid bucket-matching |
146
- | 2 | Re-ask on `None` parse | Robust to user confusion |
147
  | 3 | Skip-confirm flow ("you can skip this — say 'skip'") | Buyer autonomy |
148
- | 4 | Save profile across sessions | Returning user picks up where they left off |
149
- | 5 | Tone-match the user's energy (formal vs casual) | Conversation feels human |
 
1
+ # 05 — Needs Analysis Flow (LLM-driven sales brain)
2
 
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
+ | Version | 0.2 |
7
+ | Date | 2026-05-15 |
8
+ | Implementation | `backend/sales_brain.py` (KI-167) — the schema source still lives in `backend/needs_finder.py::GRAPH` |
9
 
10
+ ## 0. Why one LLM call per turn (not "scripted question graph + paraphraser")
11
 
12
+ A good Independent Financial Advisor opens with a stable, repeatable set of slots to fill — and adapts which slot to drive next based on what the buyer just said, what they've already told us, and what's still missing. We replicate this with **one LLM call per turn** that owns the entire fact-find surface (voice, cadence, slot-selection, multi-fact capture in a single turn), not a hardcoded state machine with a paraphraser glued on top.
13
 
14
+ The original v0.1 of this doc was an explicit `GRAPH` of canonical questions with `prompt_en` / `prompt_hi` strings. That design shipped with ADR-027 (LLM paraphraser on top of the graph) and ADR-030 (one-call brain with `<FF>` trailer + canonical fallback). Both architectures hit a hard wall: scripted prompts leaked into every fallback turn ("Got that — {slot}.") and produced robotic cadence even when the brain succeeded. KI-167 / [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) ripped it all out: no canonical_fallback, no scripted prompts, no `<FF>` trailer. Native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM) provides server-side parseable output.
15
 
16
+ What we kept and what we changed:
 
 
 
17
 
18
+ | Concern | v0.1 (scripted graph) | Current (LLM-driven sales_brain) |
19
+ |---|---|---|
20
+ | **Slot schema** | `GRAPH` of 9 `Question(id, prompt_en, prompt_hi, field, is_core, condition, parser)` entries | `GRAPH` data structure retained as the schema source; `prompt_en` / `prompt_hi` are dead strings now |
21
+ | **Question text** | Hardcoded `prompt_en` rendered to user | LLM generates natural prose in the advisor's voice per turn (no template) |
22
+ | **Slot selection** | `next_question(profile)` walked the graph in order | LLM picks `slot_driving` in its JSON response (constrained by the schema to required slots first) |
23
+ | **Multi-fact capture** | One slot per turn (a user saying "I'm 32, just myself, in Mumbai" only filled `age`) | LLM emits a `captures` map per turn — one utterance can fill 2-4 slots |
24
+ | **Auditable behavior** | Graph order was the trace | Per-turn LLM response is logged (intent, brain_used, captures, slot_driving, complete) in `40-data/llm_usage.jsonl` + `logs/turns.jsonl` |
25
+ | **Fail-soft** | Graph survived LLM degradation | Outer 25s `asyncio.wait_for` on the brain call; on total NIM + Google exhaustion the orchestrator returns the [ADR-038](../60-decisions/ADR-038-nim-only-chains.md) graceful error message to the user — no scripted reply |
26
+ | **Bilingual** | Hand-authored `prompt_en` + `prompt_hi` | Sarvam-M translation cascade on the LLM's output (English authoring + Indic translation), same UX, fewer hand-authored strings |
27
+
28
+ ## 1. The 9-slot schema (data-only — the LLM consults this through its system prompt)
29
 
30
  ```
31
  ┌─────────────────────────────┐
 
81
 
82
  ## 2. Termination criteria
83
 
84
+ The LLM sets `complete: true` in its JSON response when all required slots are filled. `backend/sales_brain_normalizer.py` double-checks: if `complete == true` while any required slot is empty, the normalizer overrides to `complete = false` — the LLM cannot prematurely declare fact-find done.
 
 
 
 
 
 
85
 
86
+ Or when the user opts out — if the user immediately asks "compare Star and HDFC ERGO" or "show me the top 3 policies for me" (KI-105 closer phrases), the intent classifier forces intent to `comparison` / `recommendation` BEFORE the fact-find branch is even reached, and the orchestrator routes straight to retrieval + Brain Main.
87
 
88
+ ## 3. Bilingual handling
 
 
 
 
 
 
89
 
90
+ The LLM authors in English; Sarvam-M translates Hinglish ↔ English at the cascade boundary. The "Indic-native, not translated" principle (problem-statement §5.3) is preserved by the LLM's native-quality output rather than by hand-authored Hindi strings the cascade itself is the bilingual surface.
91
 
92
  ## 4. Parsing user answers
93
 
94
+ The LLM produces a `captures` map per turn (e.g. `{"age": 34, "dependents": "self", "location_tier": "metro"}`). `backend/sales_brain_normalizer.py` is a pure-function post-processor that:
95
 
96
+ - Resolves field-name aliases (`location` `location_tier`)
97
+ - Coerces enums (`Bangalore` `metro`)
98
+ - Parses INR amounts (`"32 lakh"``3200000`)
99
+ - Drops null / empty values (KI-094 None-guard — a `None` capture never overwrites a filled field)
100
+ - Validates types and bounds (rejects `age: 250`)
101
 
102
+ No LLM calls in the normalizer pure rules. The orchestrator applies the normalized map via `session.update_profile_field()` exactly as before. Profile persistence remains unchanged (`backend/profile_store.save_profile()` + `backend/profile_rag.upsert_profile_chunk()` with session-scoped metadata per KI-102).
103
 
104
  ## 5. How the orchestrator uses this
105
 
106
  ```python
107
  profile = Profile.from_session(session_id)
108
+
109
+ if intent == "fact_find":
110
+ # KI-167: one LLM call per turn against FAST_BRAIN_CHAIN
111
+ outcome = await sales_brain.drive_sales_turn(
112
+ profile=profile,
113
+ history=session.chat_history[-N:],
114
+ user_text=user_text,
115
+ )
116
+ # outcome.reply, outcome.captures, outcome.slot_driving, outcome.complete
117
+ apply_normalized_captures(session, outcome.captures)
118
+ if outcome.complete:
119
+ session.free_form_session = True
120
+ reply = outcome.reply
121
  else:
122
+ # QA / comparison / recommendation Brain Main
123
+ chunks = retrieve(user_query, top_k=10)
124
+ reply = await BRAIN_CHAIN.chat(persona_prompt + history + chunks + user_text)
125
+ if intent not in {"fact_find", "recommendation"}:
126
+ # KI-171: judge skipped on fact-find + recommendation intents
127
+ verdict = await faithfulness.check(reply, chunks)
128
+ if not verdict.supported:
129
+ reply = SAFE_REFUSAL
130
  ```
131
 
 
 
132
  ## 6. What the user sees
133
 
134
+ A natural conversation, not a script. Example (real session, 2026-05-15):
135
+
136
  ```
137
+ Bot: "Hi! I'm your insurance advisor. To get you the right shortlist I'll need
138
+ to learn a bit about you and your situation. What's your name and a rough
139
+ sense of who we're looking to protect?"
140
+ User: "Hi, I'm Rohit Sar. I'm 32, just myself, living in Mumbai."
141
+ Bot: "Nice to meet you, Rohit. Got it 32, covering yourself, Mumbai-based.
142
+ Two more quick things and we can look at policies: roughly what's your
143
+ annual income, and have you had any health conditions diagnosed —
144
+ diabetes, BP, anything chronic?"
145
+ User: "around 18 lakhs, no conditions"
146
+ Bot: "Perfect. And what's drawing you to insurance right now — first policy,
147
+ top-up on employer cover, or something specific?"
148
+ User: "first policy, employer cover feels too thin"
149
+ Bot: "Makes sense. Last one: comfortable annual premium range — under 15K,
150
+ 15-30K, 30-60K, or higher?"
151
+ User: "15-30 is fine"
152
+ Bot: "Great — that's everything I need. Based on your profile, three policies
153
+ stand out: [...]"
154
  ```
155
 
156
+ Note: the LLM captured `{name, age, dependents, location_tier}` in a single turn from the user's opener. Pre-KI-167 each fact required its own turn. The exact turn count varies — some sessions finish in 3-4 turns, others in 6-7 depending on what the user volunteers.
157
+
158
  ## 7. v2 enhancements
159
 
160
  | # | Enhancement | Why |
161
  | --- | --- | --- |
162
+ | 1 | Stream the prose token-by-token to the frontend | The JSON tail is parsed server-side after stream completion would cut perceived latency below the canonical-fallback path |
163
+ | 2 | Promote the same single-call pattern to the QA path | Orchestrator's QA + fact-find branches collapse into one brain call with a richer schema (intent in the JSON block instead of out-of-band classification) |
164
  | 3 | Skip-confirm flow ("you can skip this — say 'skip'") | Buyer autonomy |
165
+ | 4 | Save profile across sessions ✅ shipped KI-040 | Returning user picks up where they left off |
166
+ | 5 | Tone-match the user's energy (formal vs casual) | The LLM already does this implicitly via the system prompt; could be explicit per-session signal |
70-docs/40-evaluation/eval-methodology.md CHANGED
@@ -32,7 +32,7 @@ For every successfully extracted policy in DuckDB, generate templated Q&A where
32
 
33
  ### Pipeline B — LLM-drafted nuanced questions (curated)
34
 
35
- For top-priority policies (top 10–20), run an LLM (Sarvam-M, with DeepSeek-V3 fallback) on the policy text with prompt: *"generate 5 buyer-style questions whose answers are explicitly in this document; include the source clause."* Each generated pair is **human spot-checked** before commit.
36
 
37
  Target: 5 × 20 = 100 questions covering multi-clause and edge-case reasoning that Pipeline A can't generate.
38
 
@@ -56,9 +56,9 @@ Each pair marked with `expected_refusal: bool`. Refusal cases test that the bot
56
 
57
  Numbers, dates, currency, durations, percentages are extracted via regex from both gold and bot reply. Exact match (after normalization) is the strongest signal. Catches "the premium is ₹15,000" hallucinations without an LLM.
58
 
59
- ### Signal 2 — LLM-judge faithfulness (Groq Llama-3.3-70B)
60
 
61
- Different model family from Sarvam-M (the brain) → **non-circular evaluation**. Judge prompt:
62
 
63
  > Given GOLD and BOT answers, output strict JSON: `{factual_match: bool, citation_present: bool, score: 0-1, reason: str}`. Be strict on partial matches.
64
 
@@ -76,7 +76,7 @@ Different model family from Sarvam-M (the brain) → **non-circular evaluation**
76
  | Citation accuracy | C3 ≥ 95% | n_correct_citations / n_non_refusal |
77
  | Refusal precision | C4 ≥ 90% | n_correct_refusals / n_expected_refusals |
78
  | Hindi parity | C8 within 5pp | factual_acc(hi) vs factual_acc(en) |
79
- | Brain winners | (router config) | factual_acc grouped by brain_used |
80
 
81
  ## 4. Output artifacts per run
82
 
@@ -100,6 +100,6 @@ Bilingual sub-set: 20 questions translated to Hindi + Hinglish via Sarvam-M with
100
  ## 7. Known limitations (transparent)
101
 
102
  - **Sample bias:** Pipeline A questions are templated, so accuracy on Pipeline A is upper-bound real-world performance. Pipeline B + C provide the harder signal.
103
- - **Single judge model:** Groq Llama is one judge. Risk of judge-specific bias. v2: 3-judge consensus.
104
  - **No human evaluation:** Cost-prohibitive for 1,100 questions. We rely on the grader; manual spot-check 5%.
105
  - **No latency budget enforcement in eval:** Latency is captured per record but doesn't gate the score. Doc 01 C1 (p50 ≤ 4s) is monitored separately.
 
32
 
33
  ### Pipeline B — LLM-drafted nuanced questions (curated)
34
 
35
+ For top-priority policies (top 10–20), run `NimChainLLM(BRAIN_CHAIN)` on the policy text with prompt: *"generate 5 buyer-style questions whose answers are explicitly in this document; include the source clause."* Each generated pair is **human spot-checked** before commit.
36
 
37
  Target: 5 × 20 = 100 questions covering multi-clause and edge-case reasoning that Pipeline A can't generate.
38
 
 
56
 
57
  Numbers, dates, currency, durations, percentages are extracted via regex from both gold and bot reply. Exact match (after normalization) is the strongest signal. Catches "the premium is ₹15,000" hallucinations without an LLM.
58
 
59
+ ### Signal 2 — LLM-judge faithfulness (JUDGE_CHAIN: NIM Mistral Large 3 675B primary)
60
 
61
+ Different model family from the Gemini-led Brain Fast / Brain Main → **non-circular evaluation**. Falls through NIM Llama-4 Maverick → OR Qwen 80B `:free` → NIM Nemotron 49B as last resort. Judge prompt:
62
 
63
  > Given GOLD and BOT answers, output strict JSON: `{factual_match: bool, citation_present: bool, score: 0-1, reason: str}`. Be strict on partial matches.
64
 
 
76
  | Citation accuracy | C3 ≥ 95% | n_correct_citations / n_non_refusal |
77
  | Refusal precision | C4 ≥ 90% | n_correct_refusals / n_expected_refusals |
78
  | Hindi parity | C8 within 5pp | factual_acc(hi) vs factual_acc(en) |
79
+ | Brain winners | (router config) | factual_acc grouped by `brain_used` — emitted values include `sales_brain::continue`, `sales_brain::complete`, `brain_main::<elected_model>`, `brain_main::graceful_error` |
80
 
81
  ## 4. Output artifacts per run
82
 
 
100
  ## 7. Known limitations (transparent)
101
 
102
  - **Sample bias:** Pipeline A questions are templated, so accuracy on Pipeline A is upper-bound real-world performance. Pipeline B + C provide the harder signal.
103
+ - **Single judge model:** NIM Mistral Large 3 675B is the elected judge primary; the fallback ladder includes Llama-4 Maverick + OR Qwen 80B + NIM Nemotron 49B for resilience but every grade run uses a single judge per question. Risk of judge-specific bias. v2: 3-judge consensus.
104
  - **No human evaluation:** Cost-prohibitive for 1,100 questions. We rely on the grader; manual spot-check 5%.
105
  - **No latency budget enforcement in eval:** Latency is captured per record but doesn't gate the score. Doc 01 C1 (p50 ≤ 4s) is monitored separately.
70-docs/40-evaluation/known-issues.md CHANGED
@@ -376,6 +376,58 @@ visually misleading — looks like a real store, isn't.
376
  **Recommendation:** (A) — once activated, the marketplace tab gets
377
  proper SQL filtering that's faster than the current N-JSON-load pattern.
378
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
379
  ### KI-019 — `kb/policies/*.md` not embedded (224 human-readable summaries left out of retrieval)
380
 
381
  **Severity:** P3
 
376
  **Recommendation:** (A) — once activated, the marketplace tab gets
377
  proper SQL filtering that's faster than the current N-JSON-load pattern.
378
 
379
+ ### KI-167 — Scripted `<FF>` trailer + canonical_fallback removed in favour of LLM-driven `sales_brain` — **FIXED** (ADR-039)
380
+
381
+ **Severity:** P0
382
+ **Source:** `backend/fact_find_brain.py` + `_canonical_fallback` + `_pick_opener` family in `backend/orchestrator.py`
383
+ **Discovered:** 2026-05-15 — user report *"zero natural LLM chat — it always defaults to the script."*
384
+
385
+ Pre-fix, every fact-find turn ran through ADR-030's one-call brain with a `<FF>...</FF>` trailer convention; on parse failure or contract violation the orchestrator fell to `_canonical_fallback`, which emitted the scripted `Question.prompt_en` of the next unfilled slot prefixed with `"Got that — {slot}."`. Real LLMs (Qwen, Nemotron, Mistral) under load regularly dropped the trailer tags (KI-090 lenient parser bought partial recovery; KI-155 / KI-156 / KI-158 / KI-161 showed it was structural) and silently violated the contract, so the scripted prefix became the dominant user-facing path.
386
+
387
+ **Fix:** Ripped out `backend/fact_find_brain.py` (441 LOC) + `_canonical_fallback` + `_pick_opener` + `_NEUTRAL_OPENERS` + `_FAMILY_OPENERS` + `_contains_self_introduction`. Replaced with `backend/sales_brain.py` — one LLM call per turn against `FAST_BRAIN_CHAIN` using native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM). New deterministic post-processor `backend/sales_brain_normalizer.py` validates and normalises the captured fields. KI-091 / KI-094 None-guards retained. See [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md).
388
+
389
+ ---
390
+
391
+ ### KI-168 — Hybrid voice capture (Web Speech + MediaRecorder + Sarvam) — **FIXED**
392
+
393
+ **Severity:** P1
394
+ **Source:** `frontend/src/lib/useStreamingVoice.ts`
395
+ **Discovered:** 2026-05-15
396
+
397
+ Pre-fix voice UX waited for the full Sarvam STT round-trip before showing the user their transcribed text — felt sluggish on the chat surface. New hybrid: Web Speech API streams interim transcripts into the chat input live, while `MediaRecorder` captures the authoritative audio blob in parallel; on browser silence-detect the blob is POSTed to `/api/transcribe` (Sarvam Saarika STT) for the authoritative transcript. Auto-submit falls back to the Web Speech transcript only if Sarvam errors. KI-173 heartbeat + KI-174 `visibilitychange` / `focus` revival hooks keep the mic alive across tab and app switches.
398
+
399
+ ---
400
+
401
+ ### KI-171 — Skip faithfulness judge on `fact_find` + `recommendation` intents — **FIXED**
402
+
403
+ **Severity:** P1
404
+ **Source:** `backend/orchestrator.py`
405
+ **Discovered:** 2026-05-15
406
+
407
+ Faithfulness Gate 4 (LLM judge) was being run on every turn including fact-find and recommendation, where there's no retrieved-chunks context to grade the reply against — burning judge calls and occasionally blocking valid replies. Now gated behind `intent ∈ {qa, comparison}` only. Recommendation replies still go through Gates 1-3 (retrieval floor, citation integrity, regex numeric grounding).
408
+
409
+ ---
410
+
411
+ ### KI-175 / KI-176 — NIM chain reorder + OpenRouter re-added as cross-provider diversity — **FIXED**
412
+
413
+ **Severity:** P2
414
+ **Source:** Chain definitions in `backend/providers/nvidia_nim_llm.py`
415
+ **Discovered:** 2026-05-15
416
+
417
+ KI-175 demoted NIM Nemotron 49B from primary to last resort across all three chains — Mistral Large 3 675B and Qwen 80B consistently outperform on the live audit. KI-176 re-added OpenRouter as a cross-provider diversity tier using OR's `models: [...]` array param for server-side fallback within the OR free pool. The NIM-only lock from [ADR-038](../60-decisions/ADR-038-nim-only-chains.md) was relaxed in scope (the `<FF>` trailer convention that motivated it no longer exists post-KI-167); only OR `:free` candidates with verified native JSON mode are included (Nemotron-3-Super 120B, Qwen 80B `:free`).
418
+
419
+ ---
420
+
421
+ ### KI-178 / KI-179 — Google Gemini Flash added as Tier 0 primary — **FIXED** (ADR-040)
422
+
423
+ **Severity:** P2
424
+ **Source:** `backend/providers/google_gemini_llm.py` (new); chain definitions in `nvidia_nim_llm.py`
425
+ **Discovered:** 2026-05-15
426
+
427
+ Operator obtained a Google AI Studio API key. KI-178 audited which OpenRouter free-tier models support `response_format` (only Nemotron-3-Super 120B + Qwen 80B + Gemma-4 31B). KI-179 added a new `LLMProvider` against Google AI Studio's REST API (`gemini-2.0-flash` + `gemini-2.5-flash`, native JSON mode via `response_mime_type=application/json`, 1500 req/day free quota). Promoted to Tier 0 primary on Brain Fast (Gemini 2.0 Flash) + Brain Main (Gemini 2.5 Flash). NIM stays as Tier 1 fallback with Mistral 675B as the strongest non-Gemini candidate; OpenRouter sits as Tier 2 diversity. Judge stays on NIM Mistral 675B (different family from Gemini — preserves brain ↔ judge family-diversity invariant). See [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md).
428
+
429
+ ---
430
+
431
  ### KI-019 — `kb/policies/*.md` not embedded (224 human-readable summaries left out of retrieval)
432
 
433
  **Severity:** P3
70-docs/40-evaluation/quality-sprint-2026-05-14.md CHANGED
@@ -85,3 +85,45 @@ ffefacf KI-013 reinforcement — persona rules 8 + 9 for demographic-aware recs
85
 
86
  Every commit pushed to BOTH `origin` (HF Space) and `github` per the
87
  triple-mirror contract in [ADR-024](../60-decisions/ADR-024-triple-mirror-code-and-data.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
 
86
  Every commit pushed to BOTH `origin` (HF Space) and `github` per the
87
  triple-mirror contract in [ADR-024](../60-decisions/ADR-024-triple-mirror-code-and-data.md).
88
+
89
+ ---
90
+
91
+ ## Follow-on sprint — 2026-05-15 (KI-167 → KI-179)
92
+
93
+ A second-day sprint after live testing flagged that the scripted `<FF>` trailer convention was leaking into every fallback turn ("zero natural LLM chat — it always defaults to the script"). The day produced one ADR (ADR-039), one provider integration (Google AI Studio), and a chain-level rewrite (ADR-040).
94
+
95
+ ### Findings & fixes by source
96
+
97
+ | KI | Severity | Issue | Resolution |
98
+ |---|---|---|---|
99
+ | KI-167 | P0 | `<FF>` trailer + `_canonical_fallback` dominated user-visible turns when LLM contract violations occurred; bot felt robotic | Ripped out `fact_find_brain.py` + canonical_fallback + scripted prompts; new `backend/sales_brain.py` with one LLM call/turn using native provider JSON mode. ADR-039. |
100
+ | KI-168 | P1 | Voice UX waited for full Sarvam STT round-trip before showing transcript | Hybrid Web Speech (interim) + MediaRecorder (authoritative blob → Sarvam STT on silence-detect). |
101
+ | KI-171 | P1 | Faithfulness judge ran on `fact_find` + `recommendation` turns where there's no retrieval context | Skip Gate 4 on those intents; Gates 1-3 still run. |
102
+ | KI-173 | P2 | Mic died when user switched tabs / minimised the app | Heartbeat keeps mic stream alive. |
103
+ | KI-174 | P2 | Mic state did not recover on `visibilitychange` / `focus` | Revival hooks reattach the mic on tab return. |
104
+ | KI-175 | P2 | NIM Nemotron 49B was a chain primary but consistently underperformed Mistral 675B / Qwen 80B | Demoted Nemotron 49B to last resort across all chains. |
105
+ | KI-176 | P2 | OpenRouter dropped from chains in ADR-038 was no longer needed after ADR-039 retired `<FF>` | Re-added OR `:free` candidates with verified JSON mode as cross-provider diversity. |
106
+ | KI-178 | P2 | Audit of which OR `:free` models support `response_format` | Llama 3.3 70B / Hermes 3 405B excluded (no native JSON mode); Nemotron-3-Super 120B + Qwen 80B `:free` + Gemma-4 31B included. |
107
+ | KI-179 | P2 | Google AI Studio key obtained; need to wire it into chains | New `backend/providers/google_gemini_llm.py`; Gemini 2.0 Flash → Brain Fast primary, Gemini 2.5 Flash → Brain Main primary. ADR-040. |
108
+
109
+ ### What changed in the architecture
110
+
111
+ - **Brain Fast (sales_brain):** Gemini 2.0 Flash → NIM Qwen 80B → NIM Mistral Large 3 675B → NIM Llama-4 Maverick → OR Nemotron-3-Super 120B → OR Qwen 80B `:free` → NIM Nemotron 49B (last resort).
112
+ - **Brain Main:** Gemini 2.5 Flash → NIM Mistral Large 3 675B → NIM Llama-4 Maverick → NIM Qwen 80B → OR Nemotron-3-Super → NIM Nemotron 49B (last resort).
113
+ - **Judge:** NIM Mistral Large 3 675B → NIM Llama-4 Maverick → OR Qwen 80B `:free` → NIM Nemotron 49B (last resort).
114
+
115
+ ### What was deleted
116
+
117
+ - `backend/fact_find_brain.py` (441 LOC).
118
+ - `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction` in `backend/orchestrator.py`.
119
+ - The `<FF>{...}</FF>` trailer convention + lenient parser ladder + `:no_trailer` / `:empty_reply` / `:llm_error` telemetry variants from `TurnResult.brain_used`.
120
+ - Dead `prompt_en` / `prompt_hi` strings in `backend/needs_finder.py::GRAPH` are no longer rendered (data structure retained for the LLM's system-prompt schema).
121
+
122
+ ### What stayed
123
+
124
+ - KI-080 sticky-primary election machinery (now scores Google / NIM / OpenRouter candidates uniformly).
125
+ - KI-084 per-phase httpx timeouts.
126
+ - KI-085 proactive credit gating (extended to Google quota).
127
+ - KI-091 / KI-094 None-guards on the profile extractor (still relevant for QA-mode turns).
128
+ - KI-102 / KI-107 session-isolated profile RAG.
129
+ - KI-106 graceful `TimeoutError` + `Exception` handling on `/api/chat`.
70-docs/60-decisions/ADR-014-groq-llama-grader.md CHANGED
@@ -1,8 +1,12 @@
1
  # ADR-014: Groq Llama-3.3-70B as eval grader
2
 
3
- **Status:** Superseded by [ADR-019](ADR-019-nim-single-provider-consolidation.md)
4
  **Date:** 2026-05-13
5
 
 
 
 
 
6
  ## Context
7
 
8
  The eval harness needed a grading LLM that was NOT the same family as the brain LLM (to avoid circular evaluation). Available options:
 
1
  # ADR-014: Groq Llama-3.3-70B as eval grader
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) (2026-05-15) — Groq is no longer in any chain. The judge primary is NIM Mistral Large 3 675B (different family from the Gemini brain — preserves the non-circular cross-family principle this ADR introduced). Originally superseded by [ADR-019](ADR-019-nim-single-provider-consolidation.md).
4
  **Date:** 2026-05-13
5
 
6
+ ## Why supersession (current)
7
+
8
+ Groq was removed from all chains in [ADR-038](ADR-038-nim-only-chains.md) (KI-160) after KI-155 demonstrated Groq Llama-3.3-70B silently ignores structured-output contracts. The brain ↔ judge family-diversity principle this ADR introduced is now expressed differently — the brain is Gemini (Google family), the judge is Mistral (Mistral family) — but the architectural intent (judge does not mark its own homework) is unchanged.
9
+
10
  ## Context
11
 
12
  The eval harness needed a grading LLM that was NOT the same family as the brain LLM (to avoid circular evaluation). Available options:
70-docs/60-decisions/ADR-016-hybrid-brain-router.md CHANGED
@@ -1,8 +1,12 @@
1
  # ADR-016: Hybrid brain router (Sarvam primary + Llama / DeepSeek fallback)
2
 
3
- **Status:** Superseded by [ADR-019](ADR-019-nim-single-provider-consolidation.md)
4
  **Date:** 2026-05-13
5
 
 
 
 
 
6
  ## Context
7
 
8
  A single brain LLM was insufficient: Sarvam-M's 2048 output cap + `<think>` tokens caused mid-JSON truncation in extraction and mid-answer truncation in advisory. But Sarvam-first is non-negotiable narrative for the assignment.
 
1
  # ADR-016: Hybrid brain router (Sarvam primary + Llama / DeepSeek fallback)
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) (2026-05-15) — the cross-provider router intent survives, but the candidate pool now leads with Google AI Studio (Gemini 2.0 / 2.5 Flash) on Brain Fast / Brain Main with NIM Mistral / Maverick / Qwen and OpenRouter `:free` as cross-provider diversity. Originally superseded by [ADR-019](ADR-019-nim-single-provider-consolidation.md).
4
  **Date:** 2026-05-13
5
 
6
+ ## Why superseded
7
+
8
+ The original Sarvam-primary + Llama/DeepSeek-fallback router was retired in two steps. ADR-019 first collapsed cross-provider sprawl onto a single NIM key. ADR-040 then re-opened the cross-provider surface — but on a different basis: native JSON-mode contract validation (`response_mime_type=application/json` / `response_format={"type":"json_object"}`) replaces empirical query-class routing as the candidate-selection rule. The current chain is Google → NIM → OpenRouter, scored by KI-080 probe-driven election rather than a static "Indic → Sarvam, comparison → DeepSeek" heuristic.
9
+
10
  ## Context
11
 
12
  A single brain LLM was insufficient: Sarvam-M's 2048 output cap + `<think>` tokens caused mid-JSON truncation in extraction and mid-answer truncation in advisory. But Sarvam-first is non-negotiable narrative for the assignment.
70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md CHANGED
@@ -1,6 +1,6 @@
1
  # ADR-019: NVIDIA NIM as the single non-Sarvam provider
2
 
3
- **Status:** Locked
4
  **Date:** 2026-05-14
5
 
6
  **Partial supersession (D-022, 2026-05-14):** NIM's DeepSeek V4-Pro / V4-Flash and Llama-4 Maverick inference pools degraded with hours-long timeouts during a production incident. The chains were re-ordered to put **Qwen 3-Next 80B** (brain primary), **Nemotron Nano 30B** (fast brain primary), and **Mistral Large 3 675B** (judge primary) at the head, with DeepSeek + Maverick retained as fallback chain entries that re-enter rotation when those pools recover. The single-provider consolidation decision in this ADR still stands (NIM remains the primary inference provider); only the *specific model picks* were swapped. See `backend/providers/nvidia_nim_llm.py` for the current chain definitions.
 
1
  # ADR-019: NVIDIA NIM as the single non-Sarvam provider
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) (2026-05-15) — the single-provider stance is retired. Google AI Studio (Gemini 2.0 / 2.5 Flash, 1500 req/day free, native JSON mode) is now Tier 0 primary on Brain Fast + Brain Main; NIM stays as Tier 1 fallback (Mistral Large 3 675B + Llama-4 Maverick + Qwen 80B) and is still the judge primary (cross-family from Gemini). OpenRouter `:free` is Tier 2 diversity. ADR-019's "cross-family judge" + "single-key per provider" + "$0 free-tier" principles are preserved across the multi-provider stack.
4
  **Date:** 2026-05-14
5
 
6
  **Partial supersession (D-022, 2026-05-14):** NIM's DeepSeek V4-Pro / V4-Flash and Llama-4 Maverick inference pools degraded with hours-long timeouts during a production incident. The chains were re-ordered to put **Qwen 3-Next 80B** (brain primary), **Nemotron Nano 30B** (fast brain primary), and **Mistral Large 3 675B** (judge primary) at the head, with DeepSeek + Maverick retained as fallback chain entries that re-enter rotation when those pools recover. The single-provider consolidation decision in this ADR still stands (NIM remains the primary inference provider); only the *specific model picks* were swapped. See `backend/providers/nvidia_nim_llm.py` for the current chain definitions.
70-docs/60-decisions/ADR-026-provider-load-balancing.md CHANGED
@@ -1,8 +1,9 @@
1
  # ADR-026 — Brain provider load-balancing (NIM ↔ Groq, 50/50)
2
 
3
- **Status:** Accepted 2026-05-14
 
4
  **Owner:** Rohit Saraf
5
- **Supersedes:** Partial override of [ADR-019](ADR-019-nim-single-provider-consolidation.md)'s single-provider stance
6
 
7
  ## Context
8
 
 
1
  # ADR-026 — Brain provider load-balancing (NIM ↔ Groq, 50/50)
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) (2026-05-15) — the 50/50 random-coin load balance was first deprecated by [ADR-031](ADR-031-sticky-primary-election.md)'s probe-driven sticky-primary election (KI-080) and is now fully retired. The current chain shape (Google AI Studio Tier 0 primary → NIM Tier 1 fallback → OpenRouter `:free` Tier 2 diversity) is decided by per-call election against probe scores rather than a static probability split.
4
+ **Date:** 2026-05-14
5
  **Owner:** Rohit Saraf
6
+ **Originally supersedes:** Partial override of [ADR-019](ADR-019-nim-single-provider-consolidation.md)'s single-provider stance
7
 
8
  ## Context
9
 
70-docs/60-decisions/ADR-030-llm-driven-fact-find.md CHANGED
@@ -1,9 +1,14 @@
1
  # ADR-030 — LLM-driven fact-find (single brain call per turn)
2
 
3
- **Status:** Accepted 2026-05-15
 
4
  **Owner:** Rohit Saraf
5
- **Supersedes:** [ADR-027](ADR-027-fact-find-llm-paraphraser.md) (the LLM paraphraser was a band-aid on the hardcoded-graph stitching; this is the full replacement.)
6
- **Related:** [ADR-022](ADR-022-conversational-profile-updates.md) (free-form profile updates — the answer-side LLM that this ADR generalizes into the whole loop)
 
 
 
 
7
 
8
  ## Context
9
 
 
1
  # ADR-030 — LLM-driven fact-find (single brain call per turn)
2
 
3
+ **Status:** Superseded by [ADR-039](ADR-039-llm-driven-sales-brain.md) (2026-05-15, KI-167) — partially superseded by [ADR-040](ADR-040-google-gemini-primary.md) (chain candidates). The "schema-aware single LLM call per turn" intuition is preserved in `backend/sales_brain.py`; the `<FF>...</FF>` trailer convention, the `_canonical_fallback` greedy slot-walker, and the scripted `Question.prompt_en` prefix are all retired in favour of native provider JSON mode (`response_mime_type=application/json` / `response_format={"type":"json_object"}`).
4
+ **Date:** 2026-05-15
5
  **Owner:** Rohit Saraf
6
+ **Originally supersedes:** [ADR-027](ADR-027-fact-find-llm-paraphraser.md) (the LLM paraphraser was a band-aid on the hardcoded-graph stitching; this was the full replacement.)
7
+ **Related:** [ADR-022](ADR-022-conversational-profile-updates.md) (free-form profile updates — the answer-side LLM that this ADR generalized into the whole loop)
8
+
9
+ ## Why superseded
10
+
11
+ The `<FF>{...}</FF>` trailer convention proved structurally fragile under load — see KI-155 / KI-156 / KI-158 / KI-161. Even with the KI-090 lenient parser ladder, real LLMs (Qwen, Nemotron, Mistral) regularly returned grammatically fine prose with no parseable trailer, causing `_canonical_fallback` to fire and the user to see the scripted `Question.prompt_en` of the next unfilled slot. The scripted prefix became the dominant user-facing path even when the LLM call had "succeeded." ADR-039 retires the trailer convention entirely in favour of native provider JSON mode; ADR-040 expanded the candidate pool to Google AI Studio + NIM + OpenRouter, all of which natively support server-enforced JSON mode.
12
 
13
  ## Context
14
 
70-docs/60-decisions/ADR-031-sticky-primary-election.md CHANGED
@@ -1,6 +1,10 @@
1
  # ADR-031 — Sticky primary election for LLM chains
2
 
3
- **Status:** Superseded by [ADR-038](ADR-038-nim-only-chains.md) — 2026-05-15 (KI-160). Cross-provider election candidates (Groq, OpenRouter) removed from all three chains after KI-155 demonstrated Groq Llama-3.3 silently ignores the `<FF>` trailer contract. Sticky-primary election mechanics still apply, but only within the NIM candidate pool. Body retained below for history.
 
 
 
 
4
 
5
  **Status (original):** Accepted — 2026-05-15
6
  **Owner:** Rohit Saraf
 
1
  # ADR-031 — Sticky primary election for LLM chains
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) — 2026-05-15 (KI-179) on the candidate-pool scope. The sticky-primary election machinery (per-chain probe + scored election + provider-diverse BACKUP) is preserved and now scores Google AI Studio / NIM / OpenRouter candidates uniformly. Previously superseded by [ADR-038](ADR-038-nim-only-chains.md) (KI-160) on the same scope. Body retained below for history.
4
+
5
+ ## Why superseded
6
+
7
+ ADR-038 had locked the candidate pool to NIM-only after KI-155 showed cross-provider candidates silently violated the `<FF>` trailer contract. ADR-039 / KI-167 retired the `<FF>` trailer entirely in favour of native provider JSON mode, which removes the structural reason for the lock. ADR-040 then added Google AI Studio (Gemini 2.0 / 2.5 Flash) as Tier 0 primary on Brain Fast / Brain Main with NIM as Tier 1 fallback and OpenRouter `:free` as Tier 2 diversity. The election scoring (latency × success_rate) and the provider-diverse BACKUP rule are unchanged.
8
 
9
  **Status (original):** Accepted — 2026-05-15
10
  **Owner:** Rohit Saraf
70-docs/60-decisions/ADR-032-llm-chain-architecture.md CHANGED
@@ -1,12 +1,12 @@
1
  # ADR-032 — LLM Chain Architecture Reference
2
 
3
- **Status:** Superseded by [ADR-038](ADR-038-nim-only-chains.md) — 2026-05-15 (KI-160). All three chains locked to NIM candidates only after KI-155. Current candidate lists:
4
 
5
- - **BRAIN:** `nvidia/llama-3.3-nemotron-super-49b-v1.5` (primary), `qwen/qwen3-next-80b-a3b-instruct` (backup), `mistralai/mistral-large-3-675b-instruct-2512` (3rd)
6
- - **FAST_BRAIN:** `qwen/qwen3-next-80b-a3b-instruct` (primary), `nvidia/llama-3.3-nemotron-super-49b-v1.5` (backup)
7
- - **JUDGE:** `meta/llama-4-maverick-17b-128e-instruct` (primary), `mistralai/mistral-large-3-675b-instruct-2512` (backup)
8
 
9
- Body below retained as architectural reference for probe / election / timeout machinery — all still in effect within the NIM-only scope.
10
 
11
  **Status (original):** Accepted — 2026-05-15
12
  **Type:** Architecture reference (not a decision ADR)
 
1
  # ADR-032 — LLM Chain Architecture Reference
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) — 2026-05-15 (KI-179) on the candidate-pool scope. The NIM-only lock from ADR-038 was relaxed once ADR-039 / KI-167 retired the `<FF>` trailer convention. Current candidate lists per ADR-040:
4
 
5
+ - **BRAIN (Fast — sales_brain):** Gemini 2.0 Flash (Google AI Studio Tier 0 primary), then NIM `qwen/qwen3-next-80b-a3b-instruct` NIM `mistralai/mistral-large-3-675b-instruct-2512` → NIM `meta/llama-4-maverick-17b-128e-instruct` → OpenRouter `nvidia/nemotron-3-super-120b-a12b:free` → OpenRouter `qwen/qwen3-next-80b-a3b-instruct:free` → NIM `nvidia/llama-3.3-nemotron-super-49b-v1.5` (last resort).
6
+ - **BRAIN (Main — synthesis / QA / comparison / recommendation):** Gemini 2.5 Flash (Tier 0 primary), then NIM Mistral 675B → NIM Maverick → NIM Qwen 80B → OpenRouter Nemotron-3-Super → NIM Nemotron 49B (last resort).
7
+ - **JUDGE:** NIM `mistralai/mistral-large-3-675b-instruct-2512` (primary), NIM Maverick (backup), OpenRouter Qwen 80B `:free`, NIM Nemotron 49B (last resort).
8
 
9
+ Body below retained as architectural reference for probe / election / timeout / credit-gating machinery — all still in effect across the multi-provider pool. Section 4 "Election algorithm" treats Google / NIM / OpenRouter candidates uniformly via `provider_of()` for the cross-provider BACKUP rule.
10
 
11
  **Status (original):** Accepted — 2026-05-15
12
  **Type:** Architecture reference (not a decision ADR)
70-docs/60-decisions/ADR-038-nim-only-chains.md CHANGED
@@ -1,6 +1,8 @@
1
  # ADR-038 — NIM-only chains (KI-160)
2
 
3
- **Status:** Accepted 2026-05-15
 
 
4
  **Owner:** Rohit Saraf
5
  **Supersedes:** [ADR-031](ADR-031-sticky-primary-election.md) (cross-provider election scope), [ADR-032](ADR-032-llm-chain-architecture.md) (candidate-pool scope)
6
  **Related KIs:** KI-155 (Groq `<FF>` contract violation, root cause), KI-160 (this lock-down), KI-080 / KI-081 / KI-084 / KI-085 / KI-087 (probe + election mechanics, retained within NIM scope)
 
1
  # ADR-038 — NIM-only chains (KI-160)
2
 
3
+ **Status:** Superseded by [ADR-040](ADR-040-google-gemini-primary.md) (2026-05-15). The NIM-only lock was relaxed once KI-167 ([ADR-039](ADR-039-llm-driven-sales-brain.md))'s sales_brain eliminated the `<FF>` trailer convention that originally motivated the lock. Native provider JSON mode (`response_format={"type":"json_object"}` on NIM, `response_mime_type=application/json` on Gemini) is now the structured-output contract, so cross-provider candidates that natively support JSON mode are safe to re-add to the election pool. The "fail-loud > fail-silent-with-garbage" principle and brain ↔ judge family-diversity invariant from this ADR are preserved in ADR-040.
4
+
5
+ **Original status:** Accepted — 2026-05-15
6
  **Owner:** Rohit Saraf
7
  **Supersedes:** [ADR-031](ADR-031-sticky-primary-election.md) (cross-provider election scope), [ADR-032](ADR-032-llm-chain-architecture.md) (candidate-pool scope)
8
  **Related KIs:** KI-155 (Groq `<FF>` contract violation, root cause), KI-160 (this lock-down), KI-080 / KI-081 / KI-084 / KI-085 / KI-087 (probe + election mechanics, retained within NIM scope)
70-docs/60-decisions/ADR-040-google-gemini-primary.md ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-040 — Google Gemini Flash as primary LLM tier (KI-179)
2
+
3
+ **Status:** Accepted — 2026-05-15
4
+ **Owner:** Rohit Saraf
5
+ **Supersedes:** [ADR-038](ADR-038-nim-only-chains.md) (NIM-only chain lock) — partially. The "fail-loud > fail-silent-with-garbage" principle and the brain ↔ judge family-diversity invariant are preserved; the strict NIM-only candidate-pool scope is relaxed now that the `<FF>` trailer convention that motivated the lock no longer exists post-[ADR-039](ADR-039-llm-driven-sales-brain.md) / KI-167.
6
+ **Related KIs:** KI-167 (sales_brain rip-out), KI-171 (judge skip on fact_find + recommendation), KI-175 (NIM chain reorder — nemotron demoted to last), KI-176 (OpenRouter `models: [...]` server-side fallback), KI-178 (live audit of OR free-tier JSON-mode support), KI-179 (this decision — Google AI Studio added as Tier 0).
7
+
8
+ ## Context
9
+
10
+ After KI-167 ([ADR-039](ADR-039-llm-driven-sales-brain.md)) eliminated the `<FF>...</FF>` trailer convention in favour of native provider JSON mode (`response_format={"type":"json_object"}` on NIM, `response_mime_type=application/json` on Gemini), the structural reason for the NIM-only lock dissolved. KI-176 added OpenRouter back as a cross-provider fallback using OR's `models: [...]` array param for server-side fallback within the OR free pool.
11
+
12
+ KI-178 then audited which OpenRouter free-tier models actually support `response_format`:
13
+
14
+ - **No Gemini variants exist on OR's free tier.**
15
+ - **Llama 3.3 70B and Hermes 3 405B on OR free do not expose `response_format`** — JSON would have to be prompt-engineered, which carries a non-trivial parse-failure rate (the very class of silent failure KI-160 / ADR-038 was instituted to prevent).
16
+ - **The only OR free-tier models with native JSON mode** are `nvidia/nemotron-3-super-120b-a12b:free`, `qwen/qwen3-next-80b-a3b-instruct:free`, and `google/gemma-4-31b:free`.
17
+
18
+ In parallel, the operator obtained a Google AI Studio API key. Google's documented free tier on Gemini 2.0 / 2.5 Flash is **1,500 requests/day, 15 requests/minute, native JSON mode via `response_mime_type=application/json`**, with conversational quality that is genuinely better than any model in the existing NIM pool for the sales-brain free-form QA + recommendation surfaces. The free quota is adequate for the demo + early-production workload (a typical fact-find conversation is ~10 turns; 1,500 req/day supports ~150 conversations/day before falling to fallbacks).
19
+
20
+ The chain shape needs to reflect this: a frontier-quality free tier exists, and routing past it would be deliberately ignoring a strictly-better primary.
21
+
22
+ ## Decision
23
+
24
+ **Add Google AI Studio as Tier 0 (primary) on the Brain Fast and Brain Main chains.** NIM stays as Tier 1 fallback with its 675B Mistral Large 3 as the strongest non-Gemini candidate. OpenRouter ($10 stays parked) sits as Tier 2 diversity pool before the nemotron-49b last resort. The judge stays NIM-primary on Mistral Large 3 675B — different family from Gemini, preserving the brain ↔ judge family-diversity invariant.
25
+
26
+ Concretely, the three chains become:
27
+
28
+ ### Brain Fast (fact-find conversation, `backend/sales_brain.py`)
29
+
30
+ 1. **PRIMARY — Google AI Studio:** `gemini-2.0-flash` (1500 req/day free, native JSON mode)
31
+ 2. Fallback 1 — NIM: `qwen3-next-80b-a3b-instruct`
32
+ 3. Fallback 2 — NIM: `mistralai/mistral-large-3-675b-instruct-2512` (675B dense)
33
+ 4. Fallback 3 — NIM: `meta/llama-4-maverick-17b-128e-instruct` (128B MoE)
34
+ 5. Fallback 4 — OpenRouter: `nvidia/nemotron-3-super-120b-a12b:free`
35
+ 6. Fallback 5 — OpenRouter: `qwen/qwen3-next-80b-a3b-instruct:free`
36
+ 7. Last resort — NIM: `nvidia/llama-3.3-nemotron-super-49b-v1.5`
37
+
38
+ ### Brain Main (orchestrator free-form QA + recommendation synthesis)
39
+
40
+ 1. **PRIMARY — Google AI Studio:** `gemini-2.5-flash` (same 1500 req/day quota, higher synthesis quality than 2.0 Flash on long-context recommendation generation)
41
+ 2. Fallback 1 — NIM: `mistralai/mistral-large-3-675b-instruct-2512`
42
+ 3. Fallback 2 — NIM: `meta/llama-4-maverick-17b-128e-instruct`
43
+ 4. Fallback 3 — NIM: `qwen/qwen3-next-80b-a3b-instruct`
44
+ 5. Fallback 4 — OpenRouter: `nvidia/nemotron-3-super-120b-a12b:free`
45
+ 6. Last resort — NIM: `nvidia/llama-3.3-nemotron-super-49b-v1.5`
46
+
47
+ ### Judge (faithfulness Gate 4 — KI-171 skips this on `fact_find` + `recommendation` queries)
48
+
49
+ 1. **PRIMARY — NIM:** `mistralai/mistral-large-3-675b-instruct-2512` (different family from the Gemini brain; preserves cross-family invariant)
50
+ 2. Fallback 1 — NIM: `meta/llama-4-maverick-17b-128e-instruct`
51
+ 3. Fallback 2 — OpenRouter: `qwen/qwen3-next-80b-a3b-instruct:free`
52
+ 4. Last resort — NIM: `nvidia/llama-3.3-nemotron-super-49b-v1.5`
53
+
54
+ A new `backend/providers/google_gemini_llm.py` wrapper (KI-179) implements the `LLMProvider` interface against Google AI Studio's REST API, matching the same `chat()` signature as `NvidiaNimLLM` and `OpenRouterLLM`. `NimChainLLM` is provider-agnostic — chains may now mix Google / NIM / OpenRouter candidates by URL.
55
+
56
+ ## Consequences
57
+
58
+ ### Positive
59
+
60
+ - **Frontier-tier conversational quality on the free path.** Gemini 2.0 Flash + 2.5 Flash are genuinely best-in-class for the sales-brain surface (one-shot JSON-mode replies with tight schemas + natural prose) and the recommendation-synthesis surface (multi-policy comparison + ranked shortlist).
61
+ - **Native JSON mode preserved.** `response_mime_type=application/json` is Google's server-enforced equivalent of NIM's `response_format={"type":"json_object"}` — the [ADR-039](ADR-039-llm-driven-sales-brain.md) structured-output guarantee survives the provider swap.
62
+ - **NIM's massive Mistral Large 3 (675B dense) stays accessible as Fallback 1** on both Brain chains. The judge chain still leads with Mistral 675B, so the rescue-by-judge path remains identical.
63
+ - **OpenRouter $10 diversity pool stays valuable.** The two OR `:free` candidates with verified JSON mode (nemotron-3-super-120b, qwen3-next-80b) cover a third-provider tier under both Google and NIM, so a simultaneous Google + NIM degradation has at least one diverse candidate to elect.
64
+ - **Family diversity preserved at the judge boundary.** Brain primaries are Gemini (Google family); judge primary is Mistral Large 3 675B (Mistral family) — the brain ↔ judge family diversity that [ADR-014](ADR-014-groq-llama-grader.md) / [ADR-032](ADR-032-llm-chain-architecture.md) require is intact.
65
+
66
+ ### Negative
67
+
68
+ - **One additional dependency** — Google AI Studio key (`GOOGLE_API_KEY`). The provider list grows from {NIM, OpenRouter, Sarvam} to {Google, NIM, OpenRouter, Sarvam}.
69
+ - **Free-tier quota cap of 1500 req/day on Gemini Flash.** Adequate for demo + early production but not unbounded. A sustained 100+ concurrent-conversation load would exhaust the quota and force fallback to NIM Mistral 675B.
70
+
71
+ ### Mitigations
72
+
73
+ - **6-level fallback chain on Brain Fast, 5-level on Brain Main.** A Google AI Studio outage / quota exhaustion / 429 simply falls to NIM Mistral 675B — the same provider family that has been production-validated since KI-160.
74
+ - **Quota visibility.** Daily request counter is visible at [aistudio.google.com](https://aistudio.google.com); admin telemetry (KI-086 `/api/admin/llm-health`) surfaces per-candidate health + degraded-until timestamps, so quota-exhaustion is observable in the existing admin panel.
75
+ - **Key rotation procedure exists.** Google AI Studio keys can be rotated in-console; the `GOOGLE_API_KEY` env in HF Space secrets is one-flip to swap.
76
+ - **Probe loop (KI-080 + KI-088 NIM semaphore) extends naturally** to the Google provider — `backend/llm_health.py` scores every candidate uniformly on `(1 / max(50, latency_ms)) * success_rate`; the elector treats provider-of-origin as opaque.
77
+
78
+ ## Alternatives considered
79
+
80
+ - **Ship Llama 3.3 70B / Hermes 3 405B from OpenRouter free tier as the new primary.** Rejected. KI-178 confirmed neither model exposes `response_format` on OR free — JSON would have to be prompt-engineered out of prose, reintroducing the class of silent failure that KI-160 / [ADR-038](ADR-038-nim-only-chains.md) was instituted to prevent. The point of native JSON mode is server-side enforcement; a model without it is structurally worse for our use case regardless of its parameter count.
81
+
82
+ - **Stay NIM-only (KI-175 state — nemotron demoted to last, Mistral 675B as new NIM primary).** Rejected. Live testing showed Gemini Flash conversational quality is genuinely better on the sales-brain surface (more natural opener variety, less repetition, fewer "Got that —" style filler patterns), and the 1500 req/day Google free tier is plentiful for the workload. Routing past a strictly-better free primary would be deliberately ignoring it.
83
+
84
+ - **Buy paid Gemini via OpenRouter's $10 balance.** Rejected. OR's paid Gemini is ~1000–2000 conversations before the $10 refill is needed; direct Google AI Studio is genuinely free at the same quality level. The OR $10 is better spent on the diversity-pool free models (nemotron-3-super-120b, qwen3-next-80b) that round out the fallback chain.
85
+
86
+ - **Make Gemini the judge too (same family across all roles).** Rejected. Brain ↔ judge family diversity ([ADR-014](ADR-014-groq-llama-grader.md) / [ADR-032](ADR-032-llm-chain-architecture.md)) is the structural defence against circular grading — the judge would mark its own homework. Keeping the judge on Mistral Large 3 675B (Mistral family) preserves the non-circular grading invariant when the brain is Gemini.
87
+
88
+ ## Reversal trigger
89
+
90
+ Re-open this ADR if (a) Google AI Studio imposes a paid-tier requirement or removes `response_mime_type` support, (b) the 1500 req/day quota is consistently exhausted before noon UTC over a 7-day window (signalling we need either a paid tier or a different primary), (c) live audit shows Gemini Flash silently violating the JSON-mode contract on any sales-brain or recommendation turn (the failure class KI-160 originally locked NIM against), or (d) the brain ↔ judge family-diversity invariant breaks (e.g. Google ships a judge-grade reasoning model and we're tempted to consolidate). Reversal would shift the Brain primaries back to NIM Mistral 675B with OpenRouter as Tier 1 — i.e. the KI-176 state — not the strict NIM-only lock of KI-160 (the `<FF>` trailer convention that motivated that lock is permanently retired).
80-audit/ENTERPRISE_AUDIT.md CHANGED
@@ -1,7 +1,7 @@
1
  # Enterprise-Grade Readiness Audit
2
 
3
  **Target deployment:** top-tier Indian insurance companies (HDFC ERGO, ICICI Lombard, Bajaj Allianz, Star Health, Tata AIG, etc.)
4
- **Audit date:** 2026-05-14
5
  **Audit window:** active — this is a living document. Background audits (100-persona simulation + full gold-QA eval) running concurrently; results merge here as they land.
6
 
7
  ---
@@ -19,8 +19,9 @@
19
  | Language-handling fairness | ⚠️ Hinglish concern was a 20-persona sampling artifact. Real outliers: tax_planner archetype (4.6 refusals) + stream style (4.3) — open. | P1 — India-market regulatory risk |
20
  | Code hygiene | ✅ Loose tmp files removed; `fact_find_normalizer` + `profile_extractor` migrated to chain pattern (KI-033) | — |
21
  | Test coverage | 🟡 15 unit tests pinning KI-018 / KI-023 / KI-080 routing + primary-election invariants. Broader coverage still open. | P1 |
22
- | Voice UX | ✅ Live default-on + clickable toggle + labeled push-to-talk + barge-in working (ADR-028) | — |
23
- | Fact-find robotic tone | ✅ LLM paraphraser w/ verifier (ADR-027) | — |
 
24
  | Secrets handling | ✅ Verified clean (.env never committed) | — |
25
 
26
  Legend: ✅ fixed / ⚠️ partial / 🟡 improving / 🔴 open
@@ -336,4 +337,21 @@ This audit so far covers items 1-3, 5 (in progress), and 6. Items 7-10 require a
336
 
337
  ---
338
 
339
- *This file regenerates as new evidence lands. Last updated: 2026-05-15 (KI-101..KI-107 landed).*
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # Enterprise-Grade Readiness Audit
2
 
3
  **Target deployment:** top-tier Indian insurance companies (HDFC ERGO, ICICI Lombard, Bajaj Allianz, Star Health, Tata AIG, etc.)
4
+ **Audit date:** 2026-05-14 (initial) · 2026-05-15 (follow-on sprint KI-167 → KI-179 landed)
5
  **Audit window:** active — this is a living document. Background audits (100-persona simulation + full gold-QA eval) running concurrently; results merge here as they land.
6
 
7
  ---
 
19
  | Language-handling fairness | ⚠️ Hinglish concern was a 20-persona sampling artifact. Real outliers: tax_planner archetype (4.6 refusals) + stream style (4.3) — open. | P1 — India-market regulatory risk |
20
  | Code hygiene | ✅ Loose tmp files removed; `fact_find_normalizer` + `profile_extractor` migrated to chain pattern (KI-033) | — |
21
  | Test coverage | 🟡 15 unit tests pinning KI-018 / KI-023 / KI-080 routing + primary-election invariants. Broader coverage still open. | P1 |
22
+ | Voice UX | ✅ Live default-on + clickable toggle + labeled push-to-talk + barge-in working (ADR-028) + hybrid Web Speech + MediaRecorder capture (KI-168) + tab-switch revival (KI-173 / KI-174) | — |
23
+ | Fact-find robotic tone | ✅ Replaced with `sales_brain` LLM-driven loop ([ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md), KI-167) — no scripted prompts, no `<FF>` trailer, native provider JSON mode | — |
24
+ | LLM stack | ✅ 3-tier chain Google → NIM → OpenRouter per [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md). Gemini 2.0/2.5 Flash primary; NIM Mistral 675B fallback; OR `:free` diversity. Judge stays NIM Mistral 675B (cross-family from Gemini). | — |
25
  | Secrets handling | ✅ Verified clean (.env never committed) | — |
26
 
27
  Legend: ✅ fixed / ⚠️ partial / 🟡 improving / 🔴 open
 
337
 
338
  ---
339
 
340
+ ## Sprint 2026-05-15 (KI-167 → KI-179) — chain rewrite + sales_brain rip-out
341
+
342
+ Second-day follow-on sprint after live user testing flagged that the scripted `<FF>` trailer convention was leaking into every fallback turn. Full timeline in [`70-docs/40-evaluation/quality-sprint-2026-05-14.md`](../70-docs/40-evaluation/quality-sprint-2026-05-14.md) (Follow-on sprint section). Highlights against the register:
343
+
344
+ | KI | Impact on register | Note |
345
+ |---|---|---|
346
+ | **KI-167** ([ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) | Closes the "scripted fact-find leakage" defect at root — `backend/fact_find_brain.py` + `_canonical_fallback` + the `<FF>` trailer convention all deleted. Replaced with `backend/sales_brain.py` (one LLM call per turn, native provider JSON mode). KI-090 lenient parser, KI-103 no_trailer loop-breaker, and the entire `fact_find_brain::fallback:*` telemetry vocabulary are retired. | Net negative LOC. |
347
+ | **KI-168 / KI-173 / KI-174** | Voice UX hardening — hybrid Web Speech (live UX) + MediaRecorder (authoritative blob → Sarvam STT) + tab-switch revival hooks. | Frontend-only. |
348
+ | **KI-171** | Skip faithfulness Gate 4 on `fact_find` + `recommendation` intents (no retrieval context to grade against). | Tightens D-006 fairness risk surface. |
349
+ | **KI-175 / KI-176** | NIM chain reorder — Nemotron 49B demoted from primary to last resort; OpenRouter re-added as cross-provider diversity (using OR's `models: [...]` server-side fallback). | Mitigates D-004 latency long-tail. |
350
+ | **KI-178** | Live audit of OR `:free` `response_format` support — Llama 3.3 70B / Hermes 3 405B excluded; only Nemotron-3-Super 120B + Qwen 80B `:free` + Gemma-4 31B included. | Closes a silent-failure class proactively. |
351
+ | **KI-179** ([ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)) | New `backend/providers/google_gemini_llm.py`; Gemini 2.0 Flash → Brain Fast primary, Gemini 2.5 Flash → Brain Main primary. NIM Tier 1 fallback, OR `:free` Tier 2 diversity. Judge stays on NIM Mistral 675B (different family from Gemini — preserves brain ↔ judge family-diversity invariant). | Tier 0 free-tier conversational quality lift. |
352
+
353
+ The 2026-05-14 scoreboard rows for "Voice UX", "Fact-find robotic tone", and the new "LLM stack" row are now fully green. D-003 (factual accuracy) and D-004 (latency) remain partially open pending a fresh full-eval re-run on the new chain shape.
354
+
355
+ ---
356
+
357
+ *This file regenerates as new evidence lands. Last updated: 2026-05-15 (sprint KI-167 → KI-179 landed; ADR-039 + ADR-040 ship).*
ARCHITECTURE.md CHANGED
@@ -1,50 +1,95 @@
1
- Insurance Sales Portfolio Expert: Architecture & Design
2
-
3
- This document details the technical architecture, operational workflows, and future roadmap for the Insurance Sales AI Agent.
4
-
5
- ---
6
-
7
- ## System Architecture
8
-
9
- The application is built using a modular approach to ensure simplicity, explainability and high performance.
10
-
11
- ### 1. Frontend & Orchestration
12
- - **Technology**:
13
- - **Role**: Serves as the user interface and central controller. It manages:
14
- - **Session State**: Tracks chat history, processed audio IDs, and indexed document status.
15
- - **Resource Caching**: Uses `@st.cache_resource` to ensure heavy models (Whisper, RAG, LLM) are loaded only once.
16
- - **Component Coordination**: Pipes data from the microphone to Whisper, then to the RAG engine, then to the LLM, and finally to the TTS engine.
17
-
18
- ### 2. Retrieval-Augmented Generation
19
- - **Technology**:
20
- - **Embeddings**:
21
- - **Workflow**:
22
- - **Ingestion**:
23
- - **Metadata Tagging**: Documents are tagged as either `Product` (policies) or `Regulatory` (annual reports).
24
- - **Vector Storage**:
25
- - **Retrieval**:
26
-
27
- ### 3. Language Processing
28
- - **Technology**:
29
- - **Role**: The "reasoning" engine.
30
- - **System Prompting**: Enforces a persona of a "Professional Insurance Advisor." It is explicitly instructed to:
31
- - Compare multiple policies.
32
- - Use specific citation tags (e.g., `[Source: ...]`).
33
- - Be concise for voice-based interactions.
34
-
35
- ### 4. Voice Pipeline
36
- - **STT (Speech-to-Text)**:
37
- - **TTS (Text-to-Speech)**:
38
-
39
- ---
40
-
41
- ## How It Works (Step-by-Step)
42
-
43
- 1. **Document Indexing**: User uploads a PDF.
44
- 2. **Voice Input**: Captures raw audio.
45
- 3. **Transcription**: Converts the audio bytes into text.
46
- 4. **Contextual Retrieval**: The text query is embedded and matched against the DB. The engine returns the most relevant snippets along with their source.
47
- 5. **Response Generation**: The LLM receives the user's question and the retrieved snippets. It synthesises a natural language answer.
48
- 6. **Voice Synthesis**: The answer is converted to voice
49
- 7. **Playback**: Audio component plays the response automatically
50
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Insurance Sales Portfolio Expert Architecture & Design
2
+
3
+ A condensed, current architecture map of the Insurance Sales AI Agent. For
4
+ authoritative system-level detail see [`70-docs/10-architecture/system-overview.md`](70-docs/10-architecture/system-overview.md);
5
+ for routing + chain mechanics see
6
+ [`70-docs/60-decisions/ADR-040-google-gemini-primary.md`](70-docs/60-decisions/ADR-040-google-gemini-primary.md).
7
+
8
+ ---
9
+
10
+ ## System architecture
11
+
12
+ ### 1. Frontend
13
+ - **Technology:** Next.js 14 (App Router) + Tailwind v4 + shadcn/ui — deployed
14
+ to the HF Space static surface.
15
+ - **Voice capture (post-KI-168 hybrid):** Web Speech API streams interim
16
+ transcripts into the chat input for live UX feel, while `MediaRecorder` runs
17
+ in parallel and captures the authoritative audio blob. On browser
18
+ silence-detect the blob is POSTed to `/api/transcribe` (Sarvam Saarika STT,
19
+ the authoritative transcript); auto-submit then falls back to the Web Speech
20
+ transcript only if Sarvam errors. KI-173 heartbeat + KI-174
21
+ `visibilitychange` / `focus` revival keep the mic alive across tab and app
22
+ switches.
23
+
24
+ ### 2. Backend (FastAPI orchestrator)
25
+ - **Technology:** FastAPI + Pydantic. Single per-turn entry `POST /api/chat`
26
+ fans out through `backend/orchestrator.py`.
27
+ - **Sales brain (`backend/sales_brain.py`, KI-167):** one LLM call per
28
+ fact-find turn against `FAST_BRAIN_CHAIN`. Uses native provider JSON mode
29
+ (`response_mime_type=application/json` on Gemini,
30
+ `response_format={"type":"json_object"}` on NIM / OpenRouter) no `<FF>`
31
+ trailer convention, no scripted canonical fallback. See
32
+ [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md).
33
+ - **Faithfulness:** 4-gate guard in `backend/faithfulness.py` (retrieval floor
34
+ → citation integrity → regex numeric grounding → cross-family judge).
35
+ Skipped on `fact_find` + `recommendation` intents per KI-171.
36
+
37
+ ### 3. LLM chains (3-tier, post-ADR-040)
38
+ | Role | Tier 0 (primary) | Tier 1 (NIM fallback) | Tier 2 (OR diversity) | Last resort |
39
+ |---|---|---|---|---|
40
+ | Brain Fast (sales_brain) | Gemini 2.0 Flash | Qwen 3-Next 80B → Mistral Large 3 675B → Llama-4 Maverick | OR Nemotron-3-Super 120B → OR Qwen 80B `:free` | NIM Nemotron 49B |
41
+ | Brain Main (synthesis) | Gemini 2.5 Flash | Mistral Large 3 675B → Llama-4 Maverick → Qwen 80B | OR Nemotron-3-Super 120B | NIM Nemotron 49B |
42
+ | Judge (faithfulness Gate 4) | NIM Mistral Large 3 675B | Llama-4 Maverick | OR Qwen 80B `:free` | NIM Nemotron 49B |
43
+
44
+ Provider keys: `GOOGLE_API_KEY` (1500 req/day free), `NVIDIA_NIM_API_KEY`
45
+ (free, 40 req/min), `OPENROUTER_API_KEY` ($10 unlocks 1000 req/day on `:free`
46
+ models). Brain judge family-diversity invariant is preserved Gemini-brain
47
+ calls are graded by Mistral.
48
+
49
+ ### 4. Retrieval & data
50
+ - **Vector store:** Chroma persistent client at `rag/vectors/` (BGE-small-en-v1.5
51
+ local 384-d embeddings — see [ADR-011](70-docs/60-decisions/ADR-011-bge-local-embeddings.md)).
52
+ - **Structured store:** DuckDB rollup of 62-field `HealthPolicy` extractions
53
+ (`rag/policies.duckdb`); the source-of-truth JSONs live in
54
+ `rag/extracted/<policy_id>.json` and curated facts in `40-data/policy_facts/`.
55
+ - **Profile RAG:** session profile is embedded as a Chroma chunk with
56
+ `session_id` metadata; KI-102 enforces strict session isolation on lookup.
57
+
58
+ ### 5. Voice pipeline
59
+ - **STT:** Sarvam Saarika v2.5 (`/api/transcribe`).
60
+ - **TTS:** Sarvam Bulbul v2 — bot replies normalised by
61
+ `backend/voice_format.py::tts_preprocess` (markdown stripped, CoT leakage
62
+ killed per KI-104, money shorthand expanded per KI-066).
63
+
64
+ ---
65
+
66
+ ## How a single turn flows
67
+
68
+ 1. **Voice input** — browser captures audio with `MediaRecorder` while Web
69
+ Speech provides interim text for the UI.
70
+ 2. **Transcription** — silence-detect POSTs the blob to `/api/transcribe`
71
+ (Sarvam Saarika STT); Web Speech transcript is the fallback.
72
+ 3. **Chat call** — `POST /api/chat` runs `classify_intent`,
73
+ profile-RAG injection, and dispatches to `sales_brain` (fact-find) or to
74
+ retrieval + Brain Main (QA / comparison / recommendation).
75
+ 4. **Brain call** — sticky-primary election (KI-080) picks one candidate from
76
+ the relevant chain; native JSON mode guarantees parseable output.
77
+ 5. **Faithfulness gates** — retrieval floor + citation integrity + regex
78
+ numeric grounding + (cross-family) judge LLM. Blocked replies land in
79
+ `logs/hallucinations.jsonl`.
80
+ 6. **TTS synthesis** — Sarvam Bulbul returns base64 audio in the same JSON
81
+ response; frontend plays it through the in-DOM `<audio>` element so
82
+ barge-in's `querySelectorAll("audio").pause()` actually finds it.
83
+ 7. **Telemetry** — every turn writes a JSONL record (intent, brain_used,
84
+ latency, faithfulness verdict, cost estimate) for replay + audit.
85
+
86
+ ---
87
+
88
+ ## Where to read next
89
+
90
+ - [`70-docs/10-architecture/system-overview.md`](70-docs/10-architecture/system-overview.md) — full system map with the canonical data flow.
91
+ - [`70-docs/10-architecture/stack-rationale.md`](70-docs/10-architecture/stack-rationale.md) — every technology pick with alternatives + reasoning.
92
+ - [`70-docs/60-decisions/ADR-040-google-gemini-primary.md`](70-docs/60-decisions/ADR-040-google-gemini-primary.md) — the current chain shape.
93
+ - [`70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md`](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) — fact-find LLM-driven sales brain.
94
+ - [`backend/README.md`](backend/README.md) — per-module entry points.
95
+ - [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md) — live defect register + production-readiness scorecard.
CLAUDE.md CHANGED
@@ -18,19 +18,23 @@ This file is read by Claude Code (and any compatible AI tool) at the start of a
18
  - **Hands-free was removed entirely** in KI-027. Anything in the codebase still referring to it is stale.
19
  - **Bot TTS plays via the in-DOM `<audio>` element** inside `Message` (autoplay-on-mount via ref'd `useEffect`). Never use `new Audio(url).play()` — those detached instances are invisible to `document.querySelectorAll("audio").pause()` in the barge-in handler.
20
 
21
- ## LLM stack (ADR-019 + ADR-026 → ADR-031 + ADR-032 → ADR-038) — KI-080 → KI-087, KI-160
22
 
23
- Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model. End-to-end spec: [ADR-032](70-docs/60-decisions/ADR-032-llm-chain-architecture.md). All three chains are **locked to NIM candidates only** as of KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) — Groq + OpenRouter were removed as election candidates after KI-155 proved Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract. Chains still preserve brain ↔ judge family diversity (Qwen / Nemotron brain ↔ Mistral / Llama-4 judge) so failovers can't produce circular grading.
24
 
25
- - **NIM-only election (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** Candidate pools per role: BRAIN — `nvidia/llama-3.3-nemotron-super-49b-v1.5` (primary), `qwen/qwen3-next-80b-a3b-instruct` (backup), `mistralai/mistral-large-3-675b-instruct-2512` (3rd). FAST_BRAIN — `qwen/qwen3-next-80b-a3b-instruct` (primary), `nvidia/llama-3.3-nemotron-super-49b-v1.5` (backup). JUDGE — `meta/llama-4-maverick-17b-128e-instruct` (primary), `mistralai/mistral-large-3-675b-instruct-2512` (backup). If all NIM candidates fail in a turn, orchestrator returns a graceful error message instead of cascading to Groq / OpenRouter — **fail-loud > fail-silent-with-garbage**.
 
 
 
 
26
  - **Probe-driven sticky primary election (KI-080, [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md), superseded by ADR-038 for candidate-pool scope).** All three chains elect a sticky PRIMARY + BACKUP from a background probe within the NIM pool. `backend/llm_health.py` scores every candidate on `(1 / max(50, latency_ms)) * success_rate` and writes the current election to process state. `NimChainLLM.chat()` calls PRIMARY once; on real-time failure it falls to BACKUP and triggers an immediate probe refresh. **Per-turn LLM call count: 1 (most cases) or 2 (PRIMARY fails real-time → BACKUP).**
27
  - **Probe cadence + per-phase timeouts (KI-084, `119e0fd`).** Probe loop ticks at `PROBE_INTERVAL_SEC = 300s`. Probe `max_tokens` cut `5 → 1`. Every chat call uses explicit `httpx.Timeout(connect=2, read=12, write=2, pool=2)` so a stuck NIM pool releases its TCP socket independently of the outer `asyncio.wait_for`. Rate-limit failures (HTTP 429 / `RateLimit` body) get a **1h sin-bin** (`DEGRADE_DURATION_LONG_S = 3600s`).
28
  - **Proactive credit gating (KI-085, `8fc7979`), now NIM-only scope.** Election is gated by `is_alive AND has_credits` so quota-exhausted NIM models are excluded BEFORE the user hits a 429. Signal source within the NIM pool: per-model local 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). Per-model rate-metering applies within the locked NIM pool.
29
- - **HF Space secrets (KI-081).** `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space repository secrets for future re-enable, but the chain config no longer references them after KI-160 they are dormant, not active election candidates. `NVIDIA_NIM_API_KEY` is the only key the chain config consults.
30
  - **Admin telemetry (KI-086, `d90f8c0`).** `GET /api/admin/llm-health` returns `{chains, candidates, recent_turns, snapshot_ts}` with per-chain elected primary/backup, per-candidate health + credits + degraded-until, and last 20 turn outcomes. Admin "LLM Chain" tab auto-refreshes every 30s and renders whatever the chain config exposes (NIM-only post-KI-160).
31
  - **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
32
  - **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
33
- - **Brain / fast-brain / judge primaries in steady state** are **NIM Nemotron-Super 49B** (brain), **NIM Qwen 3-Next 80B** (fast brain), and **NIM Llama-4 Maverick 17B/128E** (judge), per the KI-160 candidate lists. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` within the NIM pool.
34
  - **KI-079 escalation as last bite (`87ee522`) — modified by KI-167.** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget). Post-KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) the `_canonical_fallback` terminal step is gone — on heavy-brain exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message instead. Worst-case wall-clock before graceful error: 25s FAST + 15s heavy = 40s.
35
  - **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
36
  - **Lenient FF-block parser (KI-090, `11cf4b3`) — RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** Was needed because the `<FF>...</FF>` trailer convention was fragile under load. Post-KI-167 the sales brain uses NIM `response_format={"type":"json_object"}` for guaranteed structured output, so the whole `_parse_ff_block` strict → fenced → bare-JSON ladder is gone.
@@ -47,18 +51,18 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
47
  - **Chroma collection re-ingested + profile-write hardening (KI-112).** KI-111 wrapped `.query()` so the bot survived the corruption, but every embedding query was raising `InternalError: Error executing plan: Internal error: Error finding id` and silently returning empty retrieval — the bot was answering 206 policies' worth of Qs without access to any policy chunk. Root cause: a pre-KI-102 deploy wrote a `profile_anonymous` chunk with NO `session_id` metadata; that legacy row poisoned every later `coll.query(where={"doc_type": {"$ne": "profile"}})` and the damage spread across HNSW segments (full collection extraction surfaced 1580 / 7356 chunks across 148 policies as `Error getting embedding`). Fix: full re-ingest from `rag/corpus/` PDFs → clean `rag/vectors/` + two new write-time guards in `backend/profile_rag.py::upsert_profile_chunk` — (a) reject `session_id` that isn't a non-empty `str`, (b) reject any embedding whose length ≠ `embedder.dimension` or that contains `None`. Both guards log a `WARNING` and return without writing, so a future model-drift or bad-input event can't re-poison HNSW. 4 new regression tests in `tests/test_profile_rag_isolation.py::TestUpsertRejectsBadInputs`. Repaired vectors uploaded to HF dataset `rohitsar567/insurance-bot-data` via `tools/upload_vectors_to_dataset.py` so the Space rebuild picks up the clean index. Old corrupted Chroma archived at `rag/_hf_dataset_backup/rag/vectors.corrupted.<ts>/`.
48
  - **Chain budgets:** brain 20s × 35s total, fast-brain 12s × 22s total, judge 30s × 75s total. With KI-080 only PRIMARY + BACKUP consume budget in the common case — leaves headroom for KI-079 escalation. KI-084 per-phase httpx timeouts are nested inside these budgets.
49
  - **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
50
- - **Provider keys.** `NVIDIA_NIM_API_KEY` + `GROQ_API_KEY` + `OPENROUTER_API_KEY` required in `.env` (local) and HF Space environment (production KI-081).
51
 
52
- ## Fact-find loop (ADR-039, supersedes ADR-030 / ADR-027) — KI-167
53
 
54
- **One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that — {slot}."`) and replaced it with `backend/sales_brain.py` — a single LLM-driven sales agent. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-155 and follow-ups showed the canonical fallback was the *dominant* path, not the safety net. User-visible symptom: "zero natural LLM chat — it always defaults to the script."
55
 
56
- - **Single NIM call per turn** via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool, using NVIDIA NIM's **`response_format={"type": "json_object"}`** for structured output — the same JSON-mode contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`. Response shape: `{"reply": "<prose for user>", "captures": {<slot_id>: <raw>}, "slot_driving": "<slot_id|null>", "complete": <bool>}`.
57
  - **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
58
  - **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` → `location_tier`), enum normalization (`Bangalore` → `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
59
  - **Profile persistence unchanged.** Captures flow through `session.update_profile_field()` → `backend/profile_store.save_profile()` (disk durability) → `backend/profile_rag.upsert_profile_chunk()` (retrieval-side visibility). KI-102 / KI-107 / KI-112 session-isolation guards remain in force.
60
  - **No scripted prompts, no canonical_fallback, no trailer convention.** `backend/needs_finder.py::GRAPH` slot-id data stays as a schema source for the system prompt, but `Question.prompt_en` is dead text — never consulted by the fact-find branch. `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, the lenient `<FF>` parser ladder, and the `"Got that — {slot}."` prefix logic are deleted.
61
- - **Outer 25s `asyncio.wait_for` ceiling retained.** On total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error to the user (fail-loud) rather than cascading to a scripted reply. There is no scripted safety net — that is intentional.
62
  - **DELETED in KI-167:** `backend/fact_find_brain.py` (441 LOC); `_canonical_fallback` + `_pick_opener` branches in `backend/orchestrator.py`; `_ff_failed_attempts` / `_ff_skipped_slots` session fields; every `fact_find_brain::fallback:*` telemetry variant; the lenient `<FF>` / fenced-`json` / bare-JSON-tail parser ladder.
63
  - **Persona-audit fixtures rewritten** to assert on final captured state (which slots are filled with what) rather than per-turn slot order — the LLM may capture in different turn order than the pre-KI-167 rules engine did. See WS4.
64
  - **Natural-conversation escape (KI-045):** intent_change phrases / off-topic questions still exit fact-find by routing through `should_route_to_fact_find` upstream of the brain.
@@ -133,8 +137,9 @@ Three independent safety layers against ChromaDB HNSW bloat:
133
  - **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
134
  - **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
135
  - **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
136
- - **`sales_brain.py` is the fact-find handler (KI-167, [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** `backend/fact_find_brain.py` is deleted along with the `<FF>` trailer convention, `_canonical_fallback`, scripted `Question.prompt_en`, and the `"Got that — {slot}."` prefix. Never re-introduce a fallback to scripted prompts on the fact-find branch — on total NIM exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message (fail-loud > fail-silent-with-script). The KI-150 historical note (`fact_find_brain` `max_tokens` 420 → 700) is moot post-KI-167 since the prose+trailer prompt shape that needed the larger budget no longer exists.
 
137
 
138
  ---
139
 
140
- *Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General). Same day: KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) locked chains to NIM-only after KI-155's `<FF>` trailer contract violation. Same day: KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) replaced `fact_find_brain.py` (scripted slot-walker + `<FF>` trailer + `_canonical_fallback` + scripted `Question.prompt_en`) with `backend/sales_brain.py` (single LLM call per turn, NIM `response_format=json_object`, deterministic post-processor).*
 
18
  - **Hands-free was removed entirely** in KI-027. Anything in the codebase still referring to it is stale.
19
  - **Bot TTS plays via the in-DOM `<audio>` element** inside `Message` (autoplay-on-mount via ref'd `useEffect`). Never use `new Audio(url).play()` — those detached instances are invisible to `document.querySelectorAll("audio").pause()` in the barge-in handler.
20
 
21
+ ## LLM stack (ADR-019 + ADR-026 → ADR-031 + ADR-032 → ADR-038 → ADR-040) — KI-080 → KI-087, KI-160, KI-167 → KI-179
22
 
23
+ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model. End-to-end spec: [ADR-032](70-docs/60-decisions/ADR-032-llm-chain-architecture.md). Chains now mix **Google AI Studio (Gemini Flash) as Tier 0 primary, NIM as Tier 1 fallback, OpenRouter free as Tier 2 diversity pool** ([ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md) — supersedes ADR-038's NIM-only lock once KI-167 retired the `<FF>` trailer convention that motivated the lock). Native provider JSON mode (`response_format={"type":"json_object"}` on NIM, `response_mime_type=application/json` on Gemini) is the structured-output contract; cross-provider fallback is safe again as long as every candidate supports JSON mode natively. Chains preserve brain ↔ judge family diversity (Gemini brain ↔ Mistral / Llama-4 judge) so failovers can't produce circular grading.
24
 
25
+ - **Three-tier election ([ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)).** Candidate pools per role:
26
+ - **Brain Fast** (sales-brain fact-find per KI-167, fast QA): Google `gemini-2.0-flash` (PRIMARY) → NIM `qwen3-next-80b-a3b-instruct` → NIM `mistralai/mistral-large-3-675b-instruct-2512` (675B dense) → NIM `meta/llama-4-maverick-17b-128e-instruct` (128B MoE) → OpenRouter `nvidia/nemotron-3-super-120b-a12b:free` → OpenRouter `qwen/qwen3-next-80b-a3b-instruct:free` → NIM `nvidia/llama-3.3-nemotron-super-49b-v1.5` (last resort).
27
+ - **Brain Main** (free-form QA + recommendation synthesis): Google `gemini-2.5-flash` (PRIMARY) → NIM `mistralai/mistral-large-3-675b-instruct-2512` → NIM `meta/llama-4-maverick-17b-128e-instruct` → NIM `qwen/qwen3-next-80b-a3b-instruct` → OpenRouter `nvidia/nemotron-3-super-120b-a12b:free` → NIM `nvidia/llama-3.3-nemotron-super-49b-v1.5` (last resort).
28
+ - **Judge** (faithfulness Gate 4, KI-171 skips on `fact_find` + `recommendation` queries): NIM `mistralai/mistral-large-3-675b-instruct-2512` (PRIMARY — different family from Gemini brain) → NIM `meta/llama-4-maverick-17b-128e-instruct` → OpenRouter `qwen/qwen3-next-80b-a3b-instruct:free` → NIM `nvidia/llama-3.3-nemotron-super-49b-v1.5`.
29
+ Google AI Studio free tier: 1500 req/day, 15 req/min on Gemini 2.0/2.5 Flash. On total chain exhaustion, orchestrator returns a graceful error message to the user — **fail-loud > fail-silent-with-garbage** still holds at the chain-tail level.
30
  - **Probe-driven sticky primary election (KI-080, [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md), superseded by ADR-038 for candidate-pool scope).** All three chains elect a sticky PRIMARY + BACKUP from a background probe within the NIM pool. `backend/llm_health.py` scores every candidate on `(1 / max(50, latency_ms)) * success_rate` and writes the current election to process state. `NimChainLLM.chat()` calls PRIMARY once; on real-time failure it falls to BACKUP and triggers an immediate probe refresh. **Per-turn LLM call count: 1 (most cases) or 2 (PRIMARY fails real-time → BACKUP).**
31
  - **Probe cadence + per-phase timeouts (KI-084, `119e0fd`).** Probe loop ticks at `PROBE_INTERVAL_SEC = 300s`. Probe `max_tokens` cut `5 → 1`. Every chat call uses explicit `httpx.Timeout(connect=2, read=12, write=2, pool=2)` so a stuck NIM pool releases its TCP socket independently of the outer `asyncio.wait_for`. Rate-limit failures (HTTP 429 / `RateLimit` body) get a **1h sin-bin** (`DEGRADE_DURATION_LONG_S = 3600s`).
32
  - **Proactive credit gating (KI-085, `8fc7979`), now NIM-only scope.** Election is gated by `is_alive AND has_credits` so quota-exhausted NIM models are excluded BEFORE the user hits a 429. Signal source within the NIM pool: per-model local 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). Per-model rate-metering applies within the locked NIM pool.
33
+ - **HF Space secrets (KI-081, updated KI-179).** The chain config now consults **`GOOGLE_API_KEY`** (Tier 0, Brain Fast + Brain Main primary), `NVIDIA_NIM_API_KEY` (Tier 1 fallback + Judge primary), and `OPENROUTER_API_KEY` (Tier 2 diversity pool — KI-176 server-side `models: [...]` fallback within the OR free pool). `GROQ_API_KEY` remains in HF Space secrets but is dormant Groq is no longer in any chain (KI-155 `<FF>` violation root cause remains a guard even though the `<FF>` convention itself is gone).
34
  - **Admin telemetry (KI-086, `d90f8c0`).** `GET /api/admin/llm-health` returns `{chains, candidates, recent_turns, snapshot_ts}` with per-chain elected primary/backup, per-candidate health + credits + degraded-until, and last 20 turn outcomes. Admin "LLM Chain" tab auto-refreshes every 30s and renders whatever the chain config exposes (NIM-only post-KI-160).
35
  - **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated** — `_balanced_brain_chain` retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
36
  - **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup, both NIM.
37
+ - **Brain / fast-brain / judge primaries in steady state** are **Google `gemini-2.5-flash`** (Brain Main), **Google `gemini-2.0-flash`** (Brain Fast / sales-brain fact-find), and **NIM Mistral Large 3 675B** (Judge), per the ADR-040 chain. Not hardcoded — elected primary follows live `latency × success_rate × credits_available` across the full three-tier pool; on Google quota exhaustion or 429, election falls naturally to NIM Mistral 675B (the Tier 1 fallback) without operator intervention.
38
  - **KI-079 escalation as last bite (`87ee522`) — modified by KI-167.** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, `_TIMEOUT_S_ESCALATION = 15s`, 35s chain budget). Post-KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) the `_canonical_fallback` terminal step is gone — on heavy-brain exhaustion the orchestrator returns the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) graceful error message instead. Worst-case wall-clock before graceful error: 25s FAST + 15s heavy = 40s.
39
  - **NIM concurrency semaphore + serial probe (KI-088, `14ee008`).** Module-level `asyncio.Semaphore(2)` wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from `NvidiaNimLLM.chat()` — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
40
  - **Lenient FF-block parser (KI-090, `11cf4b3`) — RETIRED by KI-167 ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)).** Was needed because the `<FF>...</FF>` trailer convention was fragile under load. Post-KI-167 the sales brain uses NIM `response_format={"type":"json_object"}` for guaranteed structured output, so the whole `_parse_ff_block` strict → fenced → bare-JSON ladder is gone.
 
51
  - **Chroma collection re-ingested + profile-write hardening (KI-112).** KI-111 wrapped `.query()` so the bot survived the corruption, but every embedding query was raising `InternalError: Error executing plan: Internal error: Error finding id` and silently returning empty retrieval — the bot was answering 206 policies' worth of Qs without access to any policy chunk. Root cause: a pre-KI-102 deploy wrote a `profile_anonymous` chunk with NO `session_id` metadata; that legacy row poisoned every later `coll.query(where={"doc_type": {"$ne": "profile"}})` and the damage spread across HNSW segments (full collection extraction surfaced 1580 / 7356 chunks across 148 policies as `Error getting embedding`). Fix: full re-ingest from `rag/corpus/` PDFs → clean `rag/vectors/` + two new write-time guards in `backend/profile_rag.py::upsert_profile_chunk` — (a) reject `session_id` that isn't a non-empty `str`, (b) reject any embedding whose length ≠ `embedder.dimension` or that contains `None`. Both guards log a `WARNING` and return without writing, so a future model-drift or bad-input event can't re-poison HNSW. 4 new regression tests in `tests/test_profile_rag_isolation.py::TestUpsertRejectsBadInputs`. Repaired vectors uploaded to HF dataset `rohitsar567/insurance-bot-data` via `tools/upload_vectors_to_dataset.py` so the Space rebuild picks up the clean index. Old corrupted Chroma archived at `rag/_hf_dataset_backup/rag/vectors.corrupted.<ts>/`.
52
  - **Chain budgets:** brain 20s × 35s total, fast-brain 12s × 22s total, judge 30s × 75s total. With KI-080 only PRIMARY + BACKUP consume budget in the common case — leaves headroom for KI-079 escalation. KI-084 per-phase httpx timeouts are nested inside these budgets.
53
  - **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). **Embeddings** = local BGE-small-en-v1.5.
54
+ - **Provider keys (post-KI-179).** `GOOGLE_API_KEY` (Tier 0 — Gemini Flash primary), `NVIDIA_NIM_API_KEY` (Tier 1 fallback + Judge primary), `OPENROUTER_API_KEY` (Tier 2 diversity pool) required in `.env` (local) and HF Space environment (production). `GROQ_API_KEY` retained as dormant secret for one-flip re-enable, but no chain references it.
55
 
56
+ ## Fact-find loop (ADR-039 + ADR-040, supersedes ADR-030 / ADR-027) — KI-167 / KI-179
57
 
58
+ **One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback` greedy slot-walker + scripted `Question.prompt_en` prefixed with `"Got that — {slot}."`) and replaced it with `backend/sales_brain.py` — a single LLM-driven sales agent. KI-179 then moved the primary candidate from NIM Qwen to Google `gemini-2.0-flash` ([ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)) for best-in-class conversational quality on the free path. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-155 and follow-ups showed the canonical fallback was the *dominant* path, not the safety net. User-visible symptom: "zero natural LLM chat — it always defaults to the script."
59
 
60
+ - **Single LLM call per turn** via `NimChainLLM(FAST_BRAIN_CHAIN)` against the [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md) three-tier pool (Google Tier 0 → NIM Tier 1 → OpenRouter Tier 2). Google uses **`response_mime_type=application/json`**; NIM uses **`response_format={"type":"json_object"}`**; OpenRouter free-tier candidates (`nvidia/nemotron-3-super-120b-a12b:free`, `qwen/qwen3-next-80b-a3b-instruct:free`) also support `response_format` natively (KI-178 audit). Every candidate enforces structured output server-side — same contract already validated in `backend/translation_check.py`, `backend/faithfulness.py`, and `backend/security.py`. Response shape: `{"reply": "<prose for user>", "captures": {<slot_id>: <raw>}, "slot_driving": "<slot_id|null>", "complete": <bool>}`.
61
  - **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
62
  - **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` → `location_tier`), enum normalization (`Bangalore` → `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
63
  - **Profile persistence unchanged.** Captures flow through `session.update_profile_field()` → `backend/profile_store.save_profile()` (disk durability) → `backend/profile_rag.upsert_profile_chunk()` (retrieval-side visibility). KI-102 / KI-107 / KI-112 session-isolation guards remain in force.
64
  - **No scripted prompts, no canonical_fallback, no trailer convention.** `backend/needs_finder.py::GRAPH` slot-id data stays as a schema source for the system prompt, but `Question.prompt_en` is dead text — never consulted by the fact-find branch. `_canonical_fallback`, `_normalize_for_slot`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`, the lenient `<FF>` parser ladder, and the `"Got that — {slot}."` prefix logic are deleted.
65
+ - **Outer 25s `asyncio.wait_for` ceiling retained.** On total chain exhaustion (all of Google → NIM → OpenRouter fail in a single turn) the orchestrator returns the graceful error to the user (fail-loud) rather than cascading to a scripted reply. There is no scripted safety net — that is intentional.
66
  - **DELETED in KI-167:** `backend/fact_find_brain.py` (441 LOC); `_canonical_fallback` + `_pick_opener` branches in `backend/orchestrator.py`; `_ff_failed_attempts` / `_ff_skipped_slots` session fields; every `fact_find_brain::fallback:*` telemetry variant; the lenient `<FF>` / fenced-`json` / bare-JSON-tail parser ladder.
67
  - **Persona-audit fixtures rewritten** to assert on final captured state (which slots are filled with what) rather than per-turn slot order — the LLM may capture in different turn order than the pre-KI-167 rules engine did. See WS4.
68
  - **Natural-conversation escape (KI-045):** intent_change phrases / off-topic questions still exit fact-find by routing through `should_route_to_fact_find` upstream of the brain.
 
137
  - **The `indusind-general` slug did not exist anywhere in the codebase before 2026-05-15.** Reliance General Insurance was rebranded to IndusInd General; **KI-144** migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing `reliance-general` should either be retained as a legacy alias (one card remains under `reliance-general` for back-compat) or migrated to `indusind-general`. Do not silently merge the two — they're tracked as separate slugs.
138
  - **Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148).** The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: `rmsThreshold=18`, `voiceBandMinProp=0.20`, `noiseFloor * 1.8`. TTS preprocess now expands `k → thousand`. Anything in the codebase still assuming default-ON Live mode is stale.
139
  - **Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145).** Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
140
+ - **`sales_brain.py` is the fact-find handler (KI-167, [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md); primary candidate is Google Gemini 2.0 Flash post-KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)).** `backend/fact_find_brain.py` is deleted along with the `<FF>` trailer convention, `_canonical_fallback`, scripted `Question.prompt_en`, and the `"Got that — {slot}."` prefix. Never re-introduce a fallback to scripted prompts on the fact-find branch — on total three-tier chain exhaustion the orchestrator returns a graceful error message (fail-loud > fail-silent-with-script). The KI-150 historical note (`fact_find_brain` `max_tokens` 420 → 700) is moot post-KI-167 since the prose+trailer prompt shape that needed the larger budget no longer exists.
141
+ - **Never hardcode `NvidiaNimLLM(model=...)` post-KI-179** — chains now mix Google (`backend/providers/google_gemini_llm.py`), NIM (`backend/providers/nvidia_nim_llm.py`), and OpenRouter (`backend/providers/openrouter_llm.py`) candidates. Always go through `NimChainLLM(chain=...)` so the call surfs the full three-tier failover; bypassing the chain reintroduces the single-provider single-point-of-failure that KI-080 / KI-160 / KI-179 collectively eliminated.
142
 
143
  ---
144
 
145
+ *Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General). Same day: KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) locked chains to NIM-only after KI-155's `<FF>` trailer contract violation. Same day: KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) replaced `fact_find_brain.py` with `backend/sales_brain.py` (single LLM call per turn, native JSON mode, deterministic post-processor). Same day: KI-171 (judge skip on fact_find + recommendation), KI-175 (NIM chain reorder — nemotron demoted to last), KI-176 (OpenRouter `models: [...]` server-side fallback), KI-178 (live audit of OR free-tier JSON-mode support), KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md) added Google AI Studio (Gemini 2.0 / 2.5 Flash) as the Tier 0 primary on Brain Fast + Brain Main, with NIM as Tier 1 fallback and OpenRouter as Tier 2 diversity pool. ADR-038 is now superseded — the NIM-only lock was relaxed once the `<FF>` trailer convention that motivated it was retired.*
README.md CHANGED
@@ -64,7 +64,11 @@ Thirty-plus knowledge increments landed today. The corpus was rebuilt, the marke
64
  - **KI-149** — budget + income parser captures bare numerals like `"30000"`.
65
  - **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (one of eight band-aids on the prose+`<FF>`-trailer prompt shape; superseded by KI-167).
66
  - **KI-160** — chains locked to NIM-only ([ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)) after KI-155 demonstrated Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract.
67
- - **KI-167** — `backend/fact_find_brain.py` ripped out and replaced by `backend/sales_brain.py` ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)). Single LLM call per turn using NIM `response_format={"type":"json_object"}`; no scripted `Question.prompt_en`, no `<FF>` trailer, no `_canonical_fallback`, no `"Got that — {slot}."` prefix. Deterministic post-processor (`backend/sales_brain_normalizer.py`) normalizes captures; profile persists via `profile_store` + `profile_rag` as before.
 
 
 
 
68
 
69
  Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
70
 
@@ -92,15 +96,15 @@ A take-home is a sample of how the engineer thinks under constraint. Three thing
92
 
93
  2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
94
 
95
- 3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a NIM-only candidate pool per role (KI-160 / [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)), not a single hardcoded brain. Each role is a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) whose candidates are all NIM-hosted KI-155 demonstrated that Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract, so cross-provider fallback was removed as a silent-failure trap:
96
 
97
- - **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`), backup **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE), 3rd candidate **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`).
98
- - **`FAST_BRAIN_CHAIN`** (sales-brain fact-find turns per KI-167, QA, normalize, extract every latency-sensitive role) primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`), backup **NVIDIA Nemotron-Super 49B v1.5** (`nvidia/llama-3.3-nemotron-super-49b-v1.5`).
99
- - **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) primary **Meta Llama-4 Maverick 17B/128E** (`meta/llama-4-maverick-17b-128e-instruct`), backup **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`). Deliberately different model families from the brain pool so the judge does not mark its own homework.
100
 
101
- **NIM-only election, no cross-provider cascade (KI-160, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).** If every NIM candidate in a chain fails, orchestrator returns a graceful error message rather than falling to Groq or OpenRouter — fail-loud is preferred over fail-silent-with-garbage for structured-output contracts. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the cross-provider-fallback variant of KI-080 ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)) are both superseded. KI-085's proactive credit gating still applies within the NIM pool via a per-model 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). `GROQ_API_KEY` + `OPENROUTER_API_KEY` remain in HF Space secrets for future re-enable but the chain config no longer references them.
102
 
103
- The result: a Sarvam customer deploying this stack gets a product that *uses Sarvam exactly where Sarvam beats the world* and uses MIT-licensed open-weights frontier models for everything else — $0 inference, two independent free-tier providers, single-key-per-provider for the entire non-voice stack.
104
 
105
  The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
106
 
@@ -201,8 +205,8 @@ The bot is two flows running together — the customer's experience and the tech
201
  - `detect_language(user_text)` → `english / indic`
202
  - Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
203
 
204
- **Tech 4 — Fact-find branch (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))**
205
- - If profile empty AND intent ∈ {fact_find, recommendation, comparison}: hand the turn to `backend/sales_brain.py::drive_sales_brain()` — one NIM call with `response_format={"type":"json_object"}`, system prompt carries the 9-slot schema + current profile state. Response: `{reply, captures, slot_driving, complete}`. `backend/sales_brain_normalizer.py` normalizes captures → `session.update_profile_field()` → `profile_store.save_profile()` + `profile_rag.upsert_profile_chunk()`. Emit `reply` to the user.
206
  - Else: continue to retrieval
207
 
208
  </td></tr>
@@ -254,9 +258,9 @@ The bot is two flows running together — the customer's experience and the tech
254
  </td><td>
255
 
256
  **Tech 7 — Brain selection, `pick_brain(intent, language)`**
257
- - `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool (KI-080 / KI-160)
258
- - `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary in the NIM-only pool (typically Qwen 3-Next 80B in steady state); `fact_find` turns go through `backend/sales_brain.py` with `response_format={"type":"json_object"}` (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))
259
- - KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → NIM-pool backup). Per-chain primary refreshed every 300s by background probe. On total NIM exhaustion: graceful error to user (fail-loud, [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)).
260
 
261
  **Tech 8 — System prompt construction, `build_messages()`**
262
  - `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
@@ -376,11 +380,13 @@ The bot is two flows running together — the customer's experience and the tech
376
 
377
 
378
  ┌──────────────────────────────────────────────────────────────────────────┐
379
- │ TECH 4. Fact-find OR free-form branch (KI-167 / ADR-039)
380
  │ ------- │
381
  │ · If intent ∈ {fact_find, recommendation, comparison} AND profile │
382
  │ incomplete: │
383
- │ drive_sales_brain() — one NIM call, response_format=json_object
 
 
384
  │ → {reply, captures, slot_driving, complete} │
385
  │ normalize captures → session.update_profile_field() │
386
  │ → profile_store.save_profile() + profile_rag.upsert_profile_chunk() │
@@ -415,9 +421,14 @@ The bot is two flows running together — the customer's experience and the tech
415
  ┌──────────────────────────────────────────────────────────────────────────┐
416
  │ TECH 7. Brain selection — pick_brain(intent, language) │
417
  │ ------- │
418
- │ · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary in NIM-only pool (heavy, KI-080 / KI-160 / ADR-038)
419
- · intent {qa, fact_find} FAST_BRAIN_CHAIN; fact_find turns dispatched to sales_brain.py (KI-167 / ADR-039, JSON mode)
420
- · NIM Qwen 80B + Nemotron-Super 49B as fast-brain election candidates (NIM-only post-KI-160)
 
 
 
 
 
421
  └──────────────────────────────────────────────────────────────────────────┘
422
 
423
 
@@ -435,7 +446,7 @@ The bot is two flows running together — the customer's experience and the tech
435
  ┌──────────────────────────────────────────────────────────────────────────┐
436
  │ TECH 9. Brain LLM call → reply text │
437
  │ ------- │
438
- │ · NIM streaming chat call (V4-Pro or V4-Flash)
439
  │ · strip_think_tags() removes <think>…</think> chain-of-thought │
440
  │ · Capture brain_model_actual for non-circular judge selection │
441
  └──────────────────────────────────────────────────────────────────────────┘
@@ -521,14 +532,18 @@ The bot is two flows running together — the customer's experience and the tech
521
  │ ┌───────────────────────┼──────────────────────────┐ │
522
  │ ▼ ▼ ▼ │
523
  │ ┌──────────────┐ ┌──────────────────┐ ┌──────────────────────┐ │
524
- │ │ STRUCTURED │ │ VECTOR STORE │ │ NIM BRAIN ROUTER │ │
525
- │ │ DuckDB │ │ Chroma 0.5.20 │ │ V4-Pro (heavy) │ │
526
- │ │ 62 fields │ │ BGE-small (384d) │ │ V4-Flash (fast) │ │
527
- │ │ per policy │ │ 800/120 chunk │ │ Mistral Large 3 675B │ │
528
- │ └──────────────┘ │ +profile chunk │ │ (judge + xcheck │ │
529
- │ │ per session │ │ + Indic gates) │ │
530
- │ └──────────────────┘ │ single NIM key │ │
531
- │ │ 40 req/min · $0 │ │
 
 
 
 
532
  │ └──────────────────────┘ │
533
  └─────────────────────────────────────────────────────────────────────────┘
534
 
@@ -573,16 +588,16 @@ The bot is two flows running together — the customer's experience and the tech
573
 
574
  ### 4.3 Model stack
575
 
576
- Every LLM role is served by a **candidate pool** of models elected over by a probe-driven sticky-primary scheme ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md), end-to-end spec in [ADR-032](70-docs/60-decisions/ADR-032-llm-chain-architecture.md)), never a hardcoded single model. Chains were curated to preserve brain ↔ judge family diversity (Qwen brain ↔ Mistral judge) so any failover still produces non-circular grading. Cross-provider entries (OpenRouter, Groq) sit inside every chain so the whole reasoning stack survives a full NIM regional outage.
577
 
578
  | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
579
  |---|---|---|---|---|
580
- | **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120BMistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B DeepSeek V4-Pro OpenRouter GPT-OSS 120B Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
581
- | **Fast brain** (voice turns, sales-brain fact-find KI-167, QA, normalize, extract) | Probe-elected primary within the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) NIM-only pool typically Qwen 3-Next 80B in steady state | Qwen 3-Next 80BNVIDIA Nemotron-Super 49B v1.5 (NIM-only election candidates) | NIM | Bottleneck on these short jobs is TTFT, not capability. Sales brain (KI-167) calls this chain with `response_format={"type":"json_object"}` for guaranteed structured output. Election adapts within the NIM pool when one candidate degrades. |
582
- | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
583
- | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
584
- | **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
585
- | **Sales brain** (fact-find conversation driver — KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)) | Fast brain chain (Qwen 3-Next 80B primary in NIM-only pool) | inherits FAST_BRAIN_CHAIN | NIM | One NIM call per turn using `response_format={"type":"json_object"}`. System prompt carries the 9-slot schema + current profile state; LLM owns voice + flow + slot order, returns `{reply, captures, slot_driving, complete}`. Replaces the scripted-`prompt_en` + paraphraser + opener-rotation + `<FF>` trailer + `_canonical_fallback` machinery (ADR-027 / ADR-030 retired). |
586
  | **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
587
  | **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
588
  | **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
@@ -806,9 +821,15 @@ cp -R rag/_hf_dataset_backup/rag/* rag/ # corpus/, extracted/, vectors/
806
 
807
  ```bash
808
  cp .env.example .env
809
- # Edit .env and fill in the 8 keys (see 70-docs/60-decisions/ADR-010-secret-handling.md):
810
- # SARVAM_API_KEY, VOYAGE_API_KEY, NVIDIA_NIM_API_KEY, HF_TOKEN,
811
- # ADMIN_PASSWORD, ADMIN_IP_ALLOWLIST, GROQ_API_KEY, OPENROUTER_API_KEY
 
 
 
 
 
 
812
  chmod 600 .env
813
  ```
814
 
 
64
  - **KI-149** — budget + income parser captures bare numerals like `"30000"`.
65
  - **KI-150** — `fact_find_brain` `max_tokens` 420 → 700 (one of eight band-aids on the prose+`<FF>`-trailer prompt shape; superseded by KI-167).
66
  - **KI-160** — chains locked to NIM-only ([ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)) after KI-155 demonstrated Groq Llama-3.3 silently ignores the `<FF>` structured-output trailer contract.
67
+ - **KI-167** — `backend/fact_find_brain.py` ripped out and replaced by `backend/sales_brain.py` ([ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)). Single LLM call per turn using native JSON mode; no scripted `Question.prompt_en`, no `<FF>` trailer, no `_canonical_fallback`, no `"Got that — {slot}."` prefix. Deterministic post-processor (`backend/sales_brain_normalizer.py`) normalizes captures; profile persists via `profile_store` + `profile_rag` as before.
68
+ - **KI-171** — orchestrator skips the Judge chain on `fact_find` + `recommendation` queries (faithfulness scoring is meaningless on profile-capture turns and on multi-policy ranked shortlists where the rubric is structural, not factual grounding).
69
+ - **KI-175** — NIM chain reorder: Mistral Large 3 675B promoted to primary, nemotron-49b demoted to last resort across all three chains. The old "nemotron primary" ordering predated the 675B model's availability.
70
+ - **KI-176** — OpenRouter re-added to the chain pool with `models: [...]` server-side fallback within the OR free pool. KI-178 live-audit then confirmed which OR free-tier models actually support `response_format` (nemotron-3-super-120b, qwen3-next-80b, gemma-4 — not Llama 3.3 70B or Hermes 3 405B).
71
+ - **KI-179** — Google AI Studio added as Tier 0 primary on Brain Fast + Brain Main ([ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)). `backend/providers/google_gemini_llm.py` wrapper matches the `LLMProvider` interface; chains now span {Google → NIM → OpenRouter}. ADR-038 (NIM-only lock) is superseded — the lock was relaxed once KI-167 retired the `<FF>` trailer convention that originally motivated it. Brain Fast primary is `gemini-2.0-flash`; Brain Main primary is `gemini-2.5-flash`; Judge stays NIM Mistral Large 3 675B (different family from Gemini, preserving the brain ↔ judge non-circular grading invariant). Google's free tier: 1500 req/day, 15 req/min, native JSON via `response_mime_type=application/json`.
72
 
73
  Per-insurer card counts (166 total across 19 real insurers): HDFC ERGO 15 · National Insurance 14 · Niva Bupa 14 · Bajaj Allianz 13 · ICICI Lombard 13 · Star Health 11 · Care Health 10 · New India Assurance 9 · Tata AIG 9 · Acko 7 · Aditya Birla 7 · Royal Sundaram 7 · Cholamandalam MS 6 · Go Digit 6 · IFFCO Tokio 6 · ManipalCigna 6 · SBI General 6 · IndusInd General 3 · Oriental Insurance 3 · Reliance General 1.
74
 
 
96
 
97
  2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
98
 
99
+ 3. **Honest model picks — Sarvam where Sarvam is uniquely strong, frontier-tier free brains for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning runs on a three-tier candidate pool per role (KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)), not a single hardcoded brain. Every candidate supports native JSON mode server-side — Google's `response_mime_type=application/json`, NIM's `response_format={"type":"json_object"}`, OpenRouter's `response_format` pass-through on the audited free-tier models (KI-178). Cross-provider fallback is safe again now that the `<FF>` trailer convention that triggered KI-160 / ADR-038's NIM-only lock has been retired by KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md):
100
 
101
+ - **Brain Main** (comparison, recommendation, synthesis) — primary **Google `gemini-2.5-flash`** via Google AI Studio (1500 req/day free, native JSON mode), Tier 1 fallback **NIM Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`), then **NIM Llama-4 Maverick 17B/128E**, then **NIM Qwen 3-Next 80B**, then **OpenRouter `nvidia/nemotron-3-super-120b-a12b:free`**, last resort **NIM Nemotron-Super 49B v1.5**.
102
+ - **Brain Fast** (sales-brain fact-find turns per KI-167, fast QA)primary **Google `gemini-2.0-flash`**, Tier 1 fallback **NIM Qwen 3-Next 80B**, then **NIM Mistral Large 3 675B**, then **NIM Llama-4 Maverick 17B/128E**, then **OpenRouter `nvidia/nemotron-3-super-120b-a12b:free`** + `qwen/qwen3-next-80b-a3b-instruct:free`, last resort **NIM Nemotron-Super 49B v1.5**.
103
+ - **Judge** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader; KI-171 skips this on `fact_find` + `recommendation` queries) primary **NIM Mistral Large 3 675B** (different family from the Gemini brain — preserves cross-family non-circular grading), Tier 1 fallback **NIM Llama-4 Maverick 17B/128E**, then **OpenRouter `qwen/qwen3-next-80b-a3b-instruct:free`**, last resort **NIM Nemotron-Super 49B v1.5**.
104
 
105
+ **Three-tier election (KI-179, [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)).** Google NIM OpenRouter, with the chain falling through naturally when an upper tier 429s, exhausts quota, or fails real-time. On total chain exhaustion the orchestrator returns a graceful error message — fail-loud > fail-silent-with-garbage is preserved. The 50/50 NIM ↔ Groq rotation of KI-025 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) and the NIM-only lock of KI-160 ([ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md)) are both superseded. KI-085's proactive credit gating applies per-candidate (60-second rate-meter, gate at 35-of-40 req/min). `GROQ_API_KEY` remains in HF Space secrets but is dormant no chain references it.
106
 
107
+ The result: a Sarvam customer deploying this stack gets a product that *uses Sarvam exactly where Sarvam beats the world*, uses **Google Gemini Flash on the free path for frontier-tier conversational quality**, and uses MIT-licensed open-weights frontier models (Mistral Large 3 675B, Llama-4 Maverick, Qwen 3-Next 80B, Nemotron-Super 49B) for everything else — $0 inference, three independent free-tier providers, single-key-per-provider for the entire non-voice stack.
108
 
109
  The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
110
 
 
205
  - `detect_language(user_text)` → `english / indic`
206
  - Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
207
 
208
+ **Tech 4 — Fact-find branch (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md); primary candidate is Google Gemini 2.0 Flash post-KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md))**
209
+ - If profile empty AND intent ∈ {fact_find, recommendation, comparison}: hand the turn to `backend/sales_brain.py::drive_sales_brain()` — one LLM call against the Brain Fast chain (Google `gemini-2.0-flash` primary → NIM Qwen / Mistral 675B / Llama-4 Maverick → OpenRouter free → NIM Nemotron-49b last resort) using native JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM / OR). System prompt carries the 9-slot schema + current profile state. Response: `{reply, captures, slot_driving, complete}`. `backend/sales_brain_normalizer.py` normalizes captures → `session.update_profile_field()` → `profile_store.save_profile()` + `profile_rag.upsert_profile_chunk()`. Emit `reply` to the user.
210
  - Else: continue to retrieval
211
 
212
  </td></tr>
 
258
  </td><td>
259
 
260
  **Tech 7 — Brain selection, `pick_brain(intent, language)`**
261
+ - `intent ∈ {comparison, recommendation}` → Brain Main chain with probe-elected primary across the three-tier pool ([ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)): Google `gemini-2.5-flash` → NIM Mistral Large 3 675B → NIM Llama-4 Maverick → NIM Qwen 3-Next 80B → OpenRouter `nemotron-3-super-120b:free` → NIM Nemotron-49b
262
+ - `intent ∈ {qa, fact_find}` → Brain Fast chain: Google `gemini-2.0-flash` NIM Qwen NIM Mistral 675B NIM Llama-4 Maverick OpenRouter free pool → NIM Nemotron-49b. `fact_find` turns go through `backend/sales_brain.py` with native JSON mode (KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md))
263
+ - KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → next tier). Per-chain primary refreshed every 300s by background probe. On total three-tier exhaustion: graceful error to user (fail-loud — KI-179 preserves the [ADR-038](70-docs/60-decisions/ADR-038-nim-only-chains.md) principle even though the NIM-only candidate-pool scope is superseded).
264
 
265
  **Tech 8 — System prompt construction, `build_messages()`**
266
  - `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
 
380
 
381
 
382
  ┌──────────────────────────────────────────────────────────────────────────┐
383
+ │ TECH 4. Fact-find OR free-form branch (KI-167 / ADR-039 / ADR-040)
384
  │ ------- │
385
  │ · If intent ∈ {fact_find, recommendation, comparison} AND profile │
386
  │ incomplete: │
387
+ │ drive_sales_brain() — one LLM call against Brain Fast chain
388
+ │ (Gemini 2.0 Flash → NIM Qwen / Mistral 675B → OR free) │
389
+ │ using native JSON mode (response_mime_type / response_format) │
390
  │ → {reply, captures, slot_driving, complete} │
391
  │ normalize captures → session.update_profile_field() │
392
  │ → profile_store.save_profile() + profile_rag.upsert_profile_chunk() │
 
421
  ┌──────────────────────────────────────────────────────────────────────────┐
422
  │ TECH 7. Brain selection — pick_brain(intent, language) │
423
  │ ------- │
424
+ │ · intent ∈ {comparison, recommendation} → Brain Main chain
425
+ Gemini 2.5 FlashNIM Mistral 675B NIM Llama-4 Maverick
426
+ NIM Qwen 80B OR nemotron-3-super-120b:free NIM Nemotron-49b
427
+ │ · intent ∈ {qa, fact_find} → Brain Fast chain │
428
+ │ Gemini 2.0 Flash → NIM Qwen 80B → NIM Mistral 675B → NIM Llama-4 → │
429
+ │ OR free pool → NIM Nemotron-49b last resort │
430
+ │ · fact_find turns dispatched to sales_brain.py (KI-167 / ADR-039) │
431
+ │ · Three-tier election: Google → NIM → OpenRouter (KI-179 / ADR-040) │
432
  └──────────────────────────────────────────────────────────────────────────┘
433
 
434
 
 
446
  ┌──────────────────────────────────────────────────────────────────────────┐
447
  │ TECH 9. Brain LLM call → reply text │
448
  │ ------- │
449
+ │ · Provider-agnostic chat call via NimChainLLM (Google / NIM / OR)
450
  │ · strip_think_tags() removes <think>…</think> chain-of-thought │
451
  │ · Capture brain_model_actual for non-circular judge selection │
452
  └──────────────────────────────────────────────────────────────────────────┘
 
532
  │ ┌───────────────────────┼──────────────────────────┐ │
533
  │ ▼ ▼ ▼ │
534
  │ ┌──────────────┐ ┌──────────────────┐ ┌──────────────────────┐ │
535
+ │ │ STRUCTURED │ │ VECTOR STORE │ │ 3-TIER BRAIN ROUTER │ │
536
+ │ │ DuckDB │ │ Chroma 0.5.20 │ │ Tier 0: Google AI │ │
537
+ │ │ 62 fields │ │ BGE-small (384d) │ │ Studio (Gemini │ │
538
+ │ │ per policy │ │ 800/120 chunk │ │ 2.0 / 2.5 Flash) │ │
539
+ │ └──────────────┘ │ +profile chunk │ │ Tier 1: NIM (Mistral │ │
540
+ │ │ per session │ │ 675B, Llama-4, │ │
541
+ │ └──────────────────┘ │ Qwen 80B, Nemo-49) │ │
542
+ │ │ Tier 2: OR free pool │ │
543
+ │ │ (nemotron-120b, │ │
544
+ │ │ qwen-80b:free) │ │
545
+ │ │ KI-179 / ADR-040 │ │
546
+ │ │ $0 inference path │ │
547
  │ └──────────────────────┘ │
548
  └─────────────────────────────────────────────────────────────────────────┘
549
 
 
588
 
589
  ### 4.3 Model stack
590
 
591
+ Every LLM role is served by a **three-tier candidate pool** elected over by a probe-driven sticky-primary scheme ([ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md), end-to-end spec in [ADR-032](70-docs/60-decisions/ADR-032-llm-chain-architecture.md), current candidate-pool shape in [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)), never a hardcoded single model. Chains preserve brain ↔ judge family diversity (Gemini brain ↔ Mistral judge) so any failover still produces non-circular grading. Tier 0 (Google AI Studio) is the steady-state primary; Tier 1 (NIM) is the fallback; Tier 2 (OpenRouter free pool) is the diversity tail before the last-resort NIM Nemotron-49b. Every candidate enforces native JSON mode server-side — `response_mime_type=application/json` on Google, `response_format={"type":"json_object"}` on NIM and on the audited OR free-tier models (KI-178).
592
 
593
  | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
594
  |---|---|---|---|---|
595
+ | **Brain Main** (comparison, recommendation, synthesis) | **Google `gemini-2.5-flash`** (1500 req/day free, native JSON mode) | NIM Mistral Large 3 675B NIM Llama-4 Maverick 17B/128E NIM Qwen 3-Next 80BOpenRouter `nvidia/nemotron-3-super-120b-a12b:free`NIM Nemotron-Super 49B v1.5 (last resort) | Google + NIM + OpenRouter | Best-in-class conversational + synthesis quality on the free path; 1500 req/day adequate for demo + early production. NIM Mistral 675B Tier 1 fallback preserves dense 675B reasoning when Google quota / 429 trips. KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md). |
596
+ | **Brain Fast** (voice turns, sales-brain fact-find KI-167, QA) | **Google `gemini-2.0-flash`** (same 1500 req/day quota, lower-latency tier for fast turns) | NIM Qwen 3-Next 80B NIM Mistral Large 3 675B → NIM Llama-4 Maverick → OpenRouter `nemotron-3-super-120b:free` + `qwen3-next-80b:free` NIM Nemotron-Super 49B v1.5 (last resort) | Google + NIM + OpenRouter | Bottleneck on these short jobs is TTFT Gemini 2.0 Flash optimised for low-latency JSON. Sales brain (KI-167) calls this chain with native JSON mode for guaranteed structured output. KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md). |
597
+ | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader; KI-171 skips on `fact_find` + `recommendation`) | **NIM Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`) | NIM Llama-4 Maverick 17B/128E → OpenRouter `qwen/qwen3-next-80b-a3b-instruct:free`NIM Nemotron-Super 49B v1.5 | NIM + OpenRouter | Different family from the Gemini brain (Mistral, not Google / Llama / Qwen) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. Brain ↔ judge family diversity is the structural defence against circular grading. |
598
+ | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Brain Fast chain (Gemini 2.0 Flash primary) | inherits Brain Fast chain | Google + NIM + OpenRouter | Short prompt, structured JSON out — same chain as sales brain. |
599
+ | **Fact-find normalizer** (user answer → typed slot value) | Brain Fast chain (Gemini 2.0 Flash primary) | inherits Brain Fast chain | Google + NIM + OpenRouter | Same shape as profile extractor — narrow input, narrow JSON output. |
600
+ | **Sales brain** (fact-find conversation driver — KI-167 / [ADR-039](70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md), KI-179 / [ADR-040](70-docs/60-decisions/ADR-040-google-gemini-primary.md)) | Brain Fast chain (Gemini 2.0 Flash primary) | inherits Brain Fast chain | Google + NIM + OpenRouter | One LLM call per turn using native JSON mode. System prompt carries the 9-slot schema + current profile state; LLM owns voice + flow + slot order, returns `{reply, captures, slot_driving, complete}`. Replaces the scripted-`prompt_en` + paraphraser + opener-rotation + `<FF>` trailer + `_canonical_fallback` machinery (ADR-027 / ADR-030 retired). |
601
  | **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
602
  | **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
603
  | **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
 
821
 
822
  ```bash
823
  cp .env.example .env
824
+ # Edit .env and fill in the keys (see 70-docs/60-decisions/ADR-010-secret-handling.md):
825
+ # GOOGLE_API_KEY — Tier 0 brain (Gemini 2.0/2.5 Flash, KI-179 / ADR-040)
826
+ # NVIDIA_NIM_API_KEY — Tier 1 brain fallback + Judge primary
827
+ # OPENROUTER_API_KEY — Tier 2 diversity pool (free-tier OR models)
828
+ # SARVAM_API_KEY — STT / TTS / Indic translation (non-substitutable)
829
+ # VOYAGE_API_KEY — offline ingest embeddings only (BGE on hot path)
830
+ # HF_TOKEN — data-dataset pull at boot
831
+ # ADMIN_PASSWORD — /api/admin/* gating (post-KI-097, password-only)
832
+ # GROQ_API_KEY — DORMANT (no chain references it post-KI-160)
833
  chmod 600 .env
834
  ```
835
 
backend/README.md CHANGED
@@ -14,9 +14,11 @@ FastAPI + Pydantic service that fronts every chat turn. The HTTP entry point is
14
 
15
  | File | Role | Related ADR |
16
  | --- | --- | --- |
17
- | `needs_finder.py` | 9-slot fact-find schema (`GRAPH`) + canonical question prompts. Post-KI-070 used as **safeguard fallback only**primary path is `fact_find_brain.py::drive_fact_find` (single LLM call per turn). | [ADR-030](../70-docs/60-decisions/ADR-030-llm-driven-fact-find.md) |
18
- | `fact_find_brain.py` | KI-070 single-LLM-call fact-find driver. One brain call per turn produces natural prose + `<FF>{...}</FF>` JSON tail (captured slots + next slot + complete flag). KI-090 made the tail parser lenient (strict / fenced / bare-JSON-tail). | [ADR-030](../70-docs/60-decisions/ADR-030-llm-driven-fact-find.md), [ADR-032](../70-docs/60-decisions/ADR-032-llm-chain-architecture.md) |
19
- | ~~`question_paraphraser.py`~~ *(deleted in KI-070)* | Was an LLM rewrite of canonical slot questions (ADR-027). Superseded by `fact_find_brain.py`'s single-call architecture. | superseded by [ADR-030](../70-docs/60-decisions/ADR-030-llm-driven-fact-find.md) |
 
 
20
  | `fact_find_normalizer.py` | LLM-driven free-text → slot-value coercion (e.g. "32 lakh" → `3200000`). Goes through `NimChainLLM`, not a single client (KI-033). | — |
21
  | `profile_extractor.py` | LLM extractor that pulls profile updates out of conversational asides ("by the way, my dad has diabetes"). Chain-pattern, never a hardcoded model. | [ADR-022](../70-docs/60-decisions/ADR-022-conversational-profile-updates.md) |
22
  | `profile_store.py` | **NEW (KI-040).** Persistent named-profile JSON store under `40-data/profiles/`. O(1) name-keyed lookup; mirrors into `profile_rag` on every save. | — |
@@ -35,7 +37,15 @@ FastAPI + Pydantic service that fronts every chat turn. The HTTP entry point is
35
 
36
  ## Subdirectory
37
 
38
- `providers/` — concrete STT / TTS / LLM / embeddings client implementations. All LLM access goes through `NimChainLLM(chain=...)` from `providers/nvidia_nim_llm.py` — never instantiate a single-provider client directly. See `backend/providers/README.md`.
 
 
 
 
 
 
 
 
39
 
40
  ## Where to read for what
41
 
 
14
 
15
  | File | Role | Related ADR |
16
  | --- | --- | --- |
17
+ | `needs_finder.py` | 9-slot fact-find SCHEMA (`GRAPH`). Post-KI-167 the `prompt_en` / `prompt_hi` strings are dead text the LLM owns voice + cadence end-to-end via its system prompt. The data structure is retained as the schema source for `sales_brain.py`. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md), [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md) |
18
+ | `sales_brain.py` | **KI-167 LLM-driven fact-find driver.** One brain call per turn against `FAST_BRAIN_CHAIN` (Gemini 2.0 Flash primary) using native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM / OpenRouter). Returns `{reply, captures, slot_driving, complete}`. No `<FF>` trailer, no scripted fallback. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md), [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md) |
19
+ | `sales_brain_normalizer.py` | **KI-167 deterministic post-processor.** Pure-rule mapping of the LLM's loose `captures` dict to canonical `{field: validated_value}`: alias resolution (`location` → `location_tier`), enum coercion (`Bangalore` → `metro`), INR parsing, type / bounds validation, KI-094 null-drop. No LLM calls. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
20
+ | ~~`fact_find_brain.py`~~ *(deleted in KI-167)* | Was the ADR-030 one-call brain with `<FF>...</FF>` trailer convention + `_canonical_fallback`. Removed entirely. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
21
+ | ~~`question_paraphraser.py`~~ *(deleted in KI-070)* | Was an LLM rewrite of canonical slot questions (ADR-027). Superseded first by `fact_find_brain.py`, now by `sales_brain.py`. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
22
  | `fact_find_normalizer.py` | LLM-driven free-text → slot-value coercion (e.g. "32 lakh" → `3200000`). Goes through `NimChainLLM`, not a single client (KI-033). | — |
23
  | `profile_extractor.py` | LLM extractor that pulls profile updates out of conversational asides ("by the way, my dad has diabetes"). Chain-pattern, never a hardcoded model. | [ADR-022](../70-docs/60-decisions/ADR-022-conversational-profile-updates.md) |
24
  | `profile_store.py` | **NEW (KI-040).** Persistent named-profile JSON store under `40-data/profiles/`. O(1) name-keyed lookup; mirrors into `profile_rag` on every save. | — |
 
37
 
38
  ## Subdirectory
39
 
40
+ `providers/` — concrete STT / TTS / LLM / embeddings client implementations:
41
+
42
+ - `sarvam_stt.py` / `sarvam_tts.py` — Sarvam Saarika v2.5 + Bulbul v2.
43
+ - `google_gemini_llm.py` — Google AI Studio (`gemini-2.0-flash`, `gemini-2.5-flash`); Tier 0 primary on Brain Fast + Brain Main ([ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)).
44
+ - `nvidia_nim_llm.py` — NIM client + `NimChainLLM` dispatcher (the elector + cross-provider BACKUP machinery from KI-080). Tier 1 fallback across all chains; judge primary stays on NIM Mistral Large 3 675B.
45
+ - `openrouter_llm.py` — OpenRouter client; Tier 2 cross-provider diversity using OR's `models: [...]` server-side fallback (KI-176).
46
+ - `local_embeddings.py` — BGE-small-en-v1.5 sentence-transformers.
47
+
48
+ All LLM access goes through `NimChainLLM(chain=...)` — never instantiate a single-provider client directly. See `backend/providers/README.md`.
49
 
50
  ## Where to read for what
51
 
backend/orchestrator.py CHANGED
@@ -28,6 +28,7 @@ from backend.providers.nvidia_nim_llm import (
28
  get_brain_llm,
29
  get_fast_brain_llm,
30
  )
 
31
  from rag.retrieve import RetrievedChunk, format_for_llm_context, retrieve
32
 
33
 
@@ -297,12 +298,15 @@ def should_route_to_fact_find(
297
  def pick_brain(intent: str, language: str) -> BrainPick:
298
  """Route to the reasoning brain per D-019 (2026-05-14, tiered routing).
299
 
300
- All routes go through NVIDIA NIM (single provider, $0 cost). Tier picked
301
- by intent classification:
302
- - 'comparison' / 'recommendation' DeepSeek-V4-Pro (1.6T/49B MoE)
303
- Heavy synthesis, multi-policy reasoning. Quality > latency.
304
- - 'fact_find' / 'qa' → DeepSeek-V4-Flash (284B/13B MoE)
305
- Single-turn voice responses. Latency > quality, still frontier-tier.
 
 
 
306
 
307
  Indic queries get a Sarvam-M translation pass in `handle_turn` before
308
  retrieval, then another after reasoning to convert the English reply back
@@ -311,8 +315,8 @@ def pick_brain(intent: str, language: str) -> BrainPick:
311
  """
312
  HEAVY_INTENTS = {"comparison", "recommendation"}
313
  if intent in HEAVY_INTENTS:
314
- return BrainPick(get_brain_llm(), f"v4-pro::{intent}")
315
- return BrainPick(get_fast_brain_llm(), f"v4-flash::{intent}")
316
 
317
 
318
  # ---------- welcome-back greeting helpers ----------
 
28
  get_brain_llm,
29
  get_fast_brain_llm,
30
  )
31
+ from backend.providers.tiered_brain_llm import get_tiered_brain_llm
32
  from rag.retrieve import RetrievedChunk, format_for_llm_context, retrieve
33
 
34
 
 
298
  def pick_brain(intent: str, language: str) -> BrainPick:
299
  """Route to the reasoning brain per D-019 (2026-05-14, tiered routing).
300
 
301
+ KI-179 (2026-05-15) every route now goes through a 3-tier wrapper
302
+ (TieredBrainLLM) that prefers Google Gemini Flash, falls back to NIM,
303
+ then OpenRouter free-tier. The intent-based heavy vs fast split is
304
+ preserved at the wrapper level heavy intents get `gemini-2.5-flash`
305
+ + BRAIN_CHAIN, light intents get `gemini-2.0-flash` + FAST_BRAIN_CHAIN.
306
+
307
+ Original D-019 design (kept for context):
308
+ - 'comparison' / 'recommendation' → heavy brain (quality > latency)
309
+ - 'fact_find' / 'qa' → fast brain (latency > quality)
310
 
311
  Indic queries get a Sarvam-M translation pass in `handle_turn` before
312
  retrieval, then another after reasoning to convert the English reply back
 
315
  """
316
  HEAVY_INTENTS = {"comparison", "recommendation"}
317
  if intent in HEAVY_INTENTS:
318
+ return BrainPick(get_tiered_brain_llm(role="brain"), f"v4-pro::{intent}")
319
+ return BrainPick(get_tiered_brain_llm(role="fast_brain"), f"v4-flash::{intent}")
320
 
321
 
322
  # ---------- welcome-back greeting helpers ----------
backend/providers/google_gemini_llm.py ADDED
@@ -0,0 +1,233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Google Gemini — primary LLM tier for sales_brain + orchestrator brain (KI-179).
2
+
3
+ Google AI Studio's free tier on `gemini-2.0-flash` and `gemini-2.5-flash` gives
4
+ 1500 req/day with native JSON mode (`responseMimeType: "application/json"`).
5
+ This is best-in-class free quality for conversation and beats every model in
6
+ NIM or OpenRouter free tiers.
7
+
8
+ Wire-shape: REST API direct via httpx (no SDK dependency — matches the pattern
9
+ of nvidia_nim_llm.py and openrouter_llm.py).
10
+
11
+ Endpoint:
12
+ POST https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={api_key}
13
+
14
+ Request body:
15
+ {
16
+ "contents": [
17
+ {"role": "user", "parts": [{"text": "..."}]},
18
+ {"role": "model", "parts": [{"text": "..."}]}
19
+ ],
20
+ "systemInstruction": {"parts": [{"text": "..."}]},
21
+ "generationConfig": {
22
+ "temperature": 0.6,
23
+ "maxOutputTokens": 700,
24
+ "responseMimeType": "application/json"
25
+ }
26
+ }
27
+
28
+ Role mapping:
29
+ OpenAI shape → Gemini shape
30
+ role="system" → systemInstruction (separate top-level field)
31
+ role="user" → contents[i].role = "user"
32
+ role="assistant" → contents[i].role = "model"
33
+
34
+ JSON mode:
35
+ response_format == {"type": "json_object"} →
36
+ generationConfig.responseMimeType = "application/json"
37
+
38
+ The provider matches the existing `LLMProvider` interface so it slots
39
+ straight into both sales_brain.py (Tier 0) and the orchestrator brain main
40
+ path (Tier 0, via TieredBrainLLM wrapper).
41
+ """
42
+
43
+ from __future__ import annotations
44
+
45
+ import asyncio
46
+ import os
47
+ from typing import Optional
48
+
49
+ import httpx
50
+
51
+ from backend.providers.base import ChatMessage, LLMProvider, LLMResult
52
+
53
+
54
+ GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/models"
55
+ DEFAULT_MODEL = "gemini-2.5-flash-lite" # KI-183 — gemini-2.0-flash retired for new accounts
56
+
57
+
58
+ def _to_gemini_contents(messages: list[ChatMessage]) -> tuple[Optional[str], list[dict]]:
59
+ """Split an OpenAI-style message list into (systemInstruction, contents).
60
+
61
+ Gemini's API expects:
62
+ - `systemInstruction` as a separate top-level field (one block, the
63
+ concatenation of every system message in order)
64
+ - `contents` as a list of `{role: "user" | "model", parts: [{text: ...}]}`
65
+ entries (no "system" role allowed inside contents)
66
+
67
+ OpenAI's role names map cleanly:
68
+ - "system" → folded into systemInstruction
69
+ - "user" → contents[i].role = "user"
70
+ - "assistant" → contents[i].role = "model"
71
+
72
+ Empty / non-string content is coerced to a safe empty string so a stray
73
+ None never breaks the wire payload.
74
+ """
75
+ system_chunks: list[str] = []
76
+ contents: list[dict] = []
77
+ for m in messages:
78
+ text = m.content if isinstance(m.content, str) else str(m.content or "")
79
+ if m.role == "system":
80
+ if text:
81
+ system_chunks.append(text)
82
+ continue
83
+ if m.role == "user":
84
+ contents.append({"role": "user", "parts": [{"text": text}]})
85
+ elif m.role == "assistant":
86
+ contents.append({"role": "model", "parts": [{"text": text}]})
87
+ else:
88
+ # Unknown role — treat as user input rather than dropping it.
89
+ contents.append({"role": "user", "parts": [{"text": text}]})
90
+ system_instruction = "\n\n".join(system_chunks) if system_chunks else None
91
+ return system_instruction, contents
92
+
93
+
94
+ class GoogleGeminiLLM(LLMProvider):
95
+ """Google Gemini Flash via the AI Studio REST API.
96
+
97
+ `api_key` is read from the `GOOGLE_API_KEY` env var at *chat-time* — NOT
98
+ at module import time — so missing-key errors only surface when this
99
+ provider is actually called. That keeps cold imports cheap and lets the
100
+ Tier 0 layer cleanly fall through to Tier 1 (NIM) when the key is
101
+ unavailable (e.g., on the eval harness machine).
102
+ """
103
+
104
+ name = "gemini"
105
+
106
+ def __init__(
107
+ self,
108
+ model: str = DEFAULT_MODEL,
109
+ api_key: Optional[str] = None,
110
+ timeout: float = 25.0,
111
+ ):
112
+ # Defer the key-presence check to chat() — see class docstring.
113
+ self.api_key = api_key or os.environ.get("GOOGLE_API_KEY", "")
114
+ self.model = model
115
+ self.timeout = timeout
116
+ self.name = f"gemini::{model}"
117
+
118
+ async def chat(
119
+ self,
120
+ messages: list[ChatMessage],
121
+ temperature: float = 0.6,
122
+ max_tokens: int = 700,
123
+ response_format: Optional[dict] = None,
124
+ **kwargs, # absorb OR-specific kwargs like `models=[...]` — ignored here
125
+ ) -> LLMResult:
126
+ if not self.api_key:
127
+ raise RuntimeError(
128
+ "GOOGLE_API_KEY not set. Get a key at "
129
+ "https://aistudio.google.com/app/apikey and add "
130
+ "GOOGLE_API_KEY=... to .env"
131
+ )
132
+
133
+ system_instruction, contents = _to_gemini_contents(messages)
134
+
135
+ generation_config: dict = {
136
+ "temperature": temperature,
137
+ "maxOutputTokens": max_tokens,
138
+ }
139
+ # JSON mode — Gemini's native equivalent of OpenAI response_format.
140
+ if response_format and response_format.get("type") == "json_object":
141
+ generation_config["responseMimeType"] = "application/json"
142
+
143
+ body: dict = {
144
+ "contents": contents,
145
+ "generationConfig": generation_config,
146
+ }
147
+ if system_instruction:
148
+ body["systemInstruction"] = {"parts": [{"text": system_instruction}]}
149
+
150
+ # API key goes in the URL query param — the Google AI Studio default.
151
+ # The header form (`x-goog-api-key`) also works but is undocumented for
152
+ # the v1beta generativelanguage endpoint.
153
+ url = f"{GEMINI_BASE_URL}/{self.model}:generateContent?key={self.api_key}"
154
+ headers = {"Content-Type": "application/json"}
155
+
156
+ # Per-phase timeouts mirroring the NIM/OpenRouter pattern: a stuck
157
+ # connection releases its slot on its own deadline rather than holding
158
+ # past the outer wait_for cancellation.
159
+ client_timeout = httpx.Timeout(
160
+ connect=2.0,
161
+ read=self.timeout,
162
+ write=2.0,
163
+ pool=2.0,
164
+ )
165
+
166
+ async with httpx.AsyncClient(timeout=client_timeout) as client:
167
+ resp = await client.post(url, headers=headers, json=body)
168
+ if resp.status_code >= 400:
169
+ # Surface the Google error body so the caller's log makes the
170
+ # root cause visible (typical failure: 429 quota exceeded or
171
+ # 400 prompt-block).
172
+ detail = ""
173
+ try:
174
+ detail = resp.text[:500]
175
+ except Exception:
176
+ pass
177
+ raise httpx.HTTPStatusError(
178
+ f"Gemini API {resp.status_code}: {detail}",
179
+ request=resp.request,
180
+ response=resp,
181
+ )
182
+ payload = resp.json()
183
+
184
+ # Response shape:
185
+ # {"candidates": [{"content": {"parts": [{"text": "..."}], "role": "model"},
186
+ # "finishReason": "STOP"}],
187
+ # "usageMetadata": {"promptTokenCount": N, "candidatesTokenCount": M, ...}}
188
+ text = ""
189
+ try:
190
+ candidates = payload.get("candidates") or []
191
+ if candidates:
192
+ parts = (candidates[0].get("content") or {}).get("parts") or []
193
+ # Concatenate every text part — Gemini sometimes returns
194
+ # multiple chunks per candidate when streaming-shape leaks
195
+ # into a non-streaming response.
196
+ text = "".join(
197
+ p.get("text", "") for p in parts if isinstance(p, dict)
198
+ )
199
+ except Exception:
200
+ text = ""
201
+
202
+ usage = payload.get("usageMetadata") or {}
203
+ return LLMResult(
204
+ text=text,
205
+ model=self.model,
206
+ prompt_tokens=usage.get("promptTokenCount"),
207
+ completion_tokens=usage.get("candidatesTokenCount"),
208
+ raw=payload,
209
+ )
210
+
211
+
212
+ # ----------------------------------------------------------------------------
213
+ # Factory
214
+ # ----------------------------------------------------------------------------
215
+
216
+ def get_gemini_llm(
217
+ model: str = DEFAULT_MODEL,
218
+ timeout: float = 25.0,
219
+ ) -> GoogleGeminiLLM:
220
+ """Return a fresh GoogleGeminiLLM client.
221
+
222
+ 25s timeout matches the KI-179 brief — Gemini Flash typically responds in
223
+ 1-3s so 25s is a generous outer bound that still bails fast on a quota /
224
+ network stall.
225
+
226
+ Common models:
227
+ - "gemini-2.0-flash" → fast, conversational, sales_brain Tier 0
228
+ - "gemini-2.5-flash" → slightly heavier reasoning, brain main Tier 0
229
+ """
230
+ return GoogleGeminiLLM(model=model, timeout=timeout)
231
+
232
+
233
+ __all__ = ["GoogleGeminiLLM", "get_gemini_llm", "DEFAULT_MODEL"]
backend/providers/openrouter_llm.py CHANGED
@@ -68,13 +68,26 @@ class OpenRouterLLM(LLMProvider):
68
  temperature: float = 0.2,
69
  max_tokens: int = 1024,
70
  response_format: Optional[dict] = None,
 
71
  ) -> LLMResult:
 
 
 
 
 
 
 
 
 
 
72
  body: dict = {
73
- "model": self.model,
74
  "messages": [{"role": m.role, "content": m.content} for m in messages],
75
  "temperature": temperature,
76
  "max_tokens": max_tokens,
77
  }
 
 
78
  if response_format:
79
  body["response_format"] = response_format
80
 
@@ -138,3 +151,52 @@ class OpenRouterLLM(LLMProvider):
138
  completion_tokens=usage.get("completion_tokens"),
139
  raw=payload,
140
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  temperature: float = 0.2,
69
  max_tokens: int = 1024,
70
  response_format: Optional[dict] = None,
71
+ models: Optional[list[str]] = None,
72
  ) -> LLMResult:
73
+ """Send a chat completion to OpenRouter.
74
+
75
+ KI-176 — `models` (optional list of model ids) activates OpenRouter's
76
+ native server-side fallback: the server tries each id in order and
77
+ returns the first that succeeds. When provided, `models[0]` is also
78
+ set as the primary `model` field (OpenRouter requires both for
79
+ routing; if `models` is omitted the lone `model` field is used).
80
+ Reference: https://openrouter.ai/docs/features/model-routing
81
+ """
82
+ primary_model = models[0] if models else self.model
83
  body: dict = {
84
+ "model": primary_model,
85
  "messages": [{"role": m.role, "content": m.content} for m in messages],
86
  "temperature": temperature,
87
  "max_tokens": max_tokens,
88
  }
89
+ if models:
90
+ body["models"] = list(models)
91
  if response_format:
92
  body["response_format"] = response_format
93
 
 
151
  completion_tokens=usage.get("completion_tokens"),
152
  raw=payload,
153
  )
154
+
155
+
156
+ # ----------------------------------------------------------------------------
157
+ # KI-176 — Factory for the OpenRouter free-tier frontier pool.
158
+ #
159
+ # User loaded $10 on OpenRouter; that unlocks 1000 req/day across all `:free`
160
+ # suffix models. This factory returns an OpenRouterLLM whose `model` field is
161
+ # a sane single-model default; callers that want server-side fallback should
162
+ # pass `models=[...]` to `chat()`.
163
+ #
164
+ # Model IDs verified against the LIVE OpenRouter catalog
165
+ # (GET /api/v1/models, 2026-05-15). The model IDs from the KI-176 brief
166
+ # (gemini-2.0-flash-exp:free, llama-3.3-70b-instruct:free,
167
+ # hermes-3-llama-3.1-405b:free) were checked:
168
+ # - gemini-2.0-flash-exp:free — NOT in catalog
169
+ # - llama-3.3-70b-instruct:free — PRESENT but no response_format
170
+ # - hermes-3-llama-3.1-405b:free — PRESENT but no response_format
171
+ # So those would silently lose JSON mode. The IDs below are the actual
172
+ # free-tier set that declare `response_format` (and most also
173
+ # `structured_outputs`) in their `supported_parameters`:
174
+ # - nvidia/nemotron-3-super-120b-a12b:free (120B, structured_outputs)
175
+ # - qwen/qwen3-next-80b-a3b-instruct:free (80B, structured_outputs)
176
+ # - google/gemma-4-31b-it:free (31B, response_format)
177
+ # - minimax/minimax-m2.5:free (?, response_format)
178
+ # - google/gemma-4-26b-a4b-it:free (26B, response_format)
179
+ # ----------------------------------------------------------------------------
180
+
181
+ # Default JSON-mode-capable free-tier model. Callers usually override this by
182
+ # passing `models=[...]` to chat() — this is just the lone-model fallback.
183
+ DEFAULT_FREE_BRAIN_MODEL = "nvidia/nemotron-3-super-120b-a12b:free"
184
+
185
+
186
+ def get_openrouter_llm(
187
+ chain_name: str = "sales_brain",
188
+ model: str = DEFAULT_FREE_BRAIN_MODEL,
189
+ timeout: float = 60.0,
190
+ ) -> "OpenRouterLLM":
191
+ """Return a fresh OpenRouterLLM ready for sales_brain.
192
+
193
+ The returned client is single-use-shaped (one chat() call per turn) and
194
+ intended to be passed `models=[primary, fallback1, fallback2]` so the
195
+ OpenRouter server handles in-pool failover before the caller falls back
196
+ to the NIM chain.
197
+ """
198
+ return OpenRouterLLM(
199
+ model=model,
200
+ timeout=timeout,
201
+ chain_name=chain_name,
202
+ )
backend/providers/tiered_brain_llm.py ADDED
@@ -0,0 +1,229 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """TieredBrainLLM — 3-tier wrapper for the orchestrator's brain main path (KI-179).
2
+
3
+ Architecture:
4
+
5
+ Tier 0 — Google Gemini Flash (best free quality, 1500 req/day)
6
+ Tier 1 — NIM brain chain (NimChainLLM, multi-candidate election)
7
+ Tier 2 — OpenRouter free pool (cross-provider safety net)
8
+
9
+ Why a wrapper class instead of inlining the tiers in orchestrator.py?
10
+
11
+ The orchestrator's brain main path is one of the busiest call-sites in the
12
+ system — faithfulness gates, cross-check retry, drift recovery, eval logging
13
+ all hang off it. Inlining a 3-tier try/except cascade there would explode
14
+ the cyclomatic complexity of an already-long function. Wrapping the
15
+ 3-tier dance behind the SAME `LLMProvider.chat()` signature as `NimChainLLM`
16
+ lets us swap in the wrapper at the construction site with zero downstream
17
+ changes — the existing `await pick.provider.chat(...)` call shape is
18
+ preserved.
19
+
20
+ Match the existing `LLMProvider` interface so this drops into the same
21
+ slot the orchestrator currently fills with `get_brain_llm()` /
22
+ `get_fast_brain_llm()`. `self.model` and `self.name` are updated after a
23
+ successful call to reflect which tier actually served — preserves the
24
+ KI-080 audit trail.
25
+ """
26
+
27
+ from __future__ import annotations
28
+
29
+ import asyncio
30
+ import logging
31
+ from typing import Optional
32
+
33
+ from backend.providers.base import ChatMessage, LLMProvider, LLMResult
34
+ from backend.providers.google_gemini_llm import get_gemini_llm
35
+ from backend.providers.nvidia_nim_llm import (
36
+ get_brain_llm,
37
+ get_fast_brain_llm,
38
+ )
39
+ from backend.providers.openrouter_llm import get_openrouter_llm
40
+
41
+
42
+ # Mirrors sales_brain.py's free-tier pool: every model declares
43
+ # response_format in its supported_parameters so the OR server's native
44
+ # fallback walks an JSON-mode-aware list.
45
+ _OR_MODELS: list[str] = [
46
+ "nvidia/nemotron-3-super-120b-a12b:free",
47
+ "qwen/qwen3-next-80b-a3b-instruct:free",
48
+ "google/gemma-4-31b-it:free",
49
+ ]
50
+
51
+
52
+ class TieredBrainLLM(LLMProvider):
53
+ """3-tier brain wrapper: Gemini → NIM → OpenRouter.
54
+
55
+ `role` is one of "brain" | "fast_brain" — controls which NIM chain
56
+ factory is used for Tier 1:
57
+ - "brain" → get_brain_llm() (BRAIN_CHAIN, 20s per-link)
58
+ - "fast_brain" → get_fast_brain_llm() (FAST_BRAIN_CHAIN, 6s per-link)
59
+
60
+ `gemini_model` defaults to "gemini-2.5-flash" for the heavy brain (slightly
61
+ higher quality on complex synthesis) and can be overridden to
62
+ "gemini-2.0-flash" for the fast brain (lower latency).
63
+
64
+ `per_tier_timeout` is the outer asyncio.wait_for cap applied to EACH
65
+ tier individually. Mirrors the 30s budget the orchestrator's existing
66
+ brain call uses (see orchestrator.py KI-098).
67
+ """
68
+
69
+ name = "tiered-brain"
70
+
71
+ def __init__(
72
+ self,
73
+ role: str = "brain",
74
+ gemini_model: str = "gemini-2.5-flash",
75
+ per_tier_timeout: float = 30.0,
76
+ ):
77
+ if role not in ("brain", "fast_brain"):
78
+ raise ValueError(f"role must be 'brain' or 'fast_brain', got {role!r}")
79
+ self.role = role
80
+ self.gemini_model = gemini_model
81
+ self.per_tier_timeout = per_tier_timeout
82
+ # Start with the Tier 0 model as the "current" model — `chat()`
83
+ # rewrites this to whichever tier actually served the request.
84
+ self.model = gemini_model
85
+ self.name = f"tiered-brain::{role}::gemini:{gemini_model}"
86
+
87
+ async def chat(
88
+ self,
89
+ messages: list[ChatMessage],
90
+ temperature: float = 0.2,
91
+ max_tokens: int = 1024,
92
+ response_format: Optional[dict] = None,
93
+ **kwargs, # absorb provider-specific kwargs (e.g. OR's `models=[...]`)
94
+ ) -> LLMResult:
95
+ gemini_exc: Optional[BaseException] = None
96
+ nim_exc: Optional[BaseException] = None
97
+ or_exc: Optional[BaseException] = None
98
+
99
+ # ---- Tier 0 — Google Gemini ----
100
+ try:
101
+ gemini_llm = get_gemini_llm(model=self.gemini_model, timeout=25.0)
102
+ result = await asyncio.wait_for(
103
+ gemini_llm.chat(
104
+ messages=messages,
105
+ temperature=temperature,
106
+ max_tokens=max_tokens,
107
+ response_format=response_format,
108
+ ),
109
+ timeout=self.per_tier_timeout,
110
+ )
111
+ text = (result.text or "").strip()
112
+ if text:
113
+ # Stamp the actually-served model so faithfulness can exclude it
114
+ # from the judge chain (cross-family-grading invariant).
115
+ self.model = result.model or self.gemini_model
116
+ self.name = f"tiered-brain::{self.role}::gemini:{self.model}"
117
+ return result
118
+ # Empty text from Gemini → treat as failure, fall through.
119
+ logging.info(
120
+ "TieredBrainLLM[%s]: Gemini returned empty text, falling to NIM",
121
+ self.role,
122
+ )
123
+ except (asyncio.CancelledError, KeyboardInterrupt, SystemExit):
124
+ raise
125
+ except asyncio.TimeoutError as e:
126
+ gemini_exc = e
127
+ logging.info(
128
+ "TieredBrainLLM[%s]: Gemini timed out, falling to NIM",
129
+ self.role,
130
+ )
131
+ except Exception as e: # noqa: BLE001 — fall through on ANY provider error
132
+ gemini_exc = e
133
+ logging.info(
134
+ "TieredBrainLLM[%s]: Gemini raised %s, falling to NIM: %s",
135
+ self.role, type(e).__name__, str(e)[:200],
136
+ )
137
+
138
+ # ---- Tier 1 — NIM chain ----
139
+ try:
140
+ nim_llm = (
141
+ get_brain_llm() if self.role == "brain" else get_fast_brain_llm()
142
+ )
143
+ result = await asyncio.wait_for(
144
+ nim_llm.chat(
145
+ messages=messages,
146
+ temperature=temperature,
147
+ max_tokens=max_tokens,
148
+ response_format=response_format,
149
+ ),
150
+ timeout=self.per_tier_timeout,
151
+ )
152
+ text = (result.text or "").strip()
153
+ if text:
154
+ self.model = result.model or getattr(nim_llm, "model", "nim")
155
+ self.name = f"tiered-brain::{self.role}::nim:{self.model}"
156
+ return result
157
+ logging.info(
158
+ "TieredBrainLLM[%s]: NIM returned empty text, falling to OpenRouter",
159
+ self.role,
160
+ )
161
+ except (asyncio.CancelledError, KeyboardInterrupt, SystemExit):
162
+ raise
163
+ except asyncio.TimeoutError as e:
164
+ nim_exc = e
165
+ logging.info(
166
+ "TieredBrainLLM[%s]: NIM timed out, falling to OpenRouter",
167
+ self.role,
168
+ )
169
+ except Exception as e: # noqa: BLE001
170
+ nim_exc = e
171
+ logging.info(
172
+ "TieredBrainLLM[%s]: NIM raised %s, falling to OpenRouter: %s",
173
+ self.role, type(e).__name__, str(e)[:200],
174
+ )
175
+
176
+ # ---- Tier 2 — OpenRouter free pool ----
177
+ try:
178
+ or_llm = get_openrouter_llm(chain_name=f"orchestrator_{self.role}")
179
+ result = await asyncio.wait_for(
180
+ or_llm.chat(
181
+ messages=messages,
182
+ temperature=temperature,
183
+ max_tokens=max_tokens,
184
+ response_format=response_format,
185
+ models=_OR_MODELS,
186
+ ),
187
+ timeout=self.per_tier_timeout,
188
+ )
189
+ self.model = result.model or "openrouter"
190
+ self.name = f"tiered-brain::{self.role}::or:{self.model}"
191
+ return result
192
+ except (asyncio.CancelledError, KeyboardInterrupt, SystemExit):
193
+ raise
194
+ except Exception as e: # noqa: BLE001
195
+ or_exc = e
196
+
197
+ # All three tiers failed — raise a single combined RuntimeError so
198
+ # the orchestrator's existing outer-error handling treats it as a
199
+ # standard brain failure (it already handles `pick.provider.chat`
200
+ # raising — see the wait_for cap at orchestrator.py KI-098).
201
+ raise RuntimeError(
202
+ f"TieredBrainLLM[{self.role}] all tiers failed: "
203
+ f"gemini={type(gemini_exc).__name__ if gemini_exc else 'n/a'}, "
204
+ f"nim={type(nim_exc).__name__ if nim_exc else 'n/a'}, "
205
+ f"or={type(or_exc).__name__ if or_exc else 'n/a'}"
206
+ ) from or_exc
207
+
208
+
209
+ def get_tiered_brain_llm(
210
+ role: str = "brain",
211
+ gemini_model: Optional[str] = None,
212
+ ) -> TieredBrainLLM:
213
+ """Factory for the orchestrator's brain main path.
214
+
215
+ Defaults:
216
+ role="brain" → gemini-2.5-flash + BRAIN_CHAIN (heavy synthesis)
217
+ role="fast_brain" → gemini-2.5-flash-lite + FAST_BRAIN_CHAIN (voice latency)
218
+
219
+ KI-183 (2026-05-15) — gemini-2.0-flash retired for new accounts;
220
+ gemini-2.5-flash-lite is the replacement (faster, cleaner JSON).
221
+ """
222
+ if gemini_model is None:
223
+ gemini_model = (
224
+ "gemini-2.5-flash" if role == "brain" else "gemini-2.5-flash-lite"
225
+ )
226
+ return TieredBrainLLM(role=role, gemini_model=gemini_model)
227
+
228
+
229
+ __all__ = ["TieredBrainLLM", "get_tiered_brain_llm"]
backend/sales_brain.py CHANGED
@@ -56,10 +56,28 @@ from typing import Any, Optional
56
 
57
  from backend.needs_finder import Profile
58
  from backend.providers.base import ChatMessage
 
59
  from backend.providers.nvidia_nim_llm import get_fast_brain_llm
 
60
  from backend.sales_brain_normalizer import normalize_captures
61
 
62
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  # ----------------------------------------------------------------------------
64
  # Public result dataclass — the contract WS2 will consume.
65
  # ----------------------------------------------------------------------------
@@ -376,44 +394,150 @@ async def drive_sales_brain(
376
  messages.append(ChatMessage(role=role, content=str(content)))
377
  messages.append(ChatMessage(role="user", content=user_text or ""))
378
 
379
- # Call the fast-brain NIM chain with response_format=json_object
380
- # (singleton-via-factory see KI-080; the factory builds a fresh
381
- # NimChainLLM each call but the underlying election state is shared
382
- # via backend.llm_health module-level dicts).
383
- llm = get_fast_brain_llm()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
384
  try:
385
- result = await asyncio.wait_for(
386
- llm.chat(
387
- messages=messages,
388
- temperature=_TEMPERATURE,
389
- max_tokens=_MAX_TOKENS,
390
- response_format={"type": "json_object"},
391
- ),
392
- timeout=_TIMEOUT_S,
393
  )
394
- except asyncio.TimeoutError:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
395
  elapsed = time.time() - t0
396
  logging.warning(
397
- "sales_brain timeout (session=%s, %.1fs)", session_id, elapsed,
 
 
 
 
 
398
  )
399
- return SalesBrainResult(
400
- reply_text="",
401
- captured_updates={},
402
- ready_for_recommendations=False,
403
- brain_used="sales_brain::error:timeout",
404
- error_reason=f"timeout_after_{elapsed:.1f}s",
405
- )
406
- except Exception as e: # noqa: BLE001 — broad catch is intentional here
407
- logging.warning(
408
- "sales_brain LLM call failed (session=%s): %s: %s",
409
- session_id, type(e).__name__, str(e)[:200],
410
  )
 
411
  return SalesBrainResult(
412
  reply_text="",
413
  captured_updates={},
414
  ready_for_recommendations=False,
415
- brain_used=f"sales_brain::error:llm_error_{type(e).__name__}",
416
- error_reason=f"{type(e).__name__}: {str(e)[:200]}",
 
 
 
 
 
417
  )
418
 
419
  # Parse the JSON object from the LLM response.
@@ -510,7 +634,7 @@ async def drive_sales_brain(
510
  reply_text=retry_reply,
511
  captured_updates=captured_updates,
512
  ready_for_recommendations=ready_for_recommendations,
513
- brain_used=f"sales_brain::nim:{retry_model}::retry",
514
  raw_json=retry_parsed,
515
  error_reason=None,
516
  )
@@ -533,7 +657,7 @@ async def drive_sales_brain(
533
  reply_text=reply_text,
534
  captured_updates=captured_updates,
535
  ready_for_recommendations=ready_for_recommendations,
536
- brain_used=f"sales_brain::nim:{served_model}",
537
  raw_json=parsed,
538
  error_reason=None,
539
  )
 
56
 
57
  from backend.needs_finder import Profile
58
  from backend.providers.base import ChatMessage
59
+ from backend.providers.google_gemini_llm import get_gemini_llm
60
  from backend.providers.nvidia_nim_llm import get_fast_brain_llm
61
+ from backend.providers.openrouter_llm import get_openrouter_llm
62
  from backend.sales_brain_normalizer import normalize_captures
63
 
64
 
65
+ # ----------------------------------------------------------------------------
66
+ # KI-176 — OpenRouter free-tier frontier pool (PRIMARY tier).
67
+ #
68
+ # User loaded $10 on OpenRouter which unlocks 1000 req/day on `:free` models.
69
+ # These IDs were verified against the LIVE OpenRouter catalog and ALL declare
70
+ # `response_format` in their supported_parameters (the first two also declare
71
+ # `structured_outputs`). The list is passed to OpenRouter's native `models`
72
+ # array for server-side in-pool fallback BEFORE we fall through to NIM.
73
+ # ----------------------------------------------------------------------------
74
+ _OR_MODELS: list[str] = [
75
+ "nvidia/nemotron-3-super-120b-a12b:free", # 120B, structured_outputs
76
+ "qwen/qwen3-next-80b-a3b-instruct:free", # 80B, structured_outputs
77
+ "google/gemma-4-31b-it:free", # 31B, response_format
78
+ ]
79
+
80
+
81
  # ----------------------------------------------------------------------------
82
  # Public result dataclass — the contract WS2 will consume.
83
  # ----------------------------------------------------------------------------
 
394
  messages.append(ChatMessage(role=role, content=str(content)))
395
  messages.append(ChatMessage(role="user", content=user_text or ""))
396
 
397
+ # KI-179 (2026-05-15) — 3-tier LLM stack with Google Gemini 2.0 Flash
398
+ # PRIMARY (Tier 0), NIM fast-brain chain Tier 1, OpenRouter free-tier
399
+ # pool Tier 2. Google AI Studio's free tier is 1500 req/day with native
400
+ # JSON mode; quality > NIM/OR free tiers for conversational fact-find.
401
+ #
402
+ # Order: Gemini → NIM → OR → final error.
403
+ # `llm` is rebound to whichever provider served so the empty-reply
404
+ # retry path (KI-169, further down) re-uses the SAME provider that
405
+ # just answered — no re-tier on retry.
406
+ llm = None
407
+ result = None
408
+ served_tier: str = "" # "gemini" | "nim" | "or"
409
+ gemini_exc: Optional[BaseException] = None
410
+ nim_exc: Optional[BaseException] = None
411
+ or_exc: Optional[BaseException] = None
412
+
413
+ # ---- Tier 0 — Google Gemini 2.5 Flash Lite ----
414
+ # KI-183 (2026-05-15) — gemini-2.0-flash is retired for new accounts
415
+ # (HTTP 404 "no longer available to new users"). gemini-2.5-flash-lite
416
+ # is the supported replacement: faster (~900ms vs ~1100ms for 2.5-flash),
417
+ # cleaner JSON-mode output, same free quota.
418
+ gemini_llm = None
419
  try:
420
+ gemini_llm = get_gemini_llm(model="gemini-2.5-flash-lite", timeout=25.0)
421
+ except Exception as e: # noqa: BLE001
422
+ gemini_exc = e
423
+ logging.info(
424
+ "sales_brain: Gemini unavailable, skipping to NIM (session=%s): %s: %s",
425
+ session_id, type(e).__name__, str(e)[:200],
 
 
426
  )
427
+
428
+ if gemini_llm is not None:
429
+ try:
430
+ result = await asyncio.wait_for(
431
+ gemini_llm.chat(
432
+ messages=messages,
433
+ temperature=_TEMPERATURE,
434
+ max_tokens=_MAX_TOKENS,
435
+ response_format={"type": "json_object"},
436
+ ),
437
+ timeout=_TIMEOUT_S,
438
+ )
439
+ llm = gemini_llm
440
+ served_tier = "gemini"
441
+ except asyncio.TimeoutError as e:
442
+ gemini_exc = e
443
+ logging.info(
444
+ "sales_brain: Gemini timed out, falling back to NIM (session=%s)",
445
+ session_id,
446
+ )
447
+ except Exception as e: # noqa: BLE001
448
+ gemini_exc = e
449
+ logging.info(
450
+ "sales_brain: Gemini raised %s, falling back to NIM (session=%s): %s",
451
+ type(e).__name__, session_id, str(e)[:200],
452
+ )
453
+
454
+ # ---- Tier 1 — NIM fast-brain chain ----
455
+ if result is None:
456
+ nim_llm = get_fast_brain_llm()
457
+ try:
458
+ result = await asyncio.wait_for(
459
+ nim_llm.chat(
460
+ messages=messages,
461
+ temperature=_TEMPERATURE,
462
+ max_tokens=_MAX_TOKENS,
463
+ response_format={"type": "json_object"},
464
+ ),
465
+ timeout=_TIMEOUT_S,
466
+ )
467
+ llm = nim_llm
468
+ served_tier = "nim"
469
+ except asyncio.TimeoutError as e:
470
+ nim_exc = e
471
+ logging.info(
472
+ "sales_brain: NIM timed out, falling back to OpenRouter (session=%s)",
473
+ session_id,
474
+ )
475
+ except Exception as e: # noqa: BLE001
476
+ nim_exc = e
477
+ logging.info(
478
+ "sales_brain: NIM raised %s, falling back to OpenRouter (session=%s): %s",
479
+ type(e).__name__, session_id, str(e)[:200],
480
+ )
481
+
482
+ # ---- Tier 2 — OpenRouter free-tier frontier pool ----
483
+ if result is None:
484
+ or_llm = None
485
+ try:
486
+ or_llm = get_openrouter_llm(chain_name="sales_brain")
487
+ except Exception as e: # noqa: BLE001 — missing API key etc.
488
+ or_exc = e
489
+ logging.info(
490
+ "sales_brain: OpenRouter unavailable (session=%s): %s: %s",
491
+ session_id, type(e).__name__, str(e)[:200],
492
+ )
493
+
494
+ if or_llm is not None:
495
+ try:
496
+ result = await asyncio.wait_for(
497
+ or_llm.chat(
498
+ messages=messages,
499
+ temperature=_TEMPERATURE,
500
+ max_tokens=_MAX_TOKENS,
501
+ response_format={"type": "json_object"},
502
+ models=_OR_MODELS,
503
+ ),
504
+ timeout=_TIMEOUT_S,
505
+ )
506
+ llm = or_llm
507
+ served_tier = "or"
508
+ except asyncio.TimeoutError as e:
509
+ or_exc = e
510
+ except Exception as e: # noqa: BLE001
511
+ or_exc = e
512
+
513
+ # ---- All three tiers exhausted — final error ----
514
+ if result is None:
515
  elapsed = time.time() - t0
516
  logging.warning(
517
+ "sales_brain ALL TIERS FAILED (session=%s, %.1fs, "
518
+ "gemini_exc=%s, nim_exc=%s, or_exc=%s)",
519
+ session_id, elapsed,
520
+ f"{type(gemini_exc).__name__}" if gemini_exc else "n/a",
521
+ f"{type(nim_exc).__name__}" if nim_exc else "n/a",
522
+ f"{type(or_exc).__name__}" if or_exc else "n/a",
523
  )
524
+ # Classify reason: timeout-everywhere is its own bucket vs mixed errors.
525
+ all_timeouts = all(
526
+ isinstance(e, asyncio.TimeoutError) for e in (gemini_exc, nim_exc, or_exc)
527
+ if e is not None
 
 
 
 
 
 
 
528
  )
529
+ reason_tag = "timeout_all_tiers" if all_timeouts else "llm_error_all_tiers"
530
  return SalesBrainResult(
531
  reply_text="",
532
  captured_updates={},
533
  ready_for_recommendations=False,
534
+ brain_used=f"sales_brain::error:{reason_tag}",
535
+ error_reason=(
536
+ f"gemini_exc={type(gemini_exc).__name__ if gemini_exc else 'n/a'}; "
537
+ f"nim_exc={type(nim_exc).__name__ if nim_exc else 'n/a'}; "
538
+ f"or_exc={type(or_exc).__name__ if or_exc else 'n/a'}; "
539
+ f"elapsed={elapsed:.1f}s"
540
+ ),
541
  )
542
 
543
  # Parse the JSON object from the LLM response.
 
634
  reply_text=retry_reply,
635
  captured_updates=captured_updates,
636
  ready_for_recommendations=ready_for_recommendations,
637
+ brain_used=f"sales_brain::{served_tier}:{retry_model}::retry",
638
  raw_json=retry_parsed,
639
  error_reason=None,
640
  )
 
657
  reply_text=reply_text,
658
  captured_updates=captured_updates,
659
  ready_for_recommendations=ready_for_recommendations,
660
+ brain_used=f"sales_brain::{served_tier}:{served_model}",
661
  raw_json=parsed,
662
  error_reason=None,
663
  )
eval/README.md CHANGED
@@ -10,7 +10,7 @@ Numerical accuracy + grounding eval. Walks a fixed list of curated questions thr
10
  | --- | --- |
11
  | `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
12
  | `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
13
- | `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts + NIM Mistral Large 3 judge chain (different family from the NIM Qwen brain → non-circular). Falls through to Groq Llama only at the cross-provider tail of `JUDGE_CHAIN`. |
14
  | `results.json` | Machine-readable last-run results. |
15
  | `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
16
  | `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
@@ -39,7 +39,7 @@ python -m eval.generate_gold
39
  | Layer | What it checks | Output field |
40
  | --- | --- | --- |
41
  | Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
42
- | LLM judge (NIM Mistral Large 3 675B primary; OpenAI GPT-OSS 120B / Moonshot Kimi K2 / Llama-4 Maverick / Groq Llama-3.3-70B fallbacks) | Reply *semantically* answers the question and cites the expected source. Different family from the NIM Qwen brain → non-circular. | `judge_pass` |
43
  | Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
44
 
45
  ## Acceptance bar
@@ -51,4 +51,5 @@ Per `80-audit/ENTERPRISE_AUDIT.md` D-003: **≥ 90% on the 96-Q gold-QA set is t
51
  - [`backend/faithfulness.py`](../backend/faithfulness.py) — the 4-gate guard that produces the `blocked` verdict
52
  - [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) § "Live bot replies" — provenance trail for any flagged reply
53
  - `tools/info_source_map.py` — generator of `info_source_map.json`
54
- - [ADR-014](../70-docs/60-decisions/ADR-014-groq-llama-grader.md) (superseded — but the Groq-judge-of-different-family principle survives)
 
 
10
  | --- | --- |
11
  | `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
12
  | `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
13
+ | `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts + `JUDGE_CHAIN` (NIM Mistral Large 3 675B primary; different family from the Gemini brain → non-circular). Falls through to NIM Llama-4 Maverick → OpenRouter Qwen 80B `:free` → NIM Nemotron 49B at the tail. Emits `brain_used` like `sales_brain::continue` / `sales_brain::complete` / `brain_main::<elected_model>` / `brain_main::graceful_error`. |
14
  | `results.json` | Machine-readable last-run results. |
15
  | `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
16
  | `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
 
39
  | Layer | What it checks | Output field |
40
  | --- | --- | --- |
41
  | Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
42
+ | LLM judge (`JUDGE_CHAIN`: NIM Mistral Large 3 675B primary; NIM Llama-4 Maverick / OR Qwen 80B `:free` / NIM Nemotron 49B fallbacks per [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)) | Reply *semantically* answers the question and cites the expected source. Different family from the Gemini brain → non-circular. | `judge_pass` |
43
  | Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
44
 
45
  ## Acceptance bar
 
51
  - [`backend/faithfulness.py`](../backend/faithfulness.py) — the 4-gate guard that produces the `blocked` verdict
52
  - [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) § "Live bot replies" — provenance trail for any flagged reply
53
  - `tools/info_source_map.py` — generator of `info_source_map.json`
54
+ - [ADR-014](../70-docs/60-decisions/ADR-014-groq-llama-grader.md) (superseded — but the cross-family judge principle survives; current judge is NIM Mistral 675B, brain is Gemini)
55
+ - [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md) — current chain shape (Google → NIM → OpenRouter)
frontend/README.md CHANGED
@@ -13,12 +13,15 @@ For Claude / agent-specific rules (e.g. the "this is NOT the Next.js you know" n
13
  | `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
14
  | `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
15
  | `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)). |
 
16
  | `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
17
 
18
  ## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md))
19
 
20
  - **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
21
  - **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
 
 
22
  - **Bot TTS plays via the in-DOM `<audio>` element inside `Message`** (autoplay-on-mount via ref'd `useEffect`). **Never use `new Audio(url).play()`** — those detached instances are invisible to the `document.querySelectorAll("audio").pause()` call in the barge-in handler, so they keep playing under the user's speech.
23
 
24
  ## Tooling
 
13
  | `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
14
  | `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
15
  | `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)). |
16
+ | `src/lib/useStreamingVoice.ts` | **KI-168 hybrid voice capture.** Runs Web Speech API (interim transcripts streamed live to the chat input) and `MediaRecorder` (authoritative audio blob) in parallel; on browser silence-detect the blob is POSTed to `/api/transcribe` (Sarvam Saarika STT). Auto-submits the Sarvam transcript; falls back to the Web Speech transcript only if Sarvam errors. KI-173 heartbeat + KI-174 `visibilitychange` / `focus` revival hooks keep the mic alive across tab / app switches. |
17
  | `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
18
 
19
  ## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md))
20
 
21
  - **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
22
  - **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
23
+ - **Hybrid capture (KI-168).** Web Speech API drives the live interim transcript on the chat input; `MediaRecorder` captures the authoritative blob in parallel. On browser silence-detect the blob is POSTed to `/api/transcribe` (Sarvam) for the authoritative transcript; Web Speech is the fallback only if Sarvam errors. See `src/lib/useStreamingVoice.ts`.
24
+ - **Mic survives tab / app switches (KI-173 / KI-174).** Heartbeat keeps the mic stream alive while the tab is backgrounded; `visibilitychange` / `focus` revival hooks reattach the mic on tab return.
25
  - **Bot TTS plays via the in-DOM `<audio>` element inside `Message`** (autoplay-on-mount via ref'd `useEffect`). **Never use `new Audio(url).play()`** — those detached instances are invisible to the `document.querySelectorAll("audio").pause()` call in the barge-in handler, so they keep playing under the user's speech.
26
 
27
  ## Tooling
frontend/public/admin/llm-control.html CHANGED
@@ -727,6 +727,7 @@
727
  function $(id) { return document.getElementById(id); }
728
 
729
  function clearChildren(el) {
 
730
  while (el.firstChild) el.removeChild(el.firstChild);
731
  }
732
 
 
727
  function $(id) { return document.getElementById(id); }
728
 
729
  function clearChildren(el) {
730
+ if (!el) return;
731
  while (el.firstChild) el.removeChild(el.firstChild);
732
  }
733
 
kb/AUDIT_TRAIL.md CHANGED
@@ -30,27 +30,32 @@ PARSE CHUNK EMBED
30
 
31
  [7] [8] [9]
32
  VECTOR INDEX STRUCTURED EXTRACTION SCORECARD
33
- - Chroma persistent - Sarvam-M extraction - rules-based 6-sub-score
34
  client with HealthPolicy weighted aggregation
35
  - metadata per chunk Pydantic schema - A-F grade
36
- - rag/vectors/*.sqlite3 - DeepSeek-V3 fallback - 24 of 62 fields used
37
  → rag/extracted/*.json
38
  → rag/policies.duckdb
39
 
40
  [10] [11] [12]
41
  RETRIEVAL ORCHESTRATION FAITHFULNESS GATES
42
- - Voyage at query time - intent classifier - Gate 1: retrieval floor
43
- (BGE for matching) - brain router (Sarvam-M - Gate 2: citation integrity
44
- - top-k cosine → Llama/DeepSeek) - Gate 3: regex numeric
45
- - per-policy filter - persona prompt grounding
46
- - chat history - Gate 4: LLM-judge
 
 
 
47
 
48
  [13] [14] [15]
49
  RESPONSE TTS UI RENDER
50
- - reply_text + citations - Sarvam Bulbul - chat bubble
51
- - brain used, latency - audio response - source links per
52
- - faithfulness verdict - language match citation
53
- - scorecard (P1)
 
 
54
  ```
55
 
56
  ## 2. Every artifact at every stage
@@ -65,7 +70,7 @@ RESPONSE TTS UI RENDER
65
  | 5 CHUNK | chunks (in-memory) | _ephemeral_ | `rag/ingest.py:chunk_pages` | run ingest with `--dry-run` |
66
  | 6 EMBED | 384-dim vectors | _embedded in Chroma_ | `backend/providers/local_embeddings.py` | re-encode the same text |
67
  | 7 VECTOR INDEX | persistent sqlite | `rag/vectors/chroma.sqlite3` + HNSW binaries | `chromadb.PersistentClient` | open with chromadb client |
68
- | 8 STRUCTURED EXTRACTION | 62-field JSON per policy | `rag/extracted/<policy_id>.json` | `rag/extract.py` (Sarvam-M DeepSeek-V3 fallback) | re-run extraction; compare |
69
  | 8 STRUCTURED EXTRACTION | aggregate table | `rag/policies.duckdb` | upsert in `rag/extract.py` | `duckdb` CLI query |
70
  | 9 SCORECARD | per-policy grade | `kb/policies/<policy_id>.md` (live in code via `backend/scorecard.py`) | `rag/build_kb.py` + `backend/scorecard.py` | re-run `build_kb` |
71
  | 10 RETRIEVAL (runtime) | top-k chunks | _ephemeral, logged_ | `rag/retrieve.py` | replay query against Chroma |
@@ -88,7 +93,7 @@ RESPONSE TTS UI RENDER
88
  - **Risk if tampered**: silent retrieval poisoning. Mitigation: chunk content stored alongside the embedding; can be re-verified against source PDF on demand.
89
 
90
  ### Extracted structured data (Stage 8)
91
- - **Provenance**: LLM (Sarvam-M / DeepSeek-V3) over full PDF text + Pydantic schema
92
  - **Verifiable**: `extraction_confidence_pct` field per record + re-runnable in <30s with `python -m rag.extract --policy <id>`
93
  - **Risk if tampered**: bad downstream scorecard + filter results. Mitigation: schema validates types; manual spot-check 5%.
94
 
@@ -134,7 +139,7 @@ Every architectural decision in `70-docs/decisions.md` produces a specific artif
134
  Honesty about gaps:
135
 
136
  - **Insurer-side PDF tampering** — we trust the insurer's published PDF was real at download time. We don't have a re-fetch + diff pipeline yet. v2 enhancement.
137
- - **LLM determinism** — Sarvam-M / DeepSeek-V3 can produce slightly different extraction across runs even at `temperature=0`. We pin model versions in `backend/config.py`; for full reproducibility we'd cache extraction outputs. v2 enhancement.
138
  - **Embedding model drift** — if BGE updates, our cached vectors become stale relative to query-time embeddings. v2: pin model commit hash.
139
  - **Chroma version compatibility** — different `chromadb` releases store metadata differently (we hit `KeyError: '_type'` on first HF deploy). Pinned to `0.5.20` in `requirements.txt`.
140
 
@@ -194,5 +199,19 @@ Three back-to-back curation passes brought the `40-data/policy_facts/` directory
194
  - **EN ↔ हिं i18n** — full bilingual UI with the 13-term jargon glossary at `frontend/src/lib/i18n.ts` (mirrored to `kb/methodology/glossary.json`).
195
  - **Scorecard methodology expander** — every grade opens a transparency panel sourced from `METHODOLOGY_BLUEPRINT` (mirrored to `kb/methodology/scorecard.json`).
196
  - **Source-quote popovers** — hovering a fact on a policy card surfaces the verbatim PDF quote that backed it.
197
- - **2026-05-14 Stack A consolidation (D-019)** — 4 third-party LLM providers (Groq, OpenRouter, direct DeepSeek, Cerebras) collapsed onto a single NVIDIA NIM endpoint. Brain = Qwen 80B (current brain primary) (heavy intents) + V4-Flash (voice + fact-find). Judge = Meta Mistral Large 3 675B (different family). Sarvam scoped to voice STT/TTS + Indic translation only. Free tier, 40 req/min, no daily cap. ~600 LOC of legacy provider wiring deleted.
198
  - **2026-05-14 Space/data split (D-020)** — `rag/corpus/` (188 MB), `rag/vectors/` (129 MB), and `rag/extracted/` moved to companion HF dataset `rohitsar567/insurance-bot-data` (50 GB free quota). Dockerfile pulls them at image build time. Space repo collapses from 286 MB → ~3 MB code-only.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  [7] [8] [9]
32
  VECTOR INDEX STRUCTURED EXTRACTION SCORECARD
33
+ - Chroma persistent - NimChainLLM(BRAIN_CHAIN)- rules-based 6-sub-score
34
  client with HealthPolicy weighted aggregation
35
  - metadata per chunk Pydantic schema - A-F grade
36
+ - rag/vectors/*.sqlite3 - native JSON mode - 24 of 62 fields used
37
  → rag/extracted/*.json
38
  → rag/policies.duckdb
39
 
40
  [10] [11] [12]
41
  RETRIEVAL ORCHESTRATION FAITHFULNESS GATES
42
+ - BGE-small at query time - intent classifier - Gate 1: retrieval floor
43
+ (local CPU, 384-d) - sales_brain (fact-find) - Gate 2: citation integrity
44
+ - top-k cosine / BRAIN_CHAIN (QA, etc) - Gate 3: regex numeric
45
+ - per-policy filter - persona prompt grounding
46
+ - profile_rag (session- - chat history - Gate 4: LLM-judge
47
+ scoped, KI-102) (NIM Mistral 675B,
48
+ skipped on fact-find +
49
+ recommendation, KI-171)
50
 
51
  [13] [14] [15]
52
  RESPONSE TTS UI RENDER
53
+ - reply_text + citations - Sarvam Bulbul v2 - chat bubble
54
+ - brain used (Gemini / - voice_format strips - source links per
55
+ NIM / OR + model id), CoT + ₹ shorthand citation
56
+ latency - base64 in same JSON - scorecard (P1)
57
+ - faithfulness verdict response - in-DOM <audio> for
58
+ barge-in compatibility
59
  ```
60
 
61
  ## 2. Every artifact at every stage
 
70
  | 5 CHUNK | chunks (in-memory) | _ephemeral_ | `rag/ingest.py:chunk_pages` | run ingest with `--dry-run` |
71
  | 6 EMBED | 384-dim vectors | _embedded in Chroma_ | `backend/providers/local_embeddings.py` | re-encode the same text |
72
  | 7 VECTOR INDEX | persistent sqlite | `rag/vectors/chroma.sqlite3` + HNSW binaries | `chromadb.PersistentClient` | open with chromadb client |
73
+ | 8 STRUCTURED EXTRACTION | 62-field JSON per policy | `rag/extracted/<policy_id>.json` | `rag/extract.py` (`NimChainLLM(BRAIN_CHAIN)` with native JSON mode) | re-run extraction; compare |
74
  | 8 STRUCTURED EXTRACTION | aggregate table | `rag/policies.duckdb` | upsert in `rag/extract.py` | `duckdb` CLI query |
75
  | 9 SCORECARD | per-policy grade | `kb/policies/<policy_id>.md` (live in code via `backend/scorecard.py`) | `rag/build_kb.py` + `backend/scorecard.py` | re-run `build_kb` |
76
  | 10 RETRIEVAL (runtime) | top-k chunks | _ephemeral, logged_ | `rag/retrieve.py` | replay query against Chroma |
 
93
  - **Risk if tampered**: silent retrieval poisoning. Mitigation: chunk content stored alongside the embedding; can be re-verified against source PDF on demand.
94
 
95
  ### Extracted structured data (Stage 8)
96
+ - **Provenance**: `NimChainLLM(BRAIN_CHAIN)` over full PDF text + Pydantic schema, native JSON mode
97
  - **Verifiable**: `extraction_confidence_pct` field per record + re-runnable in <30s with `python -m rag.extract --policy <id>`
98
  - **Risk if tampered**: bad downstream scorecard + filter results. Mitigation: schema validates types; manual spot-check 5%.
99
 
 
139
  Honesty about gaps:
140
 
141
  - **Insurer-side PDF tampering** — we trust the insurer's published PDF was real at download time. We don't have a re-fetch + diff pipeline yet. v2 enhancement.
142
+ - **LLM determinism** — every brain candidate (Gemini, Qwen, Mistral, Maverick) can produce slightly different extraction across runs even at `temperature=0`. Model versions are pinned in `backend/config.py`; for full reproducibility we'd cache extraction outputs. v2 enhancement.
143
  - **Embedding model drift** — if BGE updates, our cached vectors become stale relative to query-time embeddings. v2: pin model commit hash.
144
  - **Chroma version compatibility** — different `chromadb` releases store metadata differently (we hit `KeyError: '_type'` on first HF deploy). Pinned to `0.5.20` in `requirements.txt`.
145
 
 
199
  - **EN ↔ हिं i18n** — full bilingual UI with the 13-term jargon glossary at `frontend/src/lib/i18n.ts` (mirrored to `kb/methodology/glossary.json`).
200
  - **Scorecard methodology expander** — every grade opens a transparency panel sourced from `METHODOLOGY_BLUEPRINT` (mirrored to `kb/methodology/scorecard.json`).
201
  - **Source-quote popovers** — hovering a fact on a policy card surfaces the verbatim PDF quote that backed it.
202
+ - **2026-05-14 Stack A consolidation (D-019)** — 4 third-party LLM providers (Groq, OpenRouter, direct DeepSeek, Cerebras) collapsed onto a single NVIDIA NIM endpoint. Brain = Qwen 80B (heavy intents) + Nemotron Nano 30B (voice + fact-find). Judge = Mistral Large 3 675B (different family). Sarvam scoped to voice STT/TTS + Indic translation only. Free tier, 40 req/min, no daily cap. ~600 LOC of legacy provider wiring deleted.
203
  - **2026-05-14 Space/data split (D-020)** — `rag/corpus/` (188 MB), `rag/vectors/` (129 MB), and `rag/extracted/` moved to companion HF dataset `rohitsar567/insurance-bot-data` (50 GB free quota). Dockerfile pulls them at image build time. Space repo collapses from 286 MB → ~3 MB code-only.
204
+
205
+ ## Batch — 2026-05-15
206
+
207
+ The day's sprint produced a chain-level rewrite, a provider integration, and a fact-find rip-out. See [`70-docs/40-evaluation/quality-sprint-2026-05-14.md`](../70-docs/40-evaluation/quality-sprint-2026-05-14.md) (Follow-on sprint section) for the full KI list.
208
+
209
+ - **KI-167 / [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)** — Ripped out `backend/fact_find_brain.py` (441 LOC) + `_canonical_fallback` + the `<FF>{...}</FF>` trailer convention. Replaced with `backend/sales_brain.py` — one LLM call per turn against `FAST_BRAIN_CHAIN` using native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM). New deterministic post-processor `backend/sales_brain_normalizer.py` validates and normalises captured fields.
210
+ - **KI-168** — Hybrid voice capture: Web Speech API streams interim transcripts to chat input (live UX feel) while `MediaRecorder` runs in parallel; on browser silence-detect the audio blob is POSTed to `/api/transcribe` (Sarvam Saarika v2.5) for the authoritative transcript. Auto-submits the Sarvam result; falls back to Web Speech only if Sarvam errors.
211
+ - **KI-171** — Skip faithfulness Gate 4 (LLM judge) on `fact_find` + `recommendation` intents — no retrieved context to grade against on those turns.
212
+ - **KI-173 / KI-174** — Voice heartbeat + `visibilitychange` / `focus` revival keeps the mic alive across tab / app switches.
213
+ - **KI-175 / KI-176** — Chain reorder: NIM Nemotron 49B demoted from primary to last resort across all three chains (Mistral 675B / Qwen 80B consistently outperform on live audit). OpenRouter re-added as cross-provider diversity using OR's `models: [...]` server-side fallback within the OR free pool.
214
+ - **KI-178** — Live audit of which OpenRouter free-tier models support `response_format`. Only NIM-style native JSON mode is supported on `nvidia/nemotron-3-super-120b-a12b:free`, `qwen/qwen3-next-80b-a3b-instruct:free`, and `google/gemma-4-31b:free`. Llama 3.3 70B / Hermes 3 405B on OR free do NOT — excluded from chains.
215
+ - **KI-179 / [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)** — Operator obtained a Google AI Studio API key. New `backend/providers/google_gemini_llm.py` wraps Google's REST API (`gemini-2.0-flash` for Brain Fast / sales_brain, `gemini-2.5-flash` for Brain Main / synthesis; native JSON mode via `response_mime_type=application/json`; 1500 req/day free quota). Promoted to **Tier 0 primary** on Brain Fast + Brain Main. NIM stays as Tier 1 fallback (Mistral 675B / Maverick / Qwen 80B); OpenRouter `:free` sits as Tier 2 diversity. **Judge stays on NIM Mistral Large 3 675B** (different family from Gemini — preserves brain ↔ judge family-diversity invariant). NIM-only lock from [ADR-038](../70-docs/60-decisions/ADR-038-nim-only-chains.md) relaxed (the `<FF>` trailer convention that motivated the lock is permanently retired post-KI-167). KI-080 sticky-primary election machinery extends naturally to the Google provider.
216
+
217
+ **Net architectural shape after the sprint:** 3-tier chains across Google → NIM → OpenRouter for Brain Fast + Brain Main; NIM-only with OR `:free` diversity for Judge. Sarvam unchanged (STT + TTS + Indic translation). All structured-output traffic uses native provider JSON mode; no trailer conventions remain.
rag/README.md CHANGED
@@ -12,7 +12,7 @@ Lineage for every artefact below is documented in [`kb/AUDIT_TRAIL.md`](../kb/AU
12
  | `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
13
  | `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
14
  | `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)). |
15
- | `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields). `get_brain_llm()` (Qwen 3-Next 80B primary) + `get_fast_brain_llm()` (Nemotron Nano 30B) — both `NimChainLLM` fallback chains. Writes `rag/extracted/<policy_id>.json` + upserts `policies.duckdb`. |
16
  | `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
17
  | `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
18
 
 
12
  | `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
13
  | `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
14
  | `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)). |
15
+ | `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields). `get_brain_llm()` (Brain Main: Gemini 2.5 Flash primary per [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)) + `get_fast_brain_llm()` (Brain Fast: Gemini 2.0 Flash primary) — both `NimChainLLM` fallback chains across Google → NIM → OpenRouter. Native JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM). Writes `rag/extracted/<policy_id>.json` + upserts `policies.duckdb`. |
16
  | `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
17
  | `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
18
 
tools/README.md CHANGED
@@ -41,15 +41,15 @@ Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CR
41
  | --- | --- |
42
  | `upload_to_hf.py` | Code-side push to the HF Space repo (`huggingface.co/spaces/rohitsar567/InsuranceBot`). |
43
  | `upload_corpus_to_dataset.py`, `upload_extracted_to_dataset.py`, `upload_vectors_to_dataset.py`, `upload_all_to_dataset.py` | Push specific slices of `rag/` to the companion HF Dataset `rohitsar567/insurance-bot-data`. See [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) and [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md). |
44
- | `set_hf_secrets.py` | One-shot helper that pushes the runtime secrets into the HF Space (idempotent). |
45
 
46
  ## Probes + diagnostics
47
 
48
  | Script | Provider it pokes |
49
  | --- | --- |
50
  | `sarvam_probe.py`, `sarvam_nothink_probe.py` | Sarvam-M / Saarika / Bulbul connectivity + latency. |
51
- | `groq_probe.py`, `groq_long_probe.py` | Groq Llama free-tier latency + sustained-rate test. |
52
- | `openrouter_probe.py`, `or_models.py` | OpenRouter routing + model-list inspection. |
53
  | `pdf_probe.py` | pdfplumber parse on a single PDF — first stop when extraction silently produces empty text. |
54
  | `heavy_smoke_test.py` | End-to-end smoke against the live HF Space (every provider in one call). |
55
 
 
41
  | --- | --- |
42
  | `upload_to_hf.py` | Code-side push to the HF Space repo (`huggingface.co/spaces/rohitsar567/InsuranceBot`). |
43
  | `upload_corpus_to_dataset.py`, `upload_extracted_to_dataset.py`, `upload_vectors_to_dataset.py`, `upload_all_to_dataset.py` | Push specific slices of `rag/` to the companion HF Dataset `rohitsar567/insurance-bot-data`. See [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) and [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md). |
44
+ | `set_hf_secrets.py` | One-shot helper that pushes the runtime secrets into the HF Space (idempotent). Current secret set: `GOOGLE_API_KEY` (Google AI Studio, per [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)), `NVIDIA_NIM_API_KEY`, `OPENROUTER_API_KEY`, `SARVAM_API_KEY`, plus admin password / IP allowlist. |
45
 
46
  ## Probes + diagnostics
47
 
48
  | Script | Provider it pokes |
49
  | --- | --- |
50
  | `sarvam_probe.py`, `sarvam_nothink_probe.py` | Sarvam-M / Saarika / Bulbul connectivity + latency. |
51
+ | `groq_probe.py`, `groq_long_probe.py` | Historical Groq Llama free-tier probe Groq is no longer in any production chain (removed in [ADR-038](../70-docs/60-decisions/ADR-038-nim-only-chains.md), not re-added in [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md)). Kept for benchmarking. |
52
+ | `openrouter_probe.py`, `or_models.py` | OpenRouter routing + model-list inspection. Used by KI-178 to audit which `:free` models expose `response_format`. |
53
  | `pdf_probe.py` | pdfplumber parse on a single PDF — first stop when extraction silently produces empty text. |
54
  | `heavy_smoke_test.py` | End-to-end smoke against the live HF Space (every provider in one call). |
55
 
tools/set_hf_secrets.py CHANGED
@@ -31,6 +31,7 @@ REPO_ID = "rohitsar567/InsuranceBot"
31
  SECRETS_TO_SET = [
32
  "SARVAM_API_KEY", "VOYAGE_API_KEY", "NVIDIA_NIM_API_KEY",
33
  "OPENROUTER_API_KEY", "GROQ_API_KEY",
 
34
  "ADMIN_IP_ALLOWLIST", "ADMIN_PASSWORD",
35
  ]
36
  # Truly retired providers — delete from Space to prevent confusion.
 
31
  SECRETS_TO_SET = [
32
  "SARVAM_API_KEY", "VOYAGE_API_KEY", "NVIDIA_NIM_API_KEY",
33
  "OPENROUTER_API_KEY", "GROQ_API_KEY",
34
+ "GOOGLE_API_KEY", # KI-179 (2026-05-15) — Gemini Flash via Google AI Studio
35
  "ADMIN_IP_ALLOWLIST", "ADMIN_PASSWORD",
36
  ]
37
  # Truly retired providers — delete from Space to prevent confusion.