rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
23b8fad
·
1 Parent(s): 1a166f9

chore(cleanup): purge stale narrative/tombstones/dead code — codebase reads as the current standard

Browse files

Comprehensive stale-content purge so code/docs read as if written correctly
from the start; history of record stays only in ADRs (70-docs/60-decisions),
KIs (80-audit), and the README.

Code (.py): removed mistake/removal/incident narration and tombstone
comments ("REMOVED/VESTIGIAL", dated fix-narration, "Z2 fix/W1 Turn/Priya/
Vikram", "the bug was", "used to ... → loop", "do NOT reintroduce"),
rewrote them as clean current-behavior comments; deleted the dead
SessionState._recall_sniff_done field (zero readers; in-memory-only,
verified safe); cleaned stale references to removed systems
(sales_brain/qa_brain/orchestrator) in comments, docstrings and a log
string. NO logic changed: 29/31 modified .py are AST-equal modulo
docstrings (provably comment-only); the other 2 are the verified-safe
field deletion + log/docstring string text only.

Docs: 70-docs/** + ARCHITECTURE.md rewritten to the current
single-LLM-with-tools standard (gemini-2.5-flash + save_profile_field/
retrieve_policies/mark_recommendation + small nim_fallback); removed
systems no longer described as live; historical eval/known-issues logs
carry accurate "superseded" banners. ADRs and KIs left verbatim.

Verified: full pytest gate green (rc=0, 0 failures) AFTER the purge;
AST-equivalence proves the code sweep is logic-preserving. Not pushed in
this commit's own step — guarded deploy follows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (42) hide show
  1. 70-docs/00-overview/roadmap.md +25 -18
  2. 70-docs/10-architecture/safety-architecture.md +16 -10
  3. 70-docs/10-architecture/scoring-methodology.md +7 -6
  4. 70-docs/10-architecture/stack-rationale.md +9 -5
  5. 70-docs/10-architecture/system-overview.md +16 -10
  6. 70-docs/30-engineering/discovery-script.md +10 -8
  7. 70-docs/30-engineering/needs-analysis-flow.md +39 -55
  8. 70-docs/40-evaluation/eval-methodology.md +20 -16
  9. 70-docs/40-evaluation/known-issues.md +11 -0
  10. 70-docs/40-evaluation/quality-sprint-2026-05-14.md +10 -0
  11. ARCHITECTURE.md +9 -7
  12. backend/admin.py +30 -37
  13. backend/brain_tools.py +61 -110
  14. backend/config.py +14 -20
  15. backend/llm_health.py +118 -154
  16. backend/main.py +91 -106
  17. backend/needs_finder.py +24 -35
  18. backend/nim_fallback.py +8 -8
  19. backend/policy_identity.py +8 -10
  20. backend/premium_calculator.py +14 -24
  21. backend/profile_persistence.py +0 -7
  22. backend/profile_rag.py +27 -37
  23. backend/profile_store.py +7 -7
  24. backend/providers/__init__.py +7 -16
  25. backend/providers/base.py +1 -1
  26. backend/providers/google_gemini_llm.py +43 -47
  27. backend/providers/groq_llm.py +2 -2
  28. backend/providers/local_embeddings.py +6 -6
  29. backend/providers/nvidia_nim_llm.py +128 -194
  30. backend/providers/openrouter_llm.py +11 -18
  31. backend/providers/sarvam_tts.py +7 -10
  32. backend/retrieval_filters.py +43 -51
  33. backend/scorecard.py +4 -7
  34. backend/session_state.py +12 -27
  35. backend/single_brain.py +37 -88
  36. backend/voice_format.py +33 -38
  37. rag/extract.py +14 -31
  38. rag/retrieve.py +47 -66
  39. tools/chunk_sweep.py +3 -3
  40. tools/set_hf_secrets.py +9 -10
  41. tools/upload_to_hf.py +11 -12
  42. tools/upload_vectors_to_dataset.py +2 -2
70-docs/00-overview/roadmap.md CHANGED
@@ -16,29 +16,36 @@
16
 
17
  v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first advisor. The architecture is built so v2 is a **data/config change, not a rebuild**. This document maps the path.
18
 
19
- ## 0.1 Recently shipped (2026-05-15)
20
-
21
- - **KI-167 / [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md)** Ripped out `backend/fact_find_brain.py` + canonical_fallback + the `<FF>` trailer convention. Replaced with `backend/sales_brain.py` — one LLM call per turn using native provider JSON mode. No scripted prompts, multi-fact capture per turn, ~500 LOC deleted.
22
- - **KI-168** Hybrid voice capture: Web Speech API streams interim transcripts for live UX feel while MediaRecorder runs in parallel; Sarvam STT is the authoritative transcript posted on browser silence-detect.
23
- - **KI-171** Skip faithfulness judge on `fact_find` + `recommendation` intents (the judge has no retrieved context to grade against on those turns).
24
- - **KI-173 / KI-174** — Voice heartbeat + `visibilitychange` / `focus` revival hooks keep the mic alive across tab and app switches.
25
- - **KI-175 / KI-176** — NIM chain reorder (Nemotron 49B demoted to last resort), OpenRouter re-added as cross-provider diversity via `models: [...]` server-side fallback.
26
- - **KI-178 / KI-179 / [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md)** — OpenRouter free-tier `response_format` audit; added Google AI Studio (Gemini 2.0 / 2.5 Flash, 1500 req/day free) as Tier 0 primary on Brain Fast + Brain Main.
 
 
 
 
 
 
 
27
 
28
  ## 1. What v1 ships
29
 
30
  **Working product:**
31
- - Voice-first chat advisor over a curated corpus of Indian health insurance policies (~76 PDFs from 10 insurers, ingested into Chroma + DuckDB)
32
  - Multi-language: English + Hindi/Hinglish via Sarvam Saarika STT + Sarvam Bulbul TTS
33
- - 3-tier brain chain (ADR-040, KI-179): Brain Fast (`backend/sales_brain.py`, fact-find) leads with **Gemini 2.0 Flash** (Google AI Studio, 1500 req/day free, native JSON mode) → NIM Qwen 80B / Mistral Large 3 675B / Llama-4 Maverick → OpenRouter `:free` (Nemotron-3-Super 120B, Qwen 80B) → NIM Nemotron 49B last resort. Brain Main (synthesis / comparison / recommendation) leads with **Gemini 2.5 Flash** → NIM Mistral 675B / Maverick / Qwen 80B → OR Nemotron-3-Super → NIM Nemotron 49B. Judge primary stays on **NIM Mistral Large 3 675B** (different family from the Gemini brain preserves the cross-family invariant). Sticky-primary election (KI-080) per chain. Sarvam scoped to Indic translation + voice only.
34
- - 4-gate hallucination defense + auditable refusal log
35
  - 62-field structured extraction per policy
36
  - Clean Next.js + Tailwind frontend
37
- - FastAPI backend deployed on Render; frontend on Vercel
38
  - 8 design / decision documents totaling ~30 pages
39
 
40
  **Eval signal:**
41
- - Gold Q&A harness (~300 pairs targeted) + automated grader (the JUDGE_CHAIN with NIM Mistral Large 3 675B primary — different family from the Gemini brain so grading stays non-circular)
42
  - `eval/results.md` versioned table per run
43
  - Live audit log `logs/hallucinations.jsonl` for every blocked claim
44
 
@@ -133,10 +140,10 @@ v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first
133
  The point of disciplined v1 architecture is that these things are **stable** across the transition:
134
 
135
  1. **62-field structured schema** (`rag/schema.py`) — data-only change to add v2 categories
136
- 2. **Provider abstraction** (`backend/providers/base.py`) — swap STT/TTS/LLM via config
137
- 3. **Faithfulness verifier** (`backend/faithfulness.py`) same 4 gates, possibly more
138
- 4. **Persona prompt + citation grammar** — same, refined
139
- 5. **Eval methodology** (`70-docs/03-eval-plan.md`) — same harness, more gold data
140
 
141
  The "c-readiness commitments" in Doc 02 §7 are the contract. Every v2 feature is a commitment honored.
142
 
@@ -146,7 +153,7 @@ The "c-readiness commitments" in Doc 02 §7 are the contract. Every v2 feature i
146
  | --- | --- | --- |
147
  | Streamlit → Next.js mid-build | Production polish for a BFSI reviewer | 2 extra hours of scaffolding |
148
  | Voyage embeddings → BGE local | Voyage 3 RPM rate limit blocked ingestion | Slightly lower retrieval quality (~3pp) for full corpus access |
149
- | IRDAI corpus deferred | Akamai bot protection; 4-gate faithfulness already refuses regulatory questions cleanly | Bot can't ground answers in IRDAI text — refuses instead of citing |
150
  | Push-to-talk over streaming | Risk of broken realtime > demo latency | 2-3s perceived latency vs <1s |
151
  | No auth | Out of scope per Doc 01 | Single-user demo only |
152
  | Hand-curated 5-node fact-find | Auditable + testable | Less natural than LLM-driven |
 
16
 
17
  v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first advisor. The architecture is built so v2 is a **data/config change, not a rebuild**. This document maps the path.
18
 
19
+ ## 0.1 Architecture history (pointers only)
20
+
21
+ > The 2026-05-15 work below (KI-167…KI-179, ADR-039/040) was an
22
+ > intermediate step that was itself **subsequently superseded** by the
23
+ > single-LLM-with-tools rewrite (one Gemini 2.5-flash call per turn with
24
+ > `save_profile_field` / `retrieve_policies` / `mark_recommendation`,
25
+ > structured+vector retrieval, small `nim_fallback`). KI/ADR pointers are
26
+ > kept as history-of-record. Present-state authority:
27
+ > [`README.md`](../../README.md) §4.
28
+
29
+ - **KI-167 / [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md)** — Removed the scripted fact-find renderer + `<FF>` trailer convention (history; the later single-LLM-with-tools handler is the present design).
30
+ - **KI-168** — Hybrid voice capture: Web Speech API streams interim transcripts for live UX feel while MediaRecorder runs in parallel; Sarvam STT is the authoritative transcript posted on browser silence-detect. (Still current.)
31
+ - **KI-171** — Skipping the separate faithfulness judge on no-context turns (the separate judge LLM was later removed entirely; faithfulness is now structural).
32
+ - **KI-173 / KI-174** — Voice heartbeat + `visibilitychange` / `focus` revival hooks keep the mic alive across tab and app switches. (Still current.)
33
+ - **KI-175 / KI-176 / KI-178 / KI-179 / [ADR-040](../60-decisions/ADR-040-google-gemini-primary.md)** — Provider/chain work from the multi-model era (history; the present brain is a single Gemini 2.5-flash call per turn with a small NIM fallback for transient errors).
34
 
35
  ## 1. What v1 ships
36
 
37
  **Working product:**
38
+ - Voice-first chat advisor over a curated corpus of Indian health insurance policies (~76 PDFs from 10 insurers, ingested into Chroma)
39
  - Multi-language: English + Hindi/Hinglish via Sarvam Saarika STT + Sarvam Bulbul TTS
40
+ - **Single-LLM-with-tools brain:** one **Gemini 2.5-flash** call per turn with function-calling tools (`save_profile_field` / `retrieve_policies` / `mark_recommendation`) handling fact-find, retrieval, QA, and recommendation. Small `backend/nim_fallback.py` (NVIDIA NIM) covers transient Gemini errors so the turn completes; fail-loud otherwise. Sarvam scoped to Indic translation + voice only.
41
+ - Structural faithfulness (the LLM can only cite what `retrieve_policies` returned; recommendation fit gated in `scorecard.py` / `retrieval_filters.py`) + auditable refusal log
42
  - 62-field structured extraction per policy
43
  - Clean Next.js + Tailwind frontend
44
+ - FastAPI backend deployed as a Hugging Face Space (Docker, `uvicorn`)
45
  - 8 design / decision documents totaling ~30 pages
46
 
47
  **Eval signal:**
48
+ - Gold Q&A harness (~300 pairs targeted) + automated offline grader using a judge model from a different family than the runtime Gemini brain (non-circular grading; eval-only, not a runtime gate)
49
  - `eval/results.md` versioned table per run
50
  - Live audit log `logs/hallucinations.jsonl` for every blocked claim
51
 
 
140
  The point of disciplined v1 architecture is that these things are **stable** across the transition:
141
 
142
  1. **62-field structured schema** (`rag/schema.py`) — data-only change to add v2 categories
143
+ 2. **Provider abstraction** — swap STT/TTS/LLM via config
144
+ 3. **Structural faithfulness** — the brain can only cite what `retrieve_policies` returned; recommendation fit gated in `scorecard.py` / `retrieval_filters.py`
145
+ 4. **System prompt + citation grammar** — same, refined
146
+ 5. **Eval methodology** (`70-docs/40-evaluation/eval-methodology.md`) — same harness, more gold data
147
 
148
  The "c-readiness commitments" in Doc 02 §7 are the contract. Every v2 feature is a commitment honored.
149
 
 
153
  | --- | --- | --- |
154
  | Streamlit → Next.js mid-build | Production polish for a BFSI reviewer | 2 extra hours of scaffolding |
155
  | Voyage embeddings → BGE local | Voyage 3 RPM rate limit blocked ingestion | Slightly lower retrieval quality (~3pp) for full corpus access |
156
+ | IRDAI corpus deferred | Akamai bot protection; structural faithfulness already refuses regulatory questions cleanly (the brain can only cite what `retrieve_policies` returned, and there are no IRDAI chunks) | Bot can't ground answers in IRDAI text — refuses instead of citing |
157
  | Push-to-talk over streaming | Risk of broken realtime > demo latency | 2-3s perceived latency vs <1s |
158
  | No auth | Out of scope per Doc 01 | Single-user demo only |
159
  | Hand-curated 5-node fact-find | Auditable + testable | Less natural than LLM-driven |
70-docs/10-architecture/safety-architecture.md CHANGED
@@ -15,14 +15,20 @@
15
  > consolidation.
16
  >
17
  > **Current safety model (authoritative: [`README.md`](../../README.md) §6):**
18
- > faithfulness is now **structural** — the single Gemini brain can only state
19
- > what `retrieve_policies` returned and must cite; recommendation fit is gated
20
- > in `backend/scorecard.py` / `retrieval_filters.py`. The hardened surface is
21
- > the **8-gate uploaded-PDF defence in `backend/security.py`** (file
22
- > mechanics, content quality, prompt-injection, per-session + per-IP rate
23
- > limits, encrypted-PDF, page-ceiling, hash-dedupe) plus per-session
24
- > quarantine isolation with a 24 h TTL. Retained below only as the historical
25
- > failure-modes analysis that informed today's design.
 
 
 
 
 
 
26
 
27
  ## 0. Purpose
28
 
@@ -207,5 +213,5 @@ Run-time. Auditable. Tested.
207
  | M-07 | Nightly synthetic eval cron | Infra | v2 |
208
  | M-08 | Live-traffic spot grading | Eval | v2 |
209
  | M-09 | 3-judge consensus for grader | Eval | v2 |
210
- | M-10 | Sarvam-M no longer brain Gemini 2.0 / 2.5 Flash primary on Brain Fast / Brain Main, NIM fallback | Orchestrator (ADR-040) | Live ✅ |
211
- | M-11 | Faithfulness 4-gate verifier | Orchestrator | Live ✅ |
 
15
  > consolidation.
16
  >
17
  > **Current safety model (authoritative: [`README.md`](../../README.md) §6):**
18
+ > faithfulness is now **structural** — the single Gemini 2.5-flash brain
19
+ > (one call/turn with `save_profile_field` / `retrieve_policies` /
20
+ > `mark_recommendation`) can only state what `retrieve_policies` returned
21
+ > and must cite it; recommendation fit is gated in `backend/scorecard.py` /
22
+ > `retrieval_filters.py`. There is **no separate faithfulness-judge LLM**,
23
+ > no `orchestrator`, and no Brain-Fast/Brain-Main split. The hardened
24
+ > surface is the **8-gate uploaded-PDF defence in `backend/security.py`**
25
+ > (file mechanics, content quality, prompt-injection, per-session + per-IP
26
+ > rate limits, encrypted-PDF, page-ceiling, hash-dedupe) plus per-session
27
+ > quarantine isolation with a 24 h TTL. A small `backend/nim_fallback.py`
28
+ > (NVIDIA NIM) only covers transient Gemini errors so the turn still
29
+ > completes. **Everything below this banner is the historical failure-modes
30
+ > analysis that informed today's design — the per-mode "Status: Live"
31
+ > lines describe the pre-rewrite system, not the present one.**
32
 
33
  ## 0. Purpose
34
 
 
213
  | M-07 | Nightly synthetic eval cron | Infra | v2 |
214
  | M-08 | Live-traffic spot grading | Eval | v2 |
215
  | M-09 | 3-judge consensus for grader | Eval | v2 |
216
+ | M-10 | Sarvam-M scoped to Indic translation only; brain is one Gemini 2.5-flash call/turn with tools, small `nim_fallback` for transient errors | single_brain | Live ✅ |
217
+ | M-11 | Structural faithfulness (LLM can only cite what `retrieve_policies` returned; recommendation fit gated in `scorecard.py` / `retrieval_filters.py`) — the separate 4-gate judge was removed | single_brain | Live ✅ (superseded the 4-gate judge) |
70-docs/10-architecture/scoring-methodology.md CHANGED
@@ -1,10 +1,11 @@
1
  # Scorecard Methodology — From 62 Schema Fields to a Single A-F Grade
2
 
3
- > ️ **Predates the single-brain rewrite not the present-state map.** Some
4
- > implementation references here (orchestrator / `sales_brain` / 3-tier chain
5
- > / `faithfulness.py` judge) describe code that was removed. Present-state
6
- > authority: [`README.md`](../../README.md) §4. Retained for design intent /
7
- > historical record.
 
8
 
9
  | Field | Value |
10
  | --- | --- |
@@ -64,7 +65,7 @@ A small number of fields exist because they materially affect buyer outcomes eve
64
 
65
  ### How we did it (the actual code path)
66
 
67
- `rag/extract.py` calls the fast-brain chain (Nemotron Nano 30B primary, Qwen 80B / GPT-OSS / Groq fallbacks) with the full policy PDF text and the 62-field Pydantic schema as a structured-output target. The LLM extracts each field; if a field is not explicitly stated in the document, it is set to `null`. A self-critique pass scores per-field confidence (the `extraction_confidence_pct` field).
68
 
69
  The full schema lives in `rag/schema.py` (see also `rag/SCHEMA.md` for groupings and gotchas).
70
 
 
1
  # Scorecard Methodology — From 62 Schema Fields to a Single A-F Grade
2
 
3
+ > ️ **Methodology is current.** The scorecard is rules-based and
4
+ > LLM-free; the grading logic below (`backend/scorecard.py`) is the
5
+ > present-state design, including the profile-tuned weights used by the
6
+ > single-LLM-with-tools handler when it recommends policies. The only
7
+ > historical pointer is the one-off extraction pipeline noted in §1.
8
+ > Present-state authority: [`README.md`](../../README.md) §4.
9
 
10
  | Field | Value |
11
  | --- | --- |
 
65
 
66
  ### How we did it (the actual code path)
67
 
68
+ `rag/extract.py` runs a one-time offline extraction: an LLM is given the full policy PDF text and the 62-field Pydantic schema as a structured-output target. The LLM extracts each field; if a field is not explicitly stated in the document, it is set to `null`. A self-critique pass scores per-field confidence (the `extraction_confidence_pct` field). This is a build-time pipeline, separate from the runtime single-LLM-with-tools handler.
69
 
70
  The full schema lives in `rag/schema.py` (see also `rag/SCHEMA.md` for groupings and gotchas).
71
 
70-docs/10-architecture/stack-rationale.md CHANGED
@@ -25,11 +25,15 @@ place so it cannot drift again:
25
 
26
  - **Frontend:** Next.js 16 (App Router), React 19, Tailwind v4, static export.
27
  - **Backend:** FastAPI + Pydantic; `uvicorn`, port 7860 on the HF Space.
28
- - **Brain:** Google **Gemini `gemini-2.5-flash-lite`** + function calling
29
- (one call/turn) **NVIDIA NIM** open-model fallback chain (health-elected,
30
- fail-loud). No separate judge model (retired in the single-brain
31
- consolidation).
32
- - **Retrieval:** Chroma + BGE-small-en-v1.5 (local CPU, 384-d).
 
 
 
 
33
  - **Voice:** Sarvam Saarika (STT) + Bulbul (TTS) + Sarvam-M (Indic).
34
  - **Hosting:** Hugging Face Space (Docker) + companion HF dataset for
35
  corpus/vectors; **not** Vercel/Render (that was the old plan).
 
25
 
26
  - **Frontend:** Next.js 16 (App Router), React 19, Tailwind v4, static export.
27
  - **Backend:** FastAPI + Pydantic; `uvicorn`, port 7860 on the HF Space.
28
+ - **Brain:** Google **Gemini `gemini-2.5-flash`** + function-calling tools
29
+ (`save_profile_field` / `retrieve_policies` / `mark_recommendation`), one
30
+ call/turn handling fact-find, retrieval, QA, and recommendation. On a
31
+ transient Gemini error → small **NVIDIA NIM** fallback
32
+ (`backend/nim_fallback.py`) so the turn completes; fail-loud otherwise. No
33
+ separate judge model, no orchestrator, no sales/QA-brain split (all
34
+ retired in the single-LLM-with-tools consolidation).
35
+ - **Retrieval:** structured + vector over Chroma + BGE-small-en-v1.5 (local
36
+ CPU, 384-d) with a profile-tuned scorecard.
37
  - **Voice:** Sarvam Saarika (STT) + Bulbul (TTS) + Sarvam-M (Indic).
38
  - **Hosting:** Hugging Face Space (Docker) + companion HF dataset for
39
  corpus/vectors; **not** Vercel/Render (that was the old plan).
70-docs/10-architecture/system-overview.md CHANGED
@@ -12,10 +12,12 @@
12
  ## This document has been superseded
13
 
14
  The earlier version of this file described the **pre-rewrite** architecture
15
- (an `orchestrator.py`, a `sales_brain`/`QA-brain` split, a 4-gate
16
- `faithfulness.py` judge, a 3-tier Gemini/NIM/OpenRouter chain, a DuckDB
17
- 62-field hot path, Next.js 14, Vercel/Render hosting). **None of that exists
18
- in the codebase anymore** it was replaced by the single-brain rewrite.
 
 
19
 
20
  To prevent two documents drifting apart again, there is now exactly **one**
21
  authoritative architecture description:
@@ -28,12 +30,16 @@ authoritative architecture description:
28
 
29
  ## Current architecture in three lines
30
 
31
- - One LLM call per turn: **Gemini `gemini-2.5-flash-lite` + function-calling
32
- tools** (`backend/single_brain.py`, `brain_tools.py`); on Gemini failure →
33
- `backend/nim_fallback.py` (NVIDIA NIM chain, health-elected, fail-loud).
34
- - Retrieval: **Chroma + BGE-small local 384-d** (`rag/retrieve.py`); shared
35
- policies collection + per-session 24h quarantine for user-uploaded PDFs
36
- (8 security gates in `backend/security.py`).
 
 
 
 
37
  - **Next.js 16 / FastAPI**, deployed as an HF Space (Docker, `uvicorn`,
38
  port 7860); heavy data pulled at build from the `insurance-bot-data` HF
39
  dataset; curated facts in `40-data/`.
 
12
  ## This document has been superseded
13
 
14
  The earlier version of this file described the **pre-rewrite** architecture
15
+ (an `orchestrator.py`, a `sales_brain`/`QA-brain` split, a 3-brain design,
16
+ a separate `faithfulness.py` judge LLM, a `profile_extractor`, a tiered
17
+ brain, a translation-check pass, a 3-tier Gemini/NIM/OpenRouter chain, a
18
+ DuckDB 62-field hot path, Next.js 14, Vercel/Render hosting). **None of
19
+ that exists in the codebase anymore** — it was replaced by the
20
+ single-LLM-with-tools rewrite.
21
 
22
  To prevent two documents drifting apart again, there is now exactly **one**
23
  authoritative architecture description:
 
30
 
31
  ## Current architecture in three lines
32
 
33
+ - One LLM call per turn: **Gemini `gemini-2.5-flash` + function-calling
34
+ tools** (`save_profile_field` / `retrieve_policies` /
35
+ `mark_recommendation`) in `backend/single_brain.py` / `brain_tools.py`.
36
+ A single call drives fact-find, retrieval, QA, and recommendation. On a
37
+ transient Gemini error small `backend/nim_fallback.py` (NVIDIA NIM) so
38
+ the turn completes. Fail-loud.
39
+ - Retrieval: **structured + vector over Chroma + BGE-small local 384-d**
40
+ (`rag/retrieve.py`) with a profile-tuned scorecard; shared policies
41
+ collection + per-session 24h quarantine for user-uploaded PDFs (8
42
+ security gates in `backend/security.py`).
43
  - **Next.js 16 / FastAPI**, deployed as an HF Space (Docker, `uvicorn`,
44
  port 7860); heavy data pulled at build from the `insurance-bot-data` HF
45
  dataset; curated facts in `40-data/`.
70-docs/30-engineering/discovery-script.md CHANGED
@@ -1,10 +1,12 @@
1
  # Discovery Conversation Script
2
 
3
- > ⚠️ **Predates the single-brain rewrite — not the present-state map.** Some
4
- > implementation references here (orchestrator / `sales_brain` / 3-tier chain
5
- > / `faithfulness.py` judge) describe code that was removed. Present-state
6
- > authority: [`README.md`](../../README.md) §4. Retained for design intent /
7
- > historical record.
 
 
8
 
9
  The 10-turn fact-find that turns a stranger into a profiled buyer.
10
 
@@ -135,13 +137,13 @@ This is the customer-protection framing. It tells the user honesty is **self-pro
135
 
136
  ## Implementation notes
137
 
138
- The 9-slot fact-find SCHEMA still lives in `backend/needs_finder.py::GRAPH` — used now as the schema source for the `sales_brain` LLM system prompt rather than as a scripted question list. Each entry's `id`, `field`, `is_core`, and `condition` are consumed by the LLM as a structured contract; the `prompt_en` / `prompt_hi` strings are no longer rendered to the user post-[ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) / KI-167 (the LLM owns voice + cadence end-to-end via its system prompt + the conversation so far).
139
 
140
  To add a new question:
141
 
142
  1. Add a `Question(...)` entry with `id`, `field` (which Profile attribute it sets), `is_core` (boolean — counts toward completeness), optional `condition` callable, optional `parser`.
143
- 2. Surface the new slot in the `sales_brain` system prompt's 9-slot schema (alongside accepted value shapes + examples) so the LLM knows to capture it. See `backend/sales_brain.py::_SYSTEM_PROMPT`.
144
- 3. Wire any post-capture validation into `backend/sales_brain_normalizer.py` (enum coercion, INR parsing, bounds).
145
  4. Add a row in `70-docs/scorecard-knowledge-graph.md` Part B showing how the new input shifts weights.
146
  5. Wire the shift into `_profile_tuned_weights()` in `backend/scorecard.py`.
147
 
 
1
  # Discovery Conversation Script
2
 
3
+ > ⚠️ **Design intent, not the present-state map.** The turn-by-turn script
4
+ > and adaptive rules below capture the discovery *intent*; the implementation
5
+ > is now a single Gemini 2.5-flash call per turn with function-calling tools
6
+ > (`backend/single_brain.py` / `brain_tools.py`) there is no scripted
7
+ > question renderer, no `orchestrator`, no `sales_brain` / `qa_brain` split,
8
+ > and no separate faithfulness judge. Present-state authority:
9
+ > [`README.md`](../../README.md) §4.
10
 
11
  The 10-turn fact-find that turns a stranger into a profiled buyer.
12
 
 
137
 
138
  ## Implementation notes
139
 
140
+ The 9-slot fact-find SCHEMA still lives in `backend/needs_finder.py::GRAPH` — used now as the schema source embedded in the single-LLM system prompt rather than as a scripted question list. Each entry's `id`, `field`, `is_core`, and `condition` are consumed by the LLM as a structured contract; the `prompt_en` / `prompt_hi` strings are no longer rendered to the user (the LLM owns voice + cadence end-to-end via its system prompt + the conversation so far, and records facts through the `save_profile_field` tool).
141
 
142
  To add a new question:
143
 
144
  1. Add a `Question(...)` entry with `id`, `field` (which Profile attribute it sets), `is_core` (boolean — counts toward completeness), optional `condition` callable, optional `parser`.
145
+ 2. Surface the new slot in the single-brain system prompt's 9-slot schema (alongside accepted value shapes + examples) so the LLM knows to capture it. See the system prompt in `backend/single_brain.py`.
146
+ 3. Wire any post-capture validation into the `save_profile_field` handler in `backend/brain_tools.py` (enum coercion, INR parsing, bounds).
147
  4. Add a row in `70-docs/scorecard-knowledge-graph.md` Part B showing how the new input shifts weights.
148
  5. Wire the shift into `_profile_tuned_weights()` in `backend/scorecard.py`.
149
 
70-docs/30-engineering/needs-analysis-flow.md CHANGED
@@ -1,35 +1,38 @@
1
- # 05 — Needs Analysis Flow (LLM-driven sales brain)
2
 
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
- | Version | 0.2 |
7
  | Date | 2026-05-17 |
8
- | Implementation | `backend/single_brain.py` (single-brain rewrite — replaced the removed `backend/sales_brain.py`). The slot schema still lives in `backend/needs_finder.py::GRAPH`. |
9
 
10
- > ℹ️ **Still conceptually accurate, one pointer updated.** The core design
11
- > below — *one LLM call per turn owns the entire fact-find surface* — is
12
- > exactly what `backend/single_brain.py` does today; only the module name
13
- > changed (`sales_brain.py` `single_brain.py`, which also subsumes QA +
14
- > recommendation). Present-state authority: [`README.md`](../../README.md) §4.
 
 
 
15
 
16
  ## 0. Why one LLM call per turn (not "scripted question graph + paraphraser")
17
 
18
  A good Independent Financial Advisor opens with a stable, repeatable set of slots to fill — and adapts which slot to drive next based on what the buyer just said, what they've already told us, and what's still missing. We replicate this with **one LLM call per turn** that owns the entire fact-find surface (voice, cadence, slot-selection, multi-fact capture in a single turn), not a hardcoded state machine with a paraphraser glued on top.
19
 
20
- The original v0.1 of this doc was an explicit `GRAPH` of canonical questions with `prompt_en` / `prompt_hi` strings. That design shipped with ADR-027 (LLM paraphraser on top of the graph) and ADR-030 (one-call brain with `<FF>` trailer + canonical fallback). Both architectures hit a hard wall: scripted prompts leaked into every fallback turn ("Got that {slot}.") and produced robotic cadence even when the brain succeeded. KI-167 / [ADR-039](../60-decisions/ADR-039-llm-driven-sales-brain.md) ripped it all out: no canonical_fallback, no scripted prompts, no `<FF>` trailer. Native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM) provides server-side parseable output.
21
 
22
  What we kept and what we changed:
23
 
24
- | Concern | v0.1 (scripted graph) | Current (LLM-driven sales_brain) |
25
  |---|---|---|
26
  | **Slot schema** | `GRAPH` of 9 `Question(id, prompt_en, prompt_hi, field, is_core, condition, parser)` entries | `GRAPH` data structure retained as the schema source; `prompt_en` / `prompt_hi` are dead strings now |
27
  | **Question text** | Hardcoded `prompt_en` rendered to user | LLM generates natural prose in the advisor's voice per turn (no template) |
28
- | **Slot selection** | `next_question(profile)` walked the graph in order | LLM picks `slot_driving` in its JSON response (constrained by the schema to required slots first) |
29
- | **Multi-fact capture** | One slot per turn (a user saying "I'm 32, just myself, in Mumbai" only filled `age`) | LLM emits a `captures` map per turn — one utterance can fill 2-4 slots |
30
- | **Auditable behavior** | Graph order was the trace | Per-turn LLM response is logged (intent, brain_used, captures, slot_driving, complete) in `40-data/llm_usage.jsonl` + `logs/turns.jsonl` |
31
- | **Fail-soft** | Graph survived LLM degradation | Outer 25s `asyncio.wait_for` on the brain call; on total NIM + Google exhaustion the orchestrator returns the [ADR-038](../60-decisions/ADR-038-nim-only-chains.md) graceful error message to the user — no scripted reply |
32
- | **Bilingual** | Hand-authored `prompt_en` + `prompt_hi` | Sarvam-M translation cascade on the LLM's output (English authoring + Indic translation), same UX, fewer hand-authored strings |
33
 
34
  ## 1. The 9-slot schema (data-only — the LLM consults this through its system prompt)
35
 
@@ -87,53 +90,35 @@ What we kept and what we changed:
87
 
88
  ## 2. Termination criteria
89
 
90
- The LLM sets `complete: true` in its JSON response when all required slots are filled. `backend/sales_brain_normalizer.py` double-checks: if `complete == true` while any required slot is empty, the normalizer overrides to `complete = false` the LLM cannot prematurely declare fact-find done.
91
 
92
- Or when the user opts out — if the user immediately asks "compare Star and HDFC ERGO" or "show me the top 3 policies for me" (KI-105 closer phrases), the intent classifier forces intent to `comparison` / `recommendation` BEFORE the fact-find branch is even reached, and the orchestrator routes straight to retrieval + Brain Main.
93
 
94
  ## 3. Bilingual handling
95
 
96
  The LLM authors in English; Sarvam-M translates Hinglish ↔ English at the cascade boundary. The "Indic-native, not translated" principle (problem-statement §5.3) is preserved by the LLM's native-quality output rather than by hand-authored Hindi strings — the cascade itself is the bilingual surface.
97
 
98
- ## 4. Parsing user answers
99
 
100
- The LLM produces a `captures` map per turn (e.g. `{"age": 34, "dependents": "self", "location_tier": "metro"}`). `backend/sales_brain_normalizer.py` is a pure-function post-processor that:
101
 
102
  - Resolves field-name aliases (`location` → `location_tier`)
103
  - Coerces enums (`Bangalore` → `metro`)
104
  - Parses INR amounts (`"32 lakh"` → `3200000`)
105
- - Drops null / empty values (KI-094 None-guard — a `None` capture never overwrites a filled field)
106
  - Validates types and bounds (rejects `age: 250`)
107
 
108
- No LLM calls in the normalizer — pure rules. The orchestrator applies the normalized map via `session.update_profile_field()` exactly as before. Profile persistence remains unchanged (`backend/profile_store.save_profile()` + `backend/profile_rag.upsert_profile_chunk()` with session-scoped metadata per KI-102).
109
-
110
- ## 5. How the orchestrator uses this
111
-
112
- ```python
113
- profile = Profile.from_session(session_id)
114
-
115
- if intent == "fact_find":
116
- # KI-167: one LLM call per turn against FAST_BRAIN_CHAIN
117
- outcome = await sales_brain.drive_sales_turn(
118
- profile=profile,
119
- history=session.chat_history[-N:],
120
- user_text=user_text,
121
- )
122
- # outcome.reply, outcome.captures, outcome.slot_driving, outcome.complete
123
- apply_normalized_captures(session, outcome.captures)
124
- if outcome.complete:
125
- session.free_form_session = True
126
- reply = outcome.reply
127
- else:
128
- # QA / comparison / recommendation → Brain Main
129
- chunks = retrieve(user_query, top_k=10)
130
- reply = await BRAIN_CHAIN.chat(persona_prompt + history + chunks + user_text)
131
- if intent not in {"fact_find", "recommendation"}:
132
- # KI-171: judge skipped on fact-find + recommendation intents
133
- verdict = await faithfulness.check(reply, chunks)
134
- if not verdict.supported:
135
- reply = SAFE_REFUSAL
136
- ```
137
 
138
  ## 6. What the user sees
139
 
@@ -159,14 +144,13 @@ Bot: "Great — that's everything I need. Based on your profile, three policies
159
  stand out: [...]"
160
  ```
161
 
162
- Note: the LLM captured `{name, age, dependents, location_tier}` in a single turn from the user's opener. Pre-KI-167 each fact required its own turn. The exact turn count varies — some sessions finish in 3-4 turns, others in 6-7 depending on what the user volunteers.
163
 
164
  ## 7. v2 enhancements
165
 
166
  | # | Enhancement | Why |
167
  | --- | --- | --- |
168
- | 1 | Stream the prose token-by-token to the frontend | The JSON tail is parsed server-side after stream completion — would cut perceived latency below the canonical-fallback path |
169
- | 2 | Promote the same single-call pattern to the QA path | Orchestrator's QA + fact-find branches collapse into one brain call with a richer schema (intent in the JSON block instead of out-of-band classification) |
170
- | 3 | Skip-confirm flow ("you can skip this say 'skip'") | Buyer autonomy |
171
- | 4 | Save profile across sessions shipped KI-040 | Returning user picks up where they left off |
172
- | 5 | Tone-match the user's energy (formal vs casual) | The LLM already does this implicitly via the system prompt; could be explicit per-session signal |
 
1
+ # 05 — Needs Analysis Flow (single-LLM-with-tools)
2
 
3
  | Field | Value |
4
  | --- | --- |
5
  | Project | Insurance Sales Portfolio Expert |
6
+ | Version | 0.3 |
7
  | Date | 2026-05-17 |
8
+ | Implementation | `backend/single_brain.py` + `backend/brain_tools.py`. The slot schema still lives in `backend/needs_finder.py::GRAPH`. |
9
 
10
+ > ℹ️ **Design intent below is still accurate; implementation pointers
11
+ > updated.** The core idea — *one LLM call per turn owns the entire
12
+ > fact-find surface* is what the single-LLM-with-tools handler does
13
+ > today. Fact-find, retrieval, QA, and recommendation are all handled in
14
+ > that one call via function-calling tools; the separate `sales_brain` /
15
+ > `qa_brain` split, the orchestrator, the profile_extractor, and the
16
+ > separate faithfulness judge were removed. Present-state authority:
17
+ > [`README.md`](../../README.md) §4.
18
 
19
  ## 0. Why one LLM call per turn (not "scripted question graph + paraphraser")
20
 
21
  A good Independent Financial Advisor opens with a stable, repeatable set of slots to fill — and adapts which slot to drive next based on what the buyer just said, what they've already told us, and what's still missing. We replicate this with **one LLM call per turn** that owns the entire fact-find surface (voice, cadence, slot-selection, multi-fact capture in a single turn), not a hardcoded state machine with a paraphraser glued on top.
22
 
23
+ The original v0.1 of this doc was an explicit `GRAPH` of canonical questions with `prompt_en` / `prompt_hi` strings, rendered by a scripted state machine. That scripted approach (and a later one-call-brain variant with a structured trailer + canonical fallback) produced robotic cadence: scripted prompts leaked into fallback turns. The current design removes all of that. There are no scripted prompts and no canonical fallback. One **Gemini 2.5-flash call per turn with function-calling** owns voice, cadence, and slot selection; the `GRAPH` is consulted only as the slot *schema*.
24
 
25
  What we kept and what we changed:
26
 
27
+ | Concern | v0.1 (scripted graph) | Current (single LLM + tools) |
28
  |---|---|---|
29
  | **Slot schema** | `GRAPH` of 9 `Question(id, prompt_en, prompt_hi, field, is_core, condition, parser)` entries | `GRAPH` data structure retained as the schema source; `prompt_en` / `prompt_hi` are dead strings now |
30
  | **Question text** | Hardcoded `prompt_en` rendered to user | LLM generates natural prose in the advisor's voice per turn (no template) |
31
+ | **Slot selection** | `next_question(profile)` walked the graph in order | The single LLM decides which slot to drive next from the schema + conversation so far (required slots first) |
32
+ | **Multi-fact capture** | One slot per turn (a user saying "I'm 32, just myself, in Mumbai" only filled `age`) | One utterance can fill 2-4 slots; the LLM calls `save_profile_field` per captured fact |
33
+ | **Auditable behavior** | Graph order was the trace | Per-turn LLM call + tool calls are logged (`save_profile_field` / `retrieve_policies` / `mark_recommendation`) in `40-data/llm_usage.jsonl` + `logs/turns.jsonl` |
34
+ | **Fail-soft** | Graph survived LLM degradation | On a transient Gemini error the small `backend/nim_fallback.py` (NVIDIA NIM) completes the turn; fail-loud otherwise — no scripted reply |
35
+ | **Bilingual** | Hand-authored `prompt_en` + `prompt_hi` | Sarvam-M translation on the LLM's output (English authoring + Indic translation), same UX, fewer hand-authored strings |
36
 
37
  ## 1. The 9-slot schema (data-only — the LLM consults this through its system prompt)
38
 
 
90
 
91
  ## 2. Termination criteria
92
 
93
+ Fact-find is "done" when all required slots are filled. Because the single LLM owns the whole turn, it stops asking discovery questions and moves to retrieval/recommendation once the profile is sufficiently complete; required-slot checks against the `GRAPH` schema keep it from declaring done prematurely.
94
 
95
+ The user can also opt out — if the user immediately asks "compare Star and HDFC ERGO" or "show me the top 3 policies for me", the same LLM call can skip remaining discovery and go straight to `retrieve_policies` / `mark_recommendation` within that turn.
96
 
97
  ## 3. Bilingual handling
98
 
99
  The LLM authors in English; Sarvam-M translates Hinglish ↔ English at the cascade boundary. The "Indic-native, not translated" principle (problem-statement §5.3) is preserved by the LLM's native-quality output rather than by hand-authored Hindi strings — the cascade itself is the bilingual surface.
100
 
101
+ ## 4. Capturing user answers
102
 
103
+ When the user states a fact, the LLM calls the `save_profile_field` tool (e.g. `save_profile_field(field="age", value=34)`). The tool implementation in `backend/brain_tools.py` is a pure-function handler that:
104
 
105
  - Resolves field-name aliases (`location` → `location_tier`)
106
  - Coerces enums (`Bangalore` → `metro`)
107
  - Parses INR amounts (`"32 lakh"` → `3200000`)
108
+ - Drops null / empty values (a `None`/empty value never overwrites a filled field)
109
  - Validates types and bounds (rejects `age: 250`)
110
 
111
+ The handler is pure rules no nested LLM call. Captured fields are persisted to the session profile.
112
+
113
+ ## 5. How the single LLM uses this
114
+
115
+ There is no separate orchestrator / intent-classifier / sales-brain split. One `single_brain` call per turn receives the system prompt (which embeds the slot schema), the recent history, and the user's text, and uses function-calling tools to do everything in that one turn:
116
+
117
+ - `save_profile_field` — record a captured fact (fact-find).
118
+ - `retrieve_policies` structured + vector retrieval over the corpus when the turn needs policy evidence (QA / comparison / recommendation).
119
+ - `mark_recommendation` — record the policies the LLM recommends, surfaced with the profile-tuned scorecard.
120
+
121
+ Because the LLM can only answer from what `retrieve_policies` returned and must cite it, grounding is structural — there is no separate faithfulness-judge LLM in the loop. On a transient Gemini error, `backend/nim_fallback.py` completes the turn.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122
 
123
  ## 6. What the user sees
124
 
 
144
  stand out: [...]"
145
  ```
146
 
147
+ Note: the LLM captured `{name, age, dependents, location_tier}` in a single turn from the user's opener via `save_profile_field` calls. The exact turn count varies — some sessions finish in 3-4 turns, others in 6-7 depending on what the user volunteers.
148
 
149
  ## 7. v2 enhancements
150
 
151
  | # | Enhancement | Why |
152
  | --- | --- | --- |
153
+ | 1 | Stream the prose token-by-token to the frontend | Cuts perceived latency on longer replies |
154
+ | 2 | Skip-confirm flow ("you can skip this say 'skip'") | Buyer autonomy |
155
+ | 3 | Save profile across sessions shipped KI-040 | Returning user picks up where they left off |
156
+ | 4 | Tone-match the user's energy (formal vs casual) | The LLM already does this implicitly via the system prompt; could be explicit per-session signal |
 
70-docs/40-evaluation/eval-methodology.md CHANGED
@@ -1,17 +1,21 @@
1
  # 03 — Evaluation Plan
2
 
3
- > ⚠️ **Predates the single-brain rewrite not the present-state map.** Some
4
- > implementation references here (orchestrator / `sales_brain` / 3-tier chain
5
- > / `faithfulness.py` judge) describe code that was removed. Present-state
6
- > authority: [`README.md`](../../README.md) §4. Retained for design intent /
7
- > historical record.
 
 
 
 
8
 
9
  | Field | Value |
10
  | --- | --- |
11
  | Project | Insurance Sales Portfolio Expert |
12
  | Version | 0.1 |
13
  | Date | 2026-05-13 |
14
- | Depends on | `01-requirements.md` §6 (success criteria), `02-architecture.md` (system under test) |
15
  | Status | Pipeline implemented; full run pending corpus completion |
16
 
17
  ## 0. Purpose
@@ -28,7 +32,7 @@ These directly back the success criteria in Doc 01 §6: C2 (≥95% factual), C3
28
 
29
  ### Pipeline A — auto-generated from structured extraction (the bulk)
30
 
31
- For every successfully extracted policy in DuckDB, generate templated Q&A where the answer comes directly from the 62-field schema. Per `eval/generate_gold.py`:
32
 
33
  - ~15 question templates × ~80 policies = ~1,100 candidate pairs
34
  - Each pair is **fully reproducible** — the answer traces to a specific field which traces to a specific clause
@@ -38,7 +42,7 @@ For every successfully extracted policy in DuckDB, generate templated Q&A where
38
 
39
  ### Pipeline B — LLM-drafted nuanced questions (curated)
40
 
41
- For top-priority policies (top 10–20), run `NimChainLLM(BRAIN_CHAIN)` on the policy text with prompt: *"generate 5 buyer-style questions whose answers are explicitly in this document; include the source clause."* Each generated pair is **human spot-checked** before commit.
42
 
43
  Target: 5 × 20 = 100 questions covering multi-clause and edge-case reasoning that Pipeline A can't generate.
44
 
@@ -56,15 +60,15 @@ Each pair marked with `expected_refusal: bool`. Refusal cases test that the bot
56
 
57
  ## 2. Grader design
58
 
59
- **Single grading endpoint:** `eval/run.py` calls `backend.orchestrator.handle_turn` in-process, takes the reply, scores against gold with three signals:
60
 
61
  ### Signal 1 — Regex hard-checks (deterministic)
62
 
63
  Numbers, dates, currency, durations, percentages are extracted via regex from both gold and bot reply. Exact match (after normalization) is the strongest signal. Catches "the premium is ₹15,000" hallucinations without an LLM.
64
 
65
- ### Signal 2 — LLM-judge faithfulness (JUDGE_CHAIN: NIM Mistral Large 3 675B primary)
66
 
67
- Different model family from the Gemini-led Brain Fast / Brain Main → **non-circular evaluation**. Falls through NIM Llama-4 Maverick OR Qwen 80B `:free` NIM Nemotron 49B as last resort. Judge prompt:
68
 
69
  > Given GOLD and BOT answers, output strict JSON: `{factual_match: bool, citation_present: bool, score: 0-1, reason: str}`. Be strict on partial matches.
70
 
@@ -82,11 +86,11 @@ Different model family from the Gemini-led Brain Fast / Brain Main → **non-cir
82
  | Citation accuracy | C3 ≥ 95% | n_correct_citations / n_non_refusal |
83
  | Refusal precision | C4 ≥ 90% | n_correct_refusals / n_expected_refusals |
84
  | Hindi parity | C8 within 5pp | factual_acc(hi) vs factual_acc(en) |
85
- | Brain winners | (router config) | factual_acc grouped by `brain_used` emitted values include `sales_brain::continue`, `sales_brain::complete`, `brain_main::<elected_model>`, `brain_main::graceful_error` |
86
 
87
  ## 4. Output artifacts per run
88
 
89
- - `eval/results.md` — human-readable summary with per-type, per-brain accuracy + sample misses
90
  - `eval/results.json` — machine-readable full per-question record
91
  - `logs/hallucinations.jsonl` — every blocked reply with its reason (audit log)
92
 
@@ -96,8 +100,8 @@ Different model family from the Gemini-led Brain Fast / Brain Main → **non-cir
96
  | --- | --- | --- |
97
  | Development | Manual, after meaningful changes | `python -m eval.run` |
98
  | Pre-deploy | Every PR | GitHub Actions runs eval, blocks merge if accuracy regresses |
99
- | Production | Nightly synthetic + spot-grading | Cron via Render scheduled job; live-traffic sampling via Playwright |
100
- | Post-deploy verification | Per deploy | `tests/live_verify.py` runs full eval against live Vercel URL |
101
 
102
  ## 6. Bilingual eval
103
 
@@ -106,6 +110,6 @@ Bilingual sub-set: 20 questions translated to Hindi + Hinglish via Sarvam-M with
106
  ## 7. Known limitations (transparent)
107
 
108
  - **Sample bias:** Pipeline A questions are templated, so accuracy on Pipeline A is upper-bound real-world performance. Pipeline B + C provide the harder signal.
109
- - **Single judge model:** NIM Mistral Large 3 675B is the elected judge primary; the fallback ladder includes Llama-4 Maverick + OR Qwen 80B + NIM Nemotron 49B for resilience but every grade run uses a single judge per question. Risk of judge-specific bias. v2: 3-judge consensus.
110
  - **No human evaluation:** Cost-prohibitive for 1,100 questions. We rely on the grader; manual spot-check 5%.
111
  - **No latency budget enforcement in eval:** Latency is captured per record but doesn't gate the score. Doc 01 C1 (p50 ≤ 4s) is monitored separately.
 
1
  # 03 — Evaluation Plan
2
 
3
+ > ⚠️ **Methodology is current; some implementation pointers are historical.**
4
+ > The gold-set construction, grader signals, and metrics below still
5
+ > describe how we evaluate the bot. But the *system under test* is now the
6
+ > single-LLM-with-tools handler (`backend/single_brain.py` one Gemini
7
+ > 2.5-flash call per turn with `save_profile_field` / `retrieve_policies` /
8
+ > `mark_recommendation`, structured+vector retrieval, small
9
+ > `nim_fallback.py` for transient errors). There is no `orchestrator`, no
10
+ > `sales_brain`/`qa_brain` split, no separate faithfulness-judge LLM, and no
11
+ > DuckDB hot path. Present-state authority: [`README.md`](../../README.md) §4.
12
 
13
  | Field | Value |
14
  | --- | --- |
15
  | Project | Insurance Sales Portfolio Expert |
16
  | Version | 0.1 |
17
  | Date | 2026-05-13 |
18
+ | Depends on | `01-requirements.md` §6 (success criteria); README §4 (system under test) |
19
  | Status | Pipeline implemented; full run pending corpus completion |
20
 
21
  ## 0. Purpose
 
32
 
33
  ### Pipeline A — auto-generated from structured extraction (the bulk)
34
 
35
+ For every successfully extracted policy, generate templated Q&A where the answer comes directly from the 62-field structured extraction. Per `eval/generate_gold.py`:
36
 
37
  - ~15 question templates × ~80 policies = ~1,100 candidate pairs
38
  - Each pair is **fully reproducible** — the answer traces to a specific field which traces to a specific clause
 
42
 
43
  ### Pipeline B — LLM-drafted nuanced questions (curated)
44
 
45
+ For top-priority policies (top 10–20), run an LLM on the policy text with prompt: *"generate 5 buyer-style questions whose answers are explicitly in this document; include the source clause."* Each generated pair is **human spot-checked** before commit.
46
 
47
  Target: 5 × 20 = 100 questions covering multi-clause and edge-case reasoning that Pipeline A can't generate.
48
 
 
60
 
61
  ## 2. Grader design
62
 
63
+ **Single grading endpoint:** `eval/run.py` calls the single-LLM-with-tools turn handler (`backend/single_brain.py`) in-process, takes the reply, scores against gold with three signals:
64
 
65
  ### Signal 1 — Regex hard-checks (deterministic)
66
 
67
  Numbers, dates, currency, durations, percentages are extracted via regex from both gold and bot reply. Exact match (after normalization) is the strongest signal. Catches "the premium is ₹15,000" hallucinations without an LLM.
68
 
69
+ ### Signal 2 — LLM-judge faithfulness (offline eval grader only)
70
 
71
+ The eval grader uses a **different model family** from the runtime Gemini 2.5-flash brain → **non-circular evaluation**. This judge is part of the *offline eval harness only* — it is not a runtime gate in the bot (runtime grounding is structural: the single LLM can only state what `retrieve_policies` returned and must cite it). Judge prompt:
72
 
73
  > Given GOLD and BOT answers, output strict JSON: `{factual_match: bool, citation_present: bool, score: 0-1, reason: str}`. Be strict on partial matches.
74
 
 
86
  | Citation accuracy | C3 ≥ 95% | n_correct_citations / n_non_refusal |
87
  | Refusal precision | C4 ≥ 90% | n_correct_refusals / n_expected_refusals |
88
  | Hindi parity | C8 within 5pp | factual_acc(hi) vs factual_acc(en) |
89
+ | Path winners | (config) | factual_acc grouped by which path produced the turn `gemini` (primary single-LLM call) vs `nim_fallback` (transient-error fallback) |
90
 
91
  ## 4. Output artifacts per run
92
 
93
+ - `eval/results.md` — human-readable summary with per-type, per-path accuracy + sample misses
94
  - `eval/results.json` — machine-readable full per-question record
95
  - `logs/hallucinations.jsonl` — every blocked reply with its reason (audit log)
96
 
 
100
  | --- | --- | --- |
101
  | Development | Manual, after meaningful changes | `python -m eval.run` |
102
  | Pre-deploy | Every PR | GitHub Actions runs eval, blocks merge if accuracy regresses |
103
+ | Production | Nightly synthetic + spot-grading | Scheduled job; live-traffic sampling via Playwright |
104
+ | Post-deploy verification | Per deploy | `tests/live_verify.py` runs full eval against the live HF Space URL |
105
 
106
  ## 6. Bilingual eval
107
 
 
110
  ## 7. Known limitations (transparent)
111
 
112
  - **Sample bias:** Pipeline A questions are templated, so accuracy on Pipeline A is upper-bound real-world performance. Pipeline B + C provide the harder signal.
113
+ - **Single judge model:** the offline eval grader uses one judge model (a different family from the runtime Gemini brain) per question. Risk of judge-specific bias. v2: 3-judge consensus.
114
  - **No human evaluation:** Cost-prohibitive for 1,100 questions. We rely on the grader; manual spot-check 5%.
115
  - **No latency budget enforcement in eval:** Latency is captured per record but doesn't gate the score. Doc 01 C1 (p50 ≤ 4s) is monitored separately.
70-docs/40-evaluation/known-issues.md CHANGED
@@ -1,5 +1,16 @@
1
  # Known Issues + Quality Sprint Log
2
 
 
 
 
 
 
 
 
 
 
 
 
3
  Living document. Every defect we find — whether via code review, eval audit,
4
  or production observation — lands here with severity, root cause, and the
5
  plan to fix it. Closed issues stay in the log with a `**FIXED in <sha>**`
 
1
  # Known Issues + Quality Sprint Log
2
 
3
+ > ⚠️ **Historical defect log — not the present-state map.** Entries below
4
+ > are dated incidents; many reference subsystems (`orchestrator`,
5
+ > `faithfulness.py` judge, `fact_find_brain` / `sales_brain`,
6
+ > `fact_find_normalizer`, the 3-tier Gemini/NIM/OpenRouter chain,
7
+ > sticky-primary election) that were **subsequently removed** by the
8
+ > single-LLM-with-tools rewrite (one Gemini 2.5-flash call per turn with
9
+ > `save_profile_field` / `retrieve_policies` / `mark_recommendation`,
10
+ > structured+vector retrieval, small `nim_fallback`). The log is retained
11
+ > as the quality trajectory of record; KI/ADR cross-references stay valid as
12
+ > history. Present-state authority: [`README.md`](../../README.md) §4.
13
+
14
  Living document. Every defect we find — whether via code review, eval audit,
15
  or production observation — lands here with severity, root cause, and the
16
  plan to fix it. Closed issues stay in the log with a `**FIXED in <sha>**`
70-docs/40-evaluation/quality-sprint-2026-05-14.md CHANGED
@@ -1,5 +1,15 @@
1
  # Quality Sprint — 2026-05-14
2
 
 
 
 
 
 
 
 
 
 
 
3
  One-day sprint focused on hardening the non-speech parts of the bot via
4
  code review, large-scale audit simulation, and real user-testing
5
  feedback. Every defect found in the same day shipped a fix in the same
 
1
  # Quality Sprint — 2026-05-14
2
 
3
+ > ⚠️ **Historical sprint log — not the present-state map.** This records
4
+ > dated quality work; the architecture described mid-log (`sales_brain` /
5
+ > `fact_find_brain` / orchestrator / separate faithfulness judge / 3-tier
6
+ > Gemini-2.0/2.5 + NIM/OpenRouter chain / sticky-primary election) was
7
+ > subsequently superseded by the single-LLM-with-tools rewrite (one Gemini
8
+ > 2.5-flash call per turn with `save_profile_field` / `retrieve_policies` /
9
+ > `mark_recommendation`, structured+vector retrieval, small `nim_fallback`).
10
+ > KI/ADR pointers below remain valid as history-of-record. Present-state
11
+ > authority: [`README.md`](../../README.md) §4.
12
+
13
  One-day sprint focused on hardening the non-speech parts of the bot via
14
  code review, large-scale audit simulation, and real user-testing
15
  feedback. Every defect found in the same day shipped a fix in the same
ARCHITECTURE.md CHANGED
@@ -19,13 +19,15 @@ authority.
19
  `/api/coverage`, `/api/profile*`, `/api/scorecard`, `/api/session*`,
20
  `/api/admin/*`.
21
  - **Brain** — one LLM call per turn: Google Gemini
22
- (`gemini-2.5-flash-lite`) + function-calling tools
23
- (`backend/single_brain.py`, `backend/brain_tools.py`). On Gemini
24
- failure / cold-start 503 → `backend/nim_fallback.py` (NVIDIA NIM chain,
25
- health-probed sticky-primary election in `backend/llm_health.py`).
26
- Fail-loud, never silently wrong. The legacy multi-pass design
27
- (orchestrator / sales-brain / QA-brain / faithfulness-judge) was removed
28
- it does not exist in the codebase.
 
 
29
  - **Retrieval** — Chroma vector store, BGE-small-en-v1.5 local 384-d
30
  embeddings (`rag/retrieve.py`). Shared "policies" collection (~150 plans,
31
  ~7.3k chunks, 20 insurers) + a per-session "quarantine" collection for
 
19
  `/api/coverage`, `/api/profile*`, `/api/scorecard`, `/api/session*`,
20
  `/api/admin/*`.
21
  - **Brain** — one LLM call per turn: Google Gemini
22
+ (`gemini-2.5-flash`) + function-calling tools
23
+ (`save_profile_field`, `retrieve_policies`, `mark_recommendation`) in
24
+ `backend/single_brain.py` / `backend/brain_tools.py`. A single call owns
25
+ the whole turn: fact-find, retrieval, QA, and recommendation. On a
26
+ transient Gemini error / cold-start 503 small `backend/nim_fallback.py`
27
+ (NVIDIA NIM) so the turn still completes. Fail-loud, never silently wrong.
28
+ The legacy multi-pass design (orchestrator / sales-brain / QA-brain /
29
+ separate faithfulness judge / profile_extractor / tiered brain) was
30
+ removed — it does not exist in the codebase.
31
  - **Retrieval** — Chroma vector store, BGE-small-en-v1.5 local 384-d
32
  embeddings (`rag/retrieve.py`). Shared "policies" collection (~150 plans,
33
  ~7.3k chunks, 20 insurers) + a per-session "quarantine" collection for
backend/admin.py CHANGED
@@ -13,10 +13,10 @@ Setup:
13
  Add to Space secrets (Settings → Variables and secrets):
14
  ADMIN_PASSWORD = your-strong-password-here
15
 
16
- Returns 401 Unauthorized on bad/missing password. The earlier IP allowlist
17
- gate (ADMIN_IP_ALLOWLIST + 404-to-hide-existence) was removed in KI-097
18
- because it added operational complexity (changing networks locked out)
19
- without meaningful additional security beyond a strong password.
20
  """
21
  from __future__ import annotations
22
 
@@ -67,18 +67,15 @@ def _password_ok(supplied: Optional[str]) -> bool:
67
 
68
 
69
  def _check_admin(request: Request, password: Optional[str]) -> None:
70
- """KI-097 — password-only gate. Raises 401 on bad/missing password.
71
 
72
- `request` is kept in the signature so callsites don't churn, but the
73
- function no longer inspects the client IP. Earlier dual-gate behavior
74
- (IP allowlist + 404-to-hide-existence) was removed in KI-097.
75
  """
76
- # TODO: enforce IP allowlist for hardening verified 2026-05-15 that
77
- # ADMIN_IP_ALLOWLIST is not configured on the HF Space (rohitsar567/
78
- # InsuranceBot). Gate is password-only, which is acceptable for the
79
- # current threat model per KI-097, but a future hardening pass should
80
- # re-introduce an IP allowlist as a second factor (with a documented
81
- # break-glass procedure so a network change doesn't lock ops out).
82
  if not _password_ok(password):
83
  raise HTTPException(status_code=401, detail="Unauthorized")
84
 
@@ -547,14 +544,12 @@ def _read_usage_24h() -> Optional[dict]:
547
  last USAGE_TAIL_LINES rows (typically covers ≈24h of activity at current
548
  traffic). Keeping the name aligns with the admin UI label.
549
 
550
- ADR-039/040 + Path B (2026-05-15): the multi-brain split (fast_brain /
551
- judge) and the standalone profile_extractor / paraphraser passes were
552
- DELETED. The only role the live stack now emits is `brain` (see
553
- providers.nvidia_nim_llm.get_brain_llm + llm_health.ROLES). The append-
554
- only llm_usage.jsonl still contains pre-rewrite legacy rows tagged with
555
- the retired roles, so we filter to the canonical role set exactly as
556
- /api/admin/usage already does via its `chains` dict — instead of blindly
557
- surfacing every historical role value. llm_health.ROLES is the single
558
  source of truth (also used as _LLM_HEALTH_CHAIN_ROLES below).
559
  """
560
  usage_path = settings.DATA_DIR / "llm_usage.jsonl"
@@ -633,9 +628,9 @@ _LLM_HEALTH_CHAIN_ROLES = llm_health.ROLES
633
 
634
 
635
  def _chain_names_map() -> dict[str, list[str]]:
636
- """Live (post-admin-override) chain config — read off the module so admin
637
- reorders applied earlier in the same process are reflected immediately.
638
- Three-chain collapse (2026-05-15) — only the brain role remains."""
639
  from backend.providers import nvidia_nim_llm as nim
640
  return {
641
  "brain": list(getattr(nim, "BRAIN_CHAIN", [])),
@@ -758,20 +753,20 @@ def _candidate_available_for_calls(h, now_mono: float) -> bool:
758
 
759
 
760
  def _candidate_credit_exhausted_strict(h, now_mono: float) -> bool:
761
- """KI-122 — STRICT credit-exhausted rule for a single candidate.
762
 
763
  True ONLY when ALL three hold:
764
  (1) credits_remaining is NOT None (we have a real signal)
765
  (2) credits_remaining <= credits_low_water
766
  (3) credits_reset_at is set AND in the FUTURE (we're inside an
767
  active gating window — stale or absent reset means the
768
- snapshot is no longer authoritative).
769
 
770
  This is intentionally stricter than `not _has_credits()` because the
771
  banner is louder than the elector. The elector falls through cheaply
772
- on a single bad candidate; the banner falsely scaring the operator
773
- when one quota-exhausted backup co-exists with a perfectly healthy
774
- primary is the bug we're fixing here."""
775
  if h is None:
776
  return False
777
  if h.credits_remaining is None:
@@ -987,13 +982,11 @@ async def admin_llm_health(
987
  # ---------------------------------------------------------------------------
988
 
989
  # The seven canonical "ready-to-recommend" fact-find slots. Mirrors
990
- # single_brain._FALLBACK_REQUIRED_SLOTS (the live single-brain stack's
991
- # required set after the ADR-039/040 Path B rewrite) and needs_finder.
992
- # Profile's persisted fields. `dependents` was previously missing here even
993
- # though it is a required slot and is persisted on every profile JSON —
994
- # adding/removing a slot must stay in sync with that required set. Health is
995
- # captured as a list (empty list counts as "asked but no conditions" → still
996
- # a valid captured signal once the `asked` array contains the field name).
997
  _PERSONA_DRIFT_SLOTS = ("name", "age", "dependents", "location_tier",
998
  "income_band", "primary_goal", "health_conditions")
999
 
 
13
  Add to Space secrets (Settings → Variables and secrets):
14
  ADMIN_PASSWORD = your-strong-password-here
15
 
16
+ Returns 401 Unauthorized on bad/missing password. Access is gated by a
17
+ strong password only; there is no IP allowlist (it would add operational
18
+ complexity changing networks would lock the operator out — without
19
+ meaningful additional security).
20
  """
21
  from __future__ import annotations
22
 
 
67
 
68
 
69
  def _check_admin(request: Request, password: Optional[str]) -> None:
70
+ """Password-only gate. Raises 401 on bad/missing password.
71
 
72
+ `request` is kept in the signature so callsites don't churn; the
73
+ function does not inspect the client IP.
 
74
  """
75
+ # TODO: enforce an IP allowlist as a second factor for hardening (with
76
+ # a documented break-glass procedure so a network change doesn't lock
77
+ # ops out). The gate is password-only, acceptable for the current
78
+ # threat model.
 
 
79
  if not _password_ok(password):
80
  raise HTTPException(status_code=401, detail="Unauthorized")
81
 
 
544
  last USAGE_TAIL_LINES rows (typically covers ≈24h of activity at current
545
  traffic). Keeping the name aligns with the admin UI label.
546
 
547
+ The live stack emits the `brain` role (see
548
+ providers.nvidia_nim_llm.get_brain_llm + llm_health.ROLES). The
549
+ append-only llm_usage.jsonl can contain rows tagged with non-canonical
550
+ role values, so we filter to the canonical role set — the same way
551
+ /api/admin/usage does via its `chains` dict instead of surfacing
552
+ every role value present in the file. llm_health.ROLES is the single
 
 
553
  source of truth (also used as _LLM_HEALTH_CHAIN_ROLES below).
554
  """
555
  usage_path = settings.DATA_DIR / "llm_usage.jsonl"
 
628
 
629
 
630
  def _chain_names_map() -> dict[str, list[str]]:
631
+ """Live (post-admin-override) chain config — read off the module so
632
+ admin reorders applied earlier in the same process are reflected
633
+ immediately. Only the brain role is present."""
634
  from backend.providers import nvidia_nim_llm as nim
635
  return {
636
  "brain": list(getattr(nim, "BRAIN_CHAIN", [])),
 
753
 
754
 
755
  def _candidate_credit_exhausted_strict(h, now_mono: float) -> bool:
756
+ """STRICT credit-exhausted rule for a single candidate.
757
 
758
  True ONLY when ALL three hold:
759
  (1) credits_remaining is NOT None (we have a real signal)
760
  (2) credits_remaining <= credits_low_water
761
  (3) credits_reset_at is set AND in the FUTURE (we're inside an
762
  active gating window — stale or absent reset means the
763
+ snapshot is not authoritative).
764
 
765
  This is intentionally stricter than `not _has_credits()` because the
766
  banner is louder than the elector. The elector falls through cheaply
767
+ on a single bad candidate; this strictness avoids falsely scaring the
768
+ operator when one quota-exhausted backup co-exists with a perfectly
769
+ healthy primary."""
770
  if h is None:
771
  return False
772
  if h.credits_remaining is None:
 
982
  # ---------------------------------------------------------------------------
983
 
984
  # The seven canonical "ready-to-recommend" fact-find slots. Mirrors
985
+ # brain_tools._REQUIRED_FOR_READY (the recommendation-ready required set)
986
+ # and needs_finder.Profile's persisted fields. Adding/removing a slot must
987
+ # stay in sync with that required set. Health is captured as a list (empty
988
+ # list counts as "asked but no conditions" still a valid captured signal
989
+ # once the `asked` array contains the field name).
 
 
990
  _PERSONA_DRIFT_SLOTS = ("name", "age", "dependents", "location_tier",
991
  "income_band", "primary_goal", "health_conditions")
992
 
backend/brain_tools.py CHANGED
@@ -88,41 +88,28 @@ _log = logging.getLogger(__name__)
88
 
89
  _FACTS_DIR = settings.DATA_DIR / "policy_facts"
90
  _DOCTYPE_SUFFIXES = ("__wordings", "__brochure", "__cis", "__prospectus")
91
- # KI-RECALL-POOL (2026-05-17) — width of the Chroma candidate pool pulled
92
- # BEFORE eligibility + scorecard-aware ranking. Must be wide enough that a
93
- # genuinely-strong A/B policy that is cosine-rank ~15-30 for a profile
94
- # query still enters contention (so the quality-aware filter_pipeline can
95
- # surface it). The LLM still only ever sees the top-k best-fit survivors.
96
  _RECALL_POOL = 40
97
  _FACT_KEYS = (
98
  "policy_type_indemnity_or_fixed",
99
- # KI-279 (2026-05-16) — the raw catalog type key. Some curated files
100
- # (e.g. star-health__star-hospital-cash__brochure.json) carry the
101
- # product type ONLY under `policy_type` ("hospital_cash") and have NO
102
- # `policy_type_indemnity_or_fixed` key. Without surfacing this, the
103
- # retrieval_filters fixed-benefit gate is blind and a daily-cash plan
104
- # ranks #1 for a comprehensive-indemnity buyer. Read-only; STABLE key.
105
  "policy_type",
106
  "deductible_amount",
107
  "co_payment_pct",
108
  "sum_insured_options",
109
- # KI-280 (2026-05-16) — UNIFIED recommendation-fit gate. The eligibility
110
- # gate needs these STABLE policy_facts keys (read-only):
111
  # uin_code → canonical dedup identity (1 UIN = 1 product;
112
- # collapses marketing-rename / doctype-sibling
113
- # duplicate CITED cards audit P2/P4/P7).
114
- # max_entry_age → hard eligibility for the INSURED person. When
115
- # the profile insures parents (~70), a plan
116
- # whose max entry age is 65 cannot accept them
117
- # (audit P7). The chunk already carries
118
- # max_entry_age from Chroma metadata, but the
119
- # curated facts file is the authoritative value
120
- # and overrides a missing/None chunk field.
121
  # maternity_coverage → required-feature gate: an explicit maternity /
122
- # newborn_coverage newborn need must rank confirmed plans above
123
- # unconfirmed ones (audit P3).
124
- # max_renewal_age is intentionally NOT read — it was removed from the
125
- # scorecard (lifelong renewal is the IRDAI norm; scorecard.py:606).
126
  "uin_code",
127
  "max_entry_age",
128
  "maternity_coverage",
@@ -214,21 +201,12 @@ _reviews_cache: dict = {}
214
 
215
 
216
  def _curated_facts_all() -> dict:
217
- """The FULL curated-facts layer, EXACTLY as the marketplace uses it.
218
-
219
- KI-FULLFACTS (2026-05-17) root cause of empty recommendation cards:
220
- `_scorecard_signal` used to feed build_scorecard the 7-key eligibility
221
- subset (`_load_policy_facts`), which is BELOW build_scorecard's
222
- data-completeness floor every recommended policy returned grade ""
223
- / overall 0 → the Bug #71 ≥70 gate dropped 100% of them → no cards.
224
- Meanwhile /api/policies/all scored the SAME policies correctly
225
- (HDFC Ergo Optima Restore = A/78) because it feeds the FULL curated
226
- dict from main._load_curated_facts (KI-219/251 canonical precedence).
227
- We now reuse that EXACT function (lazy import — main is loaded at
228
- request time; dodges the import cycle) so a policy's recommendation
229
- grade is identical to its marketplace grade by construction. The
230
- 7-key `_load_policy_facts` stays as-is — it is the correct input for
231
- the eligibility/dedup gate, NOT for grading."""
232
  if "d" not in _curated_all_cache:
233
  from backend.main import _load_curated_facts # lazy: avoids cycle
234
  _curated_all_cache["d"] = _load_curated_facts()
@@ -236,9 +214,9 @@ def _curated_facts_all() -> dict:
236
 
237
 
238
  def _insurer_reviews(slug: str) -> Optional[dict]:
239
- """Same reviews source the marketplace passes to build_scorecard
240
- (40-data/reviews/<slug>.json) — drives the claim-experience sub-score.
241
- Without it the rec-path grade would drift below the marketplace's."""
242
  if not slug:
243
  return None
244
  if slug in _reviews_cache:
@@ -255,12 +233,11 @@ def _insurer_reviews(slug: str) -> Optional[dict]:
255
 
256
 
257
  def _scorecard_signal(policy_id: str, profile=None) -> dict:
258
- """Best-effort {_grade, _overall_score} from backend.scorecard, using
259
- the IDENTICAL inputs as the marketplace (/api/policies/all) so a
260
- policy's recommendation grade == its marketplace grade by construction
261
- (parity is asserted by tests/test_scorecard_parity.py). READ-ONLY;
262
- degrades silently to {} on any failure (ranking still works off
263
- co-pay + SI headroom + cosine)."""
264
  try:
265
  cur = _curated_facts_all()
266
  data = None
@@ -288,15 +265,8 @@ def _scorecard_signal(policy_id: str, profile=None) -> dict:
288
  )
289
  }
290
  sc = build_scorecard(data, insurer_reviews=ir, profile=prof)
291
- # KI-280 (2026-05-16) the Scorecard dataclass field is
292
- # `overall_score` (scorecard.py:95), NOT `overall`. The previous
293
- # getattr(sc, "overall", ...) ALWAYS returned None, so every
294
- # enriched chunk reached retrieval_filters with _overall_score=None
295
- # and _fit_score fell back to the coarse letter-grade table where
296
- # every "B" == 70.0 exactly. Within-grade order then collapsed to
297
- # raw cosine — the live audit's grade/rank inversion (P1 C/65 above
298
- # B/75, P2 A/77 ranked LAST). Read the real numeric field; keep a
299
- # belt-and-braces `overall` alias for forward-compat.
300
  _overall = getattr(sc, "overall_score", None)
301
  if _overall is None:
302
  _overall = getattr(sc, "overall", None)
@@ -647,18 +617,12 @@ _qseed_cache: dict = {} # (profile_sig) -> [seed chunk dicts]
647
 
648
 
649
  def _quality_seed_candidates(profile, limit: int = 25) -> list[dict]:
650
- """KI-QUALITY-SEED (2026-05-17). Cosine retrieval is quality-blind: a
651
- genuinely-best policy whose document wording isn't cosine-similar to a
652
- generic profile query never enters the candidate pool, so the
653
- (audit-verified-correct) profile-tuned scorecard + filter_pipeline never
654
- get the chance to surface it. This deterministically seeds the pool with
655
- the catalogue's top policies by the PROFILE-TUNED scorecard overall.
656
- filter_pipeline still does precise eligibility + profile-fit ranking on
657
- the union (proven: it eligibility-passes + ranks a present A-policy #1),
658
- so this ONLY guarantees CANDIDACY — it never bypasses eligibility or
659
- fabricates a recommendation. Pure logic layer: no re-ingest, no vector
660
- DB change. Cached per profile-signature (the catalogue + a fixed profile
661
- yield a fixed ranking)."""
662
  out: list[dict] = []
663
  try:
664
  prof_sig = repr(sorted(
@@ -882,18 +846,11 @@ async def retrieve_policies(
882
  try:
883
  from rag.retrieve import retrieve as _retrieve
884
 
885
- # KI-RECALL-POOL (2026-05-17) — DECOUPLE the Chroma recall pool from
886
- # the caller's top_k. Cosine-only Chroma ranking is quality-blind: a
887
- # mediocre C/D policy whose wording matches the query out-ranks a
888
- # genuinely-better A/B policy that IS in the corpus. At top_k=8 the
889
- # A-grade policies never even enter the candidate set, so the
890
- # downstream quality-aware filter_pipeline (_fit_score, now fed REAL
891
- # scorecard grades after KI-FULLFACTS) can't surface them — it can
892
- # only re-order what was retrieved. Fix: retrieve a WIDE pool, let
893
- # eligibility + scorecard-aware ranking choose, then return the
894
- # top-k best-fit survivors (truncation happens after filter_pipeline
895
- # below). For an explicit known-policy follow-up (policy_filter_ids)
896
- # the caller already knows the policy — keep the narrow top_k.
897
  _recall_pool = (
898
  (int(top_k) if top_k else 8)
899
  if policy_filter_ids
@@ -953,13 +910,12 @@ async def retrieve_policies(
953
  guard_signal = None
954
  filtered = raw
955
  if not policy_filter_ids:
956
- # KI-QUALITY-SEED — union the cosine pool with the catalogue's
957
- # top profile-graded policies so a genuinely-best policy that
958
- # isn't cosine-similar to a generic needs-query still enters
959
- # contention. filter_pipeline then does precise eligibility +
960
- # profile-fit ranking on the union (it eligibility-passes + ranks
961
- # a present A-policy #1, proven). Cosine chunk wins on dup (it
962
- # carries real retrieved text); seeds only ADD missing candidates.
963
  if profile is not None:
964
  _seen_pids = {
965
  (c.get("policy_id") or "").strip() for c in raw
@@ -978,28 +934,23 @@ async def retrieve_policies(
978
  _log.warning("retrieval_filters.filter_pipeline failed: %s", e)
979
  filtered = raw
980
 
981
- # KI-RECALL-POOL — filter_pipeline has now eligibility-filtered +
982
- # ranked the WIDE pool best-fit-first (scorecard-aware, deduped to
983
- # ~1 chunk/policy). Return only the top-N best-fit survivors: enough
984
- # for the LLM to pick 2-4 strong recommendations AND for the
985
- # citation builder to select from, without dumping the 40-wide tail
986
- # as context. This is the ONLY truncation done AFTER quality
987
- # ranking, so the genuinely-best eligible policies (e.g. SBI Super
988
- # Health A/87) are what the LLM and the cards both see. Set on the
989
- # session cache too so prose ⇄ cited cards are gated on the SAME set.
990
  filtered = filtered[: max((int(top_k) if top_k else 8), 12)]
991
 
992
- # KI-PROSE-CARD-1to1 (2026-05-17) user requirement: "can't have a
993
- # situation where it suggests something in chat but won't generate a
994
- # card." For a RECOMMENDATION turn, apply the SAME fitness floor the
995
- # citation builder uses (_recommendation_fit) HERE, so the LLM only
996
- # ever SEES cardable policies prose cards are 1:1 by
997
- # construction (it cannot name a policy it won't card because it
998
- # never receives one). If nothing clears the floor we return none —
999
- # the LLM then honestly says "no strong match" rather than padding
1000
- # with a sub-floor policy (also the requirement). qa / follow-up
1001
- # intents are untouched: they legitimately cite supporting source
1002
- # chunks regardless of recommendation grade.
1003
  if (intent or "").lower() == "recommendation" and filtered:
1004
  try:
1005
  from backend.single_brain import _recommendation_fit
 
88
 
89
  _FACTS_DIR = settings.DATA_DIR / "policy_facts"
90
  _DOCTYPE_SUFFIXES = ("__wordings", "__brochure", "__cis", "__prospectus")
91
+ # Width of the Chroma candidate pool pulled before eligibility +
92
+ # scorecard-aware ranking wide enough that a strong policy ranked only
93
+ # mid-cosine for a generic query still enters contention. The LLM still
94
+ # only sees the top-k best-fit survivors.
 
95
  _RECALL_POOL = 40
96
  _FACT_KEYS = (
97
  "policy_type_indemnity_or_fixed",
98
+ # Raw catalog type key some curated files carry the product type
99
+ # only under `policy_type` (e.g. "hospital_cash") with no
100
+ # `policy_type_indemnity_or_fixed`; the fixed-benefit gate needs it.
 
 
 
101
  "policy_type",
102
  "deductible_amount",
103
  "co_payment_pct",
104
  "sum_insured_options",
105
+ # Eligibility-gate keys (read-only):
 
106
  # uin_code → canonical dedup identity (1 UIN = 1 product;
107
+ # collapses doctype-sibling/rename duplicates).
108
+ # max_entry_age → hard eligibility for the insured person; the
109
+ # curated value is authoritative and overrides
110
+ # a missing/None Chroma chunk field.
 
 
 
 
 
111
  # maternity_coverage → required-feature gate: an explicit maternity /
112
+ # newborn_coverage newborn need ranks confirmed plans first.
 
 
 
113
  "uin_code",
114
  "max_entry_age",
115
  "maternity_coverage",
 
201
 
202
 
203
  def _curated_facts_all() -> dict:
204
+ """The full curated-facts layer the same source the marketplace
205
+ scores from (main._load_curated_facts, KI-219/251 canonical
206
+ precedence; lazy import since main is loaded at request time). Using
207
+ it here makes a policy's recommendation grade identical to its
208
+ marketplace grade. The 7-key `_load_policy_facts` is the input for the
209
+ eligibility/dedup gate, not for grading."""
 
 
 
 
 
 
 
 
 
210
  if "d" not in _curated_all_cache:
211
  from backend.main import _load_curated_facts # lazy: avoids cycle
212
  _curated_all_cache["d"] = _load_curated_facts()
 
214
 
215
 
216
  def _insurer_reviews(slug: str) -> Optional[dict]:
217
+ """Insurer reviews (40-data/reviews/<slug>.json) passed to
218
+ build_scorecard — drives the claim-experience sub-score; the same
219
+ source the marketplace uses."""
220
  if not slug:
221
  return None
222
  if slug in _reviews_cache:
 
233
 
234
 
235
  def _scorecard_signal(policy_id: str, profile=None) -> dict:
236
+ """{_grade, _overall_score} from backend.scorecard using the same
237
+ inputs as the marketplace, so a policy's recommendation grade equals
238
+ its marketplace grade (asserted by tests/test_scorecard_parity.py).
239
+ Read-only; returns {} on any failure (ranking still works off co-pay
240
+ + SI headroom + cosine)."""
 
241
  try:
242
  cur = _curated_facts_all()
243
  data = None
 
265
  )
266
  }
267
  sc = build_scorecard(data, insurer_reviews=ir, profile=prof)
268
+ # Scorecard.overall_score is the numeric field; `overall` is a
269
+ # back-compat alias.
 
 
 
 
 
 
 
270
  _overall = getattr(sc, "overall_score", None)
271
  if _overall is None:
272
  _overall = getattr(sc, "overall", None)
 
617
 
618
 
619
  def _quality_seed_candidates(profile, limit: int = 25) -> list[dict]:
620
+ """Seed the candidate pool with the catalogue's top policies by the
621
+ profile-tuned scorecard overall, so a strong policy that isn't
622
+ cosine-similar to a generic query still enters contention. This only
623
+ ADDS candidates filter_pipeline still applies precise eligibility +
624
+ profile-fit ranking on the union; it never bypasses eligibility or
625
+ fabricates a recommendation. Cached per profile-signature."""
 
 
 
 
 
 
626
  out: list[dict] = []
627
  try:
628
  prof_sig = repr(sorted(
 
846
  try:
847
  from rag.retrieve import retrieve as _retrieve
848
 
849
+ # Decouple the Chroma recall pool from the caller's top_k: pull a
850
+ # wide pool so eligibility + scorecard-aware ranking choose from a
851
+ # broad set, then return the top-k best-fit survivors (truncated
852
+ # after filter_pipeline below). For an explicit known-policy
853
+ # follow-up (policy_filter_ids) keep the narrow top_k.
 
 
 
 
 
 
 
854
  _recall_pool = (
855
  (int(top_k) if top_k else 8)
856
  if policy_filter_ids
 
910
  guard_signal = None
911
  filtered = raw
912
  if not policy_filter_ids:
913
+ # Union the cosine pool with the catalogue's top profile-graded
914
+ # policies so a strong policy that isn't cosine-similar to a
915
+ # generic needs-query still enters contention. filter_pipeline
916
+ # then applies eligibility + profile-fit ranking on the union. A
917
+ # cosine chunk wins on a dup (it carries real retrieved text);
918
+ # seeds only add missing candidates.
 
919
  if profile is not None:
920
  _seen_pids = {
921
  (c.get("policy_id") or "").strip() for c in raw
 
934
  _log.warning("retrieval_filters.filter_pipeline failed: %s", e)
935
  filtered = raw
936
 
937
+ # filter_pipeline has eligibility-filtered + ranked the wide pool
938
+ # best-fit-first (scorecard-aware, deduped to ~1 chunk/policy).
939
+ # Return only the top-N best-fit survivors enough for the LLM to
940
+ # pick 2-4 recommendations and for the citation builder to select
941
+ # from, without the wide tail as context. This is the only
942
+ # truncation, done after quality ranking, so the LLM and the cards
943
+ # see the same best-fit set.
 
 
944
  filtered = filtered[: max((int(top_k) if top_k else 8), 12)]
945
 
946
+ # For a recommendation turn, apply the same fitness floor the
947
+ # citation builder uses (_recommendation_fit) here, so the LLM only
948
+ # sees cardable policies prose and cards stay 1:1 (it cannot name
949
+ # a policy it won't card). If nothing clears the floor the set is
950
+ # empty and the LLM says there is no strong match rather than
951
+ # padding with a sub-floor policy. qa / follow-up intents are
952
+ # untouched (they cite supporting source chunks regardless of
953
+ # recommendation grade).
 
 
 
954
  if (intent or "").lower() == "recommendation" and filtered:
955
  try:
956
  from backend.single_brain import _recommendation_fit
backend/config.py CHANGED
@@ -19,12 +19,9 @@ class Settings:
19
  # Provider keys
20
  SARVAM_API_KEY: str = os.environ.get("SARVAM_API_KEY", "")
21
  VOYAGE_API_KEY: str = os.environ.get("VOYAGE_API_KEY", "")
22
- # NVIDIA NIM — single provider hosting the entire reasoning stack:
23
- # Brain = meta/llama-3.3-70b-instruct
24
- # Judge = meta/llama-4-maverick-17b-128e-instruct (different arch from brain)
25
- # Free tier: 40 req/min, no daily cap, no card. Replaces OpenRouter +
26
- # direct DeepSeek + Cerebras + Groq (four legacy providers retired
27
- # 2026-05-14 in favor of single-provider consolidation — see D-019).
28
  NVIDIA_NIM_API_KEY: str = os.environ.get("NVIDIA_NIM_API_KEY", "")
29
 
30
  # CROSS-PROVIDER FALLBACKS — last-resort entries appended to BRAIN_CHAIN +
@@ -38,7 +35,7 @@ class Settings:
38
  OPENROUTER_API_KEY: str = os.environ.get("OPENROUTER_API_KEY", "")
39
  GROQ_API_KEY: str = os.environ.get("GROQ_API_KEY", "")
40
 
41
- # Sarvam endpoints (voice STT/TTS + Indic translation only — not brain anymore)
42
  SARVAM_BASE_URL: str = "https://api.sarvam.ai"
43
  SARVAM_STT_PATH: str = "/speech-to-text"
44
  SARVAM_TTS_PATH: str = "/text-to-speech"
@@ -48,15 +45,15 @@ class Settings:
48
  SARVAM_STT_MODEL: str = "saarika:v2.5"
49
  SARVAM_TTS_MODEL: str = "bulbul:v2"
50
  SARVAM_TTS_SPEAKER: str = "anushka" # natural female advisor voice
51
- SARVAM_LLM_MODEL: str = "sarvam-m" # used by translator.py for Indic translation
52
 
53
- # Voyage (legacy — embeddings now via local BGE; kept for back-compat with extracted/ artifacts)
 
54
  VOYAGE_MODEL: str = "voyage-3"
55
 
56
- # NVIDIA NIM (single source of truth for brain + judge — tiered routing)
57
- # 2026-05-14 brain swap (D-022): DeepSeek-V4 + Meta Llama NIM pools time out
58
- # repeatedly. Qwen 3-Next 80B + Mistral Large 3 are the working production
59
- # models on NIM free tier as of 2026-05-14.
60
  NVIDIA_NIM_BASE_URL: str = "https://integrate.api.nvidia.com/v1"
61
  NVIDIA_NIM_BRAIN_MODEL: str = "qwen/qwen3-next-80b-a3b-instruct"
62
  NVIDIA_NIM_FAST_BRAIN_MODEL: str = "qwen/qwen3-next-80b-a3b-instruct"
@@ -67,12 +64,9 @@ class Settings:
67
  EXTRACTED_DIR: Path = ROOT / "rag" / "extracted"
68
  VECTORS_DIR: Path = ROOT / "rag" / "vectors"
69
  STRUCTURED_DB: Path = ROOT / "rag" / "policies.duckdb"
70
- # Single source of truth for the curated-facts directory. Previously the
71
- # literal "40-data" was hardcoded independently in ~13 runtime modules
72
- # (admin/marketplace/scorecard/premium/profile/llm-health/brain-tools).
73
- # Every one of those expressions resolved to <repo_root>/40-data, so this
74
- # constant is a behaviour-preserving consolidation, not a path change.
75
- # The directory name is intentionally kept (parallel to 70-docs/80-audit).
76
  DATA_DIR: Path = ROOT / "40-data"
77
 
78
  # Tunables (overrideable via env vars so the hyperparameter sweep can iterate)
@@ -80,7 +74,7 @@ class Settings:
80
  CHUNK_OVERLAP_TOKENS: int = int(os.environ.get("CHUNK_OVERLAP_TOKENS", "120"))
81
  RAG_TOP_K: int = int(os.environ.get("RAG_TOP_K", "5"))
82
 
83
- # Quarantine TTL (2026-05-16) — user-uploaded PDFs live in the SEPARATE
84
  # `user_uploads_quarantine` Chroma collection. They are NOT durable
85
  # corpus; a session's upload is auto-purged after this many seconds of
86
  # no further uploads from that session, so the quarantine index can't
 
19
  # Provider keys
20
  SARVAM_API_KEY: str = os.environ.get("SARVAM_API_KEY", "")
21
  VOYAGE_API_KEY: str = os.environ.get("VOYAGE_API_KEY", "")
22
+ # NVIDIA NIM — single provider hosting the reasoning stack (brain +
23
+ # judge; concrete model IDs are set on NVIDIA_NIM_*_MODEL below).
24
+ # Free tier: 40 req/min, no daily cap, no card.
 
 
 
25
  NVIDIA_NIM_API_KEY: str = os.environ.get("NVIDIA_NIM_API_KEY", "")
26
 
27
  # CROSS-PROVIDER FALLBACKS — last-resort entries appended to BRAIN_CHAIN +
 
35
  OPENROUTER_API_KEY: str = os.environ.get("OPENROUTER_API_KEY", "")
36
  GROQ_API_KEY: str = os.environ.get("GROQ_API_KEY", "")
37
 
38
+ # Sarvam endpoints (voice STT/TTS + Indic translation only)
39
  SARVAM_BASE_URL: str = "https://api.sarvam.ai"
40
  SARVAM_STT_PATH: str = "/speech-to-text"
41
  SARVAM_TTS_PATH: str = "/text-to-speech"
 
45
  SARVAM_STT_MODEL: str = "saarika:v2.5"
46
  SARVAM_TTS_MODEL: str = "bulbul:v2"
47
  SARVAM_TTS_SPEAKER: str = "anushka" # natural female advisor voice
48
+ SARVAM_LLM_MODEL: str = "sarvam-m" # Sarvam model for Indic translation
49
 
50
+ # Voyage — embeddings run on local BGE; this is kept for back-compat
51
+ # with existing extracted/ artifacts.
52
  VOYAGE_MODEL: str = "voyage-3"
53
 
54
+ # NVIDIA NIM (single source of truth for brain + judge — tiered
55
+ # routing). Qwen 3-Next 80B + Mistral Large 3 are the production
56
+ # models on NIM free tier.
 
57
  NVIDIA_NIM_BASE_URL: str = "https://integrate.api.nvidia.com/v1"
58
  NVIDIA_NIM_BRAIN_MODEL: str = "qwen/qwen3-next-80b-a3b-instruct"
59
  NVIDIA_NIM_FAST_BRAIN_MODEL: str = "qwen/qwen3-next-80b-a3b-instruct"
 
64
  EXTRACTED_DIR: Path = ROOT / "rag" / "extracted"
65
  VECTORS_DIR: Path = ROOT / "rag" / "vectors"
66
  STRUCTURED_DB: Path = ROOT / "rag" / "policies.duckdb"
67
+ # Single source of truth for the curated-facts directory. Resolves to
68
+ # <repo_root>/40-data; the directory name is intentionally kept
69
+ # (parallel to 70-docs/80-audit).
 
 
 
70
  DATA_DIR: Path = ROOT / "40-data"
71
 
72
  # Tunables (overrideable via env vars so the hyperparameter sweep can iterate)
 
74
  CHUNK_OVERLAP_TOKENS: int = int(os.environ.get("CHUNK_OVERLAP_TOKENS", "120"))
75
  RAG_TOP_K: int = int(os.environ.get("RAG_TOP_K", "5"))
76
 
77
+ # Quarantine TTL — user-uploaded PDFs live in the SEPARATE
78
  # `user_uploads_quarantine` Chroma collection. They are NOT durable
79
  # corpus; a session's upload is auto-purged after this many seconds of
80
  # no further uploads from that session, so the quarantine index can't
backend/llm_health.py CHANGED
@@ -1,19 +1,12 @@
1
- """Live NIM model health monitor — OpenRouter-style availability filter
2
- + KI-080 sticky primary/backup election.
3
-
4
- Architectural shift (KI-080, 2026-05-15):
5
- ==========================================
6
- Pre-KI-080 the chain was iterated EVERY chat turn primary, fallback 1,
7
- fallback 2, ... until one succeeded. Under NIM per-key concurrency throttling,
8
- the first 5 NIM-hosted candidates queued together inside a single turn,
9
- burning the 22s budget before the cross-provider fallback links (Groq /
10
- OpenRouter) were ever reached. The 10-turn live probe at commit 078ff45
11
- showed 7/10 fact-find turns timing out at exactly 26.6s.
12
-
13
- KI-080 inverts the model: the background probe loop ELECTS a primary +
14
- backup per chain based on real probe latencies; chat() calls the elected
15
- primary ONCE per turn, with at most ONE real-time fallback to the elected
16
- backup. Worst case per turn drops from 5-6 LLM calls to 1-2.
17
 
18
  Probes every PROBE_INTERVAL_SEC tick (300s) with a tiny ping ("Reply with
19
  exactly: ok"), `max_tokens=1` so probe-driven token spend is negligible.
@@ -27,24 +20,23 @@ Records per model:
27
  - probe_history: last PROBE_HISTORY_LEN (ok, latency_ms) tuples; powers
28
  the success_rate signal in the election score.
29
 
30
- Election (KI-201, 2026-05-15 — supersedes KI-080/KI-087 score-based election):
31
- - Walk the chain definition (BRAIN_CHAIN — the only chain after the
32
- three-chain collapse on 2026-05-15) in priority order.
33
  CURRENT_PRIMARY = first election-eligible model.
34
  CURRENT_BACKUP = next election-eligible model after primary.
35
  - Chain hierarchy IS the truth — nemotron-49b is LAST in BRAIN_CHAIN so
36
  it only serves when qwen + mistral + maverick are all unavailable.
37
- - Latency / success_rate still gate ELIGIBILITY (probe-fresh, not in
38
- sin-bin, credits above water) but no longer drive ORDERING. Pre-KI-201
39
- the (1/latency)×success_rate score let a chain-#3 model beat chain-#0
40
- on a faster probe, violating the operator-defined hierarchy.
41
  - DEGRADED window: when chat() calls report_failure(model), that model is
42
  sidelined for either DEGRADED_WINDOW_SEC (transient, 30s) or
43
- DEGRADE_DURATION_LONG_S (rate-limit / HTTP 429, 1h — KI-084) so the
44
- same turn's failure doesn't recycle to the same broken primary on the
45
- next turn. The next probe tick reconsiders the model normally for the
46
- short window; rate-limit demotions persist past several probe ticks
47
- so the elector doesn't keep bouncing back to a quota-exhausted model.
48
 
49
  Persistence: 40-data/llm_health.json (atomic write via temp+rename).
50
 
@@ -87,57 +79,49 @@ ROOT = Path(__file__).resolve().parent.parent
87
  HEALTH_FILE = settings.DATA_DIR / "llm_health.json"
88
  HEALTH_FILE.parent.mkdir(parents=True, exist_ok=True)
89
 
90
- # KI-084 (2026-05-15) — probe cadence raised 60s → 300s. With ~25
91
- # candidates per tick × 4 token round-trip each (prompt "Reply with
92
- # exactly: ok" plus a 1-token completion), the prior 60s cadence burned
93
- # ~30-50K tokens/day on Groq alone enough to push the brain to HTTP 429
94
- # on Groq's 100K free-tier TPD cap and elect a degraded provider for
95
- # every chat call. 300s cadence keeps election responsive within 5 min
96
- # of a pool degradation (more than fast enough at our chat volume) and
97
- # drops probe-driven Groq spend to ~3K/day baseline (well inside quota).
98
  PROBE_INTERVAL_SEC = 300
99
- # A4 (2026-05-15) — cadence audit: 300s for steady-state probes is comfortably
100
- # above the audit's 90s minimum for non-failing providers. The post-failure
101
- # tighter cadence is implemented OUT-OF-BAND in `report_failure()`, which
102
- # schedules a `_reprobe_one()` immediately after every chat failure (effective
103
- # 0s after-failure cadence on the loop, not 30s). The 300s cadence then
104
- # resumes for the long-run probe stream so the steady state stays cheap.
105
- PROBE_INTERVAL_SEC_FAILING = 30 # A4 documented post-failure cadence ceiling
106
- # KI-084 completion size on probes cut from 5 → 1. The probe only
107
- # needs a non-empty 200 response to mark a candidate healthy; we never
108
- # parse the body content. max_tokens=1 keeps the same response shape +
109
- # ~50× less token spend per probe.
110
  PROBE_MAX_TOKENS = 1
111
  PROBE_TIMEOUT_SEC = 8 # per-probe HTTP timeout
112
  DOWN_AFTER_CONSECUTIVE_FAILS = 3 # 3 fails in a row = mark down
113
  PROBE_HISTORY_LEN = 5 # rolling window for success_rate signal
114
- HEALTHY_PROBE_AGE_SEC = 600 # KI-084 — election candidates need a probe
115
- # within the last 600s (tracks 300s cadence
116
- # plus headroom for one missed tick).
117
- # A4 (2026-05-15) — explicit STALE window. If a candidate hasn't been probed
118
- # in >STALE_AGE_SEC, its on-record status is rewritten to "stale" so the
119
- # router treats it as untested rather than trusting the last-known
120
  # "healthy"/"unhealthy" verdict from minutes/hours ago.
121
  STALE_AGE_SEC = HEALTHY_PROBE_AGE_SEC # alias for clarity; same threshold.
122
  DEGRADED_WINDOW_SEC = 30 # report_failure sidelines a model this long
123
  # for transient failures (timeout / 5xx).
124
 
125
- # Three-chain collapse (2026-05-15) collapsed FAST_BRAIN_CHAIN + JUDGE_CHAIN
126
- # out of the architecture. "brain" is now the sole election role; every
127
- # consumer that used to iterate ("brain", "fast_brain", "judge") now iterates
128
- # ROLES so we only have one place to add a future role.
129
  ROLES: tuple[str, ...] = ("brain",)
130
- # KI-084 — rate-limit failures (HTTP 429 + provider 'RateLimit' bodies) are
131
  # almost always the daily quota on free tiers (Groq TPD, etc.) — they do
132
  # NOT reset in 30s. Demote the model from election for an hour so the
133
  # elector falls through to a non-rate-limited provider instead of
134
  # bouncing back to the dead candidate on every chat turn.
135
  DEGRADE_DURATION_LONG_S = 3600.0
136
 
137
- # KI-085 (2026-05-15) — proactive credit tracking. KI-084 demotes a candidate
138
- # for 1h AFTER a 429 hits; that costs one user-facing failure per dead quota.
139
- # KI-085 promotes llm_health to liveness+credits so election excludes
140
- # quota-exhausted candidates BEFORE the user gets stuck behind a 429.
141
  #
142
  # Three signal sources:
143
  # 1) GROQ — response headers (x-ratelimit-remaining-tokens-day etc.) on
@@ -203,8 +187,8 @@ def _headers_for(model_id: str, api_key: str) -> dict[str, str]:
203
  return h
204
 
205
 
206
- # KI-202 (2026-05-15) — categorize a `last_error` string + optional HTTP
207
- # status code into the short operator-facing reason rendered by the admin
208
  # Health columns. Centralised here so backend and any other consumer agree
209
  # on the same vocabulary ("network issue" / "rate limit (429)" / etc.).
210
  # Use lookbehind/lookahead on non-digit so we still match e.g. "Status429",
@@ -309,8 +293,8 @@ def provider_of(model_id: str) -> str:
309
  @dataclass
310
  class ModelHealth:
311
  model: str
312
- # A4 (2026-05-15) — 'stale' added. Set by `effective_status()` when a
313
- # row hasn't been pinged in > STALE_AGE_SEC; the router then treats it
314
  # as untested instead of trusting a last-known healthy/unhealthy verdict
315
  # from minutes/hours ago. Persisted records may still carry the older
316
  # status — the elector calls effective_status() at decision time.
@@ -318,8 +302,8 @@ class ModelHealth:
318
  last_success_at: Optional[str] = None
319
  last_failure_at: Optional[str] = None
320
  last_error: Optional[str] = None
321
- # KI-202 (2026-05-15) — explicit HTTP status code from the most recent
322
- # failed probe / chat call. `last_error` is a free-form string ("timeout",
323
  # "http_429", "net: TimeoutException: ..."); `last_status_code` is the
324
  # parsed integer (or None when the failure wasn't HTTP-shaped). Surfaced
325
  # in admin Health columns so the operator can see "Off — rate limit (429)"
@@ -335,7 +319,7 @@ class ModelHealth:
335
  # KI-080 — set by report_failure(); model is excluded from election
336
  # while monotonic time < degraded_until_monotonic.
337
  degraded_until_monotonic: float = 0.0
338
- # KI-085 (2026-05-15) — proactive credit tracking. Stamped by
339
  # update_credits_from_groq / update_credits_from_openrouter (response
340
  # headers + account endpoint) and by the NIM local rate-meter. The
341
  # elector gates on `credits_remaining is None OR > credits_low_water`
@@ -364,8 +348,7 @@ def _iso_age_seconds(iso_ts: Optional[str]) -> Optional[float]:
364
 
365
 
366
  def _all_known_models() -> list[str]:
367
- """Pull every model name from BRAIN_CHAIN (the only remaining chain
368
- after the three-chain collapse on 2026-05-15)."""
369
  from backend.providers.nvidia_nim_llm import BRAIN_CHAIN
370
  seen: list[str] = []
371
  for m in BRAIN_CHAIN:
@@ -378,9 +361,8 @@ def _chain_for(chain_name: str) -> list[str]:
378
  """Resolve a chain name → live chain list. Reads off the module so
379
  runtime admin reorderings are respected by the elector.
380
 
381
- Three-chain collapse (2026-05-15): "brain" is the only valid role.
382
- Any other input is coerced to "brain" for back-compat with stragglers
383
- that still pass "fast_brain"/"judge" (CLEAN1 nim_fallback callers etc.).
384
  """
385
  from backend.providers import nvidia_nim_llm as nim
386
  return list(getattr(nim, "BRAIN_CHAIN", []))
@@ -408,9 +390,9 @@ def _load_into_memory() -> None:
408
  try:
409
  raw = json.loads(HEALTH_FILE.read_text())
410
  for k, v in raw.get("models", {}).items():
411
- # Tolerate older schema (pre-KI-080 records missing
412
- # probe_history / degraded_until_monotonic; pre-KI-085
413
- # records missing the five credits_* fields).
414
  v.setdefault("probe_history", [])
415
  v.setdefault("degraded_until_monotonic", 0.0)
416
  v.setdefault("last_status_code", None)
@@ -437,7 +419,7 @@ def save(state: Optional[dict[str, ModelHealth]] = None) -> None:
437
  """Atomic write — temp file then rename so concurrent readers never see partial.
438
 
439
  If `state` is None, persists the in-memory _STATE. The optional arg is
440
- kept for backward compatibility with the pre-KI-080 call sites in admin.py."""
441
  if state is None:
442
  _load_into_memory()
443
  with _STATE_LOCK:
@@ -452,7 +434,7 @@ def save(state: Optional[dict[str, ModelHealth]] = None) -> None:
452
 
453
 
454
  # ---------------------------------------------------------------------------
455
- # KI-080 — public API: primary/backup election + failure/success reporting
456
  # ---------------------------------------------------------------------------
457
 
458
  def _success_rate(h: ModelHealth) -> float:
@@ -478,8 +460,8 @@ def _score(h: ModelHealth) -> float:
478
 
479
 
480
  def effective_status(h: ModelHealth) -> str:
481
- """A4 (2026-05-15) — Return the routing-relevant status, applying the
482
- STALE_AGE_SEC override at read time.
483
 
484
  A stored `status` of "healthy" can mean "the last probe N hours ago
485
  said this was healthy" — which the router must NOT trust. When
@@ -501,11 +483,10 @@ def _is_election_eligible(h: ModelHealth, now_mono: float) -> bool:
501
  - status is healthy (or degraded with a recent success)
502
  - last probe was within HEALTHY_PROBE_AGE_SEC
503
  - it is NOT currently in the degraded-window sin-bin
504
- - KI-085 (2026-05-15): it has credits remaining above its low-water
505
- mark, OR no credit signal yet (cold-start = permissive).
506
- - A4 (2026-05-15): effective_status != "stale" (catches the case
507
- where status field is 'healthy' but the probe is older than
508
- STALE_AGE_SEC).
509
  """
510
  if h.degraded_until_monotonic > now_mono:
511
  return False
@@ -528,7 +509,7 @@ def _is_election_eligible(h: ModelHealth, now_mono: float) -> bool:
528
 
529
 
530
  def _has_credits(h: ModelHealth, now_mono: float) -> bool:
531
- """KI-085 — credit-gate predicate for election eligibility.
532
 
533
  Rules:
534
  - If `credits_reset_at` has elapsed, treat the signal as stale and
@@ -565,7 +546,7 @@ def _ranked_candidates(chain_name: str) -> list[ModelHealth]:
565
  return [h for _, h in eligible]
566
 
567
 
568
- # A4 (2026-05-15) — Election event log. Tracks the last elected primary/
569
  # backup per chain so we can emit a structured promotion/demotion log line
570
  # when the elected model changes. Stored in-memory only (cheap; resets on
571
  # process restart, which is fine — first post-restart election re-emits).
@@ -575,8 +556,8 @@ _LAST_ELECTION: dict[str, dict[str, Optional[str]]] = {}
575
 
576
  def _emit_election_event(chain_name: str, role: str, from_m: Optional[str],
577
  to_m: Optional[str], reason: str) -> None:
578
- """A4 (2026-05-15) — Structured log line for every primary/backup
579
- promotion or demotion. Format matches the audit brief:
580
  {event, chain, role, from, to, reason, ts}
581
  The router / admin UI / log-shipper can grep on `event=election` to
582
  rebuild the timeline of who served what when.
@@ -617,28 +598,22 @@ def _record_election(chain_name: str, role: str, new_model: Optional[str],
617
 
618
 
619
  def get_primary(chain_name: str) -> Optional[str]:
620
- """KI-201 (2026-05-15) — elect by CHAIN ORDER, not latency score.
621
 
622
  Walk the chain definition in priority order. Return the first model
623
  that's election-eligible. Chain hierarchy IS the truth — e.g. in
624
  BRAIN_CHAIN, nemotron-49b is LAST so it only serves when every
625
- higher-priority model is unavailable (KI-175 last-resort rule).
626
-
627
- Pre-KI-201 the elector picked by (1/latency) × success_rate, which
628
- meant a chain-#3 model (maverick) could beat chain-#0 (qwen) on a
629
- faster probe. Live evidence: all 4 NIM models healthy but elector
630
- picked maverick over qwen. User-stated spec: "If a model is not
631
- already in use, but at the backend everything is live, then it
632
- should always suggest it according to our set hierarchy."
633
-
634
- Latency is no longer used for primary/backup selection — eligibility
635
- filtering still uses the rolling probe state, but ordering is
636
- purely chain-positional.
637
-
638
- A4 (2026-05-15) — emits a structured `event=election` log line via
639
- `_record_election` whenever the elected primary changes for this
640
- chain, so the operator/admin/log-shipper can rebuild the promotion/
641
- demotion timeline. No log emit when the value is unchanged.
642
  """
643
  chain = _chain_for(chain_name)
644
  if not chain:
@@ -657,19 +632,18 @@ def get_primary(chain_name: str) -> Optional[str]:
657
 
658
 
659
  def get_backup(chain_name: str) -> Optional[str]:
660
- """KI-201 (2026-05-15) — backup is the SECOND eligible model in chain
661
- order, skipping the elected primary.
662
 
663
  Walks the chain definition in priority order, skips the elected
664
  primary, and returns the next eligible model. Same chain-as-truth
665
- philosophy as get_primary: the chains in nvidia_nim_llm.py were
666
- designed with family/provider diversity baked in (Qwen → Mistral →
667
- Meta → NVIDIA), so walking past the primary in chain order naturally
668
- preserves the cross-family backup invariant KI-087 used to enforce
669
- via provider_of().
670
-
671
- A4 (2026-05-15) emits a structured `event=election` log line via
672
- `_record_election` whenever the elected backup changes.
673
  """
674
  chain = _chain_for(chain_name)
675
  if not chain:
@@ -875,7 +849,7 @@ def update_credits_from_groq(chain_name: str, model: str, headers: dict) -> None
875
  Called from GroqLLM.chat() after a successful HTTP response. The
876
  `headers` dict is the response's headers (case-insensitive via httpx).
877
  We prefer the DAILY tokens signal (`x-ratelimit-remaining-tokens-day`)
878
- because Groq's free-tier daily TPD cap is what bit us in KI-084.
879
 
880
  Missing header → no-op. Malformed value → log warning + no-op.
881
  """
@@ -1097,8 +1071,8 @@ async def poll_openrouter_credits() -> Optional[dict]:
1097
 
1098
 
1099
  # ---------------------------------------------------------------------------
1100
- # Probing (mostly unchanged from pre-KI-080 extended to record
1101
- # probe_history + skip degraded-window models on the regular tick).
1102
  # ---------------------------------------------------------------------------
1103
 
1104
  async def probe_one(client: httpx.AsyncClient, model: str, api_key: str) -> tuple[bool, str, Optional[int]]:
@@ -1198,29 +1172,19 @@ async def probe_all() -> dict[str, ModelHealth]:
1198
  """One-shot probe of every known model. Updates persisted state + returns it."""
1199
  _load_into_memory()
1200
  models = _all_known_models()
1201
- # Per-provider keys are resolved inside probe_one(); we no longer
1202
- # gate the whole loop on the NIM key (KI-080) cross-provider
1203
- # candidates must keep getting probed even when NVIDIA_NIM_API_KEY
1204
- # is missing.
1205
-
1206
- # KI-088 (2026-05-15) Probe candidates serially, not in parallel.
1207
- #
1208
- # Pre-KI-088: `asyncio.gather(*probe_one(m) for m in models)` fired
1209
- # all candidates simultaneously. Even though they hit different NIM
1210
- # *model pools*, they all share the same per-API-key concurrency
1211
- # quota (~3-5 slots free-tier). A 6-candidate parallel burst every
1212
- # 300s would queue inside NIM and steal slots from in-flight user
1213
- # turns — exactly the saturation pattern the global outbound
1214
- # semaphore in nvidia_nim_llm.py is sized to prevent.
1215
- #
1216
- # With KI-088's semaphore in place, parallel gather would still
1217
- # be hard-capped at 2 in-flight but introduce no benefit over a
1218
- # serial loop. Serial is simpler, easier to reason about, and
1219
- # naturally yields control to user-traffic between candidates.
1220
  #
1221
- # Cost: 6 NIM candidates × ~2s healthy probe = ~12s, well under
1222
- # the 300s probe cadence. No sleep between candidates — the
1223
- # outbound semaphore handles pacing.
1224
  async with httpx.AsyncClient() as client:
1225
  results = []
1226
  for m in models:
@@ -1260,8 +1224,8 @@ def status_summary() -> dict:
1260
  state = load()
1261
  summary = {"updated_at": None, "by_status": {"healthy": 0, "degraded": 0, "down": 0, "stale": 0, "unknown": 0}, "models": []}
1262
  for m, h in state.items():
1263
- # A4 (2026-05-15) — `effective_status` applies the STALE_AGE_SEC
1264
- # override at read time so the admin UI / router see "stale" for
1265
  # rows whose stored 'healthy' verdict is older than STALE_AGE_SEC.
1266
  eff = effective_status(h)
1267
  summary["by_status"][eff] = summary["by_status"].get(eff, 0) + 1
@@ -1294,9 +1258,9 @@ def status_summary() -> dict:
1294
  summary["models"].sort(key=lambda x: (x["status"] != "healthy", x["model"]))
1295
  if summary["models"]:
1296
  summary["updated_at"] = max((m.get("tested_at") or "") for m in summary["models"])
1297
- # KI-080 — surface currently-elected primary/backup per chain for the
1298
- # admin UI. Cheap (a couple of dict lookups + sort over <=10 entries).
1299
- # Three-chain collapse (2026-05-15) — "brain" is the only remaining role.
1300
  summary["elections"] = {
1301
  role: {"primary": get_primary(role), "backup": get_backup(role)}
1302
  for role in ROLES
@@ -1305,14 +1269,14 @@ def status_summary() -> dict:
1305
 
1306
 
1307
  async def background_probe_loop() -> None:
1308
- """Long-running task — probes every PROBE_INTERVAL_SEC (300s; KI-084).
1309
  Started from main.py.
1310
 
1311
- KI-085 (2026-05-15) — also polls OpenRouter's account-level credits
1312
- endpoint every OPENROUTER_CREDITS_POLL_EVERY_N_TICKS ticks (10 min by
1313
- default at the 300s probe cadence). Groq + NIM signals come from the
1314
- chat hot path (response headers + local rate-meter respectively) so
1315
- only OpenRouter needs an out-of-band poll.
1316
  """
1317
  tick = 0
1318
  # Initial credits poll on startup so the elector has a non-None
 
1
+ """Live NIM model health monitor — availability filter + sticky
2
+ primary/backup election.
3
+
4
+ The background probe loop ELECTS a primary + backup per chain; chat()
5
+ calls the elected primary ONCE per turn, with at most ONE real-time
6
+ fallback to the elected backup (worst case 1-2 LLM calls per turn).
7
+ Electing out-of-band, rather than iterating the whole chain on every
8
+ turn, keeps a single turn from queuing many NIM-hosted candidates under
9
+ per-key concurrency throttling and burning the turn budget.
 
 
 
 
 
 
 
10
 
11
  Probes every PROBE_INTERVAL_SEC tick (300s) with a tiny ping ("Reply with
12
  exactly: ok"), `max_tokens=1` so probe-driven token spend is negligible.
 
20
  - probe_history: last PROBE_HISTORY_LEN (ok, latency_ms) tuples; powers
21
  the success_rate signal in the election score.
22
 
23
+ Election:
24
+ - Walk the chain definition (BRAIN_CHAIN — the only chain) in priority
25
+ order.
26
  CURRENT_PRIMARY = first election-eligible model.
27
  CURRENT_BACKUP = next election-eligible model after primary.
28
  - Chain hierarchy IS the truth — nemotron-49b is LAST in BRAIN_CHAIN so
29
  it only serves when qwen + mistral + maverick are all unavailable.
30
+ - Latency / success_rate gate ELIGIBILITY (probe-fresh, not in
31
+ sin-bin, credits above water) but do not drive ORDERING, so a faster
32
+ lower-priority model never beats a higher-priority one.
 
33
  - DEGRADED window: when chat() calls report_failure(model), that model is
34
  sidelined for either DEGRADED_WINDOW_SEC (transient, 30s) or
35
+ DEGRADE_DURATION_LONG_S (rate-limit / HTTP 429, 1h) so the same turn's
36
+ failure doesn't recycle to the same broken primary on the next turn.
37
+ The next probe tick reconsiders the model normally for the short
38
+ window; rate-limit demotions persist past several probe ticks so the
39
+ elector doesn't keep bouncing back to a quota-exhausted model.
40
 
41
  Persistence: 40-data/llm_health.json (atomic write via temp+rename).
42
 
 
79
  HEALTH_FILE = settings.DATA_DIR / "llm_health.json"
80
  HEALTH_FILE.parent.mkdir(parents=True, exist_ok=True)
81
 
82
+ # Probe cadence 300s. With ~25 candidates per tick × a 4-token round-trip
83
+ # each, a tighter cadence would burn enough tokens/day on Groq to push the
84
+ # brain to HTTP 429 on Groq's free-tier TPD cap. 300s keeps election
85
+ # responsive within 5 min of a pool degradation (fast enough at our chat
86
+ # volume) and keeps probe-driven Groq spend well inside quota.
 
 
 
87
  PROBE_INTERVAL_SEC = 300
88
+ # 300s for steady-state probes. The post-failure tighter cadence is
89
+ # implemented OUT-OF-BAND in `report_failure()`, which schedules a
90
+ # `_reprobe_one()` immediately after every chat failure (effective 0s
91
+ # after-failure cadence on the loop). The 300s cadence then resumes for
92
+ # the long-run probe stream so the steady state stays cheap.
93
+ PROBE_INTERVAL_SEC_FAILING = 30 # post-failure cadence ceiling
94
+ # The probe only needs a non-empty 200 response to mark a candidate
95
+ # healthy; the body content is never parsed. max_tokens=1 keeps the same
96
+ # response shape with minimal token spend per probe.
 
 
97
  PROBE_MAX_TOKENS = 1
98
  PROBE_TIMEOUT_SEC = 8 # per-probe HTTP timeout
99
  DOWN_AFTER_CONSECUTIVE_FAILS = 3 # 3 fails in a row = mark down
100
  PROBE_HISTORY_LEN = 5 # rolling window for success_rate signal
101
+ HEALTHY_PROBE_AGE_SEC = 600 # election candidates need a probe within
102
+ # the last 600s (tracks 300s cadence plus
103
+ # headroom for one missed tick).
104
+ # Explicit STALE window. If a candidate hasn't been probed in
105
+ # >STALE_AGE_SEC, its on-record status is rewritten to "stale" so the
106
+ # router treats it as untested rather than trusting a last-known
107
  # "healthy"/"unhealthy" verdict from minutes/hours ago.
108
  STALE_AGE_SEC = HEALTHY_PROBE_AGE_SEC # alias for clarity; same threshold.
109
  DEGRADED_WINDOW_SEC = 30 # report_failure sidelines a model this long
110
  # for transient failures (timeout / 5xx).
111
 
112
+ # "brain" is the sole election role; consumers iterate ROLES so a future
113
+ # role only needs to be added in one place.
 
 
114
  ROLES: tuple[str, ...] = ("brain",)
115
+ # Rate-limit failures (HTTP 429 + provider 'RateLimit' bodies) are
116
  # almost always the daily quota on free tiers (Groq TPD, etc.) — they do
117
  # NOT reset in 30s. Demote the model from election for an hour so the
118
  # elector falls through to a non-rate-limited provider instead of
119
  # bouncing back to the dead candidate on every chat turn.
120
  DEGRADE_DURATION_LONG_S = 3600.0
121
 
122
+ # Proactive credit tracking: election excludes quota-exhausted candidates
123
+ # BEFORE the user gets stuck behind a 429 (rather than only demoting a
124
+ # candidate for 1h after a 429 hits).
 
125
  #
126
  # Three signal sources:
127
  # 1) GROQ — response headers (x-ratelimit-remaining-tokens-day etc.) on
 
187
  return h
188
 
189
 
190
+ # Categorize a `last_error` string + optional HTTP status code into the
191
+ # short operator-facing reason rendered by the admin
192
  # Health columns. Centralised here so backend and any other consumer agree
193
  # on the same vocabulary ("network issue" / "rate limit (429)" / etc.).
194
  # Use lookbehind/lookahead on non-digit so we still match e.g. "Status429",
 
293
  @dataclass
294
  class ModelHealth:
295
  model: str
296
+ # 'stale' is set by `effective_status()` when a row hasn't been pinged
297
+ # in > STALE_AGE_SEC; the router then treats it
298
  # as untested instead of trusting a last-known healthy/unhealthy verdict
299
  # from minutes/hours ago. Persisted records may still carry the older
300
  # status — the elector calls effective_status() at decision time.
 
302
  last_success_at: Optional[str] = None
303
  last_failure_at: Optional[str] = None
304
  last_error: Optional[str] = None
305
+ # Explicit HTTP status code from the most recent failed probe / chat
306
+ # call. `last_error` is a free-form string ("timeout",
307
  # "http_429", "net: TimeoutException: ..."); `last_status_code` is the
308
  # parsed integer (or None when the failure wasn't HTTP-shaped). Surfaced
309
  # in admin Health columns so the operator can see "Off — rate limit (429)"
 
319
  # KI-080 — set by report_failure(); model is excluded from election
320
  # while monotonic time < degraded_until_monotonic.
321
  degraded_until_monotonic: float = 0.0
322
+ # Proactive credit tracking. Stamped by
323
  # update_credits_from_groq / update_credits_from_openrouter (response
324
  # headers + account endpoint) and by the NIM local rate-meter. The
325
  # elector gates on `credits_remaining is None OR > credits_low_water`
 
348
 
349
 
350
  def _all_known_models() -> list[str]:
351
+ """Pull every model name from BRAIN_CHAIN (the only chain)."""
 
352
  from backend.providers.nvidia_nim_llm import BRAIN_CHAIN
353
  seen: list[str] = []
354
  for m in BRAIN_CHAIN:
 
361
  """Resolve a chain name → live chain list. Reads off the module so
362
  runtime admin reorderings are respected by the elector.
363
 
364
+ "brain" is the only valid role. Any other input is coerced to "brain"
365
+ for back-compat with callers that pass another role name.
 
366
  """
367
  from backend.providers import nvidia_nim_llm as nim
368
  return list(getattr(nim, "BRAIN_CHAIN", []))
 
390
  try:
391
  raw = json.loads(HEALTH_FILE.read_text())
392
  for k, v in raw.get("models", {}).items():
393
+ # Tolerate older on-disk schema: records may be missing
394
+ # probe_history / degraded_until_monotonic or the five
395
+ # credits_* fields.
396
  v.setdefault("probe_history", [])
397
  v.setdefault("degraded_until_monotonic", 0.0)
398
  v.setdefault("last_status_code", None)
 
419
  """Atomic write — temp file then rename so concurrent readers never see partial.
420
 
421
  If `state` is None, persists the in-memory _STATE. The optional arg is
422
+ kept for backward compatibility with call sites in admin.py."""
423
  if state is None:
424
  _load_into_memory()
425
  with _STATE_LOCK:
 
434
 
435
 
436
  # ---------------------------------------------------------------------------
437
+ # Public API: primary/backup election + failure/success reporting
438
  # ---------------------------------------------------------------------------
439
 
440
  def _success_rate(h: ModelHealth) -> float:
 
460
 
461
 
462
  def effective_status(h: ModelHealth) -> str:
463
+ """Return the routing-relevant status, applying the STALE_AGE_SEC
464
+ override at read time.
465
 
466
  A stored `status` of "healthy" can mean "the last probe N hours ago
467
  said this was healthy" — which the router must NOT trust. When
 
483
  - status is healthy (or degraded with a recent success)
484
  - last probe was within HEALTHY_PROBE_AGE_SEC
485
  - it is NOT currently in the degraded-window sin-bin
486
+ - it has credits remaining above its low-water mark, OR no credit
487
+ signal yet (cold-start = permissive).
488
+ - effective_status != "stale" (catches the case where the status
489
+ field is 'healthy' but the probe is older than STALE_AGE_SEC).
 
490
  """
491
  if h.degraded_until_monotonic > now_mono:
492
  return False
 
509
 
510
 
511
  def _has_credits(h: ModelHealth, now_mono: float) -> bool:
512
+ """Credit-gate predicate for election eligibility.
513
 
514
  Rules:
515
  - If `credits_reset_at` has elapsed, treat the signal as stale and
 
546
  return [h for _, h in eligible]
547
 
548
 
549
+ # Election event log. Tracks the last elected primary/
550
  # backup per chain so we can emit a structured promotion/demotion log line
551
  # when the elected model changes. Stored in-memory only (cheap; resets on
552
  # process restart, which is fine — first post-restart election re-emits).
 
556
 
557
  def _emit_election_event(chain_name: str, role: str, from_m: Optional[str],
558
  to_m: Optional[str], reason: str) -> None:
559
+ """Structured log line for every primary/backup promotion or
560
+ demotion. Format:
561
  {event, chain, role, from, to, reason, ts}
562
  The router / admin UI / log-shipper can grep on `event=election` to
563
  rebuild the timeline of who served what when.
 
598
 
599
 
600
  def get_primary(chain_name: str) -> Optional[str]:
601
+ """Elect by CHAIN ORDER, not latency score.
602
 
603
  Walk the chain definition in priority order. Return the first model
604
  that's election-eligible. Chain hierarchy IS the truth — e.g. in
605
  BRAIN_CHAIN, nemotron-49b is LAST so it only serves when every
606
+ higher-priority model is unavailable (last-resort rule).
607
+
608
+ Latency is not used for primary/backup selection eligibility
609
+ filtering uses the rolling probe state, but ordering is purely
610
+ chain-positional, so the hierarchy is always honoured when models
611
+ are live.
612
+
613
+ Emits a structured `event=election` log line via `_record_election`
614
+ whenever the elected primary changes for this chain, so the
615
+ operator/admin/log-shipper can rebuild the promotion/demotion
616
+ timeline. No log emit when the value is unchanged.
 
 
 
 
 
 
617
  """
618
  chain = _chain_for(chain_name)
619
  if not chain:
 
632
 
633
 
634
  def get_backup(chain_name: str) -> Optional[str]:
635
+ """Backup is the SECOND eligible model in chain order, skipping the
636
+ elected primary.
637
 
638
  Walks the chain definition in priority order, skips the elected
639
  primary, and returns the next eligible model. Same chain-as-truth
640
+ philosophy as get_primary: the chains in nvidia_nim_llm.py are
641
+ designed with family/provider diversity (Qwen → Mistral → Meta →
642
+ NVIDIA), so walking past the primary in chain order naturally keeps
643
+ the backup on a different family from the primary.
644
+
645
+ Emits a structured `event=election` log line via `_record_election`
646
+ whenever the elected backup changes.
 
647
  """
648
  chain = _chain_for(chain_name)
649
  if not chain:
 
849
  Called from GroqLLM.chat() after a successful HTTP response. The
850
  `headers` dict is the response's headers (case-insensitive via httpx).
851
  We prefer the DAILY tokens signal (`x-ratelimit-remaining-tokens-day`)
852
+ because Groq's free-tier daily TPD cap is the binding limit.
853
 
854
  Missing header → no-op. Malformed value → log warning + no-op.
855
  """
 
1071
 
1072
 
1073
  # ---------------------------------------------------------------------------
1074
+ # Probing records probe_history + skips degraded-window models on the
1075
+ # regular tick.
1076
  # ---------------------------------------------------------------------------
1077
 
1078
  async def probe_one(client: httpx.AsyncClient, model: str, api_key: str) -> tuple[bool, str, Optional[int]]:
 
1172
  """One-shot probe of every known model. Updates persisted state + returns it."""
1173
  _load_into_memory()
1174
  models = _all_known_models()
1175
+ # Per-provider keys are resolved inside probe_one(); the loop is not
1176
+ # gated on the NIM key so cross-provider candidates keep getting
1177
+ # probed even when NVIDIA_NIM_API_KEY is missing.
1178
+
1179
+ # Probe candidates serially, not in parallel. Candidates share the
1180
+ # same per-API-key concurrency quota (~3-5 slots free-tier), so a
1181
+ # parallel burst every 300s would queue inside NIM and steal slots
1182
+ # from in-flight user turns. A serial loop is simple to reason about
1183
+ # and naturally yields control to user traffic between candidates.
 
 
 
 
 
 
 
 
 
 
1184
  #
1185
+ # Cost: ~6 NIM candidates × ~2s healthy probe = ~12s, well under the
1186
+ # 300s probe cadence. No sleep between candidates — the outbound
1187
+ # semaphore handles pacing.
1188
  async with httpx.AsyncClient() as client:
1189
  results = []
1190
  for m in models:
 
1224
  state = load()
1225
  summary = {"updated_at": None, "by_status": {"healthy": 0, "degraded": 0, "down": 0, "stale": 0, "unknown": 0}, "models": []}
1226
  for m, h in state.items():
1227
+ # `effective_status` applies the STALE_AGE_SEC override at read
1228
+ # time so the admin UI / router see "stale" for
1229
  # rows whose stored 'healthy' verdict is older than STALE_AGE_SEC.
1230
  eff = effective_status(h)
1231
  summary["by_status"][eff] = summary["by_status"].get(eff, 0) + 1
 
1258
  summary["models"].sort(key=lambda x: (x["status"] != "healthy", x["model"]))
1259
  if summary["models"]:
1260
  summary["updated_at"] = max((m.get("tested_at") or "") for m in summary["models"])
1261
+ # Surface currently-elected primary/backup per chain for the admin UI.
1262
+ # Cheap (a couple of dict lookups + sort over <=10 entries). "brain"
1263
+ # is the only role.
1264
  summary["elections"] = {
1265
  role: {"primary": get_primary(role), "backup": get_backup(role)}
1266
  for role in ROLES
 
1269
 
1270
 
1271
  async def background_probe_loop() -> None:
1272
+ """Long-running task — probes every PROBE_INTERVAL_SEC (300s).
1273
  Started from main.py.
1274
 
1275
+ Also polls OpenRouter's account-level credits endpoint every
1276
+ OPENROUTER_CREDITS_POLL_EVERY_N_TICKS ticks (10 min by default at the
1277
+ 300s probe cadence). Groq + NIM signals come from the chat hot path
1278
+ (response headers + local rate-meter respectively) so only OpenRouter
1279
+ needs an out-of-band poll.
1280
  """
1281
  tick = 0
1282
  # Initial credits poll on startup so the elector has a non-None
backend/main.py CHANGED
@@ -32,21 +32,22 @@ from backend import sum_insured as _si # SI rationalisation (D1/D3) — source-
32
  from backend.providers.sarvam_stt import SarvamSTT
33
  from backend.providers.sarvam_tts import SarvamTTS
34
 
35
- # Path B — opt-in single-LLM brain. Off by default; flip via env var.
36
- # When on we replace orchestrator.handle_turn with single_brain.handle_turn
37
- # for the /api/chat hot path and fall back to nim_fallback.handle_turn_fallback
38
- # on any SingleBrainError (so users always get a reply).
 
39
  import os as _os # local alias to avoid stomping any later `import os`
40
 
41
  USE_SINGLE_BRAIN = _os.environ.get("USE_SINGLE_BRAIN", "false").lower() in (
42
  "1", "true", "yes", "on",
43
  )
44
 
45
- # U1 Test 9 — safety net for RULE 7. If Gemini forgets to call
46
- # mark_recommendation when the user clearly commits to a policy ("I'll go
47
- # with that one", "let's do #2", "buy this"), the post-turn detector below
48
- # auto-calls mark_recommendation against session.last_recommendation_ids[:1]
49
- # so the closure event is recorded for analytics even when the LLM whiffs.
50
  # Word-boundary anchored; case-insensitive at match-time.
51
  _CLOSER_KEYWORD_RE = re.compile(
52
  r"\b(go with|i'?ll take|i will take|let'?s do|let me get|sign me up|"
@@ -144,15 +145,10 @@ class ChatResponse(BaseModel):
144
  session_id: str
145
  audio_base64: Optional[str] = None
146
  audio_mime: Optional[str] = None # X8 — "audio/wav" | "audio/mp4" | "audio/webm"
147
- # KI-278 (2026-05-16) — TTS voice-output failures used to be SWALLOWED
148
- # silently: the chat endpoint caught the Sarvam exception, logged it to
149
- # turns.jsonl, and returned audio_base64=None with NO signal to the
150
- # client. The user saw a text reply with no voice and no explanation
151
- # ("no voice in reply. wtf?"). These two fields make the failure LOUD:
152
- # the frontend renders a small inline "voice unavailable" notice under
153
- # the bot bubble (text reply is unaffected). tts_error_code is a closed
154
- # enum mirroring the STT path's contract so the client never parses raw
155
- # httpx text:
156
  # rate_limit — Sarvam 429 / insufficient_quota / no credits
157
  # service_unavailable — Sarvam 5xx / 503
158
  # auth — Sarvam 401/403 / missing SARVAM_API_KEY
@@ -171,13 +167,12 @@ class ChatResponse(BaseModel):
171
  "flash an acknowledgment + refresh the completeness panel."
172
  ),
173
  )
174
- # Issue B (KI-Z6-PROFILECOMPLETE, 2026-05-15) frontend was making a
175
- # second roundtrip to /api/profile/completeness after every chat turn to
176
- # learn whether the 7 required slots are now captured. Surface it in the
177
  # primary chat response so the UI can flip to 100% in the same render
178
- # cycle. Computed via brain_tools._REQUIRED_FOR_READY (same slot list
179
- # used by retrieve_policies' profile-complete gate) so client + server
180
- # never disagree.
 
181
  profile_complete: bool = Field(
182
  False,
183
  description=(
@@ -343,13 +338,12 @@ async def _quarantine_purge_loop() -> None:
343
  )
344
 
345
 
346
- # Issue B (KI-Z6-PROFILECOMPLETE, 2026-05-15) single source of truth for
347
- # "is this profile ready to recommend against". brain_tools._profile_complete
348
- # uses the same _REQUIRED_FOR_READY tuple; we mirror the call here instead of
349
- # duplicating the slot list so a future addition (e.g. risk_appetite) only
350
- # needs to be applied in brain_tools.
351
- # KI-Z7 (2026-05-15) Feature B helper. Distinguishes "first-time capture
352
- # on turn 1" from "stored profile recalled on turn 1". Both end with a
353
  # non-empty Profile on the session; only the latter should flip
354
  # returning_user_recalled. Heuristic: if EVERY currently-filled slot was
355
  # written THIS turn (i.e. lives in `profile_updates`), the user just typed
@@ -461,11 +455,10 @@ async def _validation_exception_handler(request: Request, exc: RequestValidation
461
 
462
 
463
  # ---------- Admin panel + LLM health background loop ----------
464
- # Mount the password-gated admin endpoints (KI-097). Unauthorized callers
465
- # get 401 Unauthorized. The earlier IP allowlist gate (ADMIN_IP_ALLOWLIST +
466
- # 404-to-hide-existence) was removed in KI-097 operationally it locked
467
- # the operator out when switching networks without adding real security
468
- # beyond a strong password.
469
  from backend import admin as _admin_router_module
470
  app.include_router(_admin_router_module.router)
471
 
@@ -797,24 +790,21 @@ async def chat(req: ChatRequest, request: Request):
797
  if preferred_codec not in _allowed_codecs:
798
  preferred_codec = "audio/wav"
799
  t_chat0 = time.time()
800
- # KI-Z7 (2026-05-15) — pre-turn snapshot for the returning-user-recall
801
- # detector below. Defaults match the "no session yet" case so the
802
- # detector simply doesn't fire on the legacy orchestrator path.
803
  _pre_turn_name: str = ""
804
  _pre_turn_idx: int = 0
805
- # KI-106 — never let an inner TimeoutError / unhandled exception bubble out
806
- # of handle_turn as a 500. C4 NRI persona saw HTTP 500s with
807
- # "Orchestrator failed: TimeoutError" because the outer non-fact-find
808
- # brain call propagated asyncio.TimeoutError from KI-099/100 wait_for
809
- # wrappers. We also wrap the whole call in an outer 45s budget so even a
810
- # pathological hang inside handle_turn surfaces as a graceful reply,
811
- # not a connection-reset to the user. 45s is generous but tighter than
812
- # HF Space's gateway timeout, so the user always gets a response.
813
  try:
814
  if USE_SINGLE_BRAIN:
815
- # Path B — one Gemini call per turn with native function-calling.
816
- # Falls back to the legacy orchestrator on SingleBrainError so a
817
- # missing GOOGLE_API_KEY / model outage never breaks the chat.
818
  from backend import single_brain
819
  from backend.session_state import get_session
820
 
@@ -826,13 +816,11 @@ async def chat(req: ChatRequest, request: Request):
826
  getattr(_sb_session.profile, "name", None) or ""
827
  ).strip()
828
  _pre_turn_idx = int(getattr(_sb_session, "turn_idx", 0) or 0)
829
- # Z2 fix Issue 3 (brain election bouncing). Once a session
830
- # has had ANY successful single_brain turn, it must stay on
831
- # single_brain for the rest of its lifetime. Falling back to
832
- # the legacy orchestrator mid-stream (which is what Priya saw
833
- # 6 turns in a row) discards everything single_brain captured
834
- # in last_recommendation_ids / last_retrieved_chunks /
835
- # slug_to_insurer and confuses the user. Sticky check below.
836
  _sb_was_sticky = getattr(_sb_session, "single_brain_sticky", False)
837
  try:
838
  turn = await asyncio.wait_for(
@@ -851,14 +839,13 @@ async def chat(req: ChatRequest, request: Request):
851
  pass
852
  except single_brain.SingleBrainError as _sb_err:
853
  if _sb_was_sticky:
854
- # Z2 belt+suspenders — session already had a clean
855
- # single_brain turn. Do NOT cross-fade to the legacy
856
- # orchestrator (loses turn state + frontend sees the
857
- # brain hop). Emit a graceful retry prompt instead.
858
  logging.warning(
859
  "single_brain failed on STICKY session (session=%s); "
860
- "emitting graceful retry, NOT falling back to "
861
- "orchestrator: %s",
862
  session_id, _sb_err,
863
  )
864
  turn = single_brain.TurnResult(
@@ -893,9 +880,8 @@ async def chat(req: ChatRequest, request: Request):
893
  timeout=20.0,
894
  )
895
  else:
896
- # Legacy orchestrator path is gone (CLEAN2 deletion). When
897
- # USE_SINGLE_BRAIN is off, route directly through the minimal
898
- # NIM fallback so the bot still serves a reply.
899
  from backend.session_state import get_session as _get_session
900
  turn = await asyncio.wait_for(
901
  nim_fallback.handle_turn_fallback(
@@ -962,7 +948,7 @@ async def chat(req: ChatRequest, request: Request):
962
  profile_complete=_compute_profile_complete(session_id),
963
  )
964
 
965
- # U1 Test 9 — server-side closer-keyword safety net for RULE 7.
966
  # If the user clearly committed to a policy this turn but Gemini did
967
  # NOT call mark_recommendation (single_brain stamps "mark_recommendation"
968
  # into turn.brain_used when the tool fires — see single_brain.py:1052),
@@ -1172,11 +1158,11 @@ async def chat(req: ChatRequest, request: Request):
1172
 
1173
  audio_b64 = None
1174
  audio_mime: Optional[str] = None
1175
- # KI-278 (2026-05-16) — when TTS fails we now propagate a STRUCTURED,
1176
- # user-facing notice instead of silently dropping the audio. The text
1177
- # reply is still returned in full; the frontend renders a small inline
1178
- # "voice unavailable" line under the bot bubble so the user understands
1179
- # WHY there's no voice (root cause this fixed: Sarvam 429 / no credits).
1180
  tts_error_code: Optional[str] = None
1181
  tts_user_message: Optional[str] = None
1182
  if req.return_audio and turn.reply_text:
@@ -1932,9 +1918,9 @@ class ProfileUpdateRequest(BaseModel):
1932
  parents_has_ped: Optional[bool] = None
1933
  health_conditions: Optional[list[str]] = None
1934
  budget_band: Optional[str] = None
1935
- # KI profile-builder UI now collects these 4; they exist on the Profile
1936
- # dataclass + chat-path save_profile_field but were being silently dropped
1937
- # from POST /api/profile because they weren't whitelisted here.
1938
  desired_sum_insured_inr: Optional[int] = None
1939
  copay_pct: Optional[int] = None
1940
  family_medical_history: Optional[list[str]] = None
@@ -1971,8 +1957,8 @@ async def session_clear(req: SessionClearRequest):
1971
  The on-disk profile JSON under `40-data/profiles/` is intentionally NOT
1972
  touched — it remains durable user data keyed by persona_id / name slug.
1973
  The next time the user volunteers their name in conversation, the
1974
- confirmation-gated recall flow (see orchestrator's `pending_profile_recall`)
1975
- will ask before merging the prior captures.
1976
 
1977
  Body : {session_id: str}
1978
  Reply: {cleared: bool, new_session_id: str}
@@ -2136,9 +2122,9 @@ async def profile_completeness_view(session_id: Optional[str] = None):
2136
  missing = [k for k, v in profile_dict.items() if k not in answered or v in (None, "", [])]
2137
  hint = None
2138
  try:
2139
- # KI-167 WS3 (2026-05-15) next_question now returns the field name
2140
- # (str) of the next missing slot, not a Question object. Frontend uses
2141
- # this as a slot-hint; the actual phrasing is generated by sales_brain.
2142
  hint = next_question(p)
2143
  except Exception:
2144
  pass
@@ -2844,12 +2830,12 @@ def _policy_corroborated_si(policy_id: str | None) -> "_si.SumInsuredView":
2844
  return _rationalise_si(data, si)
2845
 
2846
 
2847
- # KI-145 (2026-05-15) — decision-critical fields used to distinguish a
2848
- # RENAME (curated entry folds onto extracted parent) from a VARIANT (same
2849
- # UIN but materially different product — must stay as its own card).
2850
- # Same UIN ≠ same product: regulators file one "wordings" PDF that covers
2851
- # multiple marketed variants (e.g. ProHealth Prime vs ProHealth Protect
2852
- # both filed under MCIHLIP24011V072324; copay/PED/maternity/NCB differ).
2853
  _KI145_DIFF_FIELDS: tuple[str, ...] = (
2854
  "copayment_pct",
2855
  "pre_existing_disease_waiting_months",
@@ -3229,14 +3215,14 @@ async def policies_all(session_id: Optional[str] = None):
3229
  data = _merge_curated(data, curated_for_this)
3230
  seen_policy_ids.add(policy_id_local)
3231
  slug = data.get("insurer_slug", "")
3232
- # KI-132 (2026-05-15) — regulatory is not an insurer; drop entirely
3233
- # from the marketplace. IRDAI/NHA docs are still retrieved and cited
3234
- # in chat answers, they just don't appear as marketplace cards.
3235
  if slug == "regulatory":
3236
  continue
3237
- # KI-133 (2026-05-15) — dedup by product (insurer__product), so the
3238
- # wordings PDF wins and the brochure/cis variants don't generate
3239
- # duplicate cards. Pass-1 sort order guarantees wordings comes first.
3240
  product_key = _product_key_of(policy_id_local)
3241
  if product_key in seen_product_keys:
3242
  continue
@@ -4254,10 +4240,10 @@ class RecallByNameRequest(BaseModel):
4254
 
4255
  class RecallByNameResponse(BaseModel):
4256
  found: bool
4257
- # PRIVACY FIX (2026-05-16, audit). The endpoint no longer hydrates the
4258
- # session off a bare name (weak/shared key → stranger-PII leak). When a
4259
- # stored profile matches, the response asks the UI to confirm identity
4260
- # FIRST. `profile`/`predicted_band` are never returned pre-confirmation.
4261
  requires_confirmation: bool = False
4262
  name: Optional[str] = None # stored display name, for the prompt
4263
  summary: Optional[dict] = None # non-PII identity hints, for the prompt
@@ -4270,16 +4256,15 @@ class RecallByNameResponse(BaseModel):
4270
  async def recall_profile_by_name(req: RecallByNameRequest):
4271
  """Stage a stored named-profile match for `session_id` (if any).
4272
 
4273
- PRIVACY FIX (2026-05-16, audit). A bare name is a weak, shared,
4274
- guessable key. This endpoint NO LONGER auto-hydrates the session — a
4275
- second user on a shared browser/IP, or anyone stating a common first
4276
- name, must not be silently served a stranger's stored profile. When a
4277
- match exists the response carries `found=True, requires_confirmation=
4278
- True, name, summary` so the UI renders an explicit "are you <name>?"
4279
- prompt. The stored fields are applied to the session only after the
4280
- user confirms (handled on the chat affirmation path via
4281
- session_state.apply_pending_recall). Returns `found=False` when the
4282
- slug doesn't resolve to a stored file.
4283
  """
4284
  from backend.profile_persistence import recall_by_name_payload
4285
 
 
32
  from backend.providers.sarvam_stt import SarvamSTT
33
  from backend.providers.sarvam_tts import SarvamTTS
34
 
35
+ # Single-LLM brain toggle. Off by default; flip via env var. When on, the
36
+ # /api/chat hot path runs single_brain.handle_turn and falls back to
37
+ # nim_fallback.handle_turn_fallback on any SingleBrainError (so users
38
+ # always get a reply). When off, /api/chat routes directly through
39
+ # nim_fallback.handle_turn_fallback.
40
  import os as _os # local alias to avoid stomping any later `import os`
41
 
42
  USE_SINGLE_BRAIN = _os.environ.get("USE_SINGLE_BRAIN", "false").lower() in (
43
  "1", "true", "yes", "on",
44
  )
45
 
46
+ # Safety net for RULE 7. If Gemini does not call mark_recommendation when
47
+ # the user clearly commits to a policy ("I'll go with that one", "let's do
48
+ # #2", "buy this"), the post-turn detector below auto-calls
49
+ # mark_recommendation against session.last_recommendation_ids[:1] so the
50
+ # closure event is recorded for analytics.
51
  # Word-boundary anchored; case-insensitive at match-time.
52
  _CLOSER_KEYWORD_RE = re.compile(
53
  r"\b(go with|i'?ll take|i will take|let'?s do|let me get|sign me up|"
 
145
  session_id: str
146
  audio_base64: Optional[str] = None
147
  audio_mime: Optional[str] = None # X8 — "audio/wav" | "audio/mp4" | "audio/webm"
148
+ # TTS voice-output failures are surfaced to the client (text reply is
149
+ # unaffected): the frontend renders a small inline "voice unavailable"
150
+ # notice under the bot bubble. tts_error_code is a closed enum mirroring
151
+ # the STT path's contract so the client never parses raw httpx text:
 
 
 
 
 
152
  # rate_limit — Sarvam 429 / insufficient_quota / no credits
153
  # service_unavailable — Sarvam 5xx / 503
154
  # auth — Sarvam 401/403 / missing SARVAM_API_KEY
 
167
  "flash an acknowledgment + refresh the completeness panel."
168
  ),
169
  )
170
+ # Whether the 7 required profile slots are captured. Surfaced in the
 
 
171
  # primary chat response so the UI can flip to 100% in the same render
172
+ # cycle without a second roundtrip to /api/profile/completeness.
173
+ # Computed via brain_tools._REQUIRED_FOR_READY (same slot list used by
174
+ # retrieve_policies' profile-complete gate) so client + server never
175
+ # disagree.
176
  profile_complete: bool = Field(
177
  False,
178
  description=(
 
338
  )
339
 
340
 
341
+ # Single source of truth for "is this profile ready to recommend against".
342
+ # brain_tools._profile_complete uses the same _REQUIRED_FOR_READY tuple; we
343
+ # mirror the call here instead of duplicating the slot list so a future
344
+ # addition (e.g. risk_appetite) only needs to be applied in brain_tools.
345
+ # Distinguishes "first-time capture on turn 1" from "stored profile
346
+ # recalled on turn 1". Both end with a
 
347
  # non-empty Profile on the session; only the latter should flip
348
  # returning_user_recalled. Heuristic: if EVERY currently-filled slot was
349
  # written THIS turn (i.e. lives in `profile_updates`), the user just typed
 
455
 
456
 
457
  # ---------- Admin panel + LLM health background loop ----------
458
+ # Mount the password-gated admin endpoints. Unauthorized callers get
459
+ # 401 Unauthorized. Access is gated by a strong password only (no IP
460
+ # allowlist, which would lock the operator out when switching networks
461
+ # without adding real security).
 
462
  from backend import admin as _admin_router_module
463
  app.include_router(_admin_router_module.router)
464
 
 
790
  if preferred_codec not in _allowed_codecs:
791
  preferred_codec = "audio/wav"
792
  t_chat0 = time.time()
793
+ # Pre-turn snapshot for the returning-user-recall detector below.
794
+ # Defaults match the "no session yet" case.
 
795
  _pre_turn_name: str = ""
796
  _pre_turn_idx: int = 0
797
+ # Never let an inner TimeoutError / unhandled exception bubble out of
798
+ # handle_turn as a 500. The whole call is wrapped in an outer 45s
799
+ # budget so even a pathological hang inside handle_turn surfaces as a
800
+ # graceful reply, not a connection-reset to the user. 45s is generous
801
+ # but tighter than HF Space's gateway timeout, so the user always gets
802
+ # a response.
 
 
803
  try:
804
  if USE_SINGLE_BRAIN:
805
+ # One Gemini call per turn with native function-calling.
806
+ # Falls back to nim_fallback on SingleBrainError so a missing
807
+ # GOOGLE_API_KEY / model outage never breaks the chat.
808
  from backend import single_brain
809
  from backend.session_state import get_session
810
 
 
816
  getattr(_sb_session.profile, "name", None) or ""
817
  ).strip()
818
  _pre_turn_idx = int(getattr(_sb_session, "turn_idx", 0) or 0)
819
+ # Once a session has had ANY successful single_brain turn, it
820
+ # must stay on single_brain for the rest of its lifetime.
821
+ # Switching brains mid-stream would discard everything
822
+ # single_brain captured in last_recommendation_ids /
823
+ # last_retrieved_chunks / slug_to_insurer. Sticky check below.
 
 
824
  _sb_was_sticky = getattr(_sb_session, "single_brain_sticky", False)
825
  try:
826
  turn = await asyncio.wait_for(
 
839
  pass
840
  except single_brain.SingleBrainError as _sb_err:
841
  if _sb_was_sticky:
842
+ # Session already had a clean single_brain turn. Do NOT
843
+ # cross-fade to the fallback brain (loses turn state +
844
+ # frontend sees the brain hop). Emit a graceful retry
845
+ # prompt instead.
846
  logging.warning(
847
  "single_brain failed on STICKY session (session=%s); "
848
+ "emitting graceful retry, NOT falling back: %s",
 
849
  session_id, _sb_err,
850
  )
851
  turn = single_brain.TurnResult(
 
880
  timeout=20.0,
881
  )
882
  else:
883
+ # When USE_SINGLE_BRAIN is off, route directly through the
884
+ # minimal NIM fallback so the bot still serves a reply.
 
885
  from backend.session_state import get_session as _get_session
886
  turn = await asyncio.wait_for(
887
  nim_fallback.handle_turn_fallback(
 
948
  profile_complete=_compute_profile_complete(session_id),
949
  )
950
 
951
+ # Server-side closer-keyword safety net for RULE 7.
952
  # If the user clearly committed to a policy this turn but Gemini did
953
  # NOT call mark_recommendation (single_brain stamps "mark_recommendation"
954
  # into turn.brain_used when the tool fires — see single_brain.py:1052),
 
1158
 
1159
  audio_b64 = None
1160
  audio_mime: Optional[str] = None
1161
+ # When TTS fails we propagate a STRUCTURED, user-facing notice instead
1162
+ # of silently dropping the audio. The text reply is still returned in
1163
+ # full; the frontend renders a small inline "voice unavailable" line
1164
+ # under the bot bubble so the user understands why there's no voice
1165
+ # (e.g. Sarvam 429 / no credits).
1166
  tts_error_code: Optional[str] = None
1167
  tts_user_message: Optional[str] = None
1168
  if req.return_audio and turn.reply_text:
 
1918
  parents_has_ped: Optional[bool] = None
1919
  health_conditions: Optional[list[str]] = None
1920
  budget_band: Optional[str] = None
1921
+ # Collected by the profile-builder UI; also present on the Profile
1922
+ # dataclass + chat-path save_profile_field. Whitelisted here so
1923
+ # POST /api/profile accepts them.
1924
  desired_sum_insured_inr: Optional[int] = None
1925
  copay_pct: Optional[int] = None
1926
  family_medical_history: Optional[list[str]] = None
 
1957
  The on-disk profile JSON under `40-data/profiles/` is intentionally NOT
1958
  touched — it remains durable user data keyed by persona_id / name slug.
1959
  The next time the user volunteers their name in conversation, the
1960
+ confirmation-gated recall flow (see session_state's
1961
+ `pending_profile_recall`) will ask before merging the prior captures.
1962
 
1963
  Body : {session_id: str}
1964
  Reply: {cleared: bool, new_session_id: str}
 
2122
  missing = [k for k, v in profile_dict.items() if k not in answered or v in (None, "", [])]
2123
  hint = None
2124
  try:
2125
+ # next_question returns the field name (str) of the next missing
2126
+ # slot. The frontend uses it as a slot-hint; the actual phrasing
2127
+ # is generated by the single-brain LLM.
2128
  hint = next_question(p)
2129
  except Exception:
2130
  pass
 
2830
  return _rationalise_si(data, si)
2831
 
2832
 
2833
+ # Decision-critical fields that distinguish a RENAME (curated entry folds
2834
+ # onto extracted parent) from a VARIANT (same UIN but materially different
2835
+ # product — must stay as its own card). Same UIN ≠ same product:
2836
+ # regulators file one "wordings" PDF that covers multiple marketed variants
2837
+ # (e.g. ProHealth Prime vs ProHealth Protect both filed under
2838
+ # MCIHLIP24011V072324; copay/PED/maternity/NCB differ).
2839
  _KI145_DIFF_FIELDS: tuple[str, ...] = (
2840
  "copayment_pct",
2841
  "pre_existing_disease_waiting_months",
 
3215
  data = _merge_curated(data, curated_for_this)
3216
  seen_policy_ids.add(policy_id_local)
3217
  slug = data.get("insurer_slug", "")
3218
+ # Regulatory is not an insurer; drop entirely from the marketplace.
3219
+ # IRDAI/NHA docs are still retrieved and cited in chat answers, they
3220
+ # just don't appear as marketplace cards.
3221
  if slug == "regulatory":
3222
  continue
3223
+ # Dedup by product (insurer__product), so the wordings PDF wins and
3224
+ # the brochure/cis variants don't generate duplicate cards. Pass-1
3225
+ # sort order guarantees wordings comes first.
3226
  product_key = _product_key_of(policy_id_local)
3227
  if product_key in seen_product_keys:
3228
  continue
 
4240
 
4241
  class RecallByNameResponse(BaseModel):
4242
  found: bool
4243
+ # The endpoint does not hydrate the session off a bare name (weak/shared
4244
+ # key → stranger-PII leak). When a stored profile matches, the response
4245
+ # asks the UI to confirm identity FIRST. `profile`/`predicted_band` are
4246
+ # never returned pre-confirmation.
4247
  requires_confirmation: bool = False
4248
  name: Optional[str] = None # stored display name, for the prompt
4249
  summary: Optional[dict] = None # non-PII identity hints, for the prompt
 
4256
  async def recall_profile_by_name(req: RecallByNameRequest):
4257
  """Stage a stored named-profile match for `session_id` (if any).
4258
 
4259
+ A bare name is a weak, shared, guessable key, so this endpoint does not
4260
+ auto-hydrate the session — a second user on a shared browser/IP, or
4261
+ anyone stating a common first name, must not be silently served a
4262
+ stranger's stored profile. When a match exists the response carries
4263
+ `found=True, requires_confirmation=True, name, summary` so the UI
4264
+ renders an explicit "are you <name>?" prompt. The stored fields are
4265
+ applied to the session only after the user confirms (handled on the
4266
+ chat affirmation path via session_state.apply_pending_recall). Returns
4267
+ `found=False` when the slug doesn't resolve to a stored file.
 
4268
  """
4269
  from backend.profile_persistence import recall_by_name_payload
4270
 
backend/needs_finder.py CHANGED
@@ -1,13 +1,11 @@
1
  """User profile state + slot-order hint for the fact-find.
2
 
3
- KI-167 WS3 (2026-05-15) the legacy rules-engine question GRAPH and the
4
- matching `Question` dataclass have been removed. Fact-find phrasing now
5
- lives in `backend/sales_brain.py` (the single-LLM-call brain); this module
6
- keeps only:
7
 
8
- - `Profile` — accumulated user state (still imported widely)
9
  - `record_answer` — slot-write helper used by session_state
10
- - INR / budget / income parsers — used by `sales_brain_normalizer.py`
11
  - `is_field_set` — local helper for record_answer + next_question
12
  - `next_question` — returns the field NAME (str) of the next
13
  missing slot in canonical order. Used by
@@ -63,11 +61,9 @@ class Profile:
63
 
64
 
65
  # ----------------------------------------------------------------------------
66
- # KI-149 (2026-05-15) — free-text INR amount parser for budget + income.
67
- # User said "I maximum 30000 I can pay" bot re-asked budget because no
68
- # parser was attached to the budget Question and the LLM brain failed to
69
- # capture it. Bare digits ("30000"), "30 thousand", "30 grand", "₹30,000",
70
- # "1 lakh", "1.5L" must all map cleanly to a rupee amount.
71
  # ----------------------------------------------------------------------------
72
 
73
  def _parse_inr_amount(text: str) -> Optional[int]:
@@ -82,13 +78,10 @@ def _parse_inr_amount(text: str) -> Optional[int]:
82
  - strips fluff: "maximum 30000", "I can pay 30000", "around 25000"
83
  - tolerates per-year qualifiers: "/year", "per year", "p.a."
84
 
85
- KI-161 (2026-05-15) — REJECTS bare digits below ₹1000 (no plausible
86
- annual health insurance budget/income falls there) and REJECTS any
87
- text whose only number is in an age context ("29 years old", "age 29",
88
- "I am 29"). Origin: user answered the age question with "I am 29
89
- years old" and the parser wrote ₹29 into both budget_band and
90
- income_band, leading the bot to claim it captured age + income + budget
91
- from a single utterance.
92
 
93
  Returns the integer rupee amount, or None if no number is recognisable
94
  or if the only numbers in the text are clearly not currency.
@@ -209,10 +202,10 @@ def _parse_income_band(text: str) -> Optional[str]:
209
  # Engine
210
  # ----------------------------------------------------------------------------
211
 
212
- # KI-167 WS3 (2026-05-15) — canonical slot order for the fact-find hint API.
213
- # The actual question phrasing lives in `sales_brain.py`; this list only
214
- # encodes which Profile attribute we want to fill next when nothing else
215
- # is driving the conversation.
216
  _SLOT_ORDER: list[str] = [
217
  "name",
218
  "age",
@@ -238,11 +231,9 @@ def is_field_set(profile: Profile, field_name: str) -> bool:
238
  def next_question(profile: Profile) -> Optional[str]:
239
  """Return the field NAME of the next missing slot, or None if complete.
240
 
241
- KI-167 WS3 (2026-05-15) — refactored from `Optional[Question]` to
242
- `Optional[str]` after the rules-engine GRAPH was deleted. The caller in
243
- `backend/main.py:/api/profile/completeness` uses this only to hint to the
244
- frontend which slot to ask next; the actual question phrasing is now
245
- produced by `sales_brain.py`.
246
 
247
  A free-form session (user driving free questions) returns None so the
248
  hint endpoint reports "nothing to ask".
@@ -256,10 +247,9 @@ def next_question(profile: Profile) -> Optional[str]:
256
  return None
257
 
258
 
259
- # Legacy GRAPH question-id → Profile field-name aliases. Some callers
260
- # (notably `session_state.record_answer` driven by `awaiting_question_id`)
261
- # still pass the old question IDs from the deleted GRAPH. Mapping them here
262
- # keeps that call path working without resurrecting the GRAPH.
263
  _QID_TO_FIELD: dict[str, str] = {
264
  "existing_cover": "existing_cover_inr",
265
  "location": "location_tier",
@@ -271,10 +261,9 @@ _QID_TO_FIELD: dict[str, str] = {
271
  def record_answer(profile: Profile, question_id: str, raw_answer: Any) -> Profile:
272
  """Mutate profile in place with a raw answer for a named slot.
273
 
274
- KI-167 WS3 (2026-05-15) parser dispatch used to live on `Question`
275
- objects in the deleted GRAPH; with the GRAPH gone we apply the same
276
- parser map inline. `question_id` may be either a Profile attribute name
277
- (preferred) or one of the legacy GRAPH question IDs from `_QID_TO_FIELD`.
278
  """
279
  field_name = _QID_TO_FIELD.get(question_id, question_id)
280
  if not hasattr(profile, field_name):
 
1
  """User profile state + slot-order hint for the fact-find.
2
 
3
+ Fact-find phrasing lives in `backend/single_brain.py` (the single-LLM-call
4
+ brain); this module provides:
 
 
5
 
6
+ - `Profile` — accumulated user state (imported widely)
7
  - `record_answer` — slot-write helper used by session_state
8
+ - INR / budget / income parsers — used by the brain tool layer
9
  - `is_field_set` — local helper for record_answer + next_question
10
  - `next_question` — returns the field NAME (str) of the next
11
  missing slot in canonical order. Used by
 
61
 
62
 
63
  # ----------------------------------------------------------------------------
64
+ # Free-text INR amount parser for budget + income. Bare digits ("30000"),
65
+ # "30 thousand", "30 grand", "₹30,000", "1 lakh", "1.5L" all map cleanly
66
+ # to a rupee amount.
 
 
67
  # ----------------------------------------------------------------------------
68
 
69
  def _parse_inr_amount(text: str) -> Optional[int]:
 
78
  - strips fluff: "maximum 30000", "I can pay 30000", "around 25000"
79
  - tolerates per-year qualifiers: "/year", "per year", "p.a."
80
 
81
+ Rejects bare digits below ₹1000 (no plausible annual health insurance
82
+ budget/income falls there) and rejects any text whose only number is
83
+ in an age context ("29 years old", "age 29", "I am 29"), so an age
84
+ answer is never misread as a rupee amount.
 
 
 
85
 
86
  Returns the integer rupee amount, or None if no number is recognisable
87
  or if the only numbers in the text are clearly not currency.
 
202
  # Engine
203
  # ----------------------------------------------------------------------------
204
 
205
+ # Canonical slot order for the fact-find hint API. The actual question
206
+ # phrasing lives in `single_brain.py`; this list only encodes which
207
+ # Profile attribute to fill next when nothing else is driving the
208
+ # conversation.
209
  _SLOT_ORDER: list[str] = [
210
  "name",
211
  "age",
 
231
  def next_question(profile: Profile) -> Optional[str]:
232
  """Return the field NAME of the next missing slot, or None if complete.
233
 
234
+ The caller in `backend/main.py:/api/profile/completeness` uses this
235
+ only to hint to the frontend which slot to ask next; the actual
236
+ question phrasing is produced by `single_brain.py`.
 
 
237
 
238
  A free-form session (user driving free questions) returns None so the
239
  hint endpoint reports "nothing to ask".
 
247
  return None
248
 
249
 
250
+ # Question-id → Profile field-name aliases. Some callers (notably
251
+ # `session_state.record_answer` driven by `awaiting_question_id`) pass a
252
+ # question ID rather than a Profile attribute name; this maps them.
 
253
  _QID_TO_FIELD: dict[str, str] = {
254
  "existing_cover": "existing_cover_inr",
255
  "location": "location_tier",
 
261
  def record_answer(profile: Profile, question_id: str, raw_answer: Any) -> Profile:
262
  """Mutate profile in place with a raw answer for a named slot.
263
 
264
+ Applies the parser map inline. `question_id` may be either a Profile
265
+ attribute name (preferred) or one of the question IDs in
266
+ `_QID_TO_FIELD`.
 
267
  """
268
  field_name = _QID_TO_FIELD.get(question_id, question_id)
269
  if not hasattr(profile, field_name):
backend/nim_fallback.py CHANGED
@@ -1,16 +1,16 @@
1
  """Minimal NIM fallback for first-turn Gemini 503.
2
 
3
- Replaces the 5,219 LOC legacy orchestrator stack. Does ONE thing: when
4
- single_brain raises SingleBrainError on the first turn (no
5
- single_brain_sticky yet), give the user a working reply using NIM.
6
 
7
  No tools, no function-calling, no faithfulness gate, no normalizer —
8
  just a single NIM chat call with a tiny system prompt. The user gets a
9
  polite "let me think out loud while I reconnect" reply; the next turn,
10
  single_brain (Gemini) takes over and the session sticks.
11
 
12
- KI-155 / ADR-038: NIM-only. We elect via llm_health.get_primary("brain")
13
- which walks BRAIN_CHAIN in priority order over election-eligible models.
14
  If the elector has nothing eligible, we fall back to BRAIN_CHAIN[0] as a
15
  last resort so the user still gets a reply.
16
 
@@ -96,9 +96,9 @@ def _pick_model() -> str:
96
 
97
 
98
  def _flatten_history(chat_history: Optional[list[dict]]) -> list[ChatMessage]:
99
- """Translate the orchestrator-style chat_history ({role, content})
100
- into the NIM `ChatMessage` shape. Roles other than user/assistant
101
- are dropped; assistant aliases collapse to `assistant`.
102
  """
103
  out: list[ChatMessage] = []
104
  for msg in chat_history or []:
 
1
  """Minimal NIM fallback for first-turn Gemini 503.
2
 
3
+ Does ONE thing: when single_brain raises SingleBrainError on the first
4
+ turn (no single_brain_sticky yet), give the user a working reply using
5
+ NIM.
6
 
7
  No tools, no function-calling, no faithfulness gate, no normalizer —
8
  just a single NIM chat call with a tiny system prompt. The user gets a
9
  polite "let me think out loud while I reconnect" reply; the next turn,
10
  single_brain (Gemini) takes over and the session sticks.
11
 
12
+ NIM-only. We elect via llm_health.get_primary("brain") which walks
13
+ BRAIN_CHAIN in priority order over election-eligible models.
14
  If the elector has nothing eligible, we fall back to BRAIN_CHAIN[0] as a
15
  last resort so the user still gets a reply.
16
 
 
96
 
97
 
98
  def _flatten_history(chat_history: Optional[list[dict]]) -> list[ChatMessage]:
99
+ """Translate the {role, content} chat_history into the NIM
100
+ `ChatMessage` shape. Roles other than user/assistant are dropped;
101
+ assistant aliases collapse to `assistant`.
102
  """
103
  out: list[ChatMessage] = []
104
  for msg in chat_history or []:
backend/policy_identity.py CHANGED
@@ -1,20 +1,18 @@
1
- """Canonical policy identity + dedup — SINGLE SOURCE OF TRUTH (KI-280).
2
 
3
- The marketplace endpoint `main.py /api/policies/all` already established the
4
- canonical-identity rule (KI-133 / KI-141 / KI-142 / KI-145):
5
 
6
  1 unique IRDAI UIN = 1 unique product (UIN-primary invariant).
7
  When no UIN is present, the product_key = the policy_id with any
8
  trailing doctype suffix (`__wordings` / `__brochure` / `__cis` /
9
  `__prospectus`) stripped.
10
 
11
- The recommendation-fit gate (retrieval_filters.dedup_by_policy) used to key
12
- ONLY on the exact `policy_id`, so two chunks for the SAME product but with
13
- different ids — a marketing rename ("my:Optima Secure" vs
14
- "my:Optima Secure (older variant)") or two doctype siblings
15
- (`...__wordings` vs `...__brochure`) — were both emitted as separate CITED
16
- cards (audit personas P2/P4/P7). The marketplace and the recommender
17
- disagreed on what "the same policy" means.
18
 
19
  This module factors that one rule so BOTH surfaces share it. It is
20
  deliberately dependency-free (no FastAPI / Chroma import) so
 
1
+ """Canonical policy identity + dedup — SINGLE SOURCE OF TRUTH.
2
 
3
+ The canonical-identity rule:
 
4
 
5
  1 unique IRDAI UIN = 1 unique product (UIN-primary invariant).
6
  When no UIN is present, the product_key = the policy_id with any
7
  trailing doctype suffix (`__wordings` / `__brochure` / `__cis` /
8
  `__prospectus`) stripped.
9
 
10
+ Both the marketplace endpoint (`main.py /api/policies/all`) and the
11
+ recommendation-fit gate (retrieval_filters.dedup_by_policy) key on this
12
+ rule, so two chunks for the SAME product with different ids — a marketing
13
+ rename ("my:Optima Secure" vs "my:Optima Secure (older variant)") or two
14
+ doctype siblings (`...__wordings` vs `...__brochure`) — collapse to one
15
+ identity and the two surfaces agree on what "the same policy" means.
 
16
 
17
  This module factors that one rule so BOTH surfaces share it. It is
18
  deliberately dependency-free (no FastAPI / Chroma import) so
backend/premium_calculator.py CHANGED
@@ -817,35 +817,25 @@ _DEFAULT_BAND_POLICY_IDS: list[str] = [
817
 
818
 
819
  # ───────────────────────────────────────────────────────────────────────────
820
- # SI RECONCILIATION (KI-278, 2026-05-16) single source of truth for the
821
- # sum-insured the header band AND the per-settings panel both price at.
822
- #
823
- # THE HEADER≠PANEL BUG: PremiumCalculatorPanel (page.tsx) seeds its SI slider
824
- # from `desired_sum_insured_inr ?? existing_cover_inr ?? 1_000_000`, but
825
- # estimate_premium_band() used to hard-code sum_insured_default=₹10L and
826
- # IGNORE the profile entirely. So a user who stated a ₹25L target saw the
827
- # header chip priced at ₹10L (₹6,500–₹26,500) while the panel priced the
828
- # same profile at ₹25L (point ₹19,100) — contradictory numbers for one
829
- # profile. Resolving BOTH surfaces' SI from this one function makes them
830
- # reconcile by construction: the panel's point estimate now falls inside
831
- # the header band because the header band is the SAME basket priced at the
832
- # SAME profile-resolved SI.
833
  #
834
  # Precedence MUST stay byte-identical to PremiumCalculatorPanel's
835
  # useState initialiser (frontend/src/app/page.tsx ~L2417) and
836
  # PolicyPremiumWidget's initialSumInsured contract:
837
  # 1. profile.desired_sum_insured_inr (user's stated target SI)
838
  # 2. profile.existing_cover_inr (closest available signal)
839
- # 3. fallback default (legacy ₹10L)
840
  # ───────────────────────────────────────────────────────────────────────────
841
 
842
- # SI RATIONALISATION (D2, 2026-05-16) the global ₹5 L / ₹1 Cr clamp
843
- # (SI_FLOOR_INR / SI_CEILING_INR) was REMOVED. Pricing now respects each
844
- # policy's own real SI bounds and the user's actual stated target instead of
845
- # squashing every profile into one synthetic envelope. A ₹2 Cr aspiration
846
- # now prices at 2 Cr; a ₹1 L corporate top-up prices at ₹1 L. When a policy
847
- # has no published SI the caller prices against the user's
848
- # desired_sum_insured_inr (else ₹10 L default) and surfaces a disclosure.
849
 
850
 
851
  def resolve_profile_sum_insured(
@@ -864,9 +854,9 @@ def resolve_profile_sum_insured(
864
  string/None gracefully and snaps to the nearest ₹50k so the resolved SI
865
  lands on a representable slider stop.
866
 
867
- D2 (2026-05-16) the global ₹5 L /1 Cr clamp was REMOVED: the user's
868
- actual stated target is honoured (a ₹2 Cr aspiration prices at ₹2 Cr, a
869
- ₹1 L top-up at ₹1 L) rather than squashed into a synthetic envelope.
870
  """
871
  profile = profile or {}
872
 
 
817
 
818
 
819
  # ───────────────────────────────────────────────────────────────────────────
820
+ # Single source of truth for the sum-insured the header band AND the
821
+ # per-settings panel both price at. Resolving both surfaces' SI from this
822
+ # one function makes them reconcile by construction: the panel's point
823
+ # estimate falls inside the header band because the header band is the
824
+ # SAME basket priced at the SAME profile-resolved SI.
 
 
 
 
 
 
 
 
825
  #
826
  # Precedence MUST stay byte-identical to PremiumCalculatorPanel's
827
  # useState initialiser (frontend/src/app/page.tsx ~L2417) and
828
  # PolicyPremiumWidget's initialSumInsured contract:
829
  # 1. profile.desired_sum_insured_inr (user's stated target SI)
830
  # 2. profile.existing_cover_inr (closest available signal)
831
+ # 3. fallback default (₹10L)
832
  # ───────────────────────────────────────────────────────────────────────────
833
 
834
+ # Pricing respects each policy's own real SI bounds and the user's actual
835
+ # stated target rather than a global clamp: a ₹2 Cr aspiration prices at
836
+ # ₹2 Cr; a ₹1 L corporate top-up prices at ₹1 L. When a policy has no
837
+ # published SI the caller prices against the user's desired_sum_insured_inr
838
+ # (else10 L default) and surfaces a disclosure.
 
 
839
 
840
 
841
  def resolve_profile_sum_insured(
 
854
  string/None gracefully and snaps to the nearest ₹50k so the resolved SI
855
  lands on a representable slider stop.
856
 
857
+ The user's actual stated target is honoured (a2 Cr aspiration prices
858
+ at ₹2 Cr, a ₹1 L top-up at ₹1 L) rather than clamped to a synthetic
859
+ envelope.
860
  """
861
  profile = profile or {}
862
 
backend/profile_persistence.py CHANGED
@@ -230,13 +230,6 @@ def extract_potential_name(text: str) -> Optional[str]:
230
  return raw
231
 
232
 
233
- # NOTE 2026-05-17: `extract_name_in_context` + its `_ASSISTANT_ASKED_NAME_RE`
234
- # / `_BARE_NAME_ANSWER_RE` regexes (b87bd2d, KI-RECALL-FIX) were REMOVED with
235
- # the deterministic in-conversation recall scaffolding. `extract_potential_name`
236
- # (above) and `try_recall_by_name` (below) remain — they back the explicit
237
- # POST /api/profile/recall-by-name endpoint, not an in-chat short-circuit.
238
-
239
-
240
  def try_recall_by_name(session, name: str) -> bool:
241
  """Look up a stored profile by name and STAGE it for confirmation.
242
 
 
230
  return raw
231
 
232
 
 
 
 
 
 
 
 
233
  def try_recall_by_name(session, name: str) -> bool:
234
  """Look up a stored profile by name and STAGE it for confirmation.
235
 
backend/profile_rag.py CHANGED
@@ -1,11 +1,10 @@
1
  """Customer-profile-as-RAG layer.
2
 
3
- KI-118 (2026-05-15) — profile chunks are now keyed by `name_slug` (the
4
- canonicalised user name), NOT by session_id. Only NAMED users ever get
5
- embedded; anonymous sessions never write to Chroma. This eliminates the
6
- corruption surface that the session_id-keyed chunks introduced (a missing
7
- session_id metadata field on the legacy `profile_anonymous` row poisoned
8
- every subsequent retrieval query — see KI-117 boot cleanup).
9
 
10
  At retrieval time, `rag/retrieve.py::retrieve(..., profile_name_slug=...)`
11
  boosts the user's profile chunk so the LLM sees the user's context inline
@@ -39,9 +38,8 @@ _log = logging.getLogger(__name__)
39
  def profile_to_chunk_text(profile: dict) -> str:
40
  """Render the profile dict as a natural-language paragraph for the LLM.
41
 
42
- The shape matches what build_messages() in the orchestrator expects so
43
- when this chunk is retrieved alongside policy chunks, the LLM sees a
44
- coherent "USER CONTEXT" block.
45
  """
46
  parts: list[str] = ["USER CONTEXT — facts about the person asking this question:"]
47
 
@@ -124,10 +122,8 @@ def _get_collection():
124
  async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
125
  """Embed the profile paragraph and store as a single chunk in Chroma.
126
 
127
- KI-118 (2026-05-15) — keyed by `name_slug` (canonical user name) instead
128
- of `session_id`. Only NAMED users ever get embedded — anonymous chats
129
- never write to Chroma, which eliminates the corruption surface that the
130
- session_id keying introduced.
131
 
132
  Idempotent — calling this on every profile update is safe; existing
133
  chunks for the same name_slug get replaced.
@@ -138,14 +134,10 @@ async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
138
  if not text or len(text) < 30:
139
  return
140
 
141
- # KI-112 (2026-05-15) / KI-118 — input guard 1: name_slug must be a
142
- # non-empty str. Pre-fix, a missing/empty key caused upsert under id
143
- # "profile_" with `policy_id` colliding across all anonymous sessions;
144
- # the initial KI-102 deploy wrote a `profile_anonymous` chunk WITHOUT a
145
- # session_id metadata field that poisoned every subsequent query whose
146
- # `where` clause referenced session_id. Anonymous users no longer reach
147
- # this function at all (the orchestrator gates on
148
- # `session.profile.name` before calling) — this guard is belt-and-braces.
149
  if not isinstance(name_slug, str) or not name_slug.strip():
150
  _log.warning(
151
  "profile_rag.upsert_profile_chunk: refusing to write — name_slug "
@@ -157,11 +149,11 @@ async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
157
  embedder = LocalEmbeddings()
158
  [vec] = await embedder.embed([text], input_type="document")
159
 
160
- # KI-112 (2026-05-15) — input guard 2: embedding must be a list of finite
161
- # floats whose length matches the embedder's declared dimension. Pre-fix,
162
- # an empty / None / mis-shaped embedding could be added to Chroma where it
163
- # would silently corrupt HNSW (dangling pointer or shape mismatch). The
164
- # corpus uses 384-dim BAAI/bge-small-en-v1.5; any other shape is a bug.
165
  expected_dim = getattr(embedder, "dimension", None) or 384
166
  if (
167
  not isinstance(vec, (list, tuple))
@@ -184,20 +176,18 @@ async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
184
  try:
185
  coll.delete(where={"policy_id": chunk_id})
186
  except Exception as e:
187
- # Non-fatal: chunk may not exist yet. KI-107 at least log so
188
- # silent corruption of the profile store is observable.
189
  _log.debug(
190
  "profile_rag.upsert_profile_chunk: delete(where=policy_id=%s) "
191
  "non-fatal failure: %s: %s",
192
  chunk_id, type(e).__name__, str(e)[:200],
193
  )
194
 
195
- # KI-107 (2026-05-15) — wrap coll.add() in try/except. C5 port-in saw
196
- # HTTP 500 "Error finding id" cascade; one possible vector is a
197
- # transient Chroma sqlite lock during HNSW compaction when add()
198
- # interleaves with the retrieve path's get(). Make upsert non-fatal so
199
- # the chat reply still returns even if the profile-chunk write fails.
200
- # On next upsert (next profile field change), the retry will succeed.
201
  try:
202
  coll.add(
203
  ids=[chunk_id],
@@ -208,8 +198,8 @@ async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
208
  "insurer_slug": "profile",
209
  "policy_name": f"User profile ({name_slug[:16]})",
210
  "doc_type": "profile",
211
- # KI-118 (2026-05-15) stamp name_slug instead of session_id.
212
- # The retrieve path filters profile chunks via this field.
213
  "name_slug": name_slug,
214
  "source_url": "",
215
  "page_start": 0,
@@ -228,7 +218,7 @@ async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
228
 
229
 
230
  def remove_profile_chunk(name_slug: str) -> None:
231
- """Optional cleanup. KI-118 — keyed by name_slug."""
232
  if not name_slug:
233
  return
234
  try:
 
1
  """Customer-profile-as-RAG layer.
2
 
3
+ Profile chunks are keyed by `name_slug` (the canonicalised user name),
4
+ NOT by session_id. Only NAMED users ever get embedded; anonymous sessions
5
+ never write to Chroma. Keying by name_slug avoids the corruption surface
6
+ of session_id-keyed chunks (an anonymous row missing a session_id
7
+ metadata field could poison session_id-scoped retrieval queries).
 
8
 
9
  At retrieval time, `rag/retrieve.py::retrieve(..., profile_name_slug=...)`
10
  boosts the user's profile chunk so the LLM sees the user's context inline
 
38
  def profile_to_chunk_text(profile: dict) -> str:
39
  """Render the profile dict as a natural-language paragraph for the LLM.
40
 
41
+ Rendered as a coherent "USER CONTEXT" block so that when this chunk is
42
+ retrieved alongside policy chunks, the LLM sees the user's facts inline.
 
43
  """
44
  parts: list[str] = ["USER CONTEXT — facts about the person asking this question:"]
45
 
 
122
  async def upsert_profile_chunk(name_slug: str, profile_dict: dict) -> None:
123
  """Embed the profile paragraph and store as a single chunk in Chroma.
124
 
125
+ Keyed by `name_slug` (canonical user name), not `session_id`. Only
126
+ NAMED users ever get embedded — anonymous chats never write to Chroma.
 
 
127
 
128
  Idempotent — calling this on every profile update is safe; existing
129
  chunks for the same name_slug get replaced.
 
134
  if not text or len(text) < 30:
135
  return
136
 
137
+ # Input guard 1: name_slug must be a non-empty str. An empty key would
138
+ # upsert under id "profile_" and collide across anonymous sessions.
139
+ # Callers gate on `session.profile.name` before calling, so anonymous
140
+ # users do not reach this function; this guard is belt-and-braces.
 
 
 
 
141
  if not isinstance(name_slug, str) or not name_slug.strip():
142
  _log.warning(
143
  "profile_rag.upsert_profile_chunk: refusing to write — name_slug "
 
149
  embedder = LocalEmbeddings()
150
  [vec] = await embedder.embed([text], input_type="document")
151
 
152
+ # Input guard 2: embedding must be a list of finite floats whose length
153
+ # matches the embedder's declared dimension. An empty / None / mis-shaped
154
+ # embedding added to Chroma would silently corrupt HNSW (dangling
155
+ # pointer or shape mismatch). The corpus uses 384-dim
156
+ # BAAI/bge-small-en-v1.5; any other shape is rejected.
157
  expected_dim = getattr(embedder, "dimension", None) or 384
158
  if (
159
  not isinstance(vec, (list, tuple))
 
176
  try:
177
  coll.delete(where={"policy_id": chunk_id})
178
  except Exception as e:
179
+ # Non-fatal: chunk may not exist yet. Log so a silent failure of
180
+ # the profile store is observable.
181
  _log.debug(
182
  "profile_rag.upsert_profile_chunk: delete(where=policy_id=%s) "
183
  "non-fatal failure: %s: %s",
184
  chunk_id, type(e).__name__, str(e)[:200],
185
  )
186
 
187
+ # Wrap coll.add() in try/except so upsert is non-fatal: a transient
188
+ # Chroma sqlite lock during HNSW compaction (add() interleaving with
189
+ # the retrieve path's get()) must not break the chat reply. The next
190
+ # upsert (next profile field change) retries the write.
 
 
191
  try:
192
  coll.add(
193
  ids=[chunk_id],
 
198
  "insurer_slug": "profile",
199
  "policy_name": f"User profile ({name_slug[:16]})",
200
  "doc_type": "profile",
201
+ # Stamp name_slug; the retrieve path filters profile
202
+ # chunks via this field.
203
  "name_slug": name_slug,
204
  "source_url": "",
205
  "page_start": 0,
 
218
 
219
 
220
  def remove_profile_chunk(name_slug: str) -> None:
221
+ """Optional cleanup. Keyed by name_slug."""
222
  if not name_slug:
223
  return
224
  try:
backend/profile_store.py CHANGED
@@ -14,8 +14,8 @@ Architectural answer to the "embed or JSON?" question:
14
  style questions. Embedding cost = once per update, not per query.
15
 
16
  Both layers stay in sync: when `save_profile()` is called here, the
17
- orchestrator also fires `profile_rag.upsert_profile_chunk()` so the
18
- Chroma side reflects the new state.
19
 
20
  Files live under `40-data/profiles/<normalised-name>.json`. Names are
21
  normalised to lowercase + alpha-only for the filename so "Rohit" and
@@ -48,8 +48,8 @@ def _normalise_name(name: str) -> str:
48
  return cleaned[:60] # cap filename length
49
 
50
 
51
- # KI-062 (2026-05-15) — identity-defining fields used to disambiguate two
52
- # users with the same display name. Order matters for hash stability.
53
  _PERSONA_ID_FIELDS: tuple[str, ...] = (
54
  "age", "dependents", "income_band", "location_tier", "parents_age_max",
55
  )
@@ -170,7 +170,7 @@ def load_profile(name: str, *, persona_id: Optional[str] = None) -> Optional[Pro
170
  p = _path_for(name)
171
  if p and p.exists():
172
  return _load_from_path(p)
173
- # 3. (REMOVED) cross-identity display-name directory scan leak vector.
174
  return None
175
 
176
 
@@ -266,10 +266,10 @@ def save_profile(name: str, profile: Profile, *, session_id: Optional[str] = Non
266
 
267
 
268
  # ---------------------------------------------------------------------------
269
- # KI-063 (2026-05-15) — per-user policy interaction tracking.
270
  #
271
  # Three event types are tracked on the Profile:
272
- # shown — auto-logged by orchestrator when a policy is cited in a
273
  # recommendation / comparison turn that passed faithfulness.
274
  # selected — user clicked "save / shortlist" on a policy card (frontend
275
  # POSTs to /api/profile/select).
 
14
  style questions. Embedding cost = once per update, not per query.
15
 
16
  Both layers stay in sync: when `save_profile()` is called here, the
17
+ save path also fires `profile_rag.upsert_profile_chunk()` so the Chroma
18
+ side reflects the new state.
19
 
20
  Files live under `40-data/profiles/<normalised-name>.json`. Names are
21
  normalised to lowercase + alpha-only for the filename so "Rohit" and
 
48
  return cleaned[:60] # cap filename length
49
 
50
 
51
+ # Identity-defining fields used to disambiguate two users with the same
52
+ # display name. Order matters for hash stability.
53
  _PERSONA_ID_FIELDS: tuple[str, ...] = (
54
  "age", "dependents", "income_band", "location_tier", "parents_age_max",
55
  )
 
170
  p = _path_for(name)
171
  if p and p.exists():
172
  return _load_from_path(p)
173
+ # No cross-identity display-name directory scan (would be a leak vector).
174
  return None
175
 
176
 
 
266
 
267
 
268
  # ---------------------------------------------------------------------------
269
+ # Per-user policy interaction tracking.
270
  #
271
  # Three event types are tracked on the Profile:
272
+ # shown — auto-logged by the brain when a policy is cited in a
273
  # recommendation / comparison turn that passed faithfulness.
274
  # selected — user clicked "save / shortlist" on a policy card (frontend
275
  # POSTs to /api/profile/select).
backend/providers/__init__.py CHANGED
@@ -1,26 +1,17 @@
1
  """Provider clients — thin, async, behind a common interface.
2
 
3
- After the 2026-05-14 Stack A consolidation (D-019, partially superseded by
4
- D-022 on 2026-05-14 — NIM brain pool swap), the provider stack is:
5
 
6
  sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
7
  sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech, speaker anushka)
8
  sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain)
9
  local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
10
- nvidia_nim_llm.py — Chain pattern via NimChainLLM. Current primaries (D-022):
11
- BRAIN_CHAIN[0] = Qwen 3-Next 80B
12
- FAST_BRAIN_CHAIN[0] = Nemotron Nano 30B
13
- JUDGE_CHAIN[0] = Mistral Large 3 675B
14
- DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick (the
15
- original D-019 picks) are retained as fallback chain
16
- entries that re-enter rotation when NIM's pools
17
- recover from the 2026-05-14 outage.
18
- openrouter_llm.py — OpenRouter cross-provider fallback (bottom of every chain).
19
- groq_llm.py — Groq LPU; brain primary rotates 50/50 NIM ↔ Groq
20
- per call (KI-025 / ADR-026).
21
-
22
- See 70-docs/60-decisions/ADR-019, ADR-026, ADR-030 and 80-audit/ENTERPRISE_AUDIT.md
23
- (D-022 row) for the full provenance.
24
 
25
  All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
26
  parallelize provider calls without blocking the event loop.
 
1
  """Provider clients — thin, async, behind a common interface.
2
 
3
+ The provider stack:
 
4
 
5
  sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
6
  sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech, speaker anushka)
7
  sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain)
8
  local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
9
+ nvidia_nim_llm.py — Chain pattern via NimChainLLM. BRAIN_CHAIN is the
10
+ only chain: Qwen 3-Next 80B → Mistral Large 3
11
+ 675B Llama-4 Maverick 17B → Nemotron-Super 49B
12
+ (last resort).
13
+ openrouter_llm.py — OpenRouter client (cross-provider option).
14
+ groq_llm.py — Groq LPU client.
 
 
 
 
 
 
 
 
15
 
16
  All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
17
  parallelize provider calls without blocking the event loop.
backend/providers/base.py CHANGED
@@ -7,7 +7,7 @@ The interfaces are deliberately narrow:
7
  - An embeddings provider takes a list of strings and returns vectors.
8
 
9
  Concrete implementations live in sibling modules (sarvam_*, groq_*, voyage_*,
10
- openrouter_*). The orchestrator imports these by interface, never by concrete
11
  class — so swapping providers is a config change.
12
  """
13
 
 
7
  - An embeddings provider takes a list of strings and returns vectors.
8
 
9
  Concrete implementations live in sibling modules (sarvam_*, groq_*, voyage_*,
10
+ openrouter_*). Callers import these by interface, never by concrete
11
  class — so swapping providers is a config change.
12
  """
13
 
backend/providers/google_gemini_llm.py CHANGED
@@ -1,9 +1,9 @@
1
- """Google Gemini — primary LLM tier for sales_brain + orchestrator brain (KI-179).
2
 
3
- Google AI Studio's free tier on `gemini-2.0-flash` and `gemini-2.5-flash` gives
4
- 1500 req/day with native JSON mode (`responseMimeType: "application/json"`).
5
- This is best-in-class free quality for conversation and beats every model in
6
- NIM or OpenRouter free tiers.
7
 
8
  Wire-shape: REST API direct via httpx (no SDK dependency — matches the pattern
9
  of nvidia_nim_llm.py and openrouter_llm.py).
@@ -35,9 +35,8 @@ JSON mode:
35
  response_format == {"type": "json_object"} →
36
  generationConfig.responseMimeType = "application/json"
37
 
38
- The provider matches the existing `LLMProvider` interface so it slots
39
- straight into both sales_brain.py (Tier 0) and the orchestrator brain main
40
- path (Tier 0, via TieredBrainLLM wrapper).
41
  """
42
 
43
  from __future__ import annotations
@@ -57,13 +56,13 @@ from backend.providers.base import ChatMessage, LLMProvider, LLMResult
57
 
58
  GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/models"
59
  GEMINI_CACHE_URL = "https://generativelanguage.googleapis.com/v1beta/cachedContents"
60
- DEFAULT_MODEL = "gemini-2.5-flash" # KI-183 retired gemini-2.0-flash → 2.5-flash (free-tier, brain-grade — see header). NOT the -lite tier: it silently broke single-brain tool-calling (save_profile_field) and caused the live fact-find regression.
61
 
62
  # ----------------------------------------------------------------------------
63
- # KI-199 — module-level cachedContents registry.
64
  #
65
  # Keyed by `(model, sha256(system_text))` so the same base preamble shared
66
- # across sales_brain calls deduplicates onto one cache. Each value is a small
67
  # dict with the cache `name` (the server-side resource id used in subsequent
68
  # generateContent bodies as `cachedContent`) and the local-clock `expires_at`
69
  # wall-time so we can self-evict before issuing a guaranteed-miss request.
@@ -77,7 +76,7 @@ DEFAULT_MODEL = "gemini-2.5-flash" # KI-183 retired gemini-2.0-flash → 2.5-fl
77
  _CACHE_REGISTRY: dict[tuple[str, str, str], dict] = {}
78
  _CACHE_REGISTRY_LOCK = threading.Lock()
79
 
80
- # A4 (2026-05-15) — Gemini cachedContents server-side TTL ceiling. Google's
81
  # `cachedContents` resources live up to ~60min on free tier; we refresh
82
  # before that ceiling so an in-flight call never lands on an expired cache.
83
  # `CACHE_REFRESH_AGE_SEC` is the wall-clock age at which we proactively
@@ -89,11 +88,11 @@ CACHE_REFRESH_AGE_SEC = 50 * 60 # 50min — refresh BEFORE 60min server ceiling
89
  def _cache_key(model: str, system_text: str, dynamic_prefix: str = "") -> tuple[str, str, str]:
90
  """Build the registry key for a (model, system_text, dynamic_prefix) tuple.
91
 
92
- A4 (2026-05-15) Cache key collision fix: SHA256 of preamble alone is
93
- insufficient when `_dynamic_profile_block` varies per persona. The key
94
- now partitions on a separate `dynamic_prefix` hash so per-persona
95
- caches don't collide on the same static preamble hash. `dynamic_prefix`
96
- defaults to "" so existing callers retain prior behaviour.
97
 
98
  Hashing inputs rather than storing the raw string keeps the registry
99
  footprint tiny even when the preamble is multi-KB.
@@ -104,18 +103,18 @@ def _cache_key(model: str, system_text: str, dynamic_prefix: str = "") -> tuple[
104
 
105
 
106
  # ---------------------------------------------------------------------------
107
- # A4 (2026-05-15) — Normalized provider error class.
108
  #
109
  # Gemini's REST surface returns different error shapes for different failure
110
  # modes (429 rate-limit, 400 BlockedReason / SafetyRating, 404 cache not
111
- # found, 500/503 server errors). The tier wrapper needs a stable contract:
112
  # `BrainProviderError(retryable=bool)` where `retryable=True` signals the
113
  # tier should fall through to the next provider, and `retryable=False`
114
  # signals a hard error (auth / content blocked) that should surface to the
115
  # caller without burning fallback budget.
116
  # ---------------------------------------------------------------------------
117
  class BrainProviderError(RuntimeError):
118
- """Stable provider-error envelope consumed by TieredBrainLLM.
119
 
120
  Attributes:
121
  provider: short name ("gemini" / "nim" / ...)
@@ -142,7 +141,7 @@ class BrainProviderError(RuntimeError):
142
  def _classify_gemini_error(status_code: int, detail: str) -> BrainProviderError:
143
  """Map a Gemini REST response into BrainProviderError(retryable=bool).
144
 
145
- Routing rules (matches TieredBrainLLM expectations):
146
  - 429 (rate-limit / quota) → retryable (next tier)
147
  - 5xx (server errors) → retryable (next tier)
148
  - 408 / 504 (timeout) → retryable
@@ -184,9 +183,9 @@ def invalidate_cache(model: str, system_text: str, dynamic_prefix: str = "") ->
184
  alive (it will lapse on TTL), but our reference is gone so the next
185
  chat() call provisions a fresh one.
186
 
187
- A4 (2026-05-15) — `dynamic_prefix` is an optional partition arg; defaults
188
- to "" so existing callers retain prior behaviour. When supplied, only the
189
- matching (model, system_text, dynamic_prefix) entry is dropped.
190
  """
191
  key = _cache_key(model, system_text, dynamic_prefix)
192
  with _CACHE_REGISTRY_LOCK:
@@ -270,8 +269,7 @@ class GoogleGeminiLLM(LLMProvider):
270
  Re-uses an existing live cache from the module registry when the
271
  (model, system_text) pair matches AND the local `expires_at` is still
272
  in the future. Cache misses + creation failures are silent (logged at
273
- INFO) so a caching outage never breaks the main path — KI-199 brief
274
- requires fail-safe behaviour.
275
  """
276
  if not self.api_key or not system_text:
277
  return None
@@ -280,8 +278,8 @@ class GoogleGeminiLLM(LLMProvider):
280
  now = time.time()
281
  with _CACHE_REGISTRY_LOCK:
282
  entry = _CACHE_REGISTRY.get(key)
283
- # A4 (2026-05-15) — TWO-stage refresh:
284
- # (a) self-evict ~10s before LOCAL expires_at (was already here).
285
  # (b) PROACTIVELY refresh if the entry is older than
286
  # CACHE_REFRESH_AGE_SEC (50min) regardless of expires_at —
287
  # guards against server-side TTL drift on long-lived
@@ -349,8 +347,8 @@ class GoogleGeminiLLM(LLMProvider):
349
  # truth, but we shadow it locally so we self-evict before
350
  # the inevitable 4xx on an expired reference.
351
  "expires_at": now_create + ttl_seconds,
352
- # A4 (2026-05-15) — created_at lets us proactively refresh
353
- # entries that have lived past CACHE_REFRESH_AGE_SEC even
354
  # when caller set a longer TTL than Google honours.
355
  "created_at": now_create,
356
  }
@@ -410,12 +408,11 @@ class GoogleGeminiLLM(LLMProvider):
410
  # connection releases its slot on its own deadline rather than holding
411
  # past the outer wait_for cancellation.
412
  #
413
- # A4 (2026-05-15) — Timeout binding: bind the SDK/httpx read timeout
414
- # to `self.timeout - 2.0` so the underlying connection times out
415
- # ~2s BEFORE the tier-wrapper's outer wait_for cancellation. This
416
- # surfaces a clean BrainProviderError(retryable=True) instead of
417
- # asyncio.CancelledError leaking up through the cancellation chain
418
- # (which the tier wrapper has historically misclassified).
419
  read_timeout = max(2.0, self.timeout - 2.0)
420
  client_timeout = httpx.Timeout(
421
  connect=2.0,
@@ -430,9 +427,9 @@ class GoogleGeminiLLM(LLMProvider):
430
  except (asyncio.CancelledError, KeyboardInterrupt, SystemExit):
431
  raise
432
  except httpx.TimeoutException as e:
433
- # A4 (2026-05-15) — TimeoutException → retryable
434
- # BrainProviderError so the tier wrapper falls through
435
- # cleanly instead of seeing asyncio.CancelledError.
436
  raise BrainProviderError(
437
  f"Gemini timeout after {read_timeout:.1f}s (model={self.model})",
438
  provider="gemini", retryable=True, status=None,
@@ -443,7 +440,7 @@ class GoogleGeminiLLM(LLMProvider):
443
  f"Gemini transport error ({type(e).__name__}): {str(e)[:200]}",
444
  provider="gemini", retryable=True, status=None,
445
  ) from e
446
- # KI-199 — graceful fallback when a cache reference is stale.
447
  # Symptoms: 400/404 with body mentioning "cachedContent" /
448
  # "cache" / "not found". Strip the reference, re-add the inline
449
  # systemInstruction, drop the registry entry, retry once.
@@ -484,8 +481,8 @@ class GoogleGeminiLLM(LLMProvider):
484
  provider="gemini", retryable=True, status=None,
485
  ) from e
486
  if resp.status_code >= 400:
487
- # A4 (2026-05-15) — Normalize the error into BrainProviderError
488
- # so the tier wrapper sees a stable contract:
489
  # retryable=True → fall through to next tier
490
  # retryable=False → surface to caller (auth/content-block)
491
  # The original httpx.HTTPStatusError is preserved as __cause__
@@ -542,13 +539,12 @@ def get_gemini_llm(
542
  ) -> GoogleGeminiLLM:
543
  """Return a fresh GoogleGeminiLLM client.
544
 
545
- 25s timeout matches the KI-179 brief Gemini Flash typically responds in
546
- 1-3s so 25s is a generous outer bound that still bails fast on a quota /
547
- network stall.
548
 
549
  Common models:
550
- - "gemini-2.0-flash" → fast, conversational, sales_brain Tier 0
551
- - "gemini-2.5-flash" → slightly heavier reasoning, brain main Tier 0
552
  """
553
  return GoogleGeminiLLM(model=model, timeout=timeout)
554
 
 
1
+ """Google Gemini — primary LLM tier for the single-brain stack.
2
 
3
+ Google AI Studio's free tier on `gemini-2.5-flash` gives 1500 req/day
4
+ with native JSON mode (`responseMimeType: "application/json"`). This is
5
+ best-in-class free quality for conversation and beats every model in NIM
6
+ or OpenRouter free tiers.
7
 
8
  Wire-shape: REST API direct via httpx (no SDK dependency — matches the pattern
9
  of nvidia_nim_llm.py and openrouter_llm.py).
 
35
  response_format == {"type": "json_object"} →
36
  generationConfig.responseMimeType = "application/json"
37
 
38
+ The provider matches the `LLMProvider` interface so it slots straight
39
+ into the single-brain stack (single_brain.py) as the primary tier.
 
40
  """
41
 
42
  from __future__ import annotations
 
56
 
57
  GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/models"
58
  GEMINI_CACHE_URL = "https://generativelanguage.googleapis.com/v1beta/cachedContents"
59
+ DEFAULT_MODEL = "gemini-2.5-flash" # free-tier, brain-grade. NOT the -lite tier: it does not reliably support single-brain tool-calling (save_profile_field).
60
 
61
  # ----------------------------------------------------------------------------
62
+ # Module-level cachedContents registry.
63
  #
64
  # Keyed by `(model, sha256(system_text))` so the same base preamble shared
65
+ # across brain calls deduplicates onto one cache. Each value is a small
66
  # dict with the cache `name` (the server-side resource id used in subsequent
67
  # generateContent bodies as `cachedContent`) and the local-clock `expires_at`
68
  # wall-time so we can self-evict before issuing a guaranteed-miss request.
 
76
  _CACHE_REGISTRY: dict[tuple[str, str, str], dict] = {}
77
  _CACHE_REGISTRY_LOCK = threading.Lock()
78
 
79
+ # Gemini cachedContents server-side TTL ceiling. Google's
80
  # `cachedContents` resources live up to ~60min on free tier; we refresh
81
  # before that ceiling so an in-flight call never lands on an expired cache.
82
  # `CACHE_REFRESH_AGE_SEC` is the wall-clock age at which we proactively
 
88
  def _cache_key(model: str, system_text: str, dynamic_prefix: str = "") -> tuple[str, str, str]:
89
  """Build the registry key for a (model, system_text, dynamic_prefix) tuple.
90
 
91
+ The key partitions on a separate `dynamic_prefix` hash (in addition to
92
+ the preamble SHA256) so per-persona caches don't collide on the same
93
+ static preamble hash when `_dynamic_profile_block` varies per persona.
94
+ `dynamic_prefix` defaults to "" so callers that don't pass it get the
95
+ preamble-only key.
96
 
97
  Hashing inputs rather than storing the raw string keeps the registry
98
  footprint tiny even when the preamble is multi-KB.
 
103
 
104
 
105
  # ---------------------------------------------------------------------------
106
+ # Normalized provider error class.
107
  #
108
  # Gemini's REST surface returns different error shapes for different failure
109
  # modes (429 rate-limit, 400 BlockedReason / SafetyRating, 404 cache not
110
+ # found, 500/503 server errors). The brain tier needs a stable contract:
111
  # `BrainProviderError(retryable=bool)` where `retryable=True` signals the
112
  # tier should fall through to the next provider, and `retryable=False`
113
  # signals a hard error (auth / content blocked) that should surface to the
114
  # caller without burning fallback budget.
115
  # ---------------------------------------------------------------------------
116
  class BrainProviderError(RuntimeError):
117
+ """Stable provider-error envelope consumed by the brain tier.
118
 
119
  Attributes:
120
  provider: short name ("gemini" / "nim" / ...)
 
141
  def _classify_gemini_error(status_code: int, detail: str) -> BrainProviderError:
142
  """Map a Gemini REST response into BrainProviderError(retryable=bool).
143
 
144
+ Routing rules (the brain tier's expectations):
145
  - 429 (rate-limit / quota) → retryable (next tier)
146
  - 5xx (server errors) → retryable (next tier)
147
  - 408 / 504 (timeout) → retryable
 
183
  alive (it will lapse on TTL), but our reference is gone so the next
184
  chat() call provisions a fresh one.
185
 
186
+ `dynamic_prefix` is an optional partition arg; defaults to "". When
187
+ supplied, only the matching (model, system_text, dynamic_prefix)
188
+ entry is dropped.
189
  """
190
  key = _cache_key(model, system_text, dynamic_prefix)
191
  with _CACHE_REGISTRY_LOCK:
 
269
  Re-uses an existing live cache from the module registry when the
270
  (model, system_text) pair matches AND the local `expires_at` is still
271
  in the future. Cache misses + creation failures are silent (logged at
272
+ INFO) so a caching outage never breaks the main path (fail-safe).
 
273
  """
274
  if not self.api_key or not system_text:
275
  return None
 
278
  now = time.time()
279
  with _CACHE_REGISTRY_LOCK:
280
  entry = _CACHE_REGISTRY.get(key)
281
+ # TWO-stage refresh:
282
+ # (a) self-evict ~10s before LOCAL expires_at.
283
  # (b) PROACTIVELY refresh if the entry is older than
284
  # CACHE_REFRESH_AGE_SEC (50min) regardless of expires_at —
285
  # guards against server-side TTL drift on long-lived
 
347
  # truth, but we shadow it locally so we self-evict before
348
  # the inevitable 4xx on an expired reference.
349
  "expires_at": now_create + ttl_seconds,
350
+ # created_at lets us proactively refresh entries that have
351
+ # lived past CACHE_REFRESH_AGE_SEC even
352
  # when caller set a longer TTL than Google honours.
353
  "created_at": now_create,
354
  }
 
408
  # connection releases its slot on its own deadline rather than holding
409
  # past the outer wait_for cancellation.
410
  #
411
+ # Timeout binding: bind the httpx read timeout to
412
+ # `self.timeout - 2.0` so the underlying connection times out ~2s
413
+ # BEFORE the outer wait_for cancellation. This surfaces a clean
414
+ # BrainProviderError(retryable=True) instead of an
415
+ # asyncio.CancelledError leaking up through the cancellation chain.
 
416
  read_timeout = max(2.0, self.timeout - 2.0)
417
  client_timeout = httpx.Timeout(
418
  connect=2.0,
 
427
  except (asyncio.CancelledError, KeyboardInterrupt, SystemExit):
428
  raise
429
  except httpx.TimeoutException as e:
430
+ # TimeoutException → retryable BrainProviderError so the
431
+ # tier falls through cleanly instead of seeing
432
+ # asyncio.CancelledError.
433
  raise BrainProviderError(
434
  f"Gemini timeout after {read_timeout:.1f}s (model={self.model})",
435
  provider="gemini", retryable=True, status=None,
 
440
  f"Gemini transport error ({type(e).__name__}): {str(e)[:200]}",
441
  provider="gemini", retryable=True, status=None,
442
  ) from e
443
+ # Graceful fallback when a cache reference is stale.
444
  # Symptoms: 400/404 with body mentioning "cachedContent" /
445
  # "cache" / "not found". Strip the reference, re-add the inline
446
  # systemInstruction, drop the registry entry, retry once.
 
481
  provider="gemini", retryable=True, status=None,
482
  ) from e
483
  if resp.status_code >= 400:
484
+ # Normalize the error into BrainProviderError so the tier
485
+ # sees a stable contract:
486
  # retryable=True → fall through to next tier
487
  # retryable=False → surface to caller (auth/content-block)
488
  # The original httpx.HTTPStatusError is preserved as __cause__
 
539
  ) -> GoogleGeminiLLM:
540
  """Return a fresh GoogleGeminiLLM client.
541
 
542
+ Gemini Flash typically responds in 1-3s so the 25s default timeout is
543
+ a generous outer bound that still bails fast on a quota / network
544
+ stall.
545
 
546
  Common models:
547
+ - "gemini-2.5-flash" → conversational brain tier (default)
 
548
  """
549
  return GoogleGeminiLLM(model=model, timeout=timeout)
550
 
backend/providers/groq_llm.py CHANGED
@@ -104,8 +104,8 @@ class GroqLLM(LLMProvider):
104
  continue
105
  resp.raise_for_status()
106
  break
107
- # KI-085 (2026-05-15) — stamp credits_remaining from Groq's
108
- # x-ratelimit-* headers BEFORE we drop the response. Groq is
109
  # the highest-fidelity provider on this front: every successful
110
  # call returns the daily-tokens remaining, so we get a
111
  # continuously-updated election signal at zero extra cost.
 
104
  continue
105
  resp.raise_for_status()
106
  break
107
+ # Stamp credits_remaining from Groq's x-ratelimit-* headers
108
+ # BEFORE we drop the response. Groq is
109
  # the highest-fidelity provider on this front: every successful
110
  # call returns the daily-tokens remaining, so we get a
111
  # continuously-updated election signal at zero extra cost.
backend/providers/local_embeddings.py CHANGED
@@ -7,8 +7,8 @@ Default model: BAAI/bge-small-en-v1.5 — 384-dim, ~110MB, top-of-class on
7
  MTEB English benchmark for its size. Faster than 1024-dim cloud models on
8
  CPU and trivially small on GPU.
9
 
10
- The interface matches EmbeddingsProvider exactly, so the orchestrator and
11
- RAG pipeline don't change — just the import in ingest/retrieve.
12
  """
13
 
14
  from __future__ import annotations
@@ -73,10 +73,10 @@ class LocalEmbeddings(EmbeddingsProvider):
73
  texts = [f"Represent this sentence for searching relevant passages: {t}" for t in texts]
74
  # Batch size scales by device: MPS / CUDA throughput benefits from
75
  # bigger batches; CPU prefers smaller to avoid memory pressure on M1.
76
- # KI-125 (2026-05-15): bumped MPS 64→128 to halve the number of GPU
77
- # kernel launches during bulk re-ingest. 800-token chunks at 128 ≈
78
- # 100 MB per batch — still well within Mac M-series unified memory.
79
- # CPU stays at 32 to keep peak RSS bounded on machines without a GPU.
80
  batch = 128 if self.device in ("mps", "cuda") else 32
81
  vectors = self.model.encode(
82
  texts,
 
7
  MTEB English benchmark for its size. Faster than 1024-dim cloud models on
8
  CPU and trivially small on GPU.
9
 
10
+ The interface matches EmbeddingsProvider exactly, so the RAG pipeline
11
+ doesn't change — only the import in ingest/retrieve.
12
  """
13
 
14
  from __future__ import annotations
 
73
  texts = [f"Represent this sentence for searching relevant passages: {t}" for t in texts]
74
  # Batch size scales by device: MPS / CUDA throughput benefits from
75
  # bigger batches; CPU prefers smaller to avoid memory pressure on M1.
76
+ # MPS/CUDA use batch 128 to minimise GPU kernel launches during
77
+ # bulk re-ingest (800-token chunks at 128 ≈ 100 MB per batch, well
78
+ # within Mac M-series unified memory). CPU stays at 32 to keep peak
79
+ # RSS bounded on machines without a GPU.
80
  batch = 128 if self.device in ("mps", "cuda") else 32
81
  vectors = self.model.encode(
82
  texts,
backend/providers/nvidia_nim_llm.py CHANGED
@@ -5,20 +5,18 @@ POST https://integrate.api.nvidia.com/v1/chat/completions.
5
 
6
  Why NIM:
7
  - Frontier open-weights models hosted free (no card, no daily cap, 40 req/min)
8
- - Single provider replaces OpenRouter + DeepSeek-direct + Cerebras + Groq
9
  - Same Bearer-auth + OpenAI request shape — drop-in retry/backoff
10
 
11
- Role (collapsed 2026-05-15 — single brain only):
12
  - Brain (every reasoning turn): one elected candidate from BRAIN_CHAIN.
13
  The chain is ordered Qwen 3-Next → Mistral Large 3 → Llama-4 Maverick
14
- → Nemotron-Super (last resort). Election (KI-080) picks the actually
15
  healthy/fastest candidate per turn from background probe data.
16
 
17
- The FAST_BRAIN_CHAIN + JUDGE_CHAIN abstractions were removed in the
18
- three-chain collapse: voice fact-find now reuses BRAIN_CHAIN with its own
19
- budget, and the LLM-judge gate was retired in favour of in-process
20
- faithfulness checks. Sarvam stays for voice STT/TTS + Hindi/Hinglish/
21
- vernacular translation.
22
  """
23
 
24
  from __future__ import annotations
@@ -74,18 +72,12 @@ async def _append_usage(record: dict) -> None:
74
  pass
75
 
76
 
77
- # KI-088 (2026-05-15) — Global outbound NIM concurrency cap.
78
  #
79
- # Live probe at commit 6a47549 measured 20% brain-turn success despite
80
- # KI-080 election + KI-085 credit gating + KI-087 NIM-first + KI-079
81
- # escalation all live. The architecture is correct; the binding constraint
82
- # is NIM's free-tier per-key concurrency (~3-5 slots).
83
- #
84
- # We have 5 sources of NIM traffic that all stack on the same key with no
85
- # global throttle: probe loop (6-slot parallel burst every 300s), admin
86
- # tab polling (every 30s), per-user chat turn (1-2 calls), concurrent
87
- # users (N × 1-2), and the inner 4-attempt retry loop (holds a slot up
88
- # to 15s on 429/5xx backoff).
89
  #
90
  # 6+ in-flight → queue → 15-25s response times → bot's 12s outer cap
91
  # fires → user sees fallback.
@@ -99,16 +91,11 @@ async def _append_usage(record: dict) -> None:
99
  _NIM_OUTBOUND_SEMAPHORE = asyncio.Semaphore(2)
100
 
101
 
102
- # A4 (2026-05-15) — Module-level httpx.AsyncClient singleton.
103
- #
104
- # Pre-A4: every NvidiaNimLLM.chat() call did `async with httpx.AsyncClient(...)
105
- # as client: ...`, which constructed a fresh connection pool per request and
106
- # tore it down on exit. Under brain/judge concurrency this churned TCP
107
- # connections, dropped TLS sessions, and (per audit) leaked pool slots when
108
- # an outer wait_for cancellation interrupted the async-with __aexit__ before
109
- # the underlying transport finalised.
110
  #
111
- # Switch to a single shared client with explicit Limits so:
 
 
112
  # - max_connections=10 caps total outbound (NIM + probes share)
113
  # - max_keepalive_connections=5 keeps a warm pool for the hot path
114
  # The client is constructed lazily (first .chat() call) so cold imports
@@ -142,12 +129,6 @@ def _get_shared_nim_client() -> httpx.AsyncClient:
142
 
143
 
144
  NVIDIA_NIM_BASE_URL = "https://integrate.api.nvidia.com/v1"
145
- # 2026-05-14 brain swap (D-022): NIM's DeepSeek-V4 + Meta Llama inference pools
146
- # are repeatedly timing out (15-120s on chat completions, no response). Qwen
147
- # pool is consistently fast (2s response, clean structured output). Mistral
148
- # Large 3 (Reddit benchmark: 4.3s, works on free tier) is the cross-family
149
- # judge replacing the timing-out Llama-4 Maverick.
150
- #
151
  # Heavy brain (complex queries — comparison, recommendation, synthesis):
152
  # Qwen 3-Next 80B — 80B / 3B active MoE, frontier multilingual, very fast.
153
  NIM_BRAIN_MODEL = "qwen/qwen3-next-80b-a3b-instruct"
@@ -156,9 +137,9 @@ NIM_BRAIN_MODEL = "qwen/qwen3-next-80b-a3b-instruct"
156
  # different intents to the same pool. Tiered routing kept for forward-compat.
157
  NIM_FAST_BRAIN_MODEL = "qwen/qwen3-next-80b-a3b-instruct"
158
  # Judge: Mistral Large 3 — 675B dense, MIT license. Different family from
159
- # Qwen brain (Mistral vs Alibaba) so the judge sees the brain's output from
160
- # a genuinely different decision surface ("the brain doesn't mark its own
161
- # homework"). Reddit benchmark confirms ~4.3s response on NIM free tier.
162
  NIM_JUDGE_MODEL = "mistralai/mistral-large-3-675b-instruct-2512"
163
 
164
 
@@ -196,8 +177,8 @@ class NvidiaNimLLM(LLMProvider):
196
  "max_tokens": max_tokens,
197
  }
198
  if response_format:
199
- # A4 (2026-05-15) — NIM forwards OpenAI-shape response_format to
200
- # the upstream model. For models that support `nvext.guided_json`
201
  # (NIM's stricter constrained-decoding surface), surface that
202
  # too when the caller passes a JSON schema (response_format
203
  # with `json_schema`). The plain `{"type": "json_object"}` flag
@@ -227,31 +208,22 @@ class NvidiaNimLLM(LLMProvider):
227
  write=2.0,
228
  pool=2.0,
229
  )
230
- # KI-088 (2026-05-15) drop the inner 4-attempt exponential
231
- # backoff retry. The previous loop held a NIM concurrency slot
232
- # for up to ~15s on 429/5xx while sleeping between attempts,
233
- # which directly contributed to the queueing that the outer
234
- # semaphore is now sized to prevent.
235
- #
236
- # Retry was a pre-KI-080 vestige that predated the chain
237
- # election architecture. With KI-080 in place, NimChainLLM
238
- # already handles 429/5xx failover by:
239
- # (1) catching the exception in _try(),
240
- # (2) calling report_failure() so the elector demotes the
241
- # slot-blocked candidate for ~30s, and
242
- # (3) falling through to the elected backup (cross-provider
243
- # by preference) within the same turn.
244
- # KI-079 then provides one more bite via BRAIN_CHAIN if both
245
- # elected models fail. Per-call retries inside this method
246
- # would only re-queue against the same slot-starved pool,
247
- # multiplying the queueing problem the semaphore fixes.
248
  #
249
  # The semaphore wraps ONLY the HTTP round-trip — not response
250
  # parsing or usage logging — so we cap concurrent network
251
  # traffic without serialising the rest of the pipeline.
252
- # A4 (2026-05-15) — Use the module-level shared httpx.AsyncClient
253
- # (singleton with bounded connection pool) instead of constructing
254
- # a fresh client per request. Per-request timeout is applied at
255
  # .post() call time so the shared pool serves both long-timeout
256
  # chat() and short-timeout probe() calls without contention.
257
  client = _get_shared_nim_client()
@@ -295,40 +267,32 @@ class NvidiaNimLLM(LLMProvider):
295
  # cross-family-grading invariant survives any failover.
296
 
297
  BRAIN_CHAIN = [
298
- # KI-155 (2026-05-15) NIM-ONLY ENFORCEMENT. Cross-provider (Groq /
299
- # OpenRouter) fallbacks REMOVED. Groq's Llama-3.3-70B failed the `<FF>`
300
- # trailer contract during a fact-find probe and silently flipped the
301
- # entire pipeline to scripted prompts. Chain is now strictly NIM. Each
302
- # NIM candidate's `<FF>` adherence has been verified or is structurally
303
- # safer (different family / smaller routing surface). Pruned candidates
304
- # that have been "down" for 48+ consecutive probes (qwen3.5-122b,
305
- # gpt-oss-120b, deepseek-v4-pro) so the election pool only contains
306
- # demonstrably-healthy NIM models.
307
- # KI-175 (2026-05-15) — Nemotron demoted to LAST RESORT per operator
308
- # preference: nemotron is the weakest practical NIM model and should
309
- # only serve when every higher-quality model is simultaneously down.
310
- # Primary: Qwen 3-Next 80B — 5/5 recent probes ok, clean JSON, multilingual
311
  "qwen/qwen3-next-80b-a3b-instruct",
312
- # 1st fallback: Mistral Large 3 675B — recent 3/3 probes ok, different
313
- # family (mistral). Strongest fallback.
314
  "mistralai/mistral-large-3-675b-instruct-2512",
315
- # 2nd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different
316
- # family (meta), keeps the chain alive through Qwen+Mistral outage.
317
  "meta/llama-4-maverick-17b-128e-instruct",
318
- # 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — barely usable for
319
- # worst-case scenarios. Only reached when Qwen + Mistral + Maverick
320
- # are ALL down simultaneously. Kept in the pool for fail-loud > down.
321
  "nvidia/llama-3.3-nemotron-super-49b-v1.5",
322
  ]
323
 
324
- # Three-chain collapse (2026-05-15): FAST_BRAIN_CHAIN + JUDGE_CHAIN removed.
325
- # Every reasoning role now resolves to BRAIN_CHAIN above. Voice fact-find
326
- # reuses get_brain_llm() with a tighter outer wait_for cap; the LLM-judge
327
- # gate (Gate 4 / Hinglish drift) was retired in favour of in-process checks.
328
 
329
 
330
  def _classify_error(e: BaseException) -> str:
331
- """KI-084 — classify a chat exception into a stable string passed to
332
  llm_health.report_failure(). The string drives degradation duration:
333
  rate-limit failures (HTTP 429) get a 1h sin-bin; everything else 30s.
334
 
@@ -338,7 +302,7 @@ def _classify_error(e: BaseException) -> str:
338
  carries `.response.status_code == 429` we tag it `"Status429"`; for
339
  other HTTP statuses we surface e.g. `"HTTPStatusError:503"`; for
340
  non-HTTP exceptions we keep the class name (`TimeoutException`,
341
- `ReadTimeout`, etc.) — matching the pre-KI-084 contract.
342
  """
343
  cls = type(e).__name__
344
  try:
@@ -354,32 +318,22 @@ def _classify_error(e: BaseException) -> str:
354
 
355
 
356
  class NimChainLLM(LLMProvider):
357
- """KI-080 sticky-primary router across multiple candidate models.
358
-
359
- Architectural shift (KI-080, 2026-05-15):
360
- PRE-KI-080: chat() iterated the full chain sequentially every turn.
361
- Under NIM per-key concurrency throttling, the first 5 NIM-hosted
362
- candidates queued together inside ONE turn and burned the 22s
363
- budget before any cross-provider fallback was ever reached. The
364
- 10-turn live probe at commit 078ff45 showed 7/10 fact-find turns
365
- timing out at exactly 26.6s.
366
-
367
- POST-KI-080: a background probe loop ELECTS a primary + cross-
368
- provider backup per chain based on real probe latencies. chat()
369
- calls the elected primary ONCE per turn (1 LLM call). On failure,
370
- we demote the primary for ~30s and fall through to the elected
371
- backup (still 2 LLM calls max). The chain list is now the
372
- CANDIDATE POOL for election, not a per-call sequence.
373
-
374
- Worst case per turn: 2 LLM calls (was 5-6). Plus a final filter_chain
375
- refresh path is kept for the rare double-failure edge case so the
376
- pre-KI-080 graceful-degradation behaviour is preserved.
377
 
378
  `name` after a successful call reflects which model actually answered
379
- so downstream callers (orchestrator brain_used tag, eval logs) can
380
- audit which candidate produced the output. The KI-079 escalation in
381
- fact_find_brain still applies if primary+backup both fail inside
382
- one turn, fact_find_brain gets one more bite via BRAIN_CHAIN.
383
  """
384
  def __init__(self, chain: list[str], api_key: Optional[str] = None,
385
  timeout: float = 30.0, per_model_attempts: int = 1,
@@ -389,15 +343,13 @@ class NimChainLLM(LLMProvider):
389
  self.chain = chain
390
  self.api_key = api_key or getattr(settings, "NVIDIA_NIM_API_KEY", "")
391
  self.timeout = timeout
392
- # KI-021 (2026-05-14) — cumulative chain budget. If the chain has 8
393
- # fallbacks at 30s each, worst-case wall-clock is 4 min per turn — that
394
- # produced the p99 58s+ tail in the 100-persona audit. Default to a
395
- # cumulative ceiling of ~2.5× the per-link timeout so a healthy primary
396
- # always completes, but a cascading-failure chain bails fast.
397
- #
398
- # KI-080 with election we only do 1-2 LLM calls per turn, so the
399
- # budget is rarely the binding constraint anymore. Kept for the
400
- # final filter_chain-refresh fallback path + cold-start edge cases.
401
  self.total_budget_s = total_budget_s if total_budget_s is not None else max(timeout * 2.5, 30.0)
402
  self.per_model_attempts = per_model_attempts
403
  self.role = role # 'brain' | 'unknown' — flows into usage log
@@ -424,17 +376,15 @@ class NimChainLLM(LLMProvider):
424
  def _get_worker_for(self, model_id: str, timeout: float) -> LLMProvider:
425
  """Dispatch a chain entry to the NIM provider client.
426
 
427
- KI-155 (2026-05-15) NIM-ONLY ENFORCEMENT. Cross-provider
428
- (`openrouter:` / `groq:`) prefixes are explicitly rejected here even
429
- though the chains no longer contain them. This is a defense-in-depth
430
- short-circuit: if anyone (admin override, monkeypatch, future drift)
431
- injects a non-NIM candidate into a chain, the dispatcher raises
432
- rather than silently routing to a provider that has demonstrated
433
- contract drift (Groq Llama-3.3-70B / `<FF>` trailer failure).
434
-
435
- KI-085 — passes `chain_name=self._chain_name` so the credit
436
- trackers in the provider clients route their response-header
437
- signals to the right chain state.
438
  """
439
  if model_id.startswith(("openrouter:", "groq:", "or:")):
440
  raise RuntimeError(
@@ -454,17 +404,16 @@ class NimChainLLM(LLMProvider):
454
  'deepseek', 'moonshot', 'minimax', 'nvidia', 'nemotron', 'llama3',
455
  'google', 'unknown'.
456
 
457
- A4 (2026-05-15) — EVERY model in BRAIN_CHAIN must map to a known
458
- family. The `assert_family_coverage()` helper walks the chain on
459
- demand (from tests / admin / probe loop) and raises if any
460
- candidate falls through to "unknown".
461
  """
462
  m = (model_id or "").lower()
463
- # KI-220 — recognise Google's Gemini family BEFORE prefix-stripping so
464
- # ids like "google/gemini-2.5-flash" and bare "gemini-..." both land in
465
- # the "google" bucket. Without this branch the family lookup fell
466
- # through to "unknown", which meant the judge-vs-brain cross-family
467
- # invariant couldn't exclude Gemini brains from a Gemini judge.
468
  if "gemini" in m or m.startswith("google/"):
469
  return "google"
470
  # Strip provider prefix first so 'groq:llama-3.3-70b' → 'meta' (it IS Meta Llama)
@@ -534,8 +483,8 @@ class NimChainLLM(LLMProvider):
534
  exclude_models: Optional[list[str]] = None,
535
  exclude_families: Optional[list[str]] = None,
536
  ) -> LLMResult:
537
- """KI-080 — sticky primary election. Call ONE elected candidate per
538
- turn, with at most ONE real-time fallback to the elected backup if
539
  primary fails. The chain list is the candidate POOL for election —
540
  NOT a per-call sequence.
541
 
@@ -546,8 +495,8 @@ class NimChainLLM(LLMProvider):
546
  the chain in order.
547
  Final fallback (both elected models fail): trigger a synchronous
548
  probe refresh + try whatever filter_chain now offers, walking the
549
- chain in order. This path is the pre-KI-080 graceful-degradation
550
- safety net for the double-failure edge case.
551
  """
552
  from backend import llm_health
553
 
@@ -650,8 +599,8 @@ class NimChainLLM(LLMProvider):
650
  llm_health.report_success(self._chain_name, model, latency_ms)
651
  except Exception:
652
  pass
653
- # KI-085 (2026-05-15) — NIM has no clean rate-limit header, so
654
- # we maintain a local 60s rate-meter for every NIM-prefixed
655
  # candidate (i.e. anything without an openrouter:/groq: prefix).
656
  # Groq + OpenRouter stamp credits from response headers inside
657
  # their own .chat() methods.
@@ -680,7 +629,7 @@ class NimChainLLM(LLMProvider):
680
  if res is not None:
681
  return res
682
 
683
- # --- Backup attempt (KI-080 single real-time fallback) ---------------
684
  if elected_backup and elected_backup != elected_primary:
685
  res = await _try(elected_backup)
686
  if res is not None:
@@ -688,9 +637,9 @@ class NimChainLLM(LLMProvider):
688
 
689
  # --- Both elected failed — final safety net --------------------------
690
  # Trigger ONE synchronous probe refresh; whatever the refreshed
691
- # filter_chain offers, walk it in order and try anything we haven't
692
- # touched this turn. This is the pre-KI-080 graceful-degradation
693
- # behaviour preserved for the (rare) double-failure case.
694
  try:
695
  # KI-099 — bound the probe-refresh cost. On a hot user-facing turn, a
696
  # 60-80s probe walk through ~10 candidates is unacceptable. If we've
@@ -793,19 +742,15 @@ class NimChainLLM(LLMProvider):
793
  ) from last_err
794
 
795
 
796
- # KI-025 (2026-05-14) provider load-balancing.
797
- # DEPRECATED 2026-05-15 by KI-080: primary election supersedes the 50/50
798
- # rotation. The probe loop now picks the actually-faster candidate
799
- # DYNAMICALLY (real probe latencies, not a coin flip), and the elected
800
- # backup is chosen with explicit cross-provider preference — both signals
801
- # the rotation was approximating heuristically. Kept around (not deleted)
802
- # because the regression suite still pins its statistical behaviour. The
803
- # get_*_llm factories no longer call it.
804
  import random as _random
805
 
806
 
807
  def _balanced_brain_chain(base: list[str], *, groq_first_probability: float = 0.5) -> list[str]:
808
- """KI-025 — provider load-balancing (DEPRECATED 2026-05-15 by KI-080).
809
 
810
  With `groq_first_probability` (default 50%), hoist the Groq Llama entry
811
  to the head of the chain so it serves as the primary instead of the
@@ -813,17 +758,14 @@ def _balanced_brain_chain(base: list[str], *, groq_first_probability: float = 0.
813
  fallback order — Groq calls that fail (rare; LPU is very reliable)
814
  still get the full NIM fallback chain.
815
 
816
- SUPERSESSION NOTE (KI-080): the elector in `backend.llm_health` now
817
- picks the actually-faster candidate dynamically from background probe
818
- data, so this static-coin-flip rotation is no longer wired into the
819
- chat hot path. Kept exported because:
820
- (a) regression tests still pin its statistical behaviour, and
821
- (b) it remains a useful pure-function for ops / sim / overrides
822
- (e.g. wanting to force a non-elected order in a debug script).
823
-
824
- Uses per-call `random.random()` so concurrent async workers (the
825
- 100-persona audit's 4 workers, the parallel 96-Q eval's 6 workers) each
826
- flip independently — no shared mutable cycle state, no GIL races, fair
827
  distribution in aggregate. `groq_first_probability` is overrideable
828
  primarily for testing."""
829
  if _random.random() >= groq_first_probability:
@@ -837,18 +779,16 @@ def _balanced_brain_chain(base: list[str], *, groq_first_probability: float = 0.
837
 
838
 
839
  def assert_family_coverage() -> dict[str, str]:
840
- """A4 (2026-05-15) — Verify every model in BRAIN_CHAIN maps to a known
841
- family (i.e. NOT 'unknown'). Returns a {model: family} dict on success;
842
- raises RuntimeError listing any 'unknown'-mapped models on failure.
843
 
844
- KI-175 — also verifies nemotron is the TAIL (last entry) of BRAIN_CHAIN
845
- so the last-resort ordering invariant survives any chain edit.
846
 
847
- Three-chain collapse (2026-05-15): FAST_BRAIN_CHAIN + JUDGE_CHAIN have
848
- been removed, so this helper now walks BRAIN_CHAIN only.
849
-
850
- Call from admin diagnostics or tests; not invoked on import to keep
851
- cold-start cheap. Safe to run from a probe loop tick.
852
  """
853
  coverage: dict[str, str] = {}
854
  unknown: list[str] = []
@@ -877,25 +817,19 @@ def assert_family_coverage() -> dict[str, str]:
877
 
878
 
879
  def get_brain_llm() -> NimChainLLM:
880
- """Heavy brain — KI-080 sticky-primary election over BRAIN_CHAIN.
881
-
882
- Pre-KI-080: per-call _balanced_brain_chain rotation between NIM Qwen
883
- and Groq Llama (KI-025 50/50 heuristic) → 5-6 LLM calls per turn
884
- under degraded conditions.
885
 
886
- Post-KI-080: the background probe loop in backend.llm_health elects
887
- the actually-fastest candidate dynamically (probe-driven, not coin-
888
- flipped) and a cross-provider backup. chat() calls 1-2 candidates max
889
- per turn. The rotation is preserved as a deprecated pure-function in
890
- case overrides need it, but the factory no longer wires it in.
891
 
892
- KI-021 — per-link 12s (KI-080 ELECTED_CALL_TIMEOUT_S), total chain
893
- budget 35s (only binding for the final filter_chain-refresh safety net).
894
  """
895
  return NimChainLLM(chain=BRAIN_CHAIN, timeout=20.0,
896
  role="brain", total_budget_s=35.0)
897
 
898
 
899
- # Three-chain collapse (2026-05-15): get_fast_brain_llm + get_judge_llm
900
- # factories removed. Any straggler caller should import get_brain_llm()
901
- # directly. The FAST_BRAIN_CHAIN + JUDGE_CHAIN constants are also gone.
 
5
 
6
  Why NIM:
7
  - Frontier open-weights models hosted free (no card, no daily cap, 40 req/min)
8
+ - Single provider for the reasoning stack
9
  - Same Bearer-auth + OpenAI request shape — drop-in retry/backoff
10
 
11
+ Role:
12
  - Brain (every reasoning turn): one elected candidate from BRAIN_CHAIN.
13
  The chain is ordered Qwen 3-Next → Mistral Large 3 → Llama-4 Maverick
14
+ → Nemotron-Super (last resort). Election picks the actually
15
  healthy/fastest candidate per turn from background probe data.
16
 
17
+ BRAIN_CHAIN is the only chain: voice fact-find reuses it with its own
18
+ budget, and faithfulness is enforced by in-process checks. Sarvam stays
19
+ for voice STT/TTS + Hindi/Hinglish/vernacular translation.
 
 
20
  """
21
 
22
  from __future__ import annotations
 
72
  pass
73
 
74
 
75
+ # Global outbound NIM concurrency cap.
76
  #
77
+ # The binding constraint is NIM's free-tier per-key concurrency (~3-5
78
+ # slots). Multiple sources of NIM traffic stack on the same key with no
79
+ # global throttle: the probe loop, admin tab polling, per-user chat turns
80
+ # (1-2 calls), concurrent users (N × 1-2), and retry backoff.
 
 
 
 
 
 
81
  #
82
  # 6+ in-flight → queue → 15-25s response times → bot's 12s outer cap
83
  # fires → user sees fallback.
 
91
  _NIM_OUTBOUND_SEMAPHORE = asyncio.Semaphore(2)
92
 
93
 
94
+ # Module-level httpx.AsyncClient singleton.
 
 
 
 
 
 
 
95
  #
96
+ # A single shared client (rather than a fresh pool per chat() call) avoids
97
+ # churning TCP connections / dropping TLS sessions / leaking pool slots
98
+ # under concurrency. Explicit Limits:
99
  # - max_connections=10 caps total outbound (NIM + probes share)
100
  # - max_keepalive_connections=5 keeps a warm pool for the hot path
101
  # The client is constructed lazily (first .chat() call) so cold imports
 
129
 
130
 
131
  NVIDIA_NIM_BASE_URL = "https://integrate.api.nvidia.com/v1"
 
 
 
 
 
 
132
  # Heavy brain (complex queries — comparison, recommendation, synthesis):
133
  # Qwen 3-Next 80B — 80B / 3B active MoE, frontier multilingual, very fast.
134
  NIM_BRAIN_MODEL = "qwen/qwen3-next-80b-a3b-instruct"
 
137
  # different intents to the same pool. Tiered routing kept for forward-compat.
138
  NIM_FAST_BRAIN_MODEL = "qwen/qwen3-next-80b-a3b-instruct"
139
  # Judge: Mistral Large 3 — 675B dense, MIT license. Different family from
140
+ # the Qwen brain (Mistral vs Alibaba) so the judge sees the brain's output
141
+ # from a genuinely different decision surface ("the brain doesn't mark its
142
+ # own homework").
143
  NIM_JUDGE_MODEL = "mistralai/mistral-large-3-675b-instruct-2512"
144
 
145
 
 
177
  "max_tokens": max_tokens,
178
  }
179
  if response_format:
180
+ # NIM forwards OpenAI-shape response_format to the upstream
181
+ # model. For models that support `nvext.guided_json`
182
  # (NIM's stricter constrained-decoding surface), surface that
183
  # too when the caller passes a JSON schema (response_format
184
  # with `json_schema`). The plain `{"type": "json_object"}` flag
 
208
  write=2.0,
209
  pool=2.0,
210
  )
211
+ # No inner retry loop. 429/5xx failover is handled by NimChainLLM:
212
+ # (1) the exception is caught in _try(),
213
+ # (2) report_failure() demotes the slot-blocked candidate for
214
+ # ~30s, and
215
+ # (3) the elected backup (cross-provider by preference) is tried
216
+ # within the same turn.
217
+ # If both elected models fail, BRAIN_CHAIN provides one more bite.
218
+ # Per-call retries inside this method would only re-queue against
219
+ # the same slot-starved pool, multiplying the queueing the
220
+ # semaphore prevents.
 
 
 
 
 
 
 
 
221
  #
222
  # The semaphore wraps ONLY the HTTP round-trip — not response
223
  # parsing or usage logging — so we cap concurrent network
224
  # traffic without serialising the rest of the pipeline.
225
+ # Use the module-level shared httpx.AsyncClient (singleton with a
226
+ # bounded connection pool). Per-request timeout is applied at
 
227
  # .post() call time so the shared pool serves both long-timeout
228
  # chat() and short-timeout probe() calls without contention.
229
  client = _get_shared_nim_client()
 
267
  # cross-family-grading invariant survives any failover.
268
 
269
  BRAIN_CHAIN = [
270
+ # NIM-only: the chain contains only NIM-hosted candidates whose `<FF>`
271
+ # trailer adherence is verified or structurally safer (different
272
+ # family / smaller routing surface). Nemotron is LAST RESORT — it is
273
+ # the weakest practical NIM model and only serves when every
274
+ # higher-quality model is simultaneously down.
275
+ # Primary: Qwen 3-Next 80B clean JSON, multilingual.
 
 
 
 
 
 
 
276
  "qwen/qwen3-next-80b-a3b-instruct",
277
+ # 1st fallback: Mistral Large 3 675B — different family (mistral).
278
+ # Strongest fallback.
279
  "mistralai/mistral-large-3-675b-instruct-2512",
280
+ # 2nd fallback: Meta Llama-4 Maverick 17B — different family (meta),
281
+ # keeps the chain alive through a Qwen+Mistral outage.
282
  "meta/llama-4-maverick-17b-128e-instruct",
283
+ # 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — only reached when
284
+ # Qwen + Mistral + Maverick are ALL down simultaneously. Kept in the
285
+ # pool for fail-loud > down.
286
  "nvidia/llama-3.3-nemotron-super-49b-v1.5",
287
  ]
288
 
289
+ # BRAIN_CHAIN is the only chain every reasoning role resolves to it.
290
+ # Voice fact-find reuses get_brain_llm() with a tighter outer wait_for
291
+ # cap; faithfulness is enforced by in-process checks.
 
292
 
293
 
294
  def _classify_error(e: BaseException) -> str:
295
+ """Classify a chat exception into a stable string passed to
296
  llm_health.report_failure(). The string drives degradation duration:
297
  rate-limit failures (HTTP 429) get a 1h sin-bin; everything else 30s.
298
 
 
302
  carries `.response.status_code == 429` we tag it `"Status429"`; for
303
  other HTTP statuses we surface e.g. `"HTTPStatusError:503"`; for
304
  non-HTTP exceptions we keep the class name (`TimeoutException`,
305
+ `ReadTimeout`, etc.).
306
  """
307
  cls = type(e).__name__
308
  try:
 
318
 
319
 
320
  class NimChainLLM(LLMProvider):
321
+ """Sticky-primary router across multiple candidate models.
322
+
323
+ A background probe loop ELECTS a primary + backup per chain based on
324
+ real probe latencies. chat() calls the elected primary ONCE per turn
325
+ (1 LLM call). On failure, it demotes the primary for ~30s and falls
326
+ through to the elected backup (2 LLM calls max). The chain list is
327
+ the CANDIDATE POOL for election, not a per-call sequence.
328
+
329
+ Worst case per turn: 2 LLM calls. A final filter_chain refresh path
330
+ handles the rare double-failure edge case so the call still
331
+ degrades gracefully.
 
 
 
 
 
 
 
 
 
332
 
333
  `name` after a successful call reflects which model actually answered
334
+ so downstream callers (brain_used tag, eval logs) can audit which
335
+ candidate produced the output. If primary+backup both fail inside one
336
+ turn, fact_find_brain gets one more bite via BRAIN_CHAIN.
 
337
  """
338
  def __init__(self, chain: list[str], api_key: Optional[str] = None,
339
  timeout: float = 30.0, per_model_attempts: int = 1,
 
343
  self.chain = chain
344
  self.api_key = api_key or getattr(settings, "NVIDIA_NIM_API_KEY", "")
345
  self.timeout = timeout
346
+ # Cumulative chain budget. Without a ceiling, a chain of N
347
+ # fallbacks at 30s each could run for minutes per turn. Default
348
+ # to ~2.5× the per-link timeout so a healthy primary always
349
+ # completes but a cascading-failure chain bails fast. With
350
+ # election doing only 1-2 LLM calls per turn the budget is
351
+ # rarely binding; it still guards the final filter_chain-refresh
352
+ # fallback path + cold-start edge cases.
 
 
353
  self.total_budget_s = total_budget_s if total_budget_s is not None else max(timeout * 2.5, 30.0)
354
  self.per_model_attempts = per_model_attempts
355
  self.role = role # 'brain' | 'unknown' — flows into usage log
 
376
  def _get_worker_for(self, model_id: str, timeout: float) -> LLMProvider:
377
  """Dispatch a chain entry to the NIM provider client.
378
 
379
+ NIM-only: cross-provider (`openrouter:` / `groq:`) prefixes are
380
+ explicitly rejected here. This is a defense-in-depth short-circuit
381
+ if anyone (admin override, monkeypatch, future drift) injects a
382
+ non-NIM candidate into a chain, the dispatcher raises rather than
383
+ routing to a provider outside the validated pool.
384
+
385
+ Passes `chain_name=self._chain_name` so the credit trackers in the
386
+ provider clients route their response-header signals to the right
387
+ chain state.
 
 
388
  """
389
  if model_id.startswith(("openrouter:", "groq:", "or:")):
390
  raise RuntimeError(
 
404
  'deepseek', 'moonshot', 'minimax', 'nvidia', 'nemotron', 'llama3',
405
  'google', 'unknown'.
406
 
407
+ EVERY model in BRAIN_CHAIN must map to a known family. The
408
+ `assert_family_coverage()` helper walks the chain on demand (from
409
+ tests / admin / probe loop) and raises if any candidate falls
410
+ through to "unknown".
411
  """
412
  m = (model_id or "").lower()
413
+ # Recognise Google's Gemini family BEFORE prefix-stripping so ids
414
+ # like "google/gemini-2.5-flash" and bare "gemini-..." both land
415
+ # in the "google" bucket, so the judge-vs-brain cross-family
416
+ # invariant can exclude a Gemini brain from a Gemini judge.
 
417
  if "gemini" in m or m.startswith("google/"):
418
  return "google"
419
  # Strip provider prefix first so 'groq:llama-3.3-70b' → 'meta' (it IS Meta Llama)
 
483
  exclude_models: Optional[list[str]] = None,
484
  exclude_families: Optional[list[str]] = None,
485
  ) -> LLMResult:
486
+ """Sticky primary election. Call ONE elected candidate per turn,
487
+ with at most ONE real-time fallback to the elected backup if
488
  primary fails. The chain list is the candidate POOL for election —
489
  NOT a per-call sequence.
490
 
 
495
  the chain in order.
496
  Final fallback (both elected models fail): trigger a synchronous
497
  probe refresh + try whatever filter_chain now offers, walking the
498
+ chain in order. This path is the graceful-degradation safety net
499
+ for the double-failure edge case.
500
  """
501
  from backend import llm_health
502
 
 
599
  llm_health.report_success(self._chain_name, model, latency_ms)
600
  except Exception:
601
  pass
602
+ # NIM has no clean rate-limit header, so we maintain a local
603
+ # 60s rate-meter for every NIM-prefixed
604
  # candidate (i.e. anything without an openrouter:/groq: prefix).
605
  # Groq + OpenRouter stamp credits from response headers inside
606
  # their own .chat() methods.
 
629
  if res is not None:
630
  return res
631
 
632
+ # --- Backup attempt (single real-time fallback) ----------------------
633
  if elected_backup and elected_backup != elected_primary:
634
  res = await _try(elected_backup)
635
  if res is not None:
 
637
 
638
  # --- Both elected failed — final safety net --------------------------
639
  # Trigger ONE synchronous probe refresh; whatever the refreshed
640
+ # filter_chain offers, walk it in order and try anything not yet
641
+ # touched this turn. Graceful-degradation path for the (rare)
642
+ # double-failure case.
643
  try:
644
  # KI-099 — bound the probe-refresh cost. On a hot user-facing turn, a
645
  # 60-80s probe walk through ~10 candidates is unacceptable. If we've
 
742
  ) from last_err
743
 
744
 
745
+ # Provider load-balancing pure-function. Not wired into the chat hot path
746
+ # (the probe-driven elector in backend.llm_health selects candidates);
747
+ # retained because the routing regression suite pins its statistical
748
+ # behaviour and it is a useful pure-function for ops / sim / overrides.
 
 
 
 
749
  import random as _random
750
 
751
 
752
  def _balanced_brain_chain(base: list[str], *, groq_first_probability: float = 0.5) -> list[str]:
753
+ """Provider load-balancing pure-function.
754
 
755
  With `groq_first_probability` (default 50%), hoist the Groq Llama entry
756
  to the head of the chain so it serves as the primary instead of the
 
758
  fallback order — Groq calls that fail (rare; LPU is very reliable)
759
  still get the full NIM fallback chain.
760
 
761
+ Not wired into the chat hot path (the elector in `backend.llm_health`
762
+ picks candidates dynamically from background probe data). Retained
763
+ because the routing regression suite pins its statistical behaviour
764
+ and it is a useful pure-function for ops / sim / overrides (e.g.
765
+ forcing a non-elected order in a debug script).
766
+
767
+ Uses per-call `random.random()` so concurrent async workers each flip
768
+ independently — no shared mutable cycle state, no GIL races, fair
 
 
 
769
  distribution in aggregate. `groq_first_probability` is overrideable
770
  primarily for testing."""
771
  if _random.random() >= groq_first_probability:
 
779
 
780
 
781
  def assert_family_coverage() -> dict[str, str]:
782
+ """Verify every model in BRAIN_CHAIN maps to a known family (i.e. NOT
783
+ 'unknown'). Returns a {model: family} dict on success; raises
784
+ RuntimeError listing any 'unknown'-mapped models on failure.
785
 
786
+ Also verifies nemotron is the TAIL (last entry) of BRAIN_CHAIN so the
787
+ last-resort ordering invariant survives any chain edit.
788
 
789
+ Walks BRAIN_CHAIN (the only chain). Call from admin diagnostics or
790
+ tests; not invoked on import to keep cold-start cheap. Safe to run
791
+ from a probe loop tick.
 
 
792
  """
793
  coverage: dict[str, str] = {}
794
  unknown: list[str] = []
 
817
 
818
 
819
  def get_brain_llm() -> NimChainLLM:
820
+ """Heavy brain — sticky-primary election over BRAIN_CHAIN.
 
 
 
 
821
 
822
+ The background probe loop in backend.llm_health elects the
823
+ actually-fastest candidate dynamically (probe-driven) plus a backup.
824
+ chat() calls 1-2 candidates max per turn. (The _balanced_brain_chain
825
+ rotation pure-function is available for overrides but not wired in
826
+ here.)
827
 
828
+ Per-link 20s timeout, total chain budget 35s (only binding for the
829
+ final filter_chain-refresh safety net).
830
  """
831
  return NimChainLLM(chain=BRAIN_CHAIN, timeout=20.0,
832
  role="brain", total_budget_s=35.0)
833
 
834
 
835
+ # get_brain_llm() is the brain factory; BRAIN_CHAIN is the only chain.
 
 
backend/providers/openrouter_llm.py CHANGED
@@ -40,7 +40,7 @@ OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
40
  DEFAULT_MODEL = "openai/gpt-oss-120b"
41
 
42
 
43
- # A4 (2026-05-15) — OpenRouter free-pool helpers.
44
  #
45
  # `:free` suffix is OpenRouter's documented marker for zero-cost models.
46
  # We use this both to:
@@ -61,8 +61,8 @@ def is_free_model(model_id: str) -> bool:
61
 
62
 
63
  def order_free_pool(models: list[str]) -> list[str]:
64
- """A4 (2026-05-15) — Return `models` in a STABLE, deterministic order
65
- suitable for OpenRouter's `models=[...]` server-side fallback list.
66
 
67
  Sort key: (length-of-id ascending, id ascending). Shorter ids tend to
68
  correlate with smaller / cheaper models on OpenRouter's catalogue
@@ -77,7 +77,7 @@ def order_free_pool(models: list[str]) -> list[str]:
77
 
78
 
79
  def enforce_free_pool(models: list[str]) -> list[str]:
80
- """A4 (2026-05-15) — Cost guard: reject paid models in a free-pool call.
81
 
82
  Raises ValueError listing any non-`:free` entries. Returns the
83
  deterministically-ordered free pool on success.
@@ -131,7 +131,7 @@ class OpenRouterLLM(LLMProvider):
131
  routing; if `models` is omitted the lone `model` field is used).
132
  Reference: https://openrouter.ai/docs/features/model-routing
133
 
134
- A4 (2026-05-15) — `free_only=True` activates the cost guard:
135
  - rejects any non-`:free` model in `models` (raises ValueError)
136
  - reorders the survivors via `order_free_pool` for stable, smaller-
137
  first preference. The single-model `self.model` is checked too
@@ -196,8 +196,8 @@ class OpenRouterLLM(LLMProvider):
196
  continue
197
  resp.raise_for_status()
198
  break
199
- # KI-085 (2026-05-15) — opportunistic between-poll signal from
200
- # OpenRouter response headers. The authoritative truth (usd_balance)
201
  # comes from poll_openrouter_credits() every 10 min; this is a
202
  # finer-grained per-call backup that catches per-minute bucket
203
  # exhaustion before the next poll tick.
@@ -232,16 +232,9 @@ class OpenRouterLLM(LLMProvider):
232
  # a sane single-model default; callers that want server-side fallback should
233
  # pass `models=[...]` to `chat()`.
234
  #
235
- # Model IDs verified against the LIVE OpenRouter catalog
236
- # (GET /api/v1/models, 2026-05-15). The model IDs from the KI-176 brief
237
- # (gemini-2.0-flash-exp:free, llama-3.3-70b-instruct:free,
238
- # hermes-3-llama-3.1-405b:free) were checked:
239
- # - gemini-2.0-flash-exp:free — NOT in catalog
240
- # - llama-3.3-70b-instruct:free — PRESENT but no response_format
241
- # - hermes-3-llama-3.1-405b:free — PRESENT but no response_format
242
- # So those would silently lose JSON mode. The IDs below are the actual
243
- # free-tier set that declare `response_format` (and most also
244
- # `structured_outputs`) in their `supported_parameters`:
245
  # - nvidia/nemotron-3-super-120b-a12b:free (120B, structured_outputs)
246
  # - qwen/qwen3-next-80b-a3b-instruct:free (80B, structured_outputs)
247
  # - google/gemma-4-31b-it:free (31B, response_format)
@@ -259,7 +252,7 @@ def get_openrouter_llm(
259
  model: str = DEFAULT_FREE_BRAIN_MODEL,
260
  timeout: float = 60.0,
261
  ) -> "OpenRouterLLM":
262
- """Return a fresh OpenRouterLLM ready for sales_brain.
263
 
264
  The returned client is single-use-shaped (one chat() call per turn) and
265
  intended to be passed `models=[primary, fallback1, fallback2]` so the
 
40
  DEFAULT_MODEL = "openai/gpt-oss-120b"
41
 
42
 
43
+ # OpenRouter free-pool helpers.
44
  #
45
  # `:free` suffix is OpenRouter's documented marker for zero-cost models.
46
  # We use this both to:
 
61
 
62
 
63
  def order_free_pool(models: list[str]) -> list[str]:
64
+ """Return `models` in a STABLE, deterministic order suitable for
65
+ OpenRouter's `models=[...]` server-side fallback list.
66
 
67
  Sort key: (length-of-id ascending, id ascending). Shorter ids tend to
68
  correlate with smaller / cheaper models on OpenRouter's catalogue
 
77
 
78
 
79
  def enforce_free_pool(models: list[str]) -> list[str]:
80
+ """Cost guard: reject paid models in a free-pool call.
81
 
82
  Raises ValueError listing any non-`:free` entries. Returns the
83
  deterministically-ordered free pool on success.
 
131
  routing; if `models` is omitted the lone `model` field is used).
132
  Reference: https://openrouter.ai/docs/features/model-routing
133
 
134
+ `free_only=True` activates the cost guard:
135
  - rejects any non-`:free` model in `models` (raises ValueError)
136
  - reorders the survivors via `order_free_pool` for stable, smaller-
137
  first preference. The single-model `self.model` is checked too
 
196
  continue
197
  resp.raise_for_status()
198
  break
199
+ # Opportunistic between-poll signal from OpenRouter response
200
+ # headers. The authoritative truth (usd_balance)
201
  # comes from poll_openrouter_credits() every 10 min; this is a
202
  # finer-grained per-call backup that catches per-minute bucket
203
  # exhaustion before the next poll tick.
 
232
  # a sane single-model default; callers that want server-side fallback should
233
  # pass `models=[...]` to `chat()`.
234
  #
235
+ # The free-tier model IDs below are the set that declare `response_format`
236
+ # (and most also `structured_outputs`) in their `supported_parameters`,
237
+ # so JSON mode is preserved:
 
 
 
 
 
 
 
238
  # - nvidia/nemotron-3-super-120b-a12b:free (120B, structured_outputs)
239
  # - qwen/qwen3-next-80b-a3b-instruct:free (80B, structured_outputs)
240
  # - google/gemma-4-31b-it:free (31B, response_format)
 
252
  model: str = DEFAULT_FREE_BRAIN_MODEL,
253
  timeout: float = 60.0,
254
  ) -> "OpenRouterLLM":
255
+ """Return a fresh OpenRouterLLM client.
256
 
257
  The returned client is single-use-shaped (one chat() call per turn) and
258
  intended to be passed `models=[primary, fallback1, fallback2]` so the
backend/providers/sarvam_tts.py CHANGED
@@ -29,8 +29,8 @@ from backend.providers.base import TTSProvider
29
 
30
  logger = logging.getLogger(__name__)
31
 
32
- # X8 (2026-05-15) — frontend now sends `X-Preferred-Codec: audio/{wav,mp4,webm}`
33
- # header so the chat endpoint can return audio in the codec the user's browser
34
  # decodes natively. Sarvam Bulbul's text-to-speech API itself does NOT support
35
  # codec negotiation in the request body (response is always base64 WAV), so we
36
  # transcode locally with pydub (already in requirements.txt; the Dockerfile
@@ -48,14 +48,11 @@ except Exception: # pragma: no cover — pydub missing in dev shell
48
  _SUPPORTED_CODECS = {"audio/wav", "audio/mp4", "audio/webm"}
49
 
50
 
51
- # KI-278 (2026-05-16) — voice-OUTPUT error vocabulary. Mirrors the STT
52
- # closed-enum contract (sarvam_stt.STT_ERROR_*) so the chat endpoint can
53
- # surface a structured tts_error_code instead of swallowing the exception
54
- # and returning audio_base64=None with zero signal to the client. Bug
55
- # origin: Sarvam account ran out of credits → /text-to-speech returned
56
- # HTTP 429 {"code":"insufficient_quota_error"} → resp.raise_for_status()
57
- # raised → chat endpoint logged it to turns.jsonl and returned a text-only
58
- # reply with NO voice and NO explanation ("no voice in reply. wtf?").
59
  TTS_ERROR_RATE_LIMIT = "rate_limit"
60
  TTS_ERROR_SERVICE = "service_unavailable"
61
  TTS_ERROR_NETWORK = "network"
 
29
 
30
  logger = logging.getLogger(__name__)
31
 
32
+ # The frontend sends an `X-Preferred-Codec: audio/{wav,mp4,webm}` header
33
+ # so the chat endpoint can return audio in the codec the user's browser
34
  # decodes natively. Sarvam Bulbul's text-to-speech API itself does NOT support
35
  # codec negotiation in the request body (response is always base64 WAV), so we
36
  # transcode locally with pydub (already in requirements.txt; the Dockerfile
 
48
  _SUPPORTED_CODECS = {"audio/wav", "audio/mp4", "audio/webm"}
49
 
50
 
51
+ # Voice-OUTPUT error vocabulary. Mirrors the STT closed-enum contract
52
+ # (sarvam_stt.STT_ERROR_*) so the chat endpoint can surface a structured
53
+ # tts_error_code instead of returning audio_base64=None with zero signal
54
+ # to the client (e.g. when Sarvam returns HTTP 429
55
+ # {"code":"insufficient_quota_error"}).
 
 
 
56
  TTS_ERROR_RATE_LIMIT = "rate_limit"
57
  TTS_ERROR_SERVICE = "service_unavailable"
58
  TTS_ERROR_NETWORK = "network"
backend/retrieval_filters.py CHANGED
@@ -3,8 +3,7 @@
3
  This module is a SIDECAR to the retrieval path (`rag/retrieve.py` and the
4
  single-brain tool layer in `backend/brain_tools.py`). It is deliberately a
5
  separate file so retrieval-correctness work stays isolated from the
6
- turn-handling code. (Historically it also shielded the now-removed
7
- `backend/orchestrator.py`, deleted in the 2026-05-15 single-brain rewrite.)
8
 
9
  Public API:
10
  apply_profile_filter(chunks, profile) -> list[RetrievedChunk]
@@ -34,8 +33,8 @@ Public API:
34
  dedup_by_policy(chunks) -> list[RetrievedChunk]
35
  Within a top-K, keep highest-score chunk per policy_id.
36
 
37
- All functions are pure / side-effect-free; safe to call from the orchestrator
38
- or from rag/retrieve.py without import-cycle risk.
39
  """
40
 
41
  from __future__ import annotations
@@ -43,7 +42,7 @@ from __future__ import annotations
43
  import re
44
  from typing import Any, Iterable, Optional
45
 
46
- # KI-280 (2026-05-16) — shared canonical-identity / dedup rule. The same
47
  # UIN-primary + product_key invariant the marketplace endpoint uses
48
  # (main.py /api/policies/all). Imported (not reimplemented) so the
49
  # recommender and the marketplace agree on "the same policy". The module
@@ -90,7 +89,7 @@ MATERNITY_NAME_RE = re.compile(
90
  )
91
 
92
  # ---------------------------------------------------------------------------
93
- # KI-278 — Eligibility / profile-fit tunables (2026-05-16)
94
  # ---------------------------------------------------------------------------
95
  # A plan is a TOP-UP / SUPER-TOP-UP (only useful ALONGSIDE an existing base
96
  # policy or above a large out-of-pocket deductible) when ANY of these hold:
@@ -150,8 +149,7 @@ UIN_RE = re.compile(
150
  )
151
 
152
  # ---------------------------------------------------------------------------
153
- # KI-279 — fixed-benefit exclusion for comprehensive-indemnity intent
154
- # (2026-05-16)
155
  # ---------------------------------------------------------------------------
156
  # A FIXED-BENEFIT product (hospital daily cash, personal accident, critical
157
  # illness, cancer / other defined-benefit) pays a fixed lump-sum / per-day
@@ -200,13 +198,12 @@ _COMPREHENSIVE_GOAL_TOKENS = (
200
  )
201
 
202
  # ---------------------------------------------------------------------------
203
- # KI-280 — UNIFIED recommendation-fit gate tunables (2026-05-16)
204
  # ---------------------------------------------------------------------------
205
- # The audit proved recommendation-fit is SYSTEMIC: the cited-card list and
206
- # the advisory prose must be gated by the SAME fitness logic. These add the
207
- # two missing hard gates (entry-age for the INSURED person; required
208
- # feature = maternity/newborn) and the ranking signals that fix the
209
- # grade/rank inversion + the cost-objective lead.
210
 
211
  # Entry-age grace. The catalog's max_entry_age is the regulator-filed entry
212
  # age; underwriters allow a small grace band (mirrors PROFILE_AGE_TOLERANCE).
@@ -348,15 +345,14 @@ def apply_profile_filter(chunks: Iterable[Any], profile: Any) -> list[Any]:
348
  if not chunks_list:
349
  return chunks_list
350
 
351
- # KI-280 — the demographic gates (numeric age-range, senior-only,
352
- # adult-only) must reason about the age of the person the policy
353
- # actually INSURES, not the payer's. When the policy covers parents
354
- # (~70), using the paying child's age (e.g. 36) made the senior-only
355
- # rule wrongly DROP a senior-citizen plan ("min_entry_age>=60 AND
356
- # payer<50") and the numeric range wrongly KEEP a max-entry-65 plan —
357
- # the exact P7 live-audit failure. `_oldest_insured_age` returns the
358
- # eldest parent's age when insuring parents, else the profile's own
359
- # age. Falls back to the raw profile age if no resolved signal.
360
  _resolved_age = _oldest_insured_age(profile)
361
  if _resolved_age is None:
362
  _resolved_age = _profile_get(profile, "age")
@@ -713,13 +709,12 @@ def apply_eligibility_filter(chunks: Iterable[Any], profile: Any) -> list[Any]:
713
  expenses, so they are a wrong PRIMARY recommendation. They are
714
  NEVER dropped for a user who explicitly wants a supplement.
715
 
716
- 5. ENTRY-AGE GATE (KI-280) — drop a plan whose max entry age cannot
717
- accept the OLDEST person the policy must insure. When the profile
718
- covers parents (~70), the gate uses the parents' age, not the
719
- paying child's the demographic pre-filter only sees the payer's
720
- age, which is exactly why a max-entry-65 plan reached a 70yo-
721
- parents profile in the live audit (P7). Conservative: fires only
722
- when both the insured age AND the policy fact are present.
723
 
724
  Conservative on missing data: a rule only fires when BOTH the profile
725
  signal AND the policy fact are present. A chunk with no enriched facts
@@ -755,14 +750,14 @@ def apply_eligibility_filter(chunks: Iterable[Any], profile: Any) -> list[Any]:
755
  if wants_comprehensive and _is_fixed_benefit_chunk(m):
756
  continue
757
 
758
- # Rule 5 (KI-280) — HARD ELIGIBILITY: a plan whose max entry age
759
- # cannot accept the oldest insured person is structurally unusable.
760
- # This is the audit-P7 fix: a 70yo parent cannot be enrolled in a
761
- # plan whose max_entry_age is 65. We only fire when BOTH the age and
762
- # the policy fact are present (conservative on missing data) and
763
- # only when the profile actually insures someone whose age we know
764
- # (the demographic apply_profile_filter uses the *payer's* age and
765
- # never sees the parents' age — that is precisely why P7 slipped).
766
  if oldest_age is not None:
767
  maxe = _as_int(m.get("max_entry_age"))
768
  if maxe is not None and maxe + ENTRY_AGE_TOLERANCE < oldest_age:
@@ -1044,10 +1039,8 @@ def enforce_citation_grounding(chunks: Iterable[Any]) -> list[Any]:
1044
  INFORMATIONAL only — it is not required for citation grounding because
1045
  upstream call sites (e.g. brain_tools.retrieve_policies) build pruned
1046
  dicts that intentionally omit it, and the brain cites by policy
1047
- identity, not by chunk offset. Z2 live test (2026-05-15) showed 15/15
1048
- retrieve_policies calls returning 0 chunks because we required an
1049
- offset that the upstream builder never included → every chunk dropped
1050
- here even though raw retrieval was healthy.
1051
  """
1052
  kept: list[Any] = []
1053
  for ch in chunks:
@@ -1071,16 +1064,15 @@ def dedup_by_policy(chunks: Iterable[Any]) -> list[Any]:
1071
  duplicates to one chunk, keeping the highest-scoring chunk. Preserves
1072
  the order of first appearance of each kept product.
1073
 
1074
- KI-280: keying is by the SHARED canonical identity
1075
  (policy_identity.canonical_key) — UIN-primary, product_key fallback —
1076
- NOT the raw policy_id. The live audit cited the SAME product twice
1077
- because the two chunks had different policy_ids: a marketing rename
1078
- ("my:Optima Secure" vs "my:Optima Secure (older variant)" — same UIN)
1079
- or two doctype siblings ("...__wordings" vs "...__brochure" — same
1080
- product_key). policy_id-only dedup let both through (P2 New National
1081
- Parivar ×2, P4 Advanced Top Up ×2, P7 Chola ×2). This reuses the exact
1082
- rule the marketplace endpoint uses so the recommender and the
1083
- marketplace agree on "the same policy".
1084
  """
1085
  best: dict[str, Any] = {}
1086
  order: list[str] = []
 
3
  This module is a SIDECAR to the retrieval path (`rag/retrieve.py` and the
4
  single-brain tool layer in `backend/brain_tools.py`). It is deliberately a
5
  separate file so retrieval-correctness work stays isolated from the
6
+ turn-handling code.
 
7
 
8
  Public API:
9
  apply_profile_filter(chunks, profile) -> list[RetrievedChunk]
 
33
  dedup_by_policy(chunks) -> list[RetrievedChunk]
34
  Within a top-K, keep highest-score chunk per policy_id.
35
 
36
+ All functions are pure / side-effect-free; safe to call from the
37
+ single-brain tool layer or from rag/retrieve.py without import-cycle risk.
38
  """
39
 
40
  from __future__ import annotations
 
42
  import re
43
  from typing import Any, Iterable, Optional
44
 
45
+ # Shared canonical-identity / dedup rule. The same
46
  # UIN-primary + product_key invariant the marketplace endpoint uses
47
  # (main.py /api/policies/all). Imported (not reimplemented) so the
48
  # recommender and the marketplace agree on "the same policy". The module
 
89
  )
90
 
91
  # ---------------------------------------------------------------------------
92
+ # Eligibility / profile-fit tunables
93
  # ---------------------------------------------------------------------------
94
  # A plan is a TOP-UP / SUPER-TOP-UP (only useful ALONGSIDE an existing base
95
  # policy or above a large out-of-pocket deductible) when ANY of these hold:
 
149
  )
150
 
151
  # ---------------------------------------------------------------------------
152
+ # Fixed-benefit exclusion for comprehensive-indemnity intent
 
153
  # ---------------------------------------------------------------------------
154
  # A FIXED-BENEFIT product (hospital daily cash, personal accident, critical
155
  # illness, cancer / other defined-benefit) pays a fixed lump-sum / per-day
 
198
  )
199
 
200
  # ---------------------------------------------------------------------------
201
+ # Unified recommendation-fit gate tunables
202
  # ---------------------------------------------------------------------------
203
+ # Recommendation-fit is SYSTEMIC: the cited-card list and the advisory prose
204
+ # are gated by the SAME fitness logic. Two hard gates (entry-age for the
205
+ # INSURED person; required feature = maternity/newborn) plus ranking signals
206
+ # for grade/rank ordering and the cost-objective lead.
 
207
 
208
  # Entry-age grace. The catalog's max_entry_age is the regulator-filed entry
209
  # age; underwriters allow a small grace band (mirrors PROFILE_AGE_TOLERANCE).
 
345
  if not chunks_list:
346
  return chunks_list
347
 
348
+ # The demographic gates (numeric age-range, senior-only, adult-only)
349
+ # reason about the age of the person the policy actually INSURES, not
350
+ # the payer's. When the policy covers parents (~70), using the paying
351
+ # child's age (e.g. 36) would wrongly DROP a senior-citizen plan
352
+ # ("min_entry_age>=60 AND payer<50") and wrongly KEEP a max-entry-65
353
+ # plan. `_oldest_insured_age` returns the eldest parent's age when
354
+ # insuring parents, else the profile's own age. Falls back to the raw
355
+ # profile age if no resolved signal.
 
356
  _resolved_age = _oldest_insured_age(profile)
357
  if _resolved_age is None:
358
  _resolved_age = _profile_get(profile, "age")
 
709
  expenses, so they are a wrong PRIMARY recommendation. They are
710
  NEVER dropped for a user who explicitly wants a supplement.
711
 
712
+ 5. ENTRY-AGE GATE — drop a plan whose max entry age cannot accept
713
+ the OLDEST person the policy must insure. When the profile covers
714
+ parents (~70), the gate uses the parents' age, not the paying
715
+ child's (the demographic pre-filter only sees the payer's age).
716
+ Conservative: fires only when both the insured age AND the policy
717
+ fact are present.
 
718
 
719
  Conservative on missing data: a rule only fires when BOTH the profile
720
  signal AND the policy fact are present. A chunk with no enriched facts
 
750
  if wants_comprehensive and _is_fixed_benefit_chunk(m):
751
  continue
752
 
753
+ # Rule 5 — HARD ELIGIBILITY: a plan whose max entry age cannot
754
+ # accept the oldest insured person is structurally unusable (e.g. a
755
+ # 70yo parent cannot be enrolled in a plan whose max_entry_age is
756
+ # 65). Fires only when BOTH the age and the policy fact are present
757
+ # (conservative on missing data) and only when the profile actually
758
+ # insures someone whose age we know (the demographic
759
+ # apply_profile_filter uses the *payer's* age and never sees the
760
+ # parents' age).
761
  if oldest_age is not None:
762
  maxe = _as_int(m.get("max_entry_age"))
763
  if maxe is not None and maxe + ENTRY_AGE_TOLERANCE < oldest_age:
 
1039
  INFORMATIONAL only — it is not required for citation grounding because
1040
  upstream call sites (e.g. brain_tools.retrieve_policies) build pruned
1041
  dicts that intentionally omit it, and the brain cites by policy
1042
+ identity, not by chunk offset. Requiring an offset here would drop
1043
+ every chunk built by those pruned-dict call sites.
 
 
1044
  """
1045
  kept: list[Any] = []
1046
  for ch in chunks:
 
1064
  duplicates to one chunk, keeping the highest-scoring chunk. Preserves
1065
  the order of first appearance of each kept product.
1066
 
1067
+ Keying is by the SHARED canonical identity
1068
  (policy_identity.canonical_key) — UIN-primary, product_key fallback —
1069
+ NOT the raw policy_id. The same product can appear under different
1070
+ policy_ids: a marketing rename ("my:Optima Secure" vs "my:Optima
1071
+ Secure (older variant)" — same UIN) or two doctype siblings
1072
+ ("...__wordings" vs "...__brochure" — same product_key). policy_id-only
1073
+ dedup would let both through; canonical-identity keying collapses them.
1074
+ This reuses the exact rule the marketplace endpoint uses so the
1075
+ recommender and the marketplace agree on "the same policy".
 
1076
  """
1077
  best: dict[str, Any] = {}
1078
  order: list[str] = []
backend/scorecard.py CHANGED
@@ -791,17 +791,14 @@ def _profile_tuned_weights(profile: Optional[dict]) -> dict[str, float]:
791
  def profile_completeness(profile: Optional[dict]) -> float:
792
  """0.0–1.0 measure of how much we know about the buyer.
793
 
794
- KI-252 (2026-05-15): aligned with Path B's `_REQUIRED_FOR_READY` 7-slot
795
- list (see brain_tools.py + single_brain.py). Previously this checked 9
796
- weighted fields including budget_band / existing_cover_inr /
797
- parents_age_max which Path B doesn't gather, causing the UI bar to read
798
- 70% while Path B already considered the profile "ready to recommend".
799
 
800
  The 7 slots: name, age, dependents, location_tier, income_band,
801
  primary_goal, health_conditions. `name` is the identifier; the other 6
802
  are decision-critical for retrieval. Existing-cover and budget-band are
803
- captured opportunistically (Rajesh provided existing_cover_inr in turn 1)
804
- but are NOT required to recommend.
805
 
806
  Used by the frontend to GATE the personalized scorecard view — until
807
  completeness >= 0.6, we show insurer-level metrics (CSR, complaints —
 
791
  def profile_completeness(profile: Optional[dict]) -> float:
792
  """0.0–1.0 measure of how much we know about the buyer.
793
 
794
+ Aligned with the `_REQUIRED_FOR_READY` 7-slot list (see brain_tools.py
795
+ + single_brain.py) so this measure agrees with the brain's "ready to
796
+ recommend" gate.
 
 
797
 
798
  The 7 slots: name, age, dependents, location_tier, income_band,
799
  primary_goal, health_conditions. `name` is the identifier; the other 6
800
  are decision-critical for retrieval. Existing-cover and budget-band are
801
+ captured opportunistically but are NOT required to recommend.
 
802
 
803
  Used by the frontend to GATE the personalized scorecard view — until
804
  completeness >= 0.6, we show insurer-level metrics (CSR, complaints —
backend/session_state.py CHANGED
@@ -74,39 +74,24 @@ class SessionState:
74
  # "Conversation turn" column in the admin Recommendation History panel
75
  # (previously showed "—" because no caller populated the field).
76
  turn_idx: int = 0
77
- # Z2 fix Issue 3 (brain election bouncing). Priya's session hopped
78
- # sales_brain single_brain single_brain sales_brain across 6
79
- # turns even though USE_SINGLE_BRAIN=true; the Gemini 503 fallback
80
- # (Z1 retry already softens) was dropping the session back onto the
81
- # legacy orchestrator mid-stream. Belt-and-suspenders: main.py stamps
82
- # this True after the FIRST successful single_brain turn, and
83
- # subsequent SingleBrainError responses on the same session must NOT
84
- # fall through to the orchestrator — they emit a graceful retry
85
- # prompt so the session stays sticky on single_brain.
86
  single_brain_sticky: bool = False
87
- # VESTIGIAL (2026-05-17): the deterministic in-conversation recall
88
- # name-sniff that set this was REMOVED from single_brain (see the
89
- # tombstone above single_brain._HONEST_EMPTY_REPLY). Field retained,
90
- # not deleted, so previously-persisted SessionState payloads still
91
- # deserialize cleanly; nothing reads or writes it any more.
92
- _recall_sniff_done: bool = False
93
- # Bug #108 + #110 (2026-05-16) — POST-RECAP pricing & family-history
94
- # bundle re-ask gate (brain_tools.retrieve_policies).
95
- # pricing_bundle_reasked — one-shot guard. Set True the first time the
96
- # gate re-asks an unresolved bundle slot so the NEXT recommendation
97
- # retrieve proceeds even if the user skips (SOFT capture, no hard
98
- # loop).
99
  # pricing_bundle_skipped — set True by single_brain when the user
100
- # explicitly declines the pricing inputs ("just show me options",
101
- # "you decide", "skip"). Bypasses the re-ask entirely.
102
  pricing_bundle_reasked: bool = False
103
  pricing_bundle_skipped: bool = False
104
 
105
  def _flush(self) -> None:
106
- """No-op since KI-118 (2026-05-15). Disk persistence was removed; the
107
- in-memory dict is now the only store. We keep the method so existing
108
- callers (orchestrator + fact_find_brain + tests) don't have to change
109
- their write paths.
110
  """
111
  return None
112
 
 
74
  # "Conversation turn" column in the admin Recommendation History panel
75
  # (previously showed "—" because no caller populated the field).
76
  turn_idx: int = 0
77
+ # Set True after the first successful single_brain turn; a later
78
+ # SingleBrainError on the same session then emits a graceful retry
79
+ # prompt instead of switching handlers, so the session stays on
80
+ # single_brain.
 
 
 
 
 
81
  single_brain_sticky: bool = False
82
+ # Post-recap pricing & family-history bundle re-ask gate
83
+ # (brain_tools.retrieve_policies):
84
+ # pricing_bundle_reasked one-shot guard; set True the first time
85
+ # the gate re-asks an unresolved bundle slot so the next
86
+ # recommendation retrieve proceeds even if the user skips.
 
 
 
 
 
 
 
87
  # pricing_bundle_skipped — set True by single_brain when the user
88
+ # explicitly declines the pricing inputs; bypasses the re-ask.
 
89
  pricing_bundle_reasked: bool = False
90
  pricing_bundle_skipped: bool = False
91
 
92
  def _flush(self) -> None:
93
+ """No-op. Session state lives only in the in-memory dict; the
94
+ method is kept so callers' write paths don't have to change.
 
 
95
  """
96
  return None
97
 
backend/single_brain.py CHANGED
@@ -1,20 +1,19 @@
1
- """Single-brain conversation handler — Path B.
2
 
3
- One Gemini Flash call per turn (with native function-calling) replaces
4
- the previous sales_brain + qa_brain split. The LLM decides on each
5
- iteration whether to:
6
  - call `save_profile_field` to persist captured slots,
7
  - call `retrieve_policies` to pull policy chunks from Chroma,
8
  - call `mark_recommendation` to flag the policies just pitched,
9
  - or emit a final text reply.
10
 
11
- The loop iterates up to `MAX_ITERATIONS` (default 5) so the LLM can chain
12
- multiple tool calls in a single user turn before responding. Beyond that
13
- cap we synthesise a defensive reply and return.
14
 
15
- Wire-up: /api/chat → main.py.chat() → if USE_SINGLE_BRAIN: single_brain.handle_turn(...)
16
- On any SingleBrainError, the API caller is expected to fall through to the
17
- legacy `orchestrator.handle_turn` so the user always gets a reply.
18
 
19
  We call the Gemini REST API directly (httpx, like google_gemini_llm.py)
20
  rather than using the `google.generativeai` SDK so we don't need to pin
@@ -57,13 +56,10 @@ GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/models"
57
  PER_CALL_TIMEOUT_SEC = 25.0
58
 
59
  # Max iterations of the tool-call loop. Prevents runaway tool-call cycles
60
- # where the LLM keeps calling save_profile_field on the same value.
61
- # KI-Z6-NONE (2026-05-15): bumped 5 8 after W1 Turn 3 live blocker.
62
- # The Z6 "no medical issues" path used to need: save(health=none) +
63
- # retrieve → profile_incomplete → save(health=none) + retrieve → loop
64
- # exhaust. Coercer fix in brain_tools resolves the primary cause; the
65
- # extra headroom protects against the next variant where Gemini chains
66
- # 3-4 saves + 2 retrieves on a long pre-recommendation user turn.
67
  MAX_ITERATIONS = 8
68
 
69
  # Transient-error retry policy (2026-05-15 / KI-singlebrain-503).
@@ -393,12 +389,9 @@ class TurnResult:
393
  blocked: bool = False
394
  profile_updates: dict = field(default_factory=dict)
395
  followup_policy_id: Optional[str] = None
396
- # main.py stamps ChatResponse.returning_user_recalled from this. The
397
- # deterministic in-conversation recall that used to set it True was
398
- # REMOVED 2026-05-17 (see the tombstone above _HONEST_EMPTY_REPLY), so
399
- # handle_turn now always leaves this False; explicit returning-user
400
- # recall is the separate POST /api/profile/recall-by-name endpoint.
401
- # Field kept for ChatResponse / frontend compatibility.
402
  returning_user_recalled: bool = False
403
 
404
 
@@ -627,22 +620,9 @@ def _detect_language(user_text: str) -> str:
627
  return "en"
628
 
629
 
630
- # --- REMOVED 2026-05-17 deterministic recall classifier + fallback -------
631
- # `_RECALL_AFFIRM*` / `_classify_recall_answer` (the staged-recall yes/no
632
- # detector) and `_FALLBACK_SLOT_QUESTIONS` / `_FALLBACK_REQUIRED_SLOTS` /
633
- # `_synthesise_fallback` (the deterministic "ask the next missing slot"
634
- # generator) were leftover rule-based scaffolding from the pre-Path-B
635
- # rule/3-brain/judge era. They masked the single LLM: when Gemini returned
636
- # no text the fallback injected a canned "What's your name?", simulating a
637
- # fact-find that wasn't happening → an infinite loop. The single LLM
638
- # (RULE 1 → save_profile_field) is the SOLE fact-find driver now; a genuine
639
- # empty-LLM turn surfaces an HONEST error (see end of handle_turn), never a
640
- # fabricated slot-question. Do NOT reintroduce deterministic fact-find.
641
-
642
- # Honest reply for the rare genuine empty-LLM turn (should be near-zero now
643
- # that thinkingConfig is set so gemini-2.5-flash actually emits output). We
644
- # surface a transparent retry ask — NEVER a fabricated slot-question that
645
- # simulates a fact-find that did not happen.
646
  _HONEST_EMPTY_REPLY = (
647
  "I'm having trouble generating a response right now — could you "
648
  "rephrase that, or try again in a moment?"
@@ -718,28 +698,15 @@ async def _gemini_call(
718
  "tools": [{"functionDeclarations": tools}],
719
  "toolConfig": {"functionCallingConfig": {"mode": "AUTO"}},
720
  "generationConfig": {
721
- # Z2 fix — Issue 1 (mid-session amnesia). Priya T3 + Vikram T2/T4
722
- # came back with the "I lost my train of thought" template even
723
- # though slot capture succeeded. Root cause matches KI-150
724
- # (fact_find LLM, 420 → 700): when Gemini must emit prose AND a
725
- # tool-call trailer in the same response, the model hits
726
- # maxOutputTokens mid-emission, the trailer truncates, and the
727
- # caller falls through to the defensive reply. Budget breakdown
728
- # at p95: prose ~600 tok + tool-call JSON ~800 tok + 20% margin
729
- # ⇒ 1680, rounded up to a safe power-of-two-ish 2048.
730
  "temperature": 0.4,
 
 
 
731
  "maxOutputTokens": 2048,
732
- # CRITICAL (proven via A/B raw-payload test 2026-05-17):
733
- # gemini-2.5-flash is a THINKING model; gemini-2.5-flash-lite /
734
- # gemini-2.0-flash (what this bot historically ran) are NOT.
735
- # With no thinkingConfig, gemini-2.5-flash returns an EMPTY
736
- # completion (finishReason=STOP, content has no `parts`, zero
737
- # candidate tokens) for tool-calling turns — so _synthesise_
738
- # fallback injects the canned "What's your name?" and the
739
- # fact-find loops forever (the live core-flow regression).
740
- # thinkingBudget=0 disables thinking → the model behaves like
741
- # the fast non-thinking frontier model it should be and emits
742
- # the save_profile_field tool call correctly. Do NOT remove.
743
  "thinkingConfig": {"thinkingBudget": 0},
744
  },
745
  }
@@ -1080,25 +1047,16 @@ def _build_recommendation_citations(
1080
  ) -> tuple[list[dict], bool]:
1081
  """Single source of truth for the structured "CITED POLICIES" cards.
1082
 
1083
- ROOT CAUSE this addresses (KI-278): the prose the user reads and the
1084
- citation cards were derived from two different sources. Prose = the
1085
- LLM's curated shortlist (free text). Cards used to be the raw
1086
- `retrieve_policies` recall dump deduped by chunk_id and rendered in
1087
- vector-score order — so the cards listed policies the LLM never named
1088
- and dropped ones it did.
1089
-
1090
- This builds the citation set as EXACTLY the policies the assistant
1091
- actually recommended:
1092
 
1093
  1. If the LLM called `mark_recommendation(policy_ids=[...])`, those
1094
- ids are the SELECTION (which policies the assistant chose).
1095
- 2. Otherwise (Gemini skips mark_recommendation on ~70% of rec turns
1096
- per KI-254), parse the reply prose: every retrieved policy whose
1097
  name appears in `reply_text` is selected.
1098
 
1099
- KI-280 — the cited-card list and the advisory prose must be gated by
1100
- the SAME fitness logic. Two refinements over the KI-278 builder so the
1101
- cards ARE the gated, fit-ranked set:
1102
 
1103
  • CANONICAL DEDUP: `retrieved_chunks_all` is the union of every
1104
  retrieve_policies result this turn. Each result is already gated +
@@ -1521,18 +1479,10 @@ async def handle_turn(
1521
  # conversation — not a returning user.
1522
  _current_turn = int(getattr(session, "turn_idx", 1) or 1)
1523
 
1524
- # 2026-05-17 the deterministic in-conversation profile-recall
1525
- # scaffolding (the "Welcome back are you the same <name>?" confirm
1526
- # short-circuit + STEP-1 name-sniff + STEP-2 apply/discard) was REMOVED.
1527
- # It was leftover rule-based scaffolding from the pre-Path-B
1528
- # rule/3-brain/judge era: it masked the single LLM, discarded the name
1529
- # the user gave, and produced an infinite fact-find loop. The single
1530
- # LLM (RULE 1 → save_profile_field) is now the SOLE fact-find driver.
1531
- # Explicit returning-user recall lives ONLY in the separate
1532
- # POST /api/profile/recall-by-name endpoint (an explicit user action,
1533
- # not a deterministic in-chat short-circuit). `_did_recall_this_turn`
1534
- # is kept as a constant False so the TurnResult.returning_user_recalled
1535
- # field (read by main.py) stays valid without the removed machinery.
1536
  _did_recall_this_turn = False
1537
 
1538
  _has_prior_profile = any(
@@ -1545,8 +1495,7 @@ async def handle_turn(
1545
  is_returning_user = (_current_turn == 1) and _has_prior_profile
1546
 
1547
  # GOOGLE_API_KEY gate — asserted just before anything that talks to
1548
- # Gemini. (The deterministic no-LLM recall-confirm short-circuit that
1549
- # used to precede this was removed 2026-05-17.)
1550
  api_key = os.environ.get("GOOGLE_API_KEY", "").strip()
1551
  if not api_key:
1552
  raise SingleBrainError("GOOGLE_API_KEY not set")
 
1
+ """Single-brain conversation handler.
2
 
3
+ One Gemini Flash call per turn with native function-calling. The LLM
4
+ decides on each iteration whether to:
 
5
  - call `save_profile_field` to persist captured slots,
6
  - call `retrieve_policies` to pull policy chunks from Chroma,
7
  - call `mark_recommendation` to flag the policies just pitched,
8
  - or emit a final text reply.
9
 
10
+ The loop iterates up to `MAX_ITERATIONS` so the LLM can chain multiple
11
+ tool calls in a single user turn before responding. Beyond that cap an
12
+ honest retry message is returned.
13
 
14
+ Wire-up: /api/chat → main.py.chat() → single_brain.handle_turn(...).
15
+ On a SingleBrainError the caller falls through to nim_fallback so the
16
+ user always gets a reply.
17
 
18
  We call the Gemini REST API directly (httpx, like google_gemini_llm.py)
19
  rather than using the `google.generativeai` SDK so we don't need to pin
 
56
  PER_CALL_TIMEOUT_SEC = 25.0
57
 
58
  # Max iterations of the tool-call loop. Prevents runaway tool-call cycles
59
+ # where the LLM keeps calling save_profile_field on the same value. Sized
60
+ # so Gemini can chain a long pre-recommendation turn (several
61
+ # save_profile_field calls + one or two retrieve_policies) within one
62
+ # user turn.
 
 
 
63
  MAX_ITERATIONS = 8
64
 
65
  # Transient-error retry policy (2026-05-15 / KI-singlebrain-503).
 
389
  blocked: bool = False
390
  profile_updates: dict = field(default_factory=dict)
391
  followup_policy_id: Optional[str] = None
392
+ # main.py stamps ChatResponse.returning_user_recalled from this.
393
+ # handle_turn leaves it False; explicit returning-user recall is the
394
+ # separate POST /api/profile/recall-by-name endpoint.
 
 
 
395
  returning_user_recalled: bool = False
396
 
397
 
 
620
  return "en"
621
 
622
 
623
+ # Reply for a turn where the LLM returns no text and no tool calls — a
624
+ # transparent retry ask. The single LLM is the only fact-find driver; we
625
+ # never fabricate a slot-question.
 
 
 
 
 
 
 
 
 
 
 
 
 
626
  _HONEST_EMPTY_REPLY = (
627
  "I'm having trouble generating a response right now — could you "
628
  "rephrase that, or try again in a moment?"
 
698
  "tools": [{"functionDeclarations": tools}],
699
  "toolConfig": {"functionCallingConfig": {"mode": "AUTO"}},
700
  "generationConfig": {
 
 
 
 
 
 
 
 
 
701
  "temperature": 0.4,
702
+ # Sized so a turn emitting prose plus a tool-call trailer does
703
+ # not truncate mid-emission (p95 ≈ prose 600 + tool JSON 800 +
704
+ # margin).
705
  "maxOutputTokens": 2048,
706
+ # gemini-2.5-flash is a thinking model; thinkingBudget=0
707
+ # disables the internal thinking phase so it emits the tool
708
+ # call / text directly (a non-zero budget can consume the
709
+ # output allowance and return an empty completion).
 
 
 
 
 
 
 
710
  "thinkingConfig": {"thinkingBudget": 0},
711
  },
712
  }
 
1047
  ) -> tuple[list[dict], bool]:
1048
  """Single source of truth for the structured "CITED POLICIES" cards.
1049
 
1050
+ The cited-card set IS exactly the policies the assistant recommended,
1051
+ gated by the same fitness logic as the prose:
 
 
 
 
 
 
 
1052
 
1053
  1. If the LLM called `mark_recommendation(policy_ids=[...])`, those
1054
+ ids are the selection.
1055
+ 2. Otherwise, parse the reply prose: every retrieved policy whose
 
1056
  name appears in `reply_text` is selected.
1057
 
1058
+ The cited-card list and the advisory prose are gated by the SAME
1059
+ fitness logic, with:
 
1060
 
1061
  • CANONICAL DEDUP: `retrieved_chunks_all` is the union of every
1062
  retrieve_policies result this turn. Each result is already gated +
 
1479
  # conversation — not a returning user.
1480
  _current_turn = int(getattr(session, "turn_idx", 1) or 1)
1481
 
1482
+ # The single LLM (RULE 1 → save_profile_field) is the sole fact-find
1483
+ # driver. Explicit returning-user recall is the separate
1484
+ # POST /api/profile/recall-by-name endpoint, so handle_turn never sets
1485
+ # returning_user_recalled True.
 
 
 
 
 
 
 
 
1486
  _did_recall_this_turn = False
1487
 
1488
  _has_prior_profile = any(
 
1495
  is_returning_user = (_current_turn == 1) and _has_prior_profile
1496
 
1497
  # GOOGLE_API_KEY gate — asserted just before anything that talks to
1498
+ # Gemini.
 
1499
  api_key = os.environ.get("GOOGLE_API_KEY", "").strip()
1500
  if not api_key:
1501
  raise SingleBrainError("GOOGLE_API_KEY not set")
backend/voice_format.py CHANGED
@@ -3,19 +3,15 @@ spoken-language text for Sarvam Bulbul TTS.
3
 
4
  Why this exists: an unprocessed LLM reply with markdown bold, inline
5
  [Source: ...] tags, and acronyms reads like a screenshot when spoken. Users
6
- hear "asterisk asterisk bold asterisk asterisk A-Y-U-S-H pp dot 1 dash 2".
7
- That's a UX-killing bug — not a Sarvam limitation, a *us* bug.
8
-
9
- KI-104 (2026-05-15) — this module also exposes `strip_cot_preamble`, the
10
- chain-of-thought / instruction-echo stripper that runs on TEXT replies
11
- (not just TTS). Live smoke tests caught NIM reasoning models (e.g.,
12
- Qwen3-Next 80B) and the judge model leaking internal reasoning into
13
- `reply_text`. Examples: "We need to respond to user question…", "We must
14
- ground every factual claim…", "<think>...</think>The answer is X."
15
- `strip_cot_preamble` is called from `persona.strip_think_tags` so every
16
- reply path that already goes through the <think>-tag strip also gets the
17
- preamble strip — no orchestrator.py changes needed (that file is owned
18
- by another lane / KI-101).
19
 
20
  The function turns text like:
21
 
@@ -73,15 +69,15 @@ ACRONYMS = {
73
  r"\bpp\.(\d+)": r"page \1",
74
  }
75
 
76
- # KI-066 (2026-05-15) — money / range shorthand that Sarvam Bulbul reads
77
- # letter-by-letter. The user said "₹25L+" was being spoken as "two five L
78
- # plus". Order in `_normalize_money` matters: handle RANGES first, then
79
- # PLUS-SUFFIXES, then bare unit-suffixes, then standalone "+".
80
  #
81
- # KI-148 (2026-05-15) — extended to cover `k` (thousand), word-forms
82
- # (`1 lakh`, `1 crore`), `+` suffix → "above", and bare numerics like
83
- # `30000` → "30 thousand", `1,00,000` → "1 lakh". `k` is currency-gated to
84
- # avoid breaking "10K marathon".
85
  _MONEY_RANGE_L = re.compile(
86
  r"₹?\s*(\d+(?:\.\d+)?)\s*-\s*(\d+(?:\.\d+)?)\s*L\b",
87
  re.IGNORECASE,
@@ -383,13 +379,13 @@ def _truncate_for_voice(text: str, max_words: int = 60) -> str:
383
 
384
 
385
  # ============================================================================
386
- # KI-104 (2026-05-15) — chain-of-thought / instruction-echo strip
387
  # ============================================================================
388
- # Live smoke test caught LLM brain replies leaking internal reasoning into
389
- # user-visible reply_text. Three failure modes:
390
- # 1. NIM reasoning models (Qwen3-Next 80B) emit a <think>...</think> block
391
- # followed by the answer the <think> tag was sometimes missing /
392
- # malformed so the existing strip_think_tags in persona.py let it through.
393
  # 2. The faithfulness JUDGE model occasionally returns its own reasoning
394
  # instead of a clean rescue reply.
395
  # 3. The brain model misunderstands the system prompt and echoes the
@@ -400,7 +396,7 @@ def _truncate_for_voice(text: str, max_words: int = 60) -> str:
400
  # lines / first 600 chars), so substantive mid-reply content like
401
  # "We have three options: A, B, C" is preserved.
402
 
403
- # ---- Sentence-level preamble patterns (KI-104) ----
404
  #
405
  # A CoT preamble can appear as:
406
  # (a) a full line of its own: "We need to respond carefully.\n<answer>"
@@ -457,9 +453,9 @@ _BRACKET_INTERNAL = re.compile(
457
  flags=re.IGNORECASE | re.DOTALL,
458
  )
459
 
460
- # Stray, unbalanced <think> tags that persona.strip_think_tags doesn't
461
- # already handle (it requires both open and close in the same blob).
462
- # If we see an isolated </think> mid-reply, drop everything before it.
463
  _STRAY_CLOSE_THINK = re.compile(r"^.*?</think>", flags=re.DOTALL | re.IGNORECASE)
464
 
465
  # Maximum scan window for preamble. Beyond this, content is treated as
@@ -567,15 +563,14 @@ def tts_preprocess(text: str, language: str = "en", max_words: int = 60) -> str:
567
  """Public entry — turn an LLM reply into spoken-language text for TTS."""
568
  if not text:
569
  return ""
570
- # KI-104 — defense in depth: even if the reply went through
571
- # persona.strip_think_tags upstream, run the preamble strip again here
572
- # in case it's called on a path that bypasses persona (e.g., direct
573
- # TTS of a cached reply).
574
  cleaned = strip_cot_preamble(text)
575
  cleaned = _strip_markdown(cleaned)
576
- # KI-066 (2026-05-15) — currency/range shorthand expansion before
577
- # acronym handling so ₹5L becomes "5 lakhs" instead of getting caught
578
- # by the bare-L acronym path.
579
  cleaned = _normalize_money(cleaned)
580
  # #106 — AFTER money (lakh/crore already words): expand remaining bare
581
  # integers ("29" → "twenty-nine") + neutralise en/em dashes so Sarvam
 
3
 
4
  Why this exists: an unprocessed LLM reply with markdown bold, inline
5
  [Source: ...] tags, and acronyms reads like a screenshot when spoken. Users
6
+ would otherwise hear "asterisk asterisk bold asterisk asterisk A-Y-U-S-H
7
+ pp dot 1 dash 2".
8
+
9
+ This module also exposes `strip_cot_preamble`, the chain-of-thought /
10
+ instruction-echo stripper that runs on TEXT replies (not just TTS).
11
+ Reasoning models can leak internal reasoning into `reply_text` e.g.
12
+ "We need to respond to user question…", "We must ground every factual
13
+ claim…", "<think>...</think>The answer is X." `strip_cot_preamble`
14
+ removes that preamble.
 
 
 
 
15
 
16
  The function turns text like:
17
 
 
69
  r"\bpp\.(\d+)": r"page \1",
70
  }
71
 
72
+ # Money / range shorthand that Sarvam Bulbul would otherwise read
73
+ # letter-by-letter (e.g. "₹25L+" as "two five L plus"). Order in
74
+ # `_normalize_money` matters: handle RANGES first, then PLUS-SUFFIXES,
75
+ # then bare unit-suffixes, then standalone "+".
76
  #
77
+ # Also covers `k` (thousand), word-forms (`1 lakh`, `1 crore`), `+`
78
+ # suffix → "above", and bare numerics like `30000` → "30 thousand",
79
+ # `1,00,000` → "1 lakh". `k` is currency-gated to avoid breaking
80
+ # "10K marathon".
81
  _MONEY_RANGE_L = re.compile(
82
  r"₹?\s*(\d+(?:\.\d+)?)\s*-\s*(\d+(?:\.\d+)?)\s*L\b",
83
  re.IGNORECASE,
 
379
 
380
 
381
  # ============================================================================
382
+ # Chain-of-thought / instruction-echo strip
383
  # ============================================================================
384
+ # LLM replies can leak internal reasoning into user-visible reply_text in
385
+ # three modes:
386
+ # 1. Reasoning models (Qwen3-Next 80B) emit a <think>...</think> block
387
+ # followed by the answer, sometimes with a missing / malformed
388
+ # <think> tag that a plain tag strip would let through.
389
  # 2. The faithfulness JUDGE model occasionally returns its own reasoning
390
  # instead of a clean rescue reply.
391
  # 3. The brain model misunderstands the system prompt and echoes the
 
396
  # lines / first 600 chars), so substantive mid-reply content like
397
  # "We have three options: A, B, C" is preserved.
398
 
399
+ # ---- Sentence-level preamble patterns ----
400
  #
401
  # A CoT preamble can appear as:
402
  # (a) a full line of its own: "We need to respond carefully.\n<answer>"
 
453
  flags=re.IGNORECASE | re.DOTALL,
454
  )
455
 
456
+ # Stray, unbalanced <think> tags (a balanced-tag strip requires both open
457
+ # and close in the same blob). If we see an isolated </think> mid-reply,
458
+ # drop everything before it.
459
  _STRAY_CLOSE_THINK = re.compile(r"^.*?</think>", flags=re.DOTALL | re.IGNORECASE)
460
 
461
  # Maximum scan window for preamble. Beyond this, content is treated as
 
563
  """Public entry — turn an LLM reply into spoken-language text for TTS."""
564
  if not text:
565
  return ""
566
+ # Defense in depth: run the preamble strip again here in case this is
567
+ # called on a path that didn't already strip (e.g. direct TTS of a
568
+ # cached reply).
 
569
  cleaned = strip_cot_preamble(text)
570
  cleaned = _strip_markdown(cleaned)
571
+ # Currency/range shorthand expansion before acronym handling so ₹5L
572
+ # becomes "5 lakhs" instead of getting caught by the bare-L acronym
573
+ # path.
574
  cleaned = _normalize_money(cleaned)
575
  # #106 — AFTER money (lakh/crore already words): expand remaining bare
576
  # integers ("29" → "twenty-nine") + neutralise en/em dashes so Sarvam
rag/extract.py CHANGED
@@ -2,11 +2,9 @@
2
 
3
  For each PDF in rag/corpus/:
4
  1. Read full text via pdfplumber (already have per-page text from ingest)
5
- 2. Pass to LLM (NIM DeepSeek-V4-Pro primary, V4-Flash fallback) with the
6
- Pydantic schema as a structured-output target. Sarvam-M is NOT in the
7
- reasoning path per D-019 (it lives in Indic translation only — its
8
- 2048-token <think> reasoning + output cap was truncating JSON mid-stream
9
- when used as extraction fallback, causing ~20 extraction failures).
10
  3. Self-critique pass — LLM scores per-field confidence vs source text
11
  4. Validate via Pydantic
12
  5. Upsert into DuckDB `policies` table
@@ -73,13 +71,11 @@ EXTRACT_SYSTEM = """You extract structured fields from Indian health insurance p
73
 
74
 
75
  def build_extract_prompt(policy_text: str, schema_excerpt: str, policy_id: str) -> str:
76
- # Sarvam-M rejects prompts >~25k chars with HTTP 400 even though docs say
77
- # 32k tokens undocumented stricter limit. DeepSeek-V4-Pro on NIM handles
78
- # 1M context but the schema fits comfortably in 12k chars. 12k chars
79
- # (~3k tokens) captures Schedule + Key Definitions + Benefits front-matter
80
- # where the structured fields live. Exclusions section (the back half)
81
- # doesn't yield many structured fields, just policy_exclusions list which
82
- # is mostly noise.
83
  MAX_CHARS = 12_000
84
  if len(policy_text) > MAX_CHARS:
85
  # Front-bias: schedules, definitions, waiting periods, UIN, sum-insured
@@ -143,8 +139,8 @@ def load_manifest() -> dict:
143
  def json_from_llm_text(text: str) -> dict:
144
  """Strip code fences and <think> blocks, extract the first balanced {...} block."""
145
  text = text.strip()
146
- # Sarvam-M reasoning model emits <think>...</think> before the JSON; strip
147
- # them (handles both closed and unterminated think blocks when output is
148
  # truncated mid-thought).
149
  text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL)
150
  text = re.sub(r"<think>.*", "", text, flags=re.DOTALL)
@@ -265,10 +261,8 @@ async def extract_one(pdf_path: Path, manifest_entry: dict, llm_primary, llm_fal
265
  for attempt, llm in enumerate([llm_primary, llm_fallback]):
266
  try:
267
  # Hard per-attempt timeout so a hung TCP connection in httpx
268
- # pooling can't stall the whole sweep. NIM DeepSeek-V4-Pro has its
269
- # own 4-step backoff retry loop that can use up to ~5min on 429s,
270
- # so give it 120s ceiling. Sarvam (fallback) has no retries; 60s
271
- # is plenty.
272
  attempt_timeout = 180 if llm is llm_primary else 120
273
  res = await asyncio.wait_for(
274
  llm.chat(messages=messages, temperature=0.0, max_tokens=2048),
@@ -328,19 +322,8 @@ async def main():
328
  if args.limit:
329
  pdfs = pdfs[: args.limit]
330
 
331
- # Provider matrix (2026-05-14, Stack A consolidation per D-019):
332
- # - NIM DeepSeek-V4-Pro: 1M ctx, frontier MoE, MIT-licensed, free tier
333
- # 40 req/min with no daily cap. Clean JSON output, no <think> verbosity.
334
- # Used as primary because of the bigger reasoning budget.
335
- # - NIM DeepSeek-V4-Flash: 284B/13B MoE, ~3× faster than V4-Pro, same
336
- # clean JSON discipline. Used as fallback on V4-Pro ReadTimeout —
337
- # different MoE routing path so a Pro timeout doesn't always reproduce
338
- # in Flash. Sarvam-M was the old fallback but its <think> tags ate the
339
- # 2048 output cap before reaching JSON, causing ~20 truncation failures.
340
- # Three-chain collapse (2026-05-15): the separate FAST_BRAIN_CHAIN /
341
- # get_fast_brain_llm accessor was removed. One brain chain (get_brain_llm)
342
- # whose NimChainLLM already does internal multi-candidate fallback;
343
- # primary/fallback both resolve to it.
344
  primary = get_brain_llm()
345
  fallback = get_brain_llm()
346
 
 
2
 
3
  For each PDF in rag/corpus/:
4
  1. Read full text via pdfplumber (already have per-page text from ingest)
5
+ 2. Pass to the brain LLM (get_brain_llm) with the Pydantic schema as a
6
+ structured-output target. Sarvam-M is not in the reasoning path (it
7
+ is used for Indic translation only).
 
 
8
  3. Self-critique pass — LLM scores per-field confidence vs source text
9
  4. Validate via Pydantic
10
  5. Upsert into DuckDB `policies` table
 
71
 
72
 
73
  def build_extract_prompt(policy_text: str, schema_excerpt: str, policy_id: str) -> str:
74
+ # The policy text is capped at 12k chars (~3k tokens): that captures
75
+ # the Schedule + Key Definitions + Benefits front-matter where the
76
+ # structured fields live. The Exclusions section (the back half)
77
+ # doesn't yield many structured fields, just the policy_exclusions
78
+ # list which is mostly noise.
 
 
79
  MAX_CHARS = 12_000
80
  if len(policy_text) > MAX_CHARS:
81
  # Front-bias: schedules, definitions, waiting periods, UIN, sum-insured
 
139
  def json_from_llm_text(text: str) -> dict:
140
  """Strip code fences and <think> blocks, extract the first balanced {...} block."""
141
  text = text.strip()
142
+ # Reasoning models emit <think>...</think> before the JSON; strip them
143
+ # (handles both closed and unterminated think blocks when output is
144
  # truncated mid-thought).
145
  text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL)
146
  text = re.sub(r"<think>.*", "", text, flags=re.DOTALL)
 
261
  for attempt, llm in enumerate([llm_primary, llm_fallback]):
262
  try:
263
  # Hard per-attempt timeout so a hung TCP connection in httpx
264
+ # pooling can't stall the whole sweep. The primary gets a larger
265
+ # ceiling than the fallback.
 
 
266
  attempt_timeout = 180 if llm is llm_primary else 120
267
  res = await asyncio.wait_for(
268
  llm.chat(messages=messages, temperature=0.0, max_tokens=2048),
 
322
  if args.limit:
323
  pdfs = pdfs[: args.limit]
324
 
325
+ # get_brain_llm() returns a NimChainLLM that already does internal
326
+ # multi-candidate fallback, so primary and fallback both resolve to it.
 
 
 
 
 
 
 
 
 
 
 
327
  primary = get_brain_llm()
328
  fallback = get_brain_llm()
329
 
rag/retrieve.py CHANGED
@@ -31,25 +31,19 @@ def _safe_collection_get(
31
  where: Optional[dict] = None,
32
  include: Optional[list[str]] = None,
33
  ) -> Optional[dict]:
34
- """KI-107 (2026-05-15) — defensive wrapper around `collection.get(...)`.
35
-
36
- C5 port-in persona logged HTTP 500 "Error executing plan: Internal
37
- error: Error finding id" against the live HF Space. Chroma's
38
- `collection.get(ids=[...])` is documented to return empty lists when ids
39
- miss but the Rust-backed client can still raise on certain edge cases
40
- (legacy schema rows, where-clause + missing-id combinations, transient
41
- sqlite lock contention during HNSW compaction). The pre-KI-107 code
42
- wrapped these calls in bare `except: pass`, which silently masked the
43
- failure and let downstream callers index into None — producing the
44
- confusing "Error finding id" plan-executor message instead of a
45
- coherent retrieval fallback.
46
 
47
  This helper:
48
  - Returns None on ANY exception (caller MUST treat None as "miss").
49
  - Returns the raw dict (may have empty lists) on success.
50
- - Logs at WARNING level so the failure is observable in HF Space logs
51
- instead of swallowed silently. Critical for post-mortem of similar
52
- regressions.
53
  """
54
  try:
55
  kwargs: dict = {}
@@ -149,7 +143,7 @@ def _build_chunk(cid: str, doc: str, meta: dict, score: float) -> RetrievedChunk
149
  )
150
 
151
 
152
- # KI-034 (2026-05-14) — in-process retrieval cache. Keyed by
153
  # (query_text, top_k, sorted policy_ids, sorted insurer_slugs). Caps at 256
154
  # entries with FIFO eviction so memory stays bounded across long sessions.
155
  # Within a single chat session, users frequently rephrase or follow up on the
@@ -170,8 +164,8 @@ def _cache_key(
170
  insurer_slugs: Optional[list[str]],
171
  session_id: Optional[str] = None,
172
  ) -> tuple:
173
- # QUARANTINE-ISOLATION FIX (2026-05-16) — session_id MUST be part of the
174
- # cache key. The result set is session-dependent: when a session_id is
175
  # supplied the quarantine boost pass prepends that session's uploaded
176
  # PDF chunks. Without session_id in the key, a result computed for
177
  # session A (carrying A's private uploaded chunks) would be served
@@ -223,13 +217,12 @@ async def retrieve(
223
  This ensures the brain sees regulatory ceilings even when policy
224
  chunks dominate raw cosine.
225
 
226
- KI-118 (2026-05-15) — profile chunks are keyed by `profile_name_slug`
227
- (canonical user name) instead of session_id. Only named users have a
228
- profile chunk to boost. `session_id` is retained for the per-session
229
- quarantine (user-uploaded PDF) lookup but is no longer used for
230
  profile boosting.
231
  """
232
- # KI-034 — short-circuit identical-query re-asks via the LRU cache.
233
  # session_id is part of the key so one session's cached result (which
234
  # may carry that session's private quarantine chunks) is never served
235
  # to another session.
@@ -238,15 +231,12 @@ async def retrieve(
238
  if cached is not None:
239
  return cached
240
 
241
- # KI-049 (2026-05-14) — wider net for table-cell questions. The
242
- # post-fix 96-Q eval surfaced 8 failures on room-rent / sub-limit /
243
- # cap questions where the bot's reply omitted the room category
244
- # ("Single Private", "Twin Sharing", "General Ward"). Root cause:
245
- # the policy table with the cap structure is one chunk in Chroma,
246
- # but at top_k=5 it can lose to other relevant prose chunks. For
247
- # queries clearly asking about a structured cap, retrieve more
248
- # candidates so the table chunk has a higher chance of landing in
249
- # the LLM's context window.
250
  _TABLE_CELL_TRIGGERS = _re.compile(
251
  r"\b(room\s*rent|sub[\s\-]?limit|sub[\s\-]?limits|copay|co[\s\-]?pay|"
252
  r"cap\s+on|capped\s+at|sum\s+insured\s+limit|"
@@ -261,13 +251,12 @@ async def retrieve(
261
  embedder = embedder or VoyageEmbeddings()
262
  [query_vec] = await embedder.embed([query], input_type="query")
263
 
264
- # KI-102 (2026-05-15) privacy P0. The main retrieval pass MUST NEVER
265
- # return chunks of doc_type='profile' because those chunks are scoped to a
266
- # specific user's session_id. Profile chunks are exclusively surfaced via
267
- # the explicit per-session collection.get(ids=[f"profile_{session_id}"])
268
- # lookup below; allowing them through the cosine pass leaks another user's
269
- # profile into the current session (live smoke test of 15 personas caught
270
- # B4 retrieving profile chunks from smokeA_1, ki100_ve, smokeB_B2).
271
  #
272
  # We translate this to Chroma's `where` DSL using $ne (not-equals) on
273
  # doc_type. Combined with optional policy_id / insurer_slug filters via
@@ -284,17 +273,13 @@ async def retrieve(
284
 
285
  collection = get_collection()
286
 
287
- # Standard retrieval — KI-108 (2026-05-15) wraps collection.query() with
288
- # the same defensive pattern as KI-107's _safe_collection_get. Live test
289
- # post-KI-107 caught the bare query raising
290
- # `chromadb.errors.InternalError: Error executing plan: Internal error:
291
- # Error finding id` on EVERY user turn (HNSW index entry pointing to a
292
- # missing doc likely from a partial KI-102 profile-chunk write
293
- # corrupting the collection state). KI-107 only wrapped .get(); .query()
294
- # has the same failure mode. On error we degrade to empty retrieval:
295
- # the brain still answers (acknowledging it can't access the catalog)
296
- # instead of the user seeing KI-106's generic "something went wrong"
297
- # fallback for every message.
298
  out: list[RetrievedChunk] = []
299
  try:
300
  res = collection.query(
@@ -316,24 +301,20 @@ async def retrieve(
316
  effective_top_k, where, type(e).__name__, str(e)[:300],
317
  )
318
 
319
- # Profile boost pass — KI-118 (2026-05-15). Profile chunks are now
320
- # keyed by `profile_name_slug` (canonical user name), not session_id.
321
- # The orchestrator passes the slug only when the live session has a
322
- # known name; anonymous chats skip this branch entirely.
323
  if profile_name_slug:
324
  profile_chunk_id = f"profile_{profile_name_slug}"
325
- # KI-118 — filter by BOTH id AND name_slug metadata so any future
326
- # ID collision (or migration-era chunk written under a shared id)
327
- # still can't leak the wrong profile into this user's context.
328
  #
329
- # KI-107 (2026-05-15) — route through _safe_collection_get because
330
- # first-time named users (no profile saved yet) and certain transient
331
- # Chroma sqlite states cause collection.get(ids=[missing_id]) to
332
- # raise instead of returning empty lists. Before this fix the bare
333
- # `except: pass` masked the failure, the orchestrator's plan-executor
334
- # still saw an AttributeError on the swallowed-None path, and the
335
- # user got HTTP 500 "Error executing plan: Internal error: Error
336
- # finding id".
337
  prof_res = _safe_collection_get(
338
  collection,
339
  ids=[profile_chunk_id],
@@ -355,7 +336,7 @@ async def retrieve(
355
  out = [profile_chunk] + [c for c in out if c.chunk_id != profile_chunk_id]
356
  out = out[:top_k]
357
 
358
- # Quarantine boost pass — when the orchestrator passes a session_id, also
359
  # query the SEPARATE quarantine collection (user-uploaded PDFs) scoped to
360
  # this session. Results get a ×1.1 score boost so the user's own uploaded
361
  # doc ranks above generic policies when relevant. Quarantine lives in a
 
31
  where: Optional[dict] = None,
32
  include: Optional[list[str]] = None,
33
  ) -> Optional[dict]:
34
+ """Defensive wrapper around `collection.get(...)`.
35
+
36
+ Chroma's `collection.get(ids=[...])` is documented to return empty
37
+ lists when ids miss, but the Rust-backed client can still raise on
38
+ certain edge cases (legacy schema rows, where-clause + missing-id
39
+ combinations, transient sqlite lock contention during HNSW
40
+ compaction).
 
 
 
 
 
41
 
42
  This helper:
43
  - Returns None on ANY exception (caller MUST treat None as "miss").
44
  - Returns the raw dict (may have empty lists) on success.
45
+ - Logs at WARNING level so the failure is observable instead of
46
+ swallowed silently.
 
47
  """
48
  try:
49
  kwargs: dict = {}
 
143
  )
144
 
145
 
146
+ # In-process retrieval cache. Keyed by
147
  # (query_text, top_k, sorted policy_ids, sorted insurer_slugs). Caps at 256
148
  # entries with FIFO eviction so memory stays bounded across long sessions.
149
  # Within a single chat session, users frequently rephrase or follow up on the
 
164
  insurer_slugs: Optional[list[str]],
165
  session_id: Optional[str] = None,
166
  ) -> tuple:
167
+ # session_id MUST be part of the cache key (quarantine isolation).
168
+ # The result set is session-dependent: when a session_id is
169
  # supplied the quarantine boost pass prepends that session's uploaded
170
  # PDF chunks. Without session_id in the key, a result computed for
171
  # session A (carrying A's private uploaded chunks) would be served
 
217
  This ensures the brain sees regulatory ceilings even when policy
218
  chunks dominate raw cosine.
219
 
220
+ Profile chunks are keyed by `profile_name_slug` (canonical user name);
221
+ only named users have a profile chunk to boost. `session_id` is used
222
+ for the per-session quarantine (user-uploaded PDF) lookup, not for
 
223
  profile boosting.
224
  """
225
+ # Short-circuit identical-query re-asks via the LRU cache.
226
  # session_id is part of the key so one session's cached result (which
227
  # may carry that session's private quarantine chunks) is never served
228
  # to another session.
 
231
  if cached is not None:
232
  return cached
233
 
234
+ # Wider net for table-cell questions. The policy table holding the
235
+ # cap structure (room-rent / sub-limit / room category) is one chunk
236
+ # in Chroma and at top_k=5 it can lose to other relevant prose
237
+ # chunks. For queries clearly asking about a structured cap, retrieve
238
+ # more candidates so the table chunk has a higher chance of landing
239
+ # in the LLM's context window.
 
 
 
240
  _TABLE_CELL_TRIGGERS = _re.compile(
241
  r"\b(room\s*rent|sub[\s\-]?limit|sub[\s\-]?limits|copay|co[\s\-]?pay|"
242
  r"cap\s+on|capped\s+at|sum\s+insured\s+limit|"
 
251
  embedder = embedder or VoyageEmbeddings()
252
  [query_vec] = await embedder.embed([query], input_type="query")
253
 
254
+ # Privacy invariant: the main retrieval pass MUST NEVER return chunks
255
+ # of doc_type='profile' because those chunks are scoped to a specific
256
+ # user. Profile chunks are exclusively surfaced via the explicit
257
+ # per-session collection.get(ids=[f"profile_{session_id}"]) lookup
258
+ # below; allowing them through the cosine pass would leak another
259
+ # user's profile into the current session.
 
260
  #
261
  # We translate this to Chroma's `where` DSL using $ne (not-equals) on
262
  # doc_type. Combined with optional policy_id / insurer_slug filters via
 
273
 
274
  collection = get_collection()
275
 
276
+ # Standard retrieval — collection.query() is wrapped with the same
277
+ # defensive pattern as _safe_collection_get because .query() can raise
278
+ # `chromadb.errors.InternalError: Error executing plan` when an HNSW
279
+ # index entry points to a missing doc. On error we degrade to empty
280
+ # retrieval: the brain still answers (acknowledging it can't access
281
+ # the catalog) instead of the user seeing a generic "something went
282
+ # wrong" fallback for every message.
 
 
 
 
283
  out: list[RetrievedChunk] = []
284
  try:
285
  res = collection.query(
 
301
  effective_top_k, where, type(e).__name__, str(e)[:300],
302
  )
303
 
304
+ # Profile boost pass. Profile chunks are keyed by `profile_name_slug`
305
+ # (canonical user name), not session_id. The caller passes the slug
306
+ # only when the live session has a known name; anonymous chats skip
307
+ # this branch entirely.
308
  if profile_name_slug:
309
  profile_chunk_id = f"profile_{profile_name_slug}"
310
+ # Filter by BOTH id AND name_slug metadata so any ID collision
311
+ # (or migration-era chunk written under a shared id) can't leak
312
+ # the wrong profile into this user's context.
313
  #
314
+ # Route through _safe_collection_get because first-time named
315
+ # users (no profile saved yet) and certain transient Chroma
316
+ # sqlite states cause collection.get(ids=[missing_id]) to raise
317
+ # instead of returning empty lists.
 
 
 
 
318
  prof_res = _safe_collection_get(
319
  collection,
320
  ids=[profile_chunk_id],
 
336
  out = [profile_chunk] + [c for c in out if c.chunk_id != profile_chunk_id]
337
  out = out[:top_k]
338
 
339
+ # Quarantine boost pass — when the caller passes a session_id, also
340
  # query the SEPARATE quarantine collection (user-uploaded PDFs) scoped to
341
  # this session. Results get a ×1.1 score boost so the user's own uploaded
342
  # doc ranks above generic policies when relevant. Quarantine lives in a
tools/chunk_sweep.py CHANGED
@@ -30,9 +30,9 @@ ROOT = Path(__file__).resolve().parent.parent
30
  RESULTS_JSON = ROOT / "eval" / "chunk_sweep_results.json"
31
  RESULTS_MD = ROOT / "kb" / "calculations" / "chunk_sweep_results.md"
32
 
33
- # Sweep grid. Range widened 2026-05-13 prior narrow grid (600-1000) produced
34
- # identical scores across all cells because retrieval converges in that band.
35
- # This grid spans 6x range so chunk-boundary effects on retrieval are visible.
36
  GRID = [
37
  (400, 60),
38
  (600, 100),
 
30
  RESULTS_JSON = ROOT / "eval" / "chunk_sweep_results.json"
31
  RESULTS_MD = ROOT / "kb" / "calculations" / "chunk_sweep_results.md"
32
 
33
+ # Sweep grid. Spans a ~6x range so chunk-boundary effects on retrieval
34
+ # are visible (a narrow grid converges to identical scores because
35
+ # retrieval is insensitive within a small band).
36
  GRID = [
37
  (400, 60),
38
  (600, 100),
tools/set_hf_secrets.py CHANGED
@@ -1,10 +1,9 @@
1
  """Programmatically set HF Space secrets via the official HF API.
2
 
3
  No browser needed. Reads keys from local .env and pushes them as Space
4
- secrets so the deployed app can authenticate. Post-D-019 (Stack A
5
- consolidation): only Sarvam + Voyage + NVIDIA NIM keys are required.
6
- Legacy GROQ/OPENROUTER/CEREBRAS/DEEPSEEK keys are deleted from the Space
7
- to prevent confusion (they are no longer referenced by the code).
8
 
9
  Run:
10
  python tools/set_hf_secrets.py
@@ -23,15 +22,15 @@ ROOT = Path(__file__).resolve().parent.parent
23
  load_dotenv(ROOT / ".env")
24
 
25
  REPO_ID = "rohitsar567/InsuranceBot"
26
- # Active secrets read by the running code (post-D-022 + admin panel):
27
- # - SARVAM / NIM = the live providers
28
- # - VOYAGE = legacy (kept for back-compat with old extracted artifacts)
29
- # - OPENROUTER / GROQ = optional cross-provider fallbacks (re-added 2026-05-14)
30
- # - ADMIN_* = control-panel gate (IP allowlist + password)
31
  SECRETS_TO_SET = [
32
  "SARVAM_API_KEY", "VOYAGE_API_KEY", "NVIDIA_NIM_API_KEY",
33
  "OPENROUTER_API_KEY", "GROQ_API_KEY",
34
- "GOOGLE_API_KEY", # KI-179 (2026-05-15) — Gemini Flash via Google AI Studio
35
  "ADMIN_IP_ALLOWLIST", "ADMIN_PASSWORD",
36
  ]
37
  # Truly retired providers — delete from Space to prevent confusion.
 
1
  """Programmatically set HF Space secrets via the official HF API.
2
 
3
  No browser needed. Reads keys from local .env and pushes them as Space
4
+ secrets so the deployed app can authenticate. Only the keys in
5
+ SECRETS_TO_SET below are pushed; any other provider keys are deleted
6
+ from the Space to prevent confusion.
 
7
 
8
  Run:
9
  python tools/set_hf_secrets.py
 
22
  load_dotenv(ROOT / ".env")
23
 
24
  REPO_ID = "rohitsar567/InsuranceBot"
25
+ # Active secrets read by the running code:
26
+ # - SARVAM / NIM / GOOGLE = the live providers
27
+ # - VOYAGE = kept for back-compat with extracted artifacts
28
+ # - OPENROUTER / GROQ = optional cross-provider fallbacks
29
+ # - ADMIN_* = control-panel gate
30
  SECRETS_TO_SET = [
31
  "SARVAM_API_KEY", "VOYAGE_API_KEY", "NVIDIA_NIM_API_KEY",
32
  "OPENROUTER_API_KEY", "GROQ_API_KEY",
33
+ "GOOGLE_API_KEY", # Gemini Flash via Google AI Studio
34
  "ADMIN_IP_ALLOWLIST", "ADMIN_PASSWORD",
35
  ]
36
  # Truly retired providers — delete from Space to prevent confusion.
tools/upload_to_hf.py CHANGED
@@ -1,16 +1,16 @@
1
- """One-shot uploader to Hugging Face Space (post-D-019 — code-only).
2
 
3
  Pushes the code-only Space (minus build artifacts, .venv, node_modules, etc.) to
4
- huggingface.co/spaces/rohitsar567/InsuranceBot. Post-D-019 the large data
5
- (corpus PDFs + Chroma vectors + extracted JSONs) lives in the companion HF
6
  dataset rohitsar567/insurance-bot-data — the Space's Dockerfile pulls it via
7
  huggingface_hub.snapshot_download at image build time.
8
 
9
- KI-152 (2026-05-15): operates in SYNC mode. After `upload_folder` (add/replace),
10
- walks the local tree, lists the remote repo, and issues a second commit that
11
- DELETES any remote file that no longer exists locally. Prevents ghost files
12
- (renamed/deleted brochures) from lingering on the Space and being served by
13
- the Docker image after a rebuild. See KI-126 / KI-143 / KI-144 / KI-151.
14
 
15
  Run:
16
  python tools/upload_to_hf.py # full upload + sync delete
@@ -57,10 +57,9 @@ IGNORE = [
57
  "rag/corpus/**",
58
  "rag/vectors/**",
59
  "rag/extracted/**",
60
- # KI-128 (2026-05-15): backup dir + LFS pointers were leaking into the
61
- # Space push via the rag/corpus and rag/extracted symlinks, blowing the
62
- # Space's 1 GB LFS storage quota (Repository storage limit reached on
63
- # post_lfs_batch_info). The data path is HF Dataset, NOT the Space.
64
  "rag/_hf_dataset_backup/**",
65
  "**/_hf_dataset_backup/**",
66
  "rag/policies.duckdb", # LFS-tracked, comes from dataset at build
 
1
+ """One-shot code-only uploader to the Hugging Face Space.
2
 
3
  Pushes the code-only Space (minus build artifacts, .venv, node_modules, etc.) to
4
+ huggingface.co/spaces/rohitsar567/InsuranceBot. The large data (corpus
5
+ PDFs + Chroma vectors + extracted JSONs) lives in the companion HF
6
  dataset rohitsar567/insurance-bot-data — the Space's Dockerfile pulls it via
7
  huggingface_hub.snapshot_download at image build time.
8
 
9
+ Operates in SYNC mode: after `upload_folder` (add/replace), walks the
10
+ local tree, lists the remote repo, and issues a second commit that
11
+ DELETES any remote file not present locally. Prevents ghost files
12
+ (renamed/deleted brochures) from lingering on the Space and being served
13
+ by the Docker image after a rebuild.
14
 
15
  Run:
16
  python tools/upload_to_hf.py # full upload + sync delete
 
57
  "rag/corpus/**",
58
  "rag/vectors/**",
59
  "rag/extracted/**",
60
+ # Exclude the backup dir + LFS pointers: pushing them would blow the
61
+ # Space's 1 GB LFS storage quota. The data path is the HF Dataset,
62
+ # NOT the Space.
 
63
  "rag/_hf_dataset_backup/**",
64
  "**/_hf_dataset_backup/**",
65
  "rag/policies.duckdb", # LFS-tracked, comes from dataset at build
tools/upload_vectors_to_dataset.py CHANGED
@@ -2,8 +2,8 @@
2
 
3
  Run AFTER `.venv/bin/python -m rag.ingest` on the developer Mac. The Space's
4
  Dockerfile pulls rag/vectors/** at build time so the deployed container has a
5
- ready-to-serve index. As of 2026-05-14 the Space no longer auto-ingests on
6
- boot — see entrypoint.sh.
7
  """
8
  from __future__ import annotations
9
  import os
 
2
 
3
  Run AFTER `.venv/bin/python -m rag.ingest` on the developer Mac. The Space's
4
  Dockerfile pulls rag/vectors/** at build time so the deployed container has a
5
+ ready-to-serve index; the Space does not auto-ingest on boot (see
6
+ entrypoint.sh).
7
  """
8
  from __future__ import annotations
9
  import os