CLAUDE.md

CLAUDE.md — project memory for AI assistants

This file is read by Claude Code (and any compatible AI tool) at the start of a session in this repo. Keep it under ~200 lines and focused on stable, non-obvious facts a new contributor would need. For change history, look at git log, 80-audit/ENTERPRISE_AUDIT.md, and 70-docs/60-decisions/.

Project at a glance

• What: a voice-first AI advisor for Indian health insurance — RAG over a curated 206-document corpus (188 product PDFs across 21 insurer slugs + 18 regulatory IRDAI/NHA docs, 7,317 Chroma chunks post-KI-125→127 rebuild — wordings 5,401 · brochure 611 · regulatory 498 · prospectus 483 · cis 302 · curated 21 · profile 1), Sarvam STT/TTS, structural grounding (the single LLM brain quotes only what its tools returned), 21-insurer scorecard (regulatory tracked separately). Marketplace surfaces 148 catalogued cards across the 21 real insurer slugs (one card per IRDAI-filed product after KI-133 / KI-141 / KI-142 / KI-145 dedup, IndusInd-General slug per KI-144); 201 extracted JSONs + 253 curated policy_facts JSONs feed the structured side. The 21 internal Chroma slugs = 20 user-facing insurers + IndusInd-General + 1 regulatory bucket; the regulatory + profile slugs are filtered out of every user-facing count (KI-129 / KI-130 / KI-132).
• Live: https://rohitsar567-insurancebot.hf.space (HF Space; rebuild triggered on every push to origin main).
• Repos: origin is the HF Space at huggingface.co/spaces/rohitsar567/InsuranceBot. github is the mirror at github.com/rohitsar567/insurance-sales-bot. Data lives separately at huggingface.co/datasets/rohitsar567/insurance-bot-data (with a GitHub mirror that uses LFS).
• Local dev path: ~/Developer/Insurance Sales Bot/ (NOT ~/Documents/Personal/AI Work/... — the older path that occasionally shows up in stale scripts; iCloud-synced + TCC-restricted).

Voice UX (ADR-028)

One default voice mode, one fallback.

• Live ✓ (default ON) — useLiveConversation keeps the mic continuously open with VAD barge-in. The user can speak over the bot and it pauses TTS + aborts in-flight /api/chat. Pill in the toolbar is the toggle: green = on, red = off. State persists in localStorage.insurance_live_pref.
• 🎤 Push-to-talk — a labeled button. Click → suspends Live for one turn → fresh recorder with VAD silence-cutoff → submits → resumes Live (only if userPrefersLive is still on).
• Hands-free was removed entirely in KI-027. Anything in the codebase still referring to it is stale.
• Bot TTS plays via the in-DOM <audio> element inside Message (autoplay-on-mount via ref'd useEffect). Never use new Audio(url).play() — those detached instances are invisible to document.querySelectorAll("audio").pause() in the barge-in handler.

LLM stack (ADR-019 + ADR-026 → ADR-031 + ADR-032 → ADR-038 → ADR-040) — KI-080 → KI-087, KI-160, KI-167 → KI-179

Every LLM role is a NimChainLLM candidate pool, NOT a hardcoded single model. End-to-end spec: ADR-032. Chains now mix Google AI Studio (Gemini Flash) as Tier 0 primary, NIM as Tier 1 fallback, OpenRouter free as Tier 2 diversity pool (ADR-040 — supersedes ADR-038's NIM-only lock once KI-167 retired the <FF> trailer convention that motivated the lock). Native provider JSON mode (response_format={"type":"json_object"} on NIM, response_mime_type=application/json on Gemini) is the structured-output contract; cross-provider fallback is safe again as long as every candidate supports JSON mode natively. Chains preserve brain ↔ judge family diversity (Gemini brain ↔ Mistral / Llama-4 judge) so failovers can't produce circular grading.

• Three-tier election (ADR-040). Candidate pools per role:
  • Brain Fast (sales-brain fact-find per KI-167, fast QA): Google gemini-2.0-flash (PRIMARY) → NIM qwen3-next-80b-a3b-instruct → NIM mistralai/mistral-large-3-675b-instruct-2512 (675B dense) → NIM meta/llama-4-maverick-17b-128e-instruct (128B MoE) → OpenRouter nvidia/nemotron-3-super-120b-a12b:free → OpenRouter qwen/qwen3-next-80b-a3b-instruct:free → NIM nvidia/llama-3.3-nemotron-super-49b-v1.5 (last resort).
  • Brain Main (free-form QA + recommendation synthesis): Google gemini-2.5-flash (PRIMARY) → NIM mistralai/mistral-large-3-675b-instruct-2512 → NIM meta/llama-4-maverick-17b-128e-instruct → NIM qwen/qwen3-next-80b-a3b-instruct → OpenRouter nvidia/nemotron-3-super-120b-a12b:free → NIM nvidia/llama-3.3-nemotron-super-49b-v1.5 (last resort).
  • Judge (faithfulness Gate 4, KI-171 skips on fact_find + recommendation queries): NIM mistralai/mistral-large-3-675b-instruct-2512 (PRIMARY — different family from Gemini brain) → NIM meta/llama-4-maverick-17b-128e-instruct → OpenRouter qwen/qwen3-next-80b-a3b-instruct:free → NIM nvidia/llama-3.3-nemotron-super-49b-v1.5. Google AI Studio free tier: 1500 req/day, 15 req/min on Gemini 2.0/2.5 Flash. On total chain exhaustion, orchestrator returns a graceful error message to the user — fail-loud > fail-silent-with-garbage still holds at the chain-tail level.
• Probe-driven sticky primary election (KI-080, ADR-031, superseded by ADR-038 for candidate-pool scope). All three chains elect a sticky PRIMARY + BACKUP from a background probe within the NIM pool. backend/llm_health.py scores every candidate on (1 / max(50, latency_ms)) * success_rate and writes the current election to process state. NimChainLLM.chat() calls PRIMARY once; on real-time failure it falls to BACKUP and triggers an immediate probe refresh. Per-turn LLM call count: 1 (most cases) or 2 (PRIMARY fails real-time → BACKUP).
• Probe cadence + per-phase timeouts (KI-084, 119e0fd). Probe loop ticks at PROBE_INTERVAL_SEC = 300s. Probe max_tokens cut 5 → 1. Every chat call uses explicit httpx.Timeout(connect=2, read=12, write=2, pool=2) so a stuck NIM pool releases its TCP socket independently of the outer asyncio.wait_for. Rate-limit failures (HTTP 429 / RateLimit body) get a 1h sin-bin (DEGRADE_DURATION_LONG_S = 3600s).
• Proactive credit gating (KI-085, 8fc7979), now NIM-only scope. Election is gated by is_alive AND has_credits so quota-exhausted NIM models are excluded BEFORE the user hits a 429. Signal source within the NIM pool: per-model local 60-second rate-meter (gate at 35-of-40 req/min, headroom 5). Per-model rate-metering applies within the locked NIM pool.
• HF Space secrets (KI-081, updated KI-179). The chain config now consults GOOGLE_API_KEY (Tier 0, Brain Fast + Brain Main primary), NVIDIA_NIM_API_KEY (Tier 1 fallback + Judge primary), and OPENROUTER_API_KEY (Tier 2 diversity pool — KI-176 server-side models: [...] fallback within the OR free pool). GROQ_API_KEY remains in HF Space secrets but is dormant — Groq is no longer in any chain (KI-155 <FF> violation root cause remains a guard even though the <FF> convention itself is gone).
• Admin telemetry (KI-086, d90f8c0). GET /api/admin/llm-health returns {chains, candidates, recent_turns, snapshot_ts} with per-chain elected primary/backup, per-candidate health + credits + degraded-until, and last 20 turn outcomes. Admin "LLM Chain" tab auto-refreshes every 30s and renders whatever the chain config exposes (NIM-only post-KI-160).
• KI-025's 50/50 NIM ↔ Groq rotation (ADR-026) is deprecated — _balanced_brain_chain retained behind a feature flag for one-release rollback; the probe-driven NIM-only election picks the actually-faster candidate dynamically.
• Cold-start fallback. Before the first probe completes (process restart, HF Space rebuild), chain[0] is the initial primary and chain[1] is the initial backup, both NIM.
• Brain / fast-brain / judge primaries in steady state are Google gemini-2.5-flash (Brain Main), Google gemini-2.0-flash (Brain Fast / sales-brain fact-find), and NIM Mistral Large 3 675B (Judge), per the ADR-040 chain. Not hardcoded — elected primary follows live latency × success_rate × credits_available across the full three-tier pool; on Google quota exhaustion or 429, election falls naturally to NIM Mistral 675B (the Tier 1 fallback) without operator intervention.
• KI-079 escalation as last bite (87ee522) — modified by KI-167. If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on BRAIN_CHAIN (heavy brain, _TIMEOUT_S_ESCALATION = 15s, 35s chain budget). Post-KI-167 (ADR-039) the _canonical_fallback terminal step is gone — on heavy-brain exhaustion the orchestrator returns the ADR-038 graceful error message instead. Worst-case wall-clock before graceful error: 25s FAST + 15s heavy = 40s.
• NIM concurrency semaphore + serial probe (KI-088, 14ee008). Module-level asyncio.Semaphore(2) wraps every NIM HTTP call so our process never has >2 NIM requests in flight simultaneously, regardless of source (probe loop + admin polls + per-user turns all serialise through the same semaphore). Probe loop changed parallel→serial so the 6-NIM probe burst becomes a 1-slot trickle over ~12s. Inner 4-attempt exponential-backoff retry deleted from NvidiaNimLLM.chat() — KI-080 election + KI-079 escalation now handle failover. Result: latency-based failures (41s timeouts under self-saturation) dropped to zero; replaced by a parser-side bottleneck (KI-090).
• Lenient FF-block parser (KI-090, 11cf4b3) — RETIRED by KI-167 (ADR-039). Was needed because the <FF>...</FF> trailer convention was fragile under load. Post-KI-167 the sales brain uses NIM response_format={"type":"json_object"} for guaranteed structured output, so the whole _parse_ff_block strict → fenced → bare-JSON ladder is gone.
• Skip profile_extractor + faithfulness judge on fact-find turns (KI-091, 9813994). Both chains were credit-exhausted on the steady-state primary, hung fact-find turns for 20+s, and the extractor periodically returned {"name": null} which wrote into session.update_profile_field and wiped the captured name mid-session. Orchestrator short-circuits both chains behind intent == "fact_find"; the sales brain (KI-167) captures fields natively from its JSON-mode response body, and faithfulness scoring is meaningless on "what's your annual income?". QA-mode turns still run both chains. Live: name re-ask loop gone; fact-find p95 28s → 6-8s.
• Defensive None-guard in extractor merge (KI-094, f068094). Belt-and-braces companion to KI-091. On QA-mode turns the extractor still runs (correct: "I'm now 35" mid-recommendation should update the profile), but under load it periodically returns {"name": null, "age": null, ...} and the merge loop was writing every key including nulls back into the session. Added if new_value in (None, "", []): continue at the top of the extracted-fields loop in backend/orchestrator.py — null / empty-string / empty-list returns are now no-ops. Closes the same root cause (LLM-returned nulls wiping state) at a second layer; KI-091 prevents the extractor from running on fact-find turns at all, KI-094 makes it safe even when it does run.
• Remove IP allowlist from admin — password-only gate (KI-097, pending). Dropped ADMIN_IP_ALLOWLIST env + _ip_allowed() from backend/admin.py; _check_admin is now password-only against X-Admin-Password → ADMIN_PASSWORD env. Backend returns 401 Unauthorized (previously 404-to-hide). Frontend admin panel is always visible; password unlocks the live data. ADR-023 superseded — IP gating added zero security beyond a strong password and locked the operator out whenever the home IP changed.
• Drop function-local import logging in orchestrator (KI-101, 66eb4ed). Removed 6 inline import logging lines from backend/orchestrator.py; Python's scoping rule was promoting logging to function-local, causing UnboundLocalError when the asyncio.wait_for TimeoutError branch fired before reaching the inline import. Module-level import is the sole binding now.
• Profile RAG session isolation (KI-102, 4bb8da0). upsert_profile_chunk stamps session_id metadata on every chunk; retrieve() excludes doc_type == "profile" from the main pass; per-session profile lookup triple-checks meta.session_id == session_id in Python after the Chroma where-clause. Legacy chunks without session_id are silently refused (fail-closed). Cross-session PII leak (age / dependents / health conditions) closed. ADR-022 extended with session-isolation subsection.
• _canonical_fallback no_trailer loop-breaker (KI-103, 8ef5c43) — RETIRED by KI-167 (ADR-039). The _ff_failed_attempts + _ff_skipped_slots state and the entire _canonical_fallback branch are deleted; the failure mode they capped (silent <FF> trailer parse failures looping on the same slot) is gone because the sales brain uses NIM JSON mode.
• CoT / instruction-echo strip in voice_format (KI-104, 407f2a1). tts_preprocess now kills <think>...</think> blocks, **Reasoning:** / **Thought:** labels, [INTERNAL] blocks, sentence-anchored CoT starters ("Let me think...", "Step 1:..."). Emergency fallback to a generic acknowledger if the whole reply is CoT-shaped. Stops Sarvam from TTS-ing the bot's internal monologue.
• Recommendation closer wired (KI-105, 8a58fa1). RECOMMENDATION_CLOSER_PHRASES frozenset ("show me the top 3", "rank", "pitch me", "compare X vs Y") classified as recommendation / comparison BEFORE the FACT_FIND_TRIGGERS check, so a fully-fact-found user can never get bounced back into fact-find. Persona prompt gets RECOMMENDATION_CLOSER_ADDENDUM with a strict 3-policy ranked-shortlist contract (3 policies, one-line rationale each, IRDAI disclaimer, no hedging). ADR-008 extended with closer-mode subsection.
• Graceful TimeoutError + Exception on /api/chat (KI-106, 565bf31). handle_turn(...) wrapped in asyncio.wait_for(45s) with explicit except asyncio.TimeoutError + broad except Exception. Both return HTTP 200 with source="graceful_timeout" / graceful_exception" and an in-character recovery sentence instead of HTTP 500. Internal logger.exception still captures the full traceback for admin observability.
• _safe_collection_get helper for Chroma (KI-107, 3a9a14f). Wraps every collection.get(ids=[...]) and collection.get(where=...) call in backend/profile_rag.py in try / except Exception, returns None on miss with logger.warning(...). Closes the KI-102 per-session profile lookup raising on never-existed sessions on HF Space (Chroma version-dependent behaviour). None return is treated identically to a session_id mismatch — fail-closed.
• Chroma collection re-ingested + profile-write hardening (KI-112). KI-111 wrapped .query() so the bot survived the corruption, but every embedding query was raising InternalError: Error executing plan: Internal error: Error finding id and silently returning empty retrieval — the bot was answering 206 policies' worth of Qs without access to any policy chunk. Root cause: a pre-KI-102 deploy wrote a profile_anonymous chunk with NO session_id metadata; that legacy row poisoned every later coll.query(where={"doc_type": {"$ne": "profile"}}) and the damage spread across HNSW segments (full collection extraction surfaced 1580 / 7356 chunks across 148 policies as Error getting embedding). Fix: full re-ingest from rag/corpus/ PDFs → clean rag/vectors/ + two new write-time guards in backend/profile_rag.py::upsert_profile_chunk — (a) reject session_id that isn't a non-empty str, (b) reject any embedding whose length ≠ embedder.dimension or that contains None. Both guards log a WARNING and return without writing, so a future model-drift or bad-input event can't re-poison HNSW. 4 new regression tests in tests/test_profile_rag_isolation.py::TestUpsertRejectsBadInputs. Repaired vectors uploaded to HF dataset rohitsar567/insurance-bot-data via tools/upload_vectors_to_dataset.py so the Space rebuild picks up the clean index. Old corrupted Chroma archived at rag/_hf_dataset_backup/rag/vectors.corrupted.<ts>/.
• Chain budgets: brain 20s × 35s total, fast-brain 12s × 22s total, judge 30s × 75s total. With KI-080 only PRIMARY + BACKUP consume budget in the common case — leaves headroom for KI-079 escalation. KI-084 per-phase httpx timeouts are nested inside these budgets.
• STT/TTS/Translator = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M). Embeddings = local BGE-small-en-v1.5.
• Provider keys (post-KI-179). GOOGLE_API_KEY (Tier 0 — Gemini Flash primary), NVIDIA_NIM_API_KEY (Tier 1 fallback + Judge primary), OPENROUTER_API_KEY (Tier 2 diversity pool) required in .env (local) and HF Space environment (production). GROQ_API_KEY retained as dormant secret for one-flip re-enable, but no chain references it.

Fact-find loop (ADR-039 + ADR-040, supersedes ADR-030 / ADR-027) — KI-167 / KI-179

One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose. KI-167 (2026-05-15) ripped out backend/fact_find_brain.py (the <FF>{...}</FF> trailer convention + _canonical_fallback + scripted Question.prompt_en) and replaced it with backend/sales_brain.py — a single LLM-driven sales agent. KI-225 (2026-05-15) then consolidated further: sales_brain.py + qa_brain.py + faithfulness.py + persona.py + translator.py + profile_extractor.py + orchestrator.py were all removed (~5,200 LOC net) and merged into the current backend/single_brain.py — ONE Gemini-with-function-calling call per turn with tools save_profile_field, retrieve_policies, get_policy_facts, mark_recommendation. Faithfulness is now structural (the brain can only quote what its tools returned).

• Single brain call per turn via single_brain.handle_turn() calling GoogleGeminiLLM through the function-calling API. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-225 collapsed the whole brain stack onto a single Gemini call with tools. NIM remains as fallback via backend/nim_fallback.py (~100 LOC) only when GOOGLE_API_KEY is missing or Gemini hard-errors — NOT as a per-turn chain. Indic queries route to Sarvam-M via single_brain when _detect_language returns 'indic'.
• System prompt carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no prompt_en template.
• Deterministic post-processor (backend/sales_brain_normalizer.py) takes the LLM's loose captures dict and emits a {canonical_field: validated_value} map: alias resolution (location → location_tier), enum normalization (Bangalore → metro), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls — pure rules. Override: if LLM sets complete: true while any required slot is empty, force complete: false.
• Profile persistence (post-ADR-043, 2026-05-27). Captures flow through session.update_profile_field() into the in-memory SessionState.profile only. The previous backend/profile_store.save_profile() disk write and backend/profile_rag.upsert_profile_chunk() Chroma write were removed entirely when cross-session recall was retired (ADR-043). Closing the tab discards the profile.
• No scripted prompts, no canonical_fallback, no trailer convention. backend/needs_finder.py::GRAPH slot-id data stays as a schema source for the system prompt, but Question.prompt_en is dead text — never consulted by the fact-find branch. _canonical_fallback, _normalize_for_slot, _pick_opener, _NEUTRAL_OPENERS, _FAMILY_OPENERS, _contains_self_introduction, the lenient <FF> parser ladder, and the "Got that — {slot}." prefix logic are deleted.
• Outer 25s asyncio.wait_for ceiling retained. On total chain exhaustion (all of Google → NIM → OpenRouter fail in a single turn) the orchestrator returns the graceful error to the user (fail-loud) rather than cascading to a scripted reply. There is no scripted safety net — that is intentional.
• DELETED in KI-167: backend/fact_find_brain.py (441 LOC); _canonical_fallback + _pick_opener branches in backend/orchestrator.py; _ff_failed_attempts / _ff_skipped_slots session fields; every fact_find_brain::fallback:* telemetry variant; the lenient <FF> / fenced-json / bare-JSON-tail parser ladder.
• Persona-audit fixtures rewritten to assert on final captured state (which slots are filled with what) rather than per-turn slot order — the LLM may capture in different turn order than the pre-KI-167 rules engine did. See WS4.
• Natural-conversation escape (KI-045): intent_change phrases / off-topic questions still exit fact-find by routing through should_route_to_fact_find upstream of the brain.
• Indic queries route through Sarvam-M for translation on input + output; the sales brain runs in English on the translated text.

Session & profile lifecycle (ADR-043 supersedes ADR-041 + ADR-042)

• Sessions are in-memory only. ADR-043 (2026-05-27) removed the cross-session recall layer entirely. SessionState lives in process memory in session_state._sessions; idle entries are evicted after _TTL_SECONDS = 60 * 60. There is no on-disk profile store, no name-slug pointer, no persona_id JSON, no profile_rag Chroma chunk, no Welcome-Back prompt, no pending_profile_recall, no apply_pending_recall, no try_recall_by_name, no _extract_*_from_text extractors. Closing the tab or hitting Clear chat discards the profile permanently — that's the privacy contract.
• POST /api/session/clear is the canonical "Clear chat" endpoint. Body {session_id}; reply {cleared, new_session_id}. Wipes the in-memory entry via clear_session() and always mints a fresh UUID. The legacy POST /api/session/reset (KI-020) is left in place for backwards compatibility.
• State-recovery from chat_history (Bug #26, in-session only). If a container restart / >1 h idle blanked the server-side session BUT the browser is still on the same tab carrying the chat history, the brain enters STATE-RECOVERY MODE and silently re-captures the already-stated facts from history via save_profile_field. This is purely in-session resilience — it never reads disk, never crosses sessions.
• Sticky-session retry policy (kept from ADR-042). _gemini_call accepts is_sticky (read once from session.single_brain_sticky in handle_turn). Non-sticky: 1 retry @ 1.5 s (fast-fail to nim_fallback on cold-start). Sticky: 2 retries with jittered exp backoffs (1.5 s → 3 s, ±25 %). The user-facing canned reply on exhausted retries: "My model service had a brief blip on that turn — please send the same message again, it should go through now."
• Profile completeness gates on Profile.asked. Default dependents="self" pre-fill in the builder form no longer registers as "done"; profile_completeness_view + POST /api/profile mask any field not in the asked list before scoring. The "Your profile X% done" badge starts at 0% for a brand-new session and only ticks up on explicit captures.

Uploaded-PDF pipeline (ADR-044, 2026-05-27 hardening)

• Owning module: backend/uploaded_docs.py (~1100 LOC). Endpoint POST /api/upload-policy returns HTTP 200 within ~1 s after the heuristic baseline is written. LLM extraction runs in a background asyncio.create_task.
• Heuristic floor is a HARD guarantee, FAT post-KI-332. build_record() runs synchronously inside the upload HTTP call (sub-second), writing UPLOADED_DOCS_DIR/<pid>/record.json with cell-shape {value, source_pdf_path, source_quote, _confidence} BEFORE any LLM fires. Post-2026-05-27 expansion covers 28+ fields: sum-insured ladder, policy_type, min/max entry age, child entry days, lifelong renewability, grace period, free-look, geographic coverage, ICU capping, deductible, NCB cap, organ/CI/preventive/domiciliary/newborn presence, premium payment modes. Floor lifted from 47.8% to **65-70%** on PDFs containing the relevant text. Test Policy.pdf (8 MB) verified live at grade C / 65.2% comp / 6 rich sub-scores even when all LLM passes fail.
• Multi-pass per-section extraction (KI-332). For PDFs with len(text) ≥ 25_000, _multipass_extract_with_gemini() runs 7 Gemini calls in parallel via asyncio.gather. _EXTRACT_SECTIONS partitions HealthPolicy into identity/eligibility/financial/waiting_periods/coverage/limits/network_claims (~6 fields each). Each section uses a filtered _schema_excerpt_for_fields() and max_tokens=4096 — half the single-pass budget. Failure-isolated: any single section landing produces a partial dict that's strictly better than the heuristic floor when merged. Identity fields force-filled by the caller. On total multi-pass failure (0/7 sections), falls through to legacy single-pass + NIM chain.
• extract_one_for_upload() resolution order (DO NOT reorder).
  1. Hash-cache short-circuit — _find_cached_extraction(sha256(pdf_bytes)) returns a prior successful extraction → copy; llm_used='hash-cache'; ~1 s.
  2. Multi-pass per-section — fires when len(text) ≥ 25_000 chars. _multipass_extract_with_gemini runs 7 sections (_EXTRACT_SECTIONS) in parallel via asyncio.gather. Any section landing counts as success (partial HealthPolicy still merges); llm_used='gemini-2.5-flash-multipass'. Total failure → falls through to step 3.
  3. Gemini 2.5-flash single-pass — 3 attempts with jittered exp backoff (base * (2**i) * random.uniform(0.75, 1.25) for base=2.0); llm_used='gemini-2.5-flash#1|#2|#3'.
  4. NIM fallback — single attempt; llm_used='nim-fallback'.
  5. Heuristic floor — record.json already exists from step 0; card renders at floor (now ~65–70% post KI-332 expansion); status='failed'.
• Merge model. When the LLM payload lands, scalars are merged INTO the heuristic record.json (LLM value wins where non-empty, heuristic stays where LLM silent; cell-shape preserved). This is the same "extracted + curated overlay" model the catalogued 148 use via 40-data/policy_facts/.
• Status endpoint = scorecard endpoint by construction. _set_extraction_status at extraction-complete time calls backend.main._catalogue_scorecard(pid, None) — the SAME resolver /api/policies/{id}/scorecard uses primary. Falls back to build_scorecard(doc, insurer_reviews, profile=None) only when catalogue indices haven't refreshed. DO NOT call build_scorecard(doc, profile=None) without insurer_reviews — that was the 2026-05-27 bug (completeness_pct=17.4, grade=None) the cache-fix landed.
• Attribute trap. The Scorecard dataclass exposes .grade, NOT .overall_grade (only the wire ScorecardResponse model renames it). The status resolver reads _sc.grade — .overall_grade will silently return None on the dataclass.
• _MG_CACHE invalidation BEFORE scorecard resolve. _set_extraction_status bursts backend.main._MG_CACHE (the marketplace grade cache) BEFORE calling _catalogue_scorecard, so the resolver rebuilds the catalogue indices with the new card. Doing it after = stale cache.
• Provenance fields on every status response: llm_used (gemini-2.5-flash#N | nim-fallback | hash-cache | null) + llm_response_chars (size of raw LLM payload). Operator can verify which LLM landed the extraction without HF Space stdout access.
• Backfill on startup. backfill_extractions() is fired as an asyncio task from a @app.on_event("startup") hook — iterates UPLOADED_DOCS_DIR/*, skips any pid that already has rag/extracted/<pid>.json (unless force=True), runs extraction for the rest. Same logic exposed as admin endpoint POST /api/admin/upload/reextract?force=<bool>.
• Insurer detection. detect_insurer_slug() scans the first ~6000 chars of PDF text against 21 known insurer name patterns (_INSURER_NAME_PATTERNS). On hit, insurer_slug flips from generic 'user-upload' to the real slug — the Claim Experience sub-score then reads 40-data/reviews/<slug>.json (real IRDAI claim ratios). Fail-closed: no match ⇒ stays 'user-upload', no fabricated insurer name.
• Locked chat sequence (ADR-044 D4). Frontend's extractionInFlight flag gates Send + textarea + PDF button + every voice path (PTT/Sarvam/auto-fire) for the entire wait window. Choice prompt NEVER fires before card lands. Both branches of step 6 in page.tsx's handleFile push the choice prompt AFTER the prior card/fail message — DO NOT reorder.
• Post-card dive-in mode (KI-330). When the upload card lands, page.tsx calls setActiveUploadPid(r.policy_id). That pid then flows into every subsequent /api/chat request as view_context.active_policy_id. single_brain.handle_turn reads it and prepends an ACTIVE POLICY DIVE-IN block to the system instruction. DO NOT remove the wire — without it the brain pivots to "let me pull your recommendations" when the user asks waiting-period / room-rent / coverage questions about the just-uploaded PDF. Verified 9/10 grounded on 2026-05-27 audit.
• Live verification matrix (2026-05-27, commit 2a58c28): 5 PDFs (manipalcigna, hdfc-ergo, care-health, icici-lombard, star-health) × {upload, extraction, scorecard, premium baseline, premium older+PED, personalisation profile, RAG grounded answer} = 35 cells, 33 green (the 2 misses are honest: Test Policy.pdf 3/3 Gemini fails caught by heuristic floor; one "room rent" question on star-health is correctly answered but the keyword detector missed it).

Refusal precision (KI-046)

• Persona prompt now explicitly instructs the bot to refuse on fanciful / out-of-scope scenarios (space tourism, diamond-tipped surgery, fictional procedures) with a specific refusal sentence.
• Anti-pattern guarded against: "policy doesn't explicitly exclude it → maybe it's covered". This is wrong; absence-of-exclusion is not evidence-of-inclusion.

Routing invariants (ADR-N/A — orchestrator.py)

These are pinned by tests/test_routing_regression.py:

• classify_intent("What is the waiting period for PED in Activ Assure?") MUST return "qa", never "fact_find".
• should_route_to_fact_find("qa", profile_is_empty=True, ...) MUST return False — direct QA questions don't need a profile.
• The empty-profile force-route guard only applies when intent ∈ {"recommendation", "comparison"} (the CONTEXT_DEPENDENT_INTENTS frozenset).
• FACT_FIND_TRIGGERS matches with word-boundary regex (\b...\b), NOT substring — "hi" no longer fires on "which" / "this" / "high".

Retrieval cache (ADR not yet written — code self-documents)

rag/retrieve.py has an in-process LRU cache keyed by (query_normalized, top_k, sorted policy_ids, sorted insurer_slugs). Cap 256. Cache hit skips both Voyage embed + Chroma query. Invalidates on process restart.

Top-k boost for table-cell questions (KI-049): room rent / sub-limit / cap on / single-private / NCB / co-pay / day-care-limit / etc. triggers bump top_k from 5 → 10 for that one query, so the policy's structured cap-table chunk has a higher chance of landing in context. Confined to the trigger query only — does not pollute the cache for downstream non-table queries.

Repo bucket layout (KI-047 / KI-050 / KI-051)

Numbered top-level buckets for non-code artifacts (sort lexicographically in ls):

• 40-data/ ← formerly data/ — runtime/cached data. All Python string-path refs updated (KI-050). Dockerfile COPY paths updated (KI-051).
• 70-docs/ ← formerly docs/ — ADRs, design notes, decisions.
• 80-audit/ ← formerly audit_results/ — defect register + eval artifacts (this audit lives here).

Code dirs (backend/, frontend/, rag/, tools/, eval/, tests/, kb/) kept as-is — Python forbids leading-digit / hyphen package names, so renaming code dirs would break imports.

Admin panel (KI-048 / KI-052, refresh wiring fixed in KI-296 / ADR-042)

• Backend: GET /api/admin/profiles + GET /api/admin/performance, both behind _check_admin (X-Admin-Password header only, post-KI-097). Auth failure returns 401 Unauthorized. POST /api/admin/probe runs a fresh serial probe of every candidate model and updates each ModelHealth.tested_at. GET /api/admin/llm-health is read-only (KI-088) — never triggers probes.
• Frontend: admin HTML has 3 lazy-loaded tabs — Profile + Visitor Log (pulls /api/admin/profiles), Performance (pulls /api/admin/performance), LLM Chain (Refresh now button + 30 s auto-poll). Auth state preserved across tab switches.
• LLM Chain timer wiring (KI-296, 2026-05-27). Refresh now click, 30 s auto-poll, tab-entry refreshChain(), and tab re-entry partial refresh now ALL call fetchHealth() alongside fetchLlmHealth() and invoke renderUpdatedLabel(). Without this, STATE.health.updated_at (the source for the top-left "Last refresh / Next in" timer) was frozen at the login-time snapshot, so the operator could not tell from the timer whether probing was actually happening on click. The bottom-right FRESH badge reads from a different source (STATE.llmHealth.snapshot_ts) and was already reset correctly — keep both in lockstep.

Disk + storage hardening (ADR-029)

Three independent safety layers against ChromaDB HNSW bloat:

1. In-process tripwire — rag/ingest.py::_abort_if_hnsw_bloated aborts ingest if link_lists.bin > 500 MB. Called from rag/ingest.py, tools/ingest_kb_summaries.py, tools/ingest_reviews.py.
2. Hourly LaunchAgent — com.rohit.insurancebot.vectorbloat auto-deletes _hf_dataset_backup/ at > 20 GB; warns at 5 GB.
3. Disk-free tripwire — com.rohit.disk-free-tripwire alerts at < 20 GB free; critical at < 8 GB, dumps every ~/Developer subdir > 1 GB into the log.

All LaunchAgents must live under ~/Library/Scripts/, NOT ~/Documents/. macOS TCC blocks launchd from executing scripts inside iCloud-synced ~/Documents/ paths, silently exit-126.

What to read for what

• System tour: README.md (the master entry).
• Decisions with alternatives: 70-docs/60-decisions/ADR-*.md (28 ADRs as of 2026-05-15).
• Production-readiness defect register: 80-audit/ENTERPRISE_AUDIT.md.
• Data lineage: kb/AUDIT_TRAIL.md.
• Tests: tests/test_routing_regression.py (15 tests pinning routing + load-balance invariants).

Working-style note (personal memory, not a project decision)

Always parallelize independent work (per feedback_always_parallelize.md in personal memory). On any task touching this project: dispatch agents in parallel when subtasks are independent, batch tool calls in a single message when there are no dependencies. Sequential-by-default wastes wall-clock time.

Watch-outs

• Never use detached new Audio() — see "Voice UX" above.
• Never hardcode a single LLM model client (NvidiaNimLLM(model=...)) — always go through NimChainLLM(chain=...) so the call survives single-pool rate limits. (KI-033 migrated the last two stragglers — profile_extractor and fact_find_normalizer.)
• Never let new code add "hi" (or any single-word trigger) to FACT_FIND_TRIGGERS without word-boundary regex — substring matching brings back the KI-023 misrouting bug.
• Never add "qa" to CONTEXT_DEPENDENT_INTENTS — that brings back the headline KI-018 bug where QA questions get trapped in fact-find.
• Voyage free tier is 3 RPM. Affects only ingest (corpus rebuild); query-time uses Chroma vectors, no Voyage call. Don't worry about it on the hot path.
• HF Space rebuild is 5-8 min per push. Audits running against the live endpoint should be done AFTER the desired image is stably deployed, or the persona transcripts span multiple builds and become useless for A/B.
• Two image-only PDFs are explicitly EXCLUDED from the ingest pipeline: royal-sundaram/family-plus__brochure.pdf and aditya-birla/activ-one__brochure.pdf (pdfplumber returns 0 chars; OCR is out of scope). Activ One coverage is provided via the activ-health-individual wordings policy — do not re-add either brochure. KI-126 made this permanent — they are now removed from the source PDF set and the source-PDF total is 206 (188 product + 18 regulatory), with 201 extracted JSONs (the gap is the 2 image-only brochures + 3 documents whose extraction failed gracefully).
• The indusind-general slug did not exist anywhere in the codebase before 2026-05-15. Reliance General Insurance was rebranded to IndusInd General; KI-144 migrated insurer slug + policy IDs + Chroma metadata + marketplace alias mapping. Any code referencing reliance-general should either be retained as a legacy alias (one card remains under reliance-general for back-compat) or migrated to indusind-general. Do not silently merge the two — they're tracked as separate slugs.
• Voice mode now defaults OFF (KI-131 / KI-134 / KI-139 / KI-148). The Live pill renders red by default; the user must opt in. AudioContext.resume() is required to unlock TTS autoplay. VAD thresholds: rmsThreshold=18, voiceBandMinProp=0.20, noiseFloor * 1.8. TTS preprocess now expands k → thousand. Anything in the codebase still assuming default-ON Live mode is stale.
• Marketplace dedup is one card per IRDAI-filed product (KI-133 / KI-141 / KI-142 / KI-145). Aliases handle marketing renames (e.g. Reliance → IndusInd); sub-variants stay separate only when material terms differ. Card count is 166 across 19 real insurers — anything counting 138 or 188 or 206 against the marketplace is stale.
• sales_brain.py is the fact-find handler (KI-167, ADR-039; primary candidate is Google Gemini 2.0 Flash post-KI-179 / ADR-040). backend/fact_find_brain.py is deleted along with the <FF> trailer convention, _canonical_fallback, scripted Question.prompt_en, and the "Got that — {slot}." prefix. Never re-introduce a fallback to scripted prompts on the fact-find branch — on total three-tier chain exhaustion the orchestrator returns a graceful error message (fail-loud > fail-silent-with-script). The KI-150 historical note (fact_find_brain max_tokens 420 → 700) is moot post-KI-167 since the prose+trailer prompt shape that needed the larger budget no longer exists.
• Never hardcode NvidiaNimLLM(model=...) post-KI-179 — chains now mix Google (backend/providers/google_gemini_llm.py), NIM (backend/providers/nvidia_nim_llm.py), and OpenRouter (backend/providers/openrouter_llm.py) candidates. Always go through NimChainLLM(chain=...) so the call surfs the full three-tier failover; bypassing the chain reintroduces the single-provider single-point-of-failure that KI-080 / KI-160 / KI-179 collectively eliminated.

---

Last reviewed 2026-05-15 — KI-101..KI-112 landed (orchestrator stability + profile-RAG session isolation + recommendation closer + graceful chat error handling + Chroma re-ingest + profile-write hardening). Same day: KI-125..KI-150 landed (full corpus rebuild → 7,317 chunks; marketplace dedup → 166 cards; voice default OFF + VAD retune; IndusInd General slug migration from Reliance General). Same day: KI-160 / ADR-038 locked chains to NIM-only after KI-155's <FF> trailer contract violation. Same day: KI-167 / ADR-039 replaced fact_find_brain.py with backend/sales_brain.py (single LLM call per turn, native JSON mode, deterministic post-processor). Same day: KI-171 (judge skip on fact_find + recommendation), KI-175 (NIM chain reorder — nemotron demoted to last), KI-176 (OpenRouter models: [...] server-side fallback), KI-178 (live audit of OR free-tier JSON-mode support), KI-179 / ADR-040 added Google AI Studio (Gemini 2.0 / 2.5 Flash) as the Tier 0 primary on Brain Fast + Brain Main, with NIM as Tier 1 fallback and OpenRouter as Tier 2 diversity pool. ADR-038 is now superseded — the NIM-only lock was relaxed once the <FF> trailer convention that motivated it was retired.