docs: KI-080 — sync CLAUDE.md + ENTERPRISE_AUDIT + new ADR-031 + 11 stale KI-025/50-50 surface sweep

Documents the KI-080 architectural shift from "iterate every chain
candidate per call" (ADR-026 / KI-025 50/50 NIM ↔ Groq rotation) to
"probe-elected sticky primary with cross-provider backup" (ADR-031).

Files updated:
- CLAUDE.md — "LLM stack" section rewritten for KI-080 election;
KI-025 rotation marked deprecated.
- 80-audit/ENTERPRISE_AUDIT.md — KI-078/KI-079/KI-080 rows added to
"Fixes shipped today" table; executive-scorecard UX-latency + test-
coverage rows updated to cite KI-080 (supersedes KI-025).
- 70-docs/60-decisions/ADR-031-sticky-primary-election.md — NEW.
Full ADR following ADR-030 shape: Context, Decision, Alternatives,
Consequences, Files touched, Revisit at scale.
- 70-docs/60-decisions/ADR-026-provider-load-balancing.md — header
updated to "Superseded by ADR-031"; body retained as historical
record per ADR convention.
- 70-docs/60-decisions/ADR-030-llm-driven-fact-find.md — pointer to
ADR-026 swapped for ADR-031.
- README.md — 6 stale 50/50 surfaces rewritten (§1.2 model picks,
§3 demo runbook, Tech-7 brain selection, ASCII routing diagram,
§4.3 model stack table heavy + fast brain rows).
- backend/providers/README.md — nvidia_nim_llm.py + groq_llm.py
descriptions updated; test reference reframed.
- tests/README.md — TestProviderLoadBalancing marked as bypassed-
path regression pin (KI-080 election is the live behaviour).
- 70-docs/00-overview/roadmap.md — brain router bullet updated.
- 70-docs/10-architecture/safety-architecture.md — F-04 resolution
paragraph updated.

Final stale-reference grep: only 2 KI-025 mentions remain, both in
historical contexts in ENTERPRISE_AUDIT.md (KI-025's original "Fixes
shipped" row + the audit verification artifacts list). Both
appropriate to leave as historical record.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

70-docs/00-overview/roadmap.md

@@ -15,7 +15,7 @@ v1 is a **vertical slice**: 10 insurers × Health × ~80 policies × voice-first
 **Working product:**
 - Voice-first chat advisor over a curated corpus of Indian health insurance policies (~76 PDFs from 10 insurers, ingested into Chroma + DuckDB)
 - Multi-language: English + Hindi/Hinglish via Sarvam Saarika STT + Sarvam Bulbul TTS
-- Brain router (D-019): the brain chain (Qwen 80B primary, 50/50 with Groq Llama-3.3) (heavy intents) + V4-Flash (voice + fact-find), Mistral Large 3 675B judge. Sarvam-M scoped to Indic translation + voice only.
+- Brain router (D-019, KI-080): BRAIN_CHAIN with probe-elected primary across NIM Qwen 80B / Groq Llama-3.3 / OpenRouter candidates (heavy intents) + FAST_BRAIN_CHAIN with probe-elected primary across Nemotron 30B / Qwen 80B / Groq (voice + fact-find), Mistral Large 3 675B judge primary. Sarvam-M scoped to Indic translation + voice only.
 - 4-gate hallucination defense + auditable refusal log
 - 62-field structured extraction per policy
 - Clean Next.js + Tailwind frontend

70-docs/10-architecture/safety-architecture.md

@@ -50,7 +50,7 @@ In a regulated BFSI domain, the worst failure isn't "the bot looks slow" — it'
 
 **Description:** Sarvam-M emits `<think>...</think>` reasoning. If `max_tokens` is exhausted before `</think>` is reached, the reply is unusable. This was a recurring issue when Sarvam-M was the primary brain.
 **Detection:** `strip_think_tags()` checks for `<think>` without matching `</think>`.
-**Resolution (2026-05-14, D-019):** Sarvam-M moved out of the brain role entirely. The NIM brain chains (Qwen 80B / Nemotron 30B primaries, 50/50 with Groq Llama-3.3) handle all reasoning; they emit direct responses without `<think>` preambles. Sarvam-M remains only for Indic translation (Hinglish ↔ English), where its `<think>` doesn't interfere because translation outputs are short. F-04 cannot fire on the current stack.
+**Resolution (2026-05-14, D-019; refined 2026-05-15, KI-080):** Sarvam-M moved out of the brain role entirely. The NIM brain chains (Qwen 80B / Nemotron 30B / Groq Llama-3.3 / OpenRouter GPT-OSS as probe-elected candidates) handle all reasoning; they emit direct responses without `<think>` preambles. Per-turn primary is elected from a 60s background probe (KI-080, [ADR-031](../60-decisions/ADR-031-sticky-primary-election.md)) — adapts to live provider degradation. Sarvam-M remains only for Indic translation (Hinglish ↔ English), where its `<think>` doesn't interfere because translation outputs are short. F-04 cannot fire on the current stack.
 **Owner:** `backend/orchestrator.py`
 **Status:** Resolved by architecture change.
 

70-docs/60-decisions/ADR-030-llm-driven-fact-find.md

@@ -24,7 +24,7 @@ The root cause is architectural: a hardcoded state machine cannot natively expre
 
 **One brain call per fact-find turn produces a natural conversational reply plus a JSON tail block describing what was captured and what's next.**
 
-Implementation: `backend/fact_find_brain.py::drive_fact_find()`. The single call is `NimChainLLM(FAST_BRAIN_CHAIN, total_budget_s=12s)` so it benefits from the [ADR-026](ADR-026-provider-load-balancing.md) NIM ↔ Groq 50/50 rotation and the cumulative budget ceiling from KI-021.
+Implementation: `backend/fact_find_brain.py::drive_fact_find()`. The single call is `NimChainLLM(FAST_BRAIN_CHAIN, total_budget_s=22s)` so it benefits from the [ADR-031](ADR-031-sticky-primary-election.md) probe-driven primary election (which supersedes ADR-026's static 50/50 rotation) and the cumulative budget ceiling from KI-021.
 
 System prompt contains:
 

70-docs/60-decisions/ADR-031-sticky-primary-election.md

@@ -0,0 +1,90 @@
+# ADR-031 — Sticky primary election for LLM chains
+
+**Status:** Accepted — 2026-05-15
+**Owner:** Rohit Saraf
+**Supersedes:** None (extends [ADR-019](ADR-019-nim-single-provider-consolidation.md) + [ADR-026](ADR-026-provider-load-balancing.md))
+**Deprecates:** [ADR-026](ADR-026-provider-load-balancing.md)'s `_balanced_brain_chain` 50/50 NIM ↔ Groq rotation (kept as a feature-flagged bypass branch for one-release rollback; default-off).
+**Related:** [ADR-026](ADR-026-provider-load-balancing.md), [ADR-030](ADR-030-llm-driven-fact-find.md)
+
+## Context
+
+Live probe at commits `078ff45` / `87ee522` (2026-05-15) showed **70% of fact-find turns timing out at the chain-budget ceiling**. Telemetry stamped every failure as `fallback:timeout`. Root cause: `NimChainLLM.chat()` iterated every chain candidate sequentially per call — under sustained NIM concurrency throttling, the 5 NIM-hosted models in `FAST_BRAIN_CHAIN` all queued together, burning the 22s `total_budget_s` before the chain reached the cross-provider Groq fallback. Each fact-find turn was costing 5-6 NIM calls (every NIM candidate timing out one after the other) before anything actually responded.
+
+User observation that triggered this ADR: *"If background probes elect one LLM, why does each chat turn try every candidate and eat the rate/concurrency limit?"*
+
+The probe loop in `backend/llm_health.py` was already running every 60s and writing per-candidate latency + success scores — but only the admin tab read it. The hot path ignored it. `filter_chain()` was the only consumer, and it only filtered "known-dead" candidates; it didn't elect a preferred one.
+
+KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](ADR-026-provider-load-balancing.md)) was a static coin flip — fair in aggregate but blind to live degradation. When NIM was throttled, half of all brain calls still went to NIM first and ate the throttle queue before falling to Groq.
+
+## Decision
+
+**`NimChainLLM.chat()` uses probe-driven primary election.** Per call, the chain consults the latest probe state and:
+
+1. Calls the **elected PRIMARY** (highest-scored candidate at probe time).
+2. On real-time failure (timeout, HTTP error, structured failure), falls to the **elected BACKUP** — guaranteed to be a *different provider* from PRIMARY (NIM ↔ Groq ↔ OpenRouter).
+3. Triggers an asynchronous probe refresh so the next call uses fresh signal.
+4. If BACKUP also fails in the same turn, raises — orchestrator catches and applies KI-079 escalation to `BRAIN_CHAIN`, then `_canonical_fallback`.
+
+The election runs in `backend/llm_health.py`:
+
+- Score = inverse-latency × success-rate over the last few probes.
+- Sort descending. Top score = PRIMARY.
+- BACKUP = highest-scoring candidate from a *different provider* than PRIMARY. If only one provider has live candidates, BACKUP = second-highest candidate from PRIMARY's provider (graceful degradation).
+- Election runs at the end of every probe cycle (60s).
+
+### Cold-start fallback
+
+Before the first probe completes (process restart, HF Space rebuild), no election exists. The chain falls back to `chain[0]` as PRIMARY and `chain[1]` as BACKUP — i.e. the static chain order acts as the cold-start prior. The probe loop runs immediately on startup, so cold-start lasts < 60s in normal operation.
+
+### Per-chain election
+
+Each of the three chains gets its own election: `BRAIN_CHAIN` / `FAST_BRAIN_CHAIN` / `JUDGE_CHAIN`. A NIM Qwen 80B that's slow for the brain may still be the fast-brain primary if its latency on shorter prompts beats Nemotron at that moment. Probe runs once per chain so signal is per-role-specific.
+
+### Provider-aware backup election
+
+The provider-diversity rule on BACKUP is **mandatory, not advisory**. The whole point is that if PRIMARY is failing because NIM is throttled, BACKUP must NOT also be on NIM — otherwise the second call queues in the same throttle window. Implementation: every candidate carries a provider tag (`nim` / `groq` / `openrouter`); election iterates the sorted list and picks the first candidate with a different provider tag than PRIMARY.
+
+### Telemetry
+
+Every `chat()` call stamps the result with the actually-used model + whether BACKUP fired. Admin tab's existing "LLM Chain" tab can read the election state and surface each chain's current PRIMARY + BACKUP + last probe timestamp.
+
+## Alternatives considered
+
+| Option | Why rejected |
+|---|---|
+| **Keep ADR-026's 50/50 rotation, just add health filtering** (skip known-dead before flipping the coin) | Doesn't fix the case where NIM is *slow*, not dead. A 5s NIM call followed by a 1.5s Groq fallback is still 6.5s; probe-election would have just called Groq for 1.5s. Health filtering alone is a strict subset of what election does. |
+| **Per-call retry budget instead of per-call election** (let `chat()` keep iterating but cap at 2 calls) | Equivalent to BACKUP semantics but blind to which 2 candidates to pick. Election + 2-call cap is the same compute cost with strictly better candidate choice. |
+| **Stream PRIMARY + BACKUP in parallel, take the first to respond** (hedged request) | Doubles outbound rate-limit consumption on every call. The point of KI-080 is to *reduce* per-turn calls; hedging goes the wrong direction. Reconsider if NIM moves to paid tier where rate cap stops biting. |
+| **Hardcode PRIMARY per chain (e.g. Groq for brain, Nemotron for fast-brain) and skip the probe** | Loses adaptation. When Groq's free tier degrades (which it does, especially during US business hours), the probe lets us notice within 60s and elect NIM. Hardcoding pins us to whichever provider was best at deploy time. |
+| **Move to paid NIM and side-step the rate cap entirely** | Real recurring cost. Same answer as ADR-019 / ADR-026 — user opted out. Election is the free-tier-compatible alternative. |
+| **Probe-driven election — chosen.** Adaptive, single-call common case, two-call worst case in-turn, KI-079 as final escalation, provider-diverse backup by construction. | — |
+
+## Consequences
+
+| Win | Cost |
+|---|---|
+| **Per-turn LLM call count drops from 5-6 to 1 (most cases) or 2 (real-time failure).** Direct fix for the 70% timeout rate observed in live probe at `87ee522`. | One state read per `chat()` call. In-memory, negligible vs network call. |
+| **Adaptive to live degradation.** When NIM throttles, the probe notices within 60s and elects Groq; when Groq's LPU saturates, the probe elects back to NIM. KI-025's static 50/50 had no such feedback. | Probe load adds ~3 LLM calls per minute per chain on top of user traffic. Bounded; within free-tier budgets on both providers. |
+| **Provider-diverse backup is structural, not stochastic.** A NIM-PRIMARY call that fails always falls to a non-NIM BACKUP. Pre-KI-080, the chain's static order meant a Qwen-failed call might fall to another NIM model first. | If a provider has only one live candidate, BACKUP can't be provider-diverse and degrades to "second-highest same-provider". Documented graceful degradation. |
+| **KI-025's `_balanced_brain_chain` is bypassed** but retained as a feature-flag branch for one-release rollback. | ~30 LOC dead-pathway in `nvidia_nim_llm.py`. Slated for deletion in v1.1. |
+| **KI-079 escalation still applies.** If both PRIMARY and BACKUP fail in one turn (rare, would require simultaneous NIM + Groq degradation), orchestrator retries once on `BRAIN_CHAIN` (heavy brain, 35s budget) before `_canonical_fallback`. Election narrows the common case; KI-079 + canonical guard the tail. | None — same code path as before KI-080. |
+| All three chains' family-diversity invariants hold (Qwen brain candidates ↔ Mistral judge candidates; probe never elects a Mistral as brain or Qwen as judge — election is *within* each chain's candidate list, not across chains). | — |
+
+## Files touched (commit `6159c54`)
+
+- **Modified:**
+  - `backend/providers/nvidia_nim_llm.py` — `NimChainLLM.chat()` refactored: replaces sequential candidate-loop with elected-primary + elected-backup path. `_call_one()` extracted as the single-model HTTP call. `_balanced_brain_chain` retained but bypassed by default.
+  - `backend/llm_health.py` — extended for primary/backup election. Probe cadence 5min → 60s. New public API: `get_primary` / `get_backup` / `report_failure` / `report_success` / `provider_of`. Election state held in process memory with thread lock + degraded-until timestamps.
+- **Unchanged:**
+  - `backend/needs_finder.py::GRAPH` and `_canonical_fallback` — KI-070 / KI-072 / KI-074 paths survive.
+  - Chain definitions in `nvidia_nim_llm.py` (`BRAIN_CHAIN`, `FAST_BRAIN_CHAIN`, `JUDGE_CHAIN`) — same candidate lists, same family-diversity.
+  - `backend/llm_health.py::filter_chain` — still skips known-dead candidates as a belt-and-braces guard before election runs.
+
+## Revisit at scale (v2)
+
+- **Per-intent election.** Currently election is per-chain; with traffic shape data, we could elect different primaries for `qa` vs `comparison` vs `recommendation` (longer outputs benefit from Qwen, shorter from Nemotron — current single-primary-per-chain is a compromise).
+- **Probe cadence tuning.** 60s probe interval is fine at current traffic. At >100 concurrent users, 30s probe + 5min EWMA on score would adapt faster to mid-session degradation.
+- **Hedged calls** for `comparison` / `recommendation` intent where latency dominates UX (call PRIMARY + BACKUP in parallel, take first valid response). Rejected for v1 because of rate-limit consumption; reconsider on paid NIM.
+- **Cross-chain provider-budget arbitration.** Today each chain elects independently. If all three elect NIM, total NIM load goes up. A v2 election could enforce a per-minute provider-call budget across chains and force one chain to elect Groq if NIM budget is near cap.
+- **Multi-worker election.** Current `_STATE` is in-process memory; multi-worker deployments would each elect independently. Acceptable today (HF Space single-worker); port to a shared store if uvicorn workers > 1.
+- **Delete `_balanced_brain_chain` in v1.1** after one stable release confirms zero regressions from the bypass.

80-audit/ENTERPRISE_AUDIT.md

@@ -14,11 +14,11 @@
 | Data pipeline integrity | ✅ Fixed (HF Hub canonical restored + in-process HNSW tripwire) | — |
 | Operational observability | ⚠️ Partial (silent-LaunchAgent regression fixed; broader `except Exception:` audit pending) | P1 |
 | Product quality (factual accuracy) | 🟡 Routing fix shipped (KI-018 / KI-023). 5-Q smoke: 60% (was 0%). Full 96-Q post-fix landing. | **P0 — needs ≥90% for deployment** |
-| UX latency | 🟡 Chain-budget cap installed (KI-021), NIM↔Groq balance (KI-025), fast-brain reorder (KI-035). Clean 100-persona re-run owed for clean numbers. | **P0 — needs p95 < 3s** |
+| UX latency | 🟡 Chain-budget cap installed (KI-021), probe-driven primary election (KI-080, supersedes KI-025), fast-brain reorder (KI-035). Per-turn LLM calls dropped from 5-6 → 1-2. | **P0 — needs p95 < 3s** |
 | Profile-capture / slot-filling | ✅ Was a telemetry bug (KI-019), not a slot-filler bug. Fact-find branch now reports profile_updates. | — |
 | Language-handling fairness | ⚠️ Hinglish concern was a 20-persona sampling artifact. Real outliers: tax_planner archetype (4.6 refusals) + stream style (4.3) — open. | P1 — India-market regulatory risk |
 | Code hygiene | ✅ Loose tmp files removed; `fact_find_normalizer` + `profile_extractor` migrated to chain pattern (KI-033) | — |
-| Test coverage | 🟡 15 unit tests pinning KI-018 / KI-023 / KI-025 routing + load-balance invariants. Broader coverage still open. | P1 |
+| Test coverage | 🟡 15 unit tests pinning KI-018 / KI-023 / KI-080 routing + primary-election invariants. Broader coverage still open. | P1 |
 | Voice UX | ✅ Live default-on + clickable toggle + labeled push-to-talk + barge-in working (ADR-028) | — |
 | Fact-find robotic tone | ✅ LLM paraphraser w/ verifier (ADR-027) | — |
 | Secrets handling | ✅ Verified clean (.env never committed) | — |
@@ -265,6 +265,9 @@ The full eval was sending **every** QA question to `needs_finder`. Sample bot an
 | **KI-075** | **`5fc01a7`** | **Root cause of "still robotic" UX.** Live probe showed 4 of 5 fact-find turns hit `_TIMEOUT_S = 12s` asyncio.wait_for cap at exactly 13.2s latency — NIM cold-start eats 10-15s after a Space rebuild. Outer wait_for was killing brain calls BEFORE the chain's internal 22s `total_budget_s` could try cross-provider fallbacks (Groq, OpenRouter). Bumped `_TIMEOUT_S` to 25s. Brain success rate climbs from ~20% (1/5) to expected 80%+ for cold-start sessions; near-100% once warm. |
 | KI-076 | (HF dataset) | Disabled the `rohitsar567/insurance-bot-data` dataset viewer by uploading a README with `viewer: false` YAML frontmatter. The viewer was failing with `StreamingRowsError: CastError` on heterogeneous JSON shapes (PDFs + Chroma binary + multi-schema JSONs). Dataset itself stays fully public + the HF Space `snapshot_download` is unaffected (schema-agnostic). Page now shows clean "Viewer disabled" notice + the new README we wrote. |
 | KI-077 | `2bb3898` | "Build your profile" panel: added Name input field at top with "captured from chat" badge when populated. Backend `/api/profile/completeness` + `/api/profile` POST + `UserProfile` TypeScript type all extended with `name`. Panel pre-fills every field from the session's captured chat state via existing `initialProfile`. New `useEffect` keeps panel in sync when chat captures fields while panel is open. On Save, name persists to the named-profile JSON store (KI-040/062) so the user is auto-recognised on return visits. |
+| KI-078 | `078ff45` | LLM chain hardening: per-link timeout 12s → 6s so chain can try 3-4 candidates inside `total_budget_s=22s` instead of 1. Narrowed `except Exception` to re-raise `CancelledError`/`KeyboardInterrupt`/`SystemExit` so `asyncio.wait_for` actually bubbles. New `_fallback_reason` stamped on `FactFindOutcome` and surfaced as `fact_find_brain::fallback:timeout` / `:no_trailer` / `:empty_reply` / `:llm_error` in `TurnResult.brain_used` for production telemetry. |
+| KI-079 | `87ee522` | Two-layer chain hardening for fact-find. (1) FAST_BRAIN_CHAIN reorder: Groq Llama-3.3-70B promoted from position #5 to #2 (right after Nemotron primary) so cross-provider fallback is reached in ~6-7s of budget instead of ~20s. (2) Heavy-chain escalation: when `drive_fact_find()` raises `TimeoutError` on fast-brain, orchestrator retries once on `BRAIN_CHAIN` (Qwen 80B primary, 35s budget) before falling to `_canonical_fallback`. Adds `fallback:timeout_after_escalation` / `:llm_error_after_escalation` to telemetry vocabulary. Now serves as the last-bite safety net once KI-080's primary election is in place. |
+| **KI-080** | **`6159c54`** | **Sticky primary election for LLM chains ([ADR-031](../70-docs/60-decisions/ADR-031-sticky-primary-election.md)).** `NimChainLLM.chat()` refactored from "iterate every chain candidate sequentially per call" to "call the probe-elected PRIMARY once; on real-time failure, call the cross-provider BACKUP once and re-trigger the probe." `backend/llm_health.py` runs a 60s background probe loop that scores every candidate on (latency × success rate) and elects a sticky PRIMARY + provider-diverse BACKUP for each of `BRAIN_CHAIN` / `FAST_BRAIN_CHAIN` / `JUDGE_CHAIN`. **Per-turn LLM call count drops from 5-6 (sustained NIM degradation, every candidate queued + timed out) to 1 (most cases) or 2 (primary fails real-time → backup + re-probe).** ADR-026's `_balanced_brain_chain` (KI-025 50/50 NIM ↔ Groq rotation) is deprecated — the probe-driven election dynamically picks the actually-faster candidate instead of a fixed 50/50 coin. Code retained as a bypassed branch behind a feature flag for one-release rollback. Cold-start fallback (no probes complete yet) uses `chain[0]` as the initial primary. KI-079 escalation still applies as the last bite if both primary AND backup fail in the same turn. Inline tests: 7/7 OK (cold-start, lowest-latency primary, cross-provider backup, demote-on-failure, score-update, primary-success makes 1 call, primary-fail/backup-success makes exactly 2 calls). `tests/test_routing_regression.py` → 15/15 pass. |
 | D-001 | (multi) | ChromaDB HNSW bloat 3-layer prevention (ADR-029) |
 | D-002 | (LaunchAgent edit) | Three silently-failing LaunchAgent scripts fixed |
 | D-009 | `bcb7079` | Removed `tmp_*.py` debug files from repo root |

CLAUDE.md

@@ -18,16 +18,18 @@ This file is read by Claude Code (and any compatible AI tool) at the start of a
 - **Hands-free was removed entirely** in KI-027. Anything in the codebase still referring to it is stale.
 - **Bot TTS plays via the in-DOM `<audio>` element** inside `Message` (autoplay-on-mount via ref'd `useEffect`). Never use `new Audio(url).play()` — those detached instances are invisible to `document.querySelectorAll("audio").pause()` in the barge-in handler.
 
-## LLM stack (ADR-019 + ADR-026 partial supersession)
+## LLM stack (ADR-019 + ADR-026 → ADR-031 supersession) — KI-080
 
 Every LLM role is a `NimChainLLM` fallback chain, NOT a hardcoded single model. Chains preserve brain↔judge family diversity (Qwen brain ↔ Mistral judge) so failovers can't accidentally produce circular grading.
 
-- **Brain primary** rotates 50/50 between **NIM Qwen 80B** and **Groq Llama-3.3-70B** via per-call `random.random()`. Effectively 2× throughput across two independent rate caps.
-- **Fast-brain primary** is **NIM Nemotron Nano 30B** (~1.6s TTFT) with Qwen 80B as next fallback. Fast brain serves: fact-find, QA, paraphrase, normalize, extract — every latency-sensitive role.
-- **Judge** = Mistral Large 3 675B primary. Different family from brain → non-circular grading.
+- **Probe-driven sticky primary election (KI-080, [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md)).** All three chains (`BRAIN_CHAIN`, `FAST_BRAIN_CHAIN`, `JUDGE_CHAIN`) elect a sticky PRIMARY + provider-diverse BACKUP from a background probe. `backend/llm_health.py` runs a 60s probe loop that scores every candidate on (latency × success rate) and writes the current election to process state. `NimChainLLM.chat()` no longer iterates every candidate per call — it calls PRIMARY once; on real-time failure it falls to BACKUP (cross-provider by construction) and triggers an immediate probe refresh. **Per-turn LLM call count: 1 (most cases) or 2 (primary fails real-time → backup + re-probe).** Pre-KI-080 worst case under sustained NIM concurrency throttling was 5-6 NIM calls per turn, all queued and timing out.
+- **KI-025's 50/50 NIM ↔ Groq rotation ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) is deprecated.** `_balanced_brain_chain` is retained in `backend/providers/nvidia_nim_llm.py` behind a feature flag for one-release rollback but is bypassed by default; the probe-driven election picks the actually-faster candidate dynamically instead of a fixed coin flip.
+- **Cold-start fallback.** Before the first probe completes (process restart, HF Space rebuild), `chain[0]` is the initial primary and `chain[1]` is the initial backup; the probe overwrites both within 60s.
+- **Brain / fast-brain / judge primaries in steady state** are currently **NIM Qwen 80B** (heavy brain), **NIM Nemotron Nano 30B** (fast brain, ~1.6s TTFT), **Mistral Large 3 675B** (judge — different family from brain → non-circular grading). These are the typical probe winners but are no longer hardcoded — the elected primary follows live latency × success scores.
+- **KI-079 escalation as last bite.** If both PRIMARY and BACKUP fail in a single fact-find turn, orchestrator retries once on `BRAIN_CHAIN` (heavy brain, 35s budget) before falling to `_canonical_fallback`. Final guardrail; fires after probe + backup are exhausted, not before.
 - **STT/TTS/Translator** = Sarvam (Saarika v2.5 / Bulbul v2 / Sarvam-M).
 - **Embeddings** = local BGE-small-en-v1.5 (`backend/providers/local_embeddings.py`). Voyage is configured in `.env` for ingest if needed but not on the hot path.
-- **Chain budgets:** brain 20s × 35s total, fast-brain 12s × 22s total, judge 30s × 75s total. Per-link timeout is dynamically clipped to remaining budget.
+- **Chain budgets:** brain 20s × 35s total, fast-brain 12s × 22s total, judge 30s × 75s total. Per-link timeout is dynamically clipped to remaining budget. With KI-080, only PRIMARY + BACKUP consume the budget in the common case — leaves headroom for KI-079 escalation.
 
 ## Fact-find loop (ADR-030, supersedes ADR-027) — KI-070
 

README.md

@@ -54,7 +54,7 @@ Live URL: **https://rohitsar567-insurancebot.hf.space**. For each: try voice and
 | # | Question | What you should see | Why this question |
 |---|---|---|---|
 | 1 | *"What's the pre-existing disease waiting period under Care Supreme?"* | Specific number + `[Source: care-health/care-supreme/wordings, p.18]`. Brain: fast-brain chain (Nemotron 30B primary). | Single-field lookup — easiest competence check. |
-| 2 | *"Compare cataract waiting period in ICICI Elevate vs HDFC Optima Secure."* | Two-policy comparison with citations from both PDFs. Brain: BRAIN_CHAIN (Qwen 80B / Groq Llama-3.3 50/50). | Multi-policy reasoning — tests retrieval + tiered brain routing. |
+| 2 | *"Compare cataract waiting period in ICICI Elevate vs HDFC Optima Secure."* | Two-policy comparison with citations from both PDFs. Brain: BRAIN_CHAIN with probe-elected primary (KI-080). | Multi-policy reasoning — tests retrieval + tiered brain routing. |
 | 3 | *"Care Supreme mein PED ka waiting period kya hai?"* (Hinglish) | Answer in Hinglish with citations preserved. Brain tag includes `cascade::sarvam-trans+...` if drift gates fire. | Indic cascade + 3-gate drift verification. |
 | 4 | *"What does IRDAI say about cataract waiting-period caps under the 2024 Master Circular?"* | Cited answer from `irdai-master-circular-health-2024.pdf`. | Demonstrates the IRDAI corpus rescue past Akamai (ADR-017). |
 | 5 | *"Does Bajaj Silver Health cover space-tourism injuries?"* | **Safe refusal.** | Adversarial out-of-corpus — refusal is the correct behaviour. |
@@ -233,9 +233,9 @@ The bot is two flows running together — the customer's experience and the tech
 </td><td>
 
 **Tech 7 — Brain selection, `pick_brain(intent, language)`**
-- `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN (Qwen 80B primary, 50/50 with Groq Llama-3.3)
-- `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN (Nemotron Nano 30B primary)
-- Per-call rotation across two providers' independent rate caps
+- `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN with probe-elected primary across NIM / Groq / OpenRouter candidates (KI-080)
+- `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN with probe-elected primary (typically Nemotron Nano 30B or Groq Llama-3.3 in steady state)
+- KI-080 election: 1 LLM call per turn (most cases) or 2 (primary fails real-time → cross-provider backup). Per-chain primary refreshed every 60s by background probe.
 
 **Tech 8 — System prompt construction, `build_messages()`**
 - `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
@@ -391,9 +391,9 @@ The bot is two flows running together — the customer's experience and the tech
 ┌──────────────────────────────────────────────────────────────────────────┐
 │  TECH 7.  Brain selection — pick_brain(intent, language)                  │
 │  -------                                                                  │
-│  · intent ∈ {comparison, recommendation} → the brain chain (Qwen 80B primary, 50/50 rotated with Groq Llama-3.3) (heavy)        │
-│  · intent ∈ {qa, fact_find} → the fast-brain chain (Nemotron Nano 30B primary, ~1.6s TTFT) (fast)                    │
-│  · All via integrate.api.nvidia.com (single NIM API key)                  │
+│  · intent ∈ {comparison, recommendation} → BRAIN_CHAIN with probe-elected primary across NIM/Groq/OpenRouter (heavy, KI-080)      │
+│  · intent ∈ {qa, fact_find} → FAST_BRAIN_CHAIN with probe-elected primary (fast, KI-080)                                          │
+│  · NIM Qwen 80B / Nemotron 30B + Groq Llama-3.3 + OpenRouter GPT-OSS as election candidates                                       │
 └──────────────────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
@@ -552,8 +552,8 @@ Every LLM role is served by a **fallback chain** of candidate models (`backend/p
 
 | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
 |---|---|---|---|---|
-| **Heavy brain** (comparison, recommendation, synthesis) | Qwen 3-Next 80B *or* Groq Llama-3.3-70B (50/50 per-call rotation, KI-025) | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + Groq + OpenRouter | Qwen 80B is the fastest reliable frontier-tier MoE on NIM free tier (~2s TTFT, clean JSON, multilingual). Groq LPU is the lowest-TTFT free-tier option; 50/50 split doubles sustained throughput across two independent rate caps ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)). |
-| **Fast brain** (voice turns, fact-find, QA, paraphrase, normalize, extract) | Nemotron Nano 30B *or* Groq Llama-3.3-70B (50/50 per-call rotation) | Qwen 3-Next 80B → GPT-OSS 120B → Qwen 3.5 122B → DeepSeek V4-Flash → Groq Llama-3.3-70B | NIM + Groq | Bottleneck on these short jobs is TTFT, not capability. Nemotron Nano hits ~1.6s; Qwen 80B is ~2-3s. Reordered for latency in KI-035. |
+| **Heavy brain** (comparison, recommendation, synthesis) | Probe-elected primary (KI-080) — typically Qwen 3-Next 80B or Groq Llama-3.3-70B in steady state | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B (all as election candidates; probe scores them every 60s) | NIM + Groq + OpenRouter | Probe-driven election picks the actually-faster candidate dynamically (latency × success-rate score), with provider-diverse BACKUP. Pre-KI-080 static 50/50 ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)) was deprecated in favour of [ADR-031](70-docs/60-decisions/ADR-031-sticky-primary-election.md). |
+| **Fast brain** (voice turns, fact-find, QA, paraphrase, normalize, extract) | Probe-elected primary (KI-080) — typically Nemotron Nano 30B or Groq Llama-3.3-70B in steady state | Qwen 3-Next 80B → GPT-OSS 120B → Qwen 3.5 122B → DeepSeek V4-Flash → Groq Llama-3.3-70B (election candidates) | NIM + Groq | Bottleneck on these short jobs is TTFT, not capability. Nemotron Nano hits ~1.6s; Qwen 80B is ~2-3s. Reordered for latency in KI-035. Election adds adaptation: when one provider degrades, the probe re-elects within 60s. |
 | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
 | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
 | **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |

backend/providers/README.md

@@ -7,8 +7,8 @@ Every external model is fronted by a small typed client here. The orchestrator a
 | File | Provider | Role | Notes |
 | --- | --- | --- | --- |
 | `base.py` | — | Abstract `LLM`, `STT`, `TTS`, `Embeddings` Protocols. Every concrete client conforms. | — |
-| `nvidia_nim_llm.py` | NVIDIA NIM | Core chain runner — `NimChainLLM(chain=[...])` walks a fallback ladder under a wall-clock budget. Exposes `get_brain_llm()`, `get_fast_brain_llm()`, `get_judge_llm()`. Also home of `_balanced_brain_chain()` (50/50 NIM ↔ Groq rotator). | [ADR-019](../../70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md), [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md) |
-| `groq_llm.py` | Groq | Single-call Llama-3.3-70B client. Used as the 50% load-balance primary for the brain chain, never standalone. | [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md) |
+| `nvidia_nim_llm.py` | NVIDIA NIM | Core chain runner — `NimChainLLM(chain=[...])` uses probe-driven primary election (KI-080): calls the elected PRIMARY once per turn, falls to elected BACKUP on real-time failure. Exposes `get_brain_llm()`, `get_fast_brain_llm()`, `get_judge_llm()`. Legacy `_balanced_brain_chain()` (50/50 NIM ↔ Groq rotator) retained as a bypassed feature-flag branch for one-release rollback. | [ADR-019](../../70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md), [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) (supersedes [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md)) |
+| `groq_llm.py` | Groq | Single-call Llama-3.3-70B client. Used as cross-provider backup election candidate (KI-080) for both brain + fast-brain chains. | [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) |
 | `openrouter_llm.py` | OpenRouter | Multi-model fallback rung (DeepSeek-V3 etc.) for chains; rarely the primary in production. | — |
 | `sarvam_llm.py` | Sarvam-M | Indic-aware LLM; on the judge / translator fallback chains and used by `backend/translator.py`. | [ADR-006](../../70-docs/60-decisions/ADR-006-sarvam-first-stack.md) |
 | `sarvam_stt.py` | Sarvam Saarika v2.5 | Speech-to-text (10 Indic languages + English). | ADR-006 |
@@ -36,5 +36,5 @@ Per-link timeout is dynamically clipped to remaining budget.
 ## Related
 
 - [ADR-006](../../70-docs/60-decisions/ADR-006-sarvam-first-stack.md), [ADR-011](../../70-docs/60-decisions/ADR-011-bge-local-embeddings.md), [ADR-019](../../70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md), [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md)
-- `tests/test_routing_regression.py::TestProviderLoadBalancing` — pins the 50/50 split
+- `tests/test_routing_regression.py::TestProviderLoadBalancing` — pins the legacy 50/50 split (kept as a bypassed-by-default invariant; KI-080's probe-driven election supersedes it for live traffic)
 - `40-data/llm_health.json` — last health-probe snapshot surfaced in the admin tab

tests/README.md

@@ -6,7 +6,7 @@ Deliberately small. The bulk of behavioural quality lives in `eval/` (gold-QA ac
 
 | File | Role |
 | --- | --- |
-| `test_routing_regression.py` | 15 `unittest` cases pinning the KI-018 / KI-023 / KI-025 fixes — see "Routing invariants" in the root `CLAUDE.md`. Includes `TestProviderLoadBalancing` which asserts the 50/50 NIM ↔ Groq split holds over 1000 seeded calls ([ADR-026](../70-docs/60-decisions/ADR-026-provider-load-balancing.md)). |
+| `test_routing_regression.py` | 15 `unittest` cases pinning the KI-018 / KI-023 / KI-080 invariants — see "Routing invariants" in the root `CLAUDE.md`. Includes legacy `TestProviderLoadBalancing` which asserts the 50/50 NIM ↔ Groq split when `_balanced_brain_chain` is invoked directly; with KI-080 ([ADR-031](../70-docs/60-decisions/ADR-031-sticky-primary-election.md)) live traffic uses probe-elected primary instead, but the rotation invariant is retained as a regression pin for the bypassed legacy path. |
 | `live_verify.py` | End-to-end production drift detector. Hits the **deployed** API with a 20-Q gold subset and asserts HTTP 200, non-empty `reply_text`, ≥1 citation, faithfulness pass, and Doc-01 latency budget (p95 ≤ 7000ms). Writes `tests/live_results_<ts>.md`. Cron-able for nightly. |
 
 ## What each test pins
@@ -16,7 +16,7 @@ Deliberately small. The bulk of behavioural quality lives in `eval/` (gold-QA ac
 | KI-018 (D-003) | `classify_intent("What is the waiting period for PED in Activ Assure?")` returns `"qa"` and `should_route_to_fact_find` returns `False` on empty profile. | Headline 30% gold-QA accuracy bug — direct QA was force-routed to fact-find. |
 | KI-018 | `CONTEXT_DEPENDENT_INTENTS = {"recommendation", "comparison"}` — no `"qa"`. | Adding `"qa"` re-introduces the headline bug. |
 | KI-023 | `FACT_FIND_TRIGGERS` uses word-boundary regex, not substring. | Stops `"hi"` firing on `"which"` / `"this"` / `"high"`. |
-| ADR-026 / KI-025 | `_balanced_brain_chain(..., groq_first_probability=0.5)` lands Groq-primary between 400 and 600 of 1000 seeded calls. | Catches the shared-counter pathology where every brain call lands on one provider. |
+| ADR-026 / KI-025 (legacy; superseded by [ADR-031](../70-docs/60-decisions/ADR-031-sticky-primary-election.md) / KI-080) | `_balanced_brain_chain(..., groq_first_probability=0.5)` lands Groq-primary between 400 and 600 of 1000 seeded calls. | Catches the shared-counter pathology where every brain call lands on one provider. Test retained as a bypassed-path regression pin. |
 
 ## Running