perf: KI-034 retrieval cache + KI-035 reorder FAST_BRAIN_CHAIN

KI-034 — Retrieval cache. In-process LRU keyed by
(query_text_normalized, top_k, sorted policy_ids, sorted insurer_slugs).
Cap 256 entries; FIFO/LRU eviction so memory stays bounded across long
sessions. Skips both the Voyage embed call AND the Chroma query +
regulatory/review boost passes on cache hit.

Typical chat session has lots of near-identical follow-ups ("waiting
period?" → "PED waiting period?" → "diabetes waiting period?"). Cache key
is lowercased + stripped so trivial reformatting hits the cache too.
Cache invalidates implicitly on process restart (HF Space rebuild).

Smoke: _cache_key("what is the waiting period?", 5, None, None) ==
_cache_key("WHAT IS the Waiting Period? ", 5, None, None) → True
_cache_get returns ['mock_chunk'] for either form.

KI-035 — FAST_BRAIN_CHAIN reorder. Fast brain serves the latency-sensitive
roles: fact-find, QA, paraphrase, normalize, extract. Throughput-wise
they're all sub-second by content size, so TTFT dominates perceived
latency, not raw capability. Nemotron Nano 30B benches at ~1.6s TTFT vs
Qwen 80B's ~2-3s, so it now sits at position 0; Qwen 80B drops to
position 1 (still serves on Nemotron pool degradation).

Before: Qwen 80B → Nemotron 30B → GPT-OSS → Qwen 122B → DeepSeek → Groq
After: Nemotron 30B → Qwen 80B → GPT-OSS → Qwen 122B → DeepSeek → Groq

Combined effect on fact-find turns:
• Each turn typically makes 2 fast-brain calls
(normalize_answer + paraphrase_question) + 1 brain call (the actual
reply). With Nemotron primary on those 2 fast calls, ~1s saved per
turn — meaningful when persona audits average 9-10s p50 today.

KI-025's NIM↔Groq 50/50 rotation still applies on top: half of all
fast-brain calls now start at Groq Llama-3.3-70B (sub-1s TTFT on LPU)
regardless of the chain reorder.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/providers/nvidia_nim_llm.py

@@ -220,8 +220,14 @@ BRAIN_CHAIN = [
 # is the lowest-TTFT free-tier option, so a fast-brain fall-through to it is
 # still acceptable from a latency-budget standpoint.
 FAST_BRAIN_CHAIN = [
-    "qwen/qwen3-next-80b-a3b-instruct",
-    "nvidia/nemotron-3-nano-30b-a3b",     # 1.6s response per Reddit benchmark
+    # KI-035 (2026-05-14) — reordered for latency. Fast brain serves
+    # fact-find + QA + paraphrase + normalize + extract: every single one
+    # of these is a sub-second job by content size, so the bottleneck IS
+    # TTFT, not capability. Nemotron Nano 30B hits ~1.6s; Qwen 80B is
+    # ~2-3s. Moved Nemotron to primary; Qwen 80B stays as next fallback so
+    # if Nemotron's NIM pool degrades we still get quality.
+    "nvidia/nemotron-3-nano-30b-a3b",     # ~1.6s TTFT (Reddit bench), NIM
+    "qwen/qwen3-next-80b-a3b-instruct",   # ~2-3s, NIM
     "openai/gpt-oss-120b",
     "qwen/qwen3.5-122b-a10b",
     "deepseek-ai/deepseek-v4-flash",

rag/retrieve.py

@@ -102,6 +102,49 @@ def _build_chunk(cid: str, doc: str, meta: dict, score: float) -> RetrievedChunk
     )
 
 
+# KI-034 (2026-05-14) — in-process retrieval cache. Keyed by
+# (query_text, top_k, sorted policy_ids, sorted insurer_slugs). Caps at 256
+# entries with FIFO eviction so memory stays bounded across long sessions.
+# Within a single chat session, users frequently rephrase or follow up on the
+# same topic ("what's the waiting period?" → "and the pre-existing diseases
+# waiting period?" → "and the diabetes-specific one?"). When the cache key
+# matches, we skip the Voyage embed call AND the Chroma collection.query() —
+# saving the round-trip + the Voyage 3 RPM tax. Cache invalidates implicitly
+# at process restart (HF Space deploy / reload).
+from collections import OrderedDict as _OrderedDict
+_RETRIEVAL_CACHE: "_OrderedDict[tuple, list]" = _OrderedDict()
+_RETRIEVAL_CACHE_MAX = 256
+
+
+def _cache_key(
+    query: str,
+    top_k: int,
+    policy_ids: Optional[list[str]],
+    insurer_slugs: Optional[list[str]],
+) -> tuple:
+    return (
+        (query or "").strip().lower(),
+        int(top_k),
+        tuple(sorted(policy_ids)) if policy_ids else None,
+        tuple(sorted(insurer_slugs)) if insurer_slugs else None,
+    )
+
+
+def _cache_get(key: tuple) -> Optional[list]:
+    if key not in _RETRIEVAL_CACHE:
+        return None
+    # LRU bump
+    val = _RETRIEVAL_CACHE.pop(key)
+    _RETRIEVAL_CACHE[key] = val
+    return val
+
+
+def _cache_set(key: tuple, value: list) -> None:
+    _RETRIEVAL_CACHE[key] = value
+    while len(_RETRIEVAL_CACHE) > _RETRIEVAL_CACHE_MAX:
+        _RETRIEVAL_CACHE.popitem(last=False)  # evict oldest
+
+
 async def retrieve(
     query: str,
     top_k: int = settings.RAG_TOP_K,
@@ -121,6 +164,12 @@ async def retrieve(
     This ensures the brain sees regulatory ceilings even when policy
     chunks dominate raw cosine.
     """
+    # KI-034 — short-circuit identical-query re-asks via the LRU cache.
+    cache_key = _cache_key(query, top_k, policy_ids, insurer_slugs)
+    cached = _cache_get(cache_key)
+    if cached is not None:
+        return cached
+
     embedder = embedder or VoyageEmbeddings()
     [query_vec] = await embedder.embed([query], input_type="query")
 
@@ -269,6 +318,9 @@ async def retrieve(
             # Review boost is additive; failure shouldn't kill the main result
             pass
 
+    # KI-034 — populate cache with the FINAL merged result so subsequent
+    # identical queries skip Voyage embed + Chroma query + boost passes.
+    _cache_set(cache_key, out)
     return out