fix(retrieval): KI-243 — CRITICAL: relax citation_grounding (chunk_offset optional)

Z2 5-persona test showed 15/15 retrieve_policies calls returned 0 chunks.
0/5 personas reached recommendation. Total blocker.

Z3 diagnosis (via /tmp/diag_retrieval.py): raw rag.retrieve returned 8
healthy chunks for the Rajesh profile (52, Pune, family of 2, 40L,
diabetes+BP). enforce_citation_grounding then dropped 8/8 because:

- The filter required chunk_offset or chunk_idx as a non-negative int
- brain_tools.retrieve_policies's dict-builder copies these keys from
RetrievedChunk: chunk_id / policy_id / policy_name / insurer_slug /
doc_type / source_url / chunk_text / score / min_entry_age / max_entry_age
- Neither chunk_idx nor chunk_offset is in that list — they're stored
on the dataclass but not copied to the dict the filter sees.

Brains cite policies by identity (policy_id, policy_name, UIN), not by
chunk offset. The grounding filter only needs to verify the chunk has
a real policy attached. Relaxed to:
- require: policy_id (non-empty str) + policy_name (non-empty str)
- drop: chunk_offset / chunk_idx requirements entirely

Diag rerun after fix: 8/8 chunks pass enforce_citation_grounding.
apply_profile_filter then correctly passes all 8 (Rajesh age=52 inside
every policy's entry_age band). No over-constraint regression.

Unblocks Path B end-to-end recommendations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/retrieval_filters.py

@@ -326,28 +326,30 @@ def empty_retrieval_guard(
 # ---------------------------------------------------------------------------
 
 def enforce_citation_grounding(chunks: Iterable[Any]) -> list[Any]:
-    """Drop chunks missing any of the three citation-critical fields.
-
-    The "chunk_offset" requirement maps onto either `chunk_offset` (newer
-    ingestions) or the existing `chunk_idx` (legacy + current). At least
-    one must be a non-negative int.
+    """Drop chunks missing citation-critical fields.
+
+    A citable chunk MUST expose:
+      - policy_id   (non-empty str)
+      - policy_name (non-empty str)
+
+    The chunk offset field (`chunk_offset` or legacy `chunk_idx`) is
+    INFORMATIONAL only — it is not required for citation grounding because
+    upstream call sites (e.g. brain_tools.retrieve_policies) build pruned
+    dicts that intentionally omit it, and the brain cites by policy
+    identity, not by chunk offset. Z2 live test (2026-05-15) showed 15/15
+    retrieve_policies calls returning 0 chunks because we required an
+    offset that the upstream builder never included → every chunk dropped
+    here even though raw retrieval was healthy.
     """
     kept: list[Any] = []
     for ch in chunks:
         m = _meta(ch)
         pid = m.get("policy_id")
         pname = m.get("policy_name")
-        # accept either field name; chunk_idx is the existing schema in rag/retrieve.py
-        offset = m.get("chunk_offset")
-        if offset is None:
-            offset = m.get("chunk_idx")
-
         if not pid or not isinstance(pid, str):
             continue
         if not pname or not isinstance(pname, str):
             continue
-        if not isinstance(offset, int) or offset < 0:
-            continue
         kept.append(ch)
     return kept