rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
28b6114
·
1 Parent(s): 930e11e

fix: KI-046 — explicit refusal on adversarial/fanciful out-of-corpus questions

Browse files

The 93-of-96 post-fix eval surfaced refusal precision as the single
biggest failure mode: 11 of the 42 failures (26%) were "did not refuse
when expected" — mostly variants of:

• "Does X cover injuries from space tourism?"
• "What is the maximum claim amount for diamond-tipped surgical
equipment?"
• "What is the IRDAI mandate on dental coverage that X must follow?"

These are intentional adversarial tests of refusal hygiene. The bot was
treating them as ordinary coverage questions, retrieving the closest-
related policy text, and answering with mis-selling-shaped reasoning
like "the policy doesn't explicitly exclude this, so it might be
covered."

Fix: new rule 1a in `ADVISOR_SYSTEM_PROMPT_V1`. The bot is now
instructed to refuse cleanly when the question asks about a fanciful
or obviously-out-of-corpus scenario, with the exact refusal phrasing:

"I don't have grounded evidence for that in any of the policy
documents I've indexed — I'd rather not speculate. Is there a
different coverage question I can help with?"

Explicit guard against the "policy doesn't say it's excluded → maybe
it's covered" reasoning that BFSI auditors flag as mis-selling.

Expected eval lift: if the refusal triggers correctly on the 11 OOS
questions, factual accuracy moves from 54.8% → ~66.7%. Real number
landing in the next eval run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. backend/persona.py +6 -0
backend/persona.py CHANGED
@@ -23,6 +23,12 @@ ABSOLUTE RULES (these are non-negotiable)
23
  "I don't see that covered in this policy document. Would you like me to check what IS covered in this category?"
24
  Hallucinated facts in BFSI = mis-selling = regulated offense.
25
 
 
 
 
 
 
 
26
  2. CITATION GRAMMAR. End every factual claim with an inline citation.
27
  - For policy clauses: [Source: <Policy Name> (<insurer-slug>), p.<page>]
28
  - For regulatory mandates: [Regulation: <Doc Name> (IRDAI / Govt), §<section>]
 
23
  "I don't see that covered in this policy document. Would you like me to check what IS covered in this category?"
24
  Hallucinated facts in BFSI = mis-selling = regulated offense.
25
 
26
+ 1a. REFUSE ADVERSARIAL / FANCIFUL / OUT-OF-CORPUS QUESTIONS (KI-046).
27
+ Some questions ask about scenarios that no reasonable health insurance policy would address — space tourism injuries, diamond-tipped surgical equipment, injuries from a meteor strike, etc. The retrieved context will NOT contain these — they're tests of refusal hygiene.
28
+ When the question asks about something that is OBVIOUSLY not in any real policy (or any IRDAI regulation), refuse cleanly:
29
+ "I don't have grounded evidence for that in any of the policy documents I've indexed — I'd rather not speculate. Is there a different coverage question I can help with?"
30
+ Do NOT try to reason "well, the policy doesn't say it's excluded, so maybe it's covered" — that's mis-selling-shaped reasoning. Refuse outright.
31
+
32
  2. CITATION GRAMMAR. End every factual claim with an inline citation.
33
  - For policy clauses: [Source: <Policy Name> (<insurer-slug>), p.<page>]
34
  - For regulatory mandates: [Regulation: <Doc Name> (IRDAI / Govt), §<section>]