fix(safety): KI-104 — strip chain-of-thought / instruction-echo leakage from replies

Live smoke test caught NIM reasoning models (Qwen3-Next 80B) and the
faithfulness judge leaking internal reasoning into user-visible
reply_text. Examples seen verbatim in production:
- "We need to respond to user question..."
- "We must ground every factual claim..."
- "We need to follow instructions. The user asks... According to
conversation rules..."

The existing persona.strip_think_tags only handled balanced
<think>...</think> tags. Bare instruction-echo and stray scratchpad
labels (**Reasoning:**, [INTERNAL]) slipped past every gate.

Adds backend/voice_format.strip_cot_preamble — a conservative
sentence-level stripper that:

1. Removes labelled reasoning lines (**Reasoning:**, **Thought:**,
**Plan:**, [INTERNAL]...[/INTERNAL]) — same-line only, never
consumes the next line (which is typically the answer).
2. Drops everything before a stray </think> tag (defense-in-depth for
unbalanced cases that strip_think_tags lets through).
3. Sentence-walks the first ~600 chars and drops leading sentences
matching a CoT starter pattern ("We need to...", "We must...",
"Let me think...", "According to conversation rules...",
"Step N:", "To answer this...", "Following the instructions...",
etc.) until the first substantive sentence.
4. Returns a safe emergency reply if the entire output was CoT.

Wired into persona.strip_think_tags so every reply path that already
runs the <think> strip — orchestrator.py (text replies), translator.py
(Indic), tts_preprocess (voice) — gets the preamble strip for free.
No orchestrator.py changes (owned by another lane).

Conservative design — mid-reply phrases like "We have three options"
or "Let me think about your needs" beyond the scan window survive.
24 unit tests in tests/test_voice_format.py cover both bug repros
and false-positive guards.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/persona.py

@@ -138,6 +138,13 @@ def strip_think_tags(text: str) -> str:
       - truncated reasoning (no </think>):  <think>... cut off  →  fallback message
       - reasoning followed by clean answer:  <think>...</think> answer  →  answer
       - well-formed with extra text after close: take only text after </think>
+
+    KI-104 (2026-05-15) — after the <think> strip, also run
+    `strip_cot_preamble` to kill instruction-echo / scratchpad lines that
+    leaked outside of `<think>` tags. Live smoke caught Qwen3-Next 80B and
+    the judge model leaking "We need to respond to user question…",
+    "We must ground every factual claim…", and bare reasoning labels
+    (`**Reasoning:**`, `[INTERNAL]…`) into user-visible reply_text.
     """
     if "<think>" in text.lower() and "</think>" not in text.lower():
         # Reasoning was truncated mid-thought — no final answer was produced.
@@ -148,4 +155,11 @@ def strip_think_tags(text: str) -> str:
     # If anything else got truncated, fall back gracefully.
     if not cleaned:
         return "I'm thinking through that. Could you rephrase or ask a follow-up?"
+
+    # KI-104 — second-layer strip for CoT / instruction-echo leakage that
+    # didn't come wrapped in <think> tags. Imported locally to avoid an
+    # import cycle (voice_format has no persona deps; persona has no
+    # voice_format deps at module level).
+    from backend.voice_format import strip_cot_preamble
+    cleaned = strip_cot_preamble(cleaned)
     return cleaned

backend/voice_format.py

@@ -6,6 +6,17 @@ Why this exists: an unprocessed LLM reply with markdown bold, inline
 hear "asterisk asterisk bold asterisk asterisk A-Y-U-S-H pp dot 1 dash 2".
 That's a UX-killing bug — not a Sarvam limitation, a *us* bug.
 
+KI-104 (2026-05-15) — this module also exposes `strip_cot_preamble`, the
+chain-of-thought / instruction-echo stripper that runs on TEXT replies
+(not just TTS). Live smoke tests caught NIM reasoning models (e.g.,
+Qwen3-Next 80B) and the judge model leaking internal reasoning into
+`reply_text`. Examples: "We need to respond to user question…", "We must
+ground every factual claim…", "<think>...</think>The answer is X."
+`strip_cot_preamble` is called from `persona.strip_think_tags` so every
+reply path that already goes through the <think>-tag strip also gets the
+preamble strip — no orchestrator.py changes needed (that file is owned
+by another lane / KI-101).
+
 The function turns text like:
 
   "**Direct answer:**
@@ -161,11 +172,197 @@ def _truncate_for_voice(text: str, max_words: int = 60) -> str:
     return truncated + " More details are on screen."
 
 
+# ============================================================================
+# KI-104 (2026-05-15) — chain-of-thought / instruction-echo strip
+# ============================================================================
+# Live smoke test caught LLM brain replies leaking internal reasoning into
+# user-visible reply_text. Three failure modes:
+#   1. NIM reasoning models (Qwen3-Next 80B) emit a <think>...</think> block
+#      followed by the answer — the <think> tag was sometimes missing /
+#      malformed so the existing strip_think_tags in persona.py let it through.
+#   2. The faithfulness JUDGE model occasionally returns its own reasoning
+#      instead of a clean rescue reply.
+#   3. The brain model misunderstands the system prompt and echoes the
+#      instruction prose ("We need to respond to user question…").
+#
+# The strip below is CONSERVATIVE — it only kills CoT preamble lines that
+# appear BEFORE the first natural-sounding sentence (within the first ~6
+# lines / first 600 chars), so substantive mid-reply content like
+# "We have three options: A, B, C" is preserved.
+
+# ---- Sentence-level preamble patterns (KI-104) ----
+#
+# A CoT preamble can appear as:
+#   (a) a full line of its own: "We need to respond carefully.\n<answer>"
+#   (b) a leading sentence INSIDE the first line: "We need to respond to
+#       user question. Here's the actual answer."
+#
+# We handle both by sentence-splitting the top of the reply and dropping
+# leading sentences that match a CoT-starter pattern, until we hit a
+# substantive sentence.
+#
+# Sentence-starter patterns. These match from the START of a sentence
+# (no MULTILINE anchor — we apply them sentence-by-sentence). Keep these
+# specific enough to avoid false positives on legitimate prose.
+# NOTE: don't append a trailing `\b` to the alternation — `\b` after `:` or
+# after a digit followed by `:` is NOT a word boundary, which silently
+# breaks `Step \d+\s*:`. Each alternative carries its own anchor where one
+# is needed.
+_COT_SENTENCE_STARTERS = re.compile(
+    r"^\s*(?:"
+    r"We need to(?:\s+respond|\s+answer|\s+follow|\s+ground|\s+check|\s+ensure|\s+make sure|\s+consider|\s+think|\s+address)\b"
+    r"|We must\b"
+    r"|We should (?:respond|answer|follow|ground|check|ensure|make sure|consider|think|address|cite)\b"
+    r"|According to (?:conversation rules|the instructions|the guidelines|the system prompt|the rules|policy guidelines)\b"
+    r"|The user (?:asks|is asking|wants|needs|wants to know)\b"
+    r"|Let me (?:think|consider|analyze|break this down|work through)\b"
+    r"|I (?:will|need to|should|must) (?:think|consider|analyze|respond|answer|check|ground|follow)\b"
+    r"|First,?\s+I(?:'ll| will| need to| should| must)\b"
+    r"|To answer this(?:\s+question)?\b"
+    r"|Step \d+\s*:"
+    r"|Following the instructions\b"
+    r"|As per the (?:guidelines|instructions|rules|system prompt)\b"
+    r"|Per the (?:guidelines|instructions|rules)\b"
+    r"|Okay,?\s+(?:let me|so the user|so I)\b"
+    r"|Alright,?\s+(?:let me|so the user|so I)\b"
+    r"|So,?\s+the user\b"
+    r"|Thinking about this\b"
+    r"|My (?:thought|reasoning|plan|approach) (?:process )?(?:is|here)\b"
+    r")",
+    flags=re.IGNORECASE,
+)
+
+# Sentence splitter — split on ". " / "! " / "? " / "\n" but keep the
+# delimiter attached to the preceding sentence so we can rejoin losslessly.
+_SENTENCE_SPLIT = re.compile(r"(?<=[.!?])\s+|\n+")
+
+# Labelled reasoning blocks. Match only the SAME-LINE label content; do
+# not consume the next line (which is usually the real answer).
+_LABELLED_REASONING_LINE = re.compile(
+    r"^[ \t]*(?:\*\*)?(?:Reasoning|Thought|Plan|Internal|Scratch(?:pad)?|Chain[- ]of[- ]thought|CoT)(?:\*\*)?\s*:\s*[^\n]*$",
+    flags=re.IGNORECASE | re.MULTILINE,
+)
+_BRACKET_INTERNAL = re.compile(
+    r"\[(?:INTERNAL|REASONING|THOUGHT|PLAN|CoT)\].*?\[/(?:INTERNAL|REASONING|THOUGHT|PLAN|CoT)\]",
+    flags=re.IGNORECASE | re.DOTALL,
+)
+
+# Stray, unbalanced <think> tags that persona.strip_think_tags doesn't
+# already handle (it requires both open and close in the same blob).
+# If we see an isolated </think> mid-reply, drop everything before it.
+_STRAY_CLOSE_THINK = re.compile(r"^.*?</think>", flags=re.DOTALL | re.IGNORECASE)
+
+# Maximum scan window for preamble. Beyond this, content is treated as
+# substantive prose even if it matches a starter pattern — protects
+# legitimate mid-reply phrasing like "Let me think about your three options".
+_PREAMBLE_SCAN_LINES = 6
+_PREAMBLE_SCAN_CHARS = 600
+
+# Fallback when stripping removes the entire reply — better than empty.
+_EMERGENCY_REPLY = (
+    "Let me think about this — could you ask me again in a moment?"
+)
+
+
+def _drop_leading_cot_sentences(text: str) -> str:
+    """Sentence-by-sentence strip of CoT preamble at the top of a reply.
+
+    Split the first ~600 chars into sentences. Drop leading sentences that
+    match a CoT starter pattern. Stop at the first substantive sentence.
+    Rejoin and prepend to whatever's left of the reply.
+    """
+    if not text:
+        return text
+
+    # Only walk the first window — anything beyond is presumed substantive.
+    head = text[:_PREAMBLE_SCAN_CHARS]
+    tail = text[_PREAMBLE_SCAN_CHARS:]
+
+    # Track delimiters so we rejoin without losing them.
+    sentences: list[str] = []
+    last_end = 0
+    for m in _SENTENCE_SPLIT.finditer(head):
+        sentence = head[last_end : m.start()]
+        delim = m.group(0)
+        sentences.append(sentence + delim)
+        last_end = m.end()
+    # Final trailing chunk (no terminating delimiter).
+    if last_end < len(head):
+        sentences.append(head[last_end:])
+
+    # Walk and drop CoT starters.
+    drop_index = 0
+    while drop_index < len(sentences) and drop_index < _PREAMBLE_SCAN_LINES:
+        s = sentences[drop_index]
+        stripped = s.strip()
+        if not stripped:
+            drop_index += 1
+            continue
+        if _COT_SENTENCE_STARTERS.match(stripped):
+            drop_index += 1
+            continue
+        break
+
+    if drop_index == 0:
+        return text
+
+    rebuilt_head = "".join(sentences[drop_index:])
+    return (rebuilt_head + tail).lstrip()
+
+
+def strip_cot_preamble(text: str) -> str:
+    """Strip chain-of-thought / instruction-echo leakage from a model reply.
+
+    Conservative rules (in order):
+      1. Drop labelled reasoning lines (`**Reasoning:** …`, `[INTERNAL]…[/INTERNAL]`).
+         These are SAME-LINE strips — we never consume the next line, which
+         is typically the real answer.
+      2. If a stray `</think>` appears (no opening `<think>`), drop
+         everything up to and including it.
+      3. Sentence-walk the first ~600 chars; drop leading sentences that
+         match a CoT starter pattern. Stop at the first substantive sentence
+         — substantive content is preserved verbatim.
+      4. If the whole reply gets stripped, return `_EMERGENCY_REPLY`.
+
+    Args:
+      text: Raw model output (post-<think>-strip but pre-user-display).
+
+    Returns:
+      Cleaned reply with internal reasoning removed. Never empty.
+    """
+    if not text or not str(text).strip():
+        return _EMERGENCY_REPLY
+
+    cleaned = text
+
+    # Rule 1 — kill labelled reasoning blocks. Same-line only.
+    cleaned = _LABELLED_REASONING_LINE.sub("", cleaned)
+    cleaned = _BRACKET_INTERNAL.sub("", cleaned)
+
+    # Rule 2 — stray close-think tag: drop everything before it.
+    if "</think>" in cleaned.lower() and "<think>" not in cleaned.lower():
+        cleaned = _STRAY_CLOSE_THINK.sub("", cleaned, count=1).lstrip()
+
+    # Rule 3 — sentence-level CoT preamble strip.
+    cleaned = _drop_leading_cot_sentences(cleaned)
+
+    # Rule 4 — emergency fallback if the whole reply was CoT.
+    if not cleaned or not cleaned.strip():
+        return _EMERGENCY_REPLY
+
+    return cleaned
+
+
 def tts_preprocess(text: str, language: str = "en", max_words: int = 60) -> str:
     """Public entry — turn an LLM reply into spoken-language text for TTS."""
     if not text:
         return ""
-    cleaned = _strip_markdown(text)
+    # KI-104 — defense in depth: even if the reply went through
+    # persona.strip_think_tags upstream, run the preamble strip again here
+    # in case it's called on a path that bypasses persona (e.g., direct
+    # TTS of a cached reply).
+    cleaned = strip_cot_preamble(text)
+    cleaned = _strip_markdown(cleaned)
     # KI-066 (2026-05-15) — currency/range shorthand expansion before
     # acronym handling so ₹5L becomes "5 lakhs" instead of getting caught
     # by the bare-L acronym path.

tests/test_voice_format.py

@@ -0,0 +1,255 @@
+"""KI-104 (2026-05-15) — chain-of-thought / instruction-echo strip tests.
+
+Live smoke caught the brain LLM leaking internal reasoning into the
+user-visible reply_text. Examples seen verbatim in production:
+  - "We need to respond to user question..."
+  - "We must ground every factual claim..."
+  - "We need to follow instructions. The user asks... According to
+     conversation rules..."
+
+The faithfulness judge passed these through. Users would see them as
+broken/embarrassing output.
+
+These tests lock in:
+  1. `strip_cot_preamble` removes CoT preamble before substantive content.
+  2. `<think>...</think>` blocks are removed (defense-in-depth — the
+     primary strip is in persona.strip_think_tags).
+  3. Labelled reasoning blocks (`**Reasoning:**`, `[INTERNAL]`) are removed.
+  4. Substantive content starting with a CoT-like phrase ("Sure, here are
+     3 plans...") is NOT a false positive.
+  5. tts_preprocess invokes the strip end-to-end.
+  6. persona.strip_think_tags invokes the strip end-to-end.
+
+Run as a script (no pytest dep):
+    cd /Users/rohitsar/Developer/Insurance\\ Sales\\ Bot
+    .venv/bin/python -m unittest tests.test_voice_format -v
+"""
+
+from __future__ import annotations
+
+import unittest
+
+from backend.voice_format import (
+    strip_cot_preamble,
+    tts_preprocess,
+    _EMERGENCY_REPLY,
+)
+from backend.persona import strip_think_tags
+
+
+class StripCotPreambleTests(unittest.TestCase):
+    """Direct unit tests on strip_cot_preamble."""
+
+    # ---------- core production-bug repros (verbatim from smoke test) ----------
+
+    def test_we_need_to_respond_preamble_dropped(self):
+        """The headline production bug — kill the 'We need to respond' line."""
+        inp = "We need to respond to user question. Here's the actual answer."
+        out = strip_cot_preamble(inp)
+        self.assertTrue(
+            out.startswith("Here's the actual answer"),
+            f"Expected actual answer at start, got: {out!r}",
+        )
+        self.assertNotIn("We need to respond", out)
+
+    def test_we_must_ground_preamble_dropped(self):
+        """Second observed leak — 'We must ground every factual claim'."""
+        inp = (
+            "We must ground every factual claim in retrieved chunks.\n"
+            "HDFC ERGO Optima Secure has a 36-month waiting period for "
+            "pre-existing diseases."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("We must ground", out)
+        self.assertIn("HDFC ERGO", out)
+
+    def test_we_need_follow_instructions_multi_line_preamble(self):
+        """Third observed leak — multiple CoT lines stacked."""
+        inp = (
+            "We need to follow instructions.\n"
+            "The user asks about pre-existing disease waiting periods.\n"
+            "According to conversation rules, we cite policy text.\n"
+            "The waiting period is 36 months under Optima Secure."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("We need to follow", out)
+        self.assertNotIn("The user asks", out)
+        self.assertNotIn("According to conversation rules", out)
+        self.assertIn("36 months", out)
+
+    # ---------- <think> tag handling (defense in depth) ----------
+
+    def test_stray_close_think_tag_dropped(self):
+        """If a stray </think> appears with no opening tag, drop everything before it."""
+        inp = "foo bar baz</think>The answer is X."
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, "The answer is X.")
+
+    # ---------- labelled reasoning blocks ----------
+
+    def test_reasoning_label_block_dropped(self):
+        inp = (
+            "**Reasoning:** I need to check the retrieved chunks.\n"
+            "The deductible is ₹5,000 per claim."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("Reasoning:", out)
+        self.assertIn("deductible", out)
+
+    def test_bracket_internal_block_dropped(self):
+        inp = (
+            "[INTERNAL]Let me check the chunks first[/INTERNAL]\n"
+            "Yes, dental is covered up to ₹10,000."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("INTERNAL", out)
+        self.assertNotIn("Let me check the chunks", out)
+        self.assertIn("dental is covered", out)
+
+    def test_plan_label_block_dropped(self):
+        inp = (
+            "**Plan:** Answer briefly, cite policy clause 3.2.\n"
+            "The maximum sum insured is ₹1 crore."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("Plan:", out)
+        self.assertIn("1 crore", out)
+
+    # ---------- starter-phrase preamble (Step 1, To answer this, etc.) ----------
+
+    def test_step_numbered_preamble_dropped(self):
+        inp = (
+            "Step 1: Identify the policy.\n"
+            "Step 2: Find the clause.\n"
+            "Maternity waiting period is 24 months."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("Step 1:", out)
+        self.assertNotIn("Step 2:", out)
+        self.assertIn("Maternity", out)
+
+    def test_to_answer_this_preamble_dropped(self):
+        inp = "To answer this question, I'll check the chunks.\nDental is covered."
+        out = strip_cot_preamble(inp)
+        self.assertTrue(out.startswith("Dental"))
+
+    def test_first_ill_preamble_dropped(self):
+        inp = "First, I'll review the retrieved policy text.\nThe waiting period is 36 months."
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("First, I'll", out)
+        self.assertIn("36 months", out)
+
+    def test_let_me_think_preamble_dropped(self):
+        inp = "Let me think about this carefully.\nThe answer is yes — OPD is included."
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("Let me think about", out)
+        self.assertIn("OPD is included", out)
+
+    def test_following_instructions_dropped(self):
+        inp = "Following the instructions, I will cite each claim.\nCoverage is comprehensive."
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("Following the instructions", out)
+        self.assertIn("Coverage is comprehensive", out)
+
+    def test_as_per_guidelines_dropped(self):
+        inp = "As per the guidelines, citations are required.\nThe premium is ₹15,000."
+        out = strip_cot_preamble(inp)
+        self.assertNotIn("As per the guidelines", out)
+        self.assertIn("premium", out)
+
+    # ---------- false-positive guards ----------
+
+    def test_legit_reply_unchanged_sure_here_are_3_plans(self):
+        """Don't false-positive a substantive opener that starts with 'Sure'."""
+        inp = "Sure, here are 3 plans to consider:\n1. HDFC ERGO Optima Secure\n2. Star Comprehensive\n3. Niva Bupa ReAssure"
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, inp)
+
+    def test_legit_reply_unchanged_yes_dental_is_covered(self):
+        inp = "Yes, dental treatment is covered under Optima Secure subject to a sub-limit of ₹10,000 per year."
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, inp)
+
+    def test_legit_reply_unchanged_we_have_three_options_midreply(self):
+        """The phrase 'We have three options:' is substantive content, not CoT."""
+        inp = "Based on your needs profile, we have three options: A, B, and C."
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, inp)
+
+    def test_legit_reply_starting_with_the_user(self):
+        """A reply that legitimately begins 'The user manual says...' must survive."""
+        # Note: 'The user manual' does NOT match the starter regex (which
+        # requires 'The user asks/is asking/wants/needs').
+        inp = "The user manual for Optima Secure is available at hdfcergo.com."
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, inp)
+
+    def test_legit_reply_with_inline_let_me_think(self):
+        """'Let me think' beyond the scan window must not be stripped."""
+        inp = (
+            "Optima Secure offers a sum insured of ₹1 crore with "
+            "unlimited restore. It covers daycare, road ambulance, and "
+            "ayurveda. Let me think about which plan suits you best — "
+            "I'll need your age and city to recommend."
+        )
+        out = strip_cot_preamble(inp)
+        # The 'Let me think' is mid-reply, beyond line 1, so it survives.
+        self.assertIn("Let me think", out)
+        self.assertIn("Optima Secure", out)
+
+    # ---------- empty / edge cases ----------
+
+    def test_empty_input_returns_emergency_reply(self):
+        self.assertEqual(strip_cot_preamble(""), _EMERGENCY_REPLY)
+        self.assertEqual(strip_cot_preamble("   \n  "), _EMERGENCY_REPLY)
+
+    def test_all_cot_returns_emergency_reply(self):
+        """If the WHOLE reply is CoT and nothing substantive remains."""
+        inp = (
+            "We need to respond to user question.\n"
+            "We must check the chunks.\n"
+            "Let me think.\n"
+            "Step 1: identify policy."
+        )
+        out = strip_cot_preamble(inp)
+        self.assertEqual(out, _EMERGENCY_REPLY)
+
+    def test_none_input_safe(self):
+        """Conservative: None / falsy inputs should not crash."""
+        self.assertEqual(strip_cot_preamble(None), _EMERGENCY_REPLY)  # type: ignore[arg-type]
+
+
+class TtsPreprocessIntegrationTests(unittest.TestCase):
+    """End-to-end: tts_preprocess invokes strip_cot_preamble."""
+
+    def test_tts_strips_cot_before_markdown(self):
+        inp = "We need to respond to user question.\n**Yes**, OPD is covered."
+        out = tts_preprocess(inp, language="en")
+        self.assertNotIn("We need to respond", out)
+        self.assertIn("Yes", out)
+        # Markdown bold was also stripped:
+        self.assertNotIn("**", out)
+
+
+class PersonaStripThinkTagsIntegrationTests(unittest.TestCase):
+    """End-to-end: persona.strip_think_tags invokes strip_cot_preamble too."""
+
+    def test_think_block_plus_cot_preamble_both_stripped(self):
+        inp = (
+            "<think>I need to check the chunks first.</think>\n"
+            "We need to respond carefully.\n"
+            "The waiting period is 36 months."
+        )
+        out = strip_think_tags(inp)
+        self.assertNotIn("<think>", out)
+        self.assertNotIn("We need to respond", out)
+        self.assertIn("36 months", out)
+
+    def test_clean_reply_passes_through_unchanged(self):
+        inp = "Dental treatment is covered up to ₹10,000 per year under Optima Secure."
+        out = strip_think_tags(inp)
+        self.assertEqual(out, inp)
+
+
+if __name__ == "__main__":
+    unittest.main()