fix(observability): KI-001..006 — log silent failures + fail-CLOSED judge

KI-001 (P0) — Gate 4 LLM judge now fails CLOSED by default
Previously when the judge LLM call raised (rate limit, parse, network),
Gate 4 returned supported=True ("fail-open") — an unsupported claim
that should have been blocked leaked through. In BFSI this is the
wrong default. New behaviour: log the failure + return supported=False
with reason "judge_unavailable_failclosed". Override with
FAITHFULNESS_FAIL_CLOSED=0 in dev/smoke if you need the old behaviour.

KI-002 (P1) — session_state._flush() now logs disk-write failures
Was swallowing the exception silently → user's profile disappeared on
Space restart with no signal. Now logs warning with session_id + error
type so HF Space logs show flush failure rate.

KI-003 (P1) — session_state._load_from_disk() logs schema-drift failures
Was returning None silently when the on-disk JSON didn't match the
current Profile dataclass. Now logs so we can detect when schema
evolves in a way that breaks existing sessions.

KI-004 (P1) — Indic translator failure now logs
Was falling through to send Indic text to English brain silently.
Logs the language + session + error so we can tune Sarvam fallback.

KI-005 (P1) — Profile-chunk upsert failure now logs
Was swallowing Chroma write failures silently → subsequent retrieval
saw stale profile. Logs the failure so we know when Chroma is
unhappy.

KI-006 (P2) — Profile extractor LLM failure now logs
Conversational mid-chat profile updates silently disabled on extractor
failure. Now logs so we know how often the extractor flakes.

None of these changes alter functional behaviour beyond KI-001 (which
flips the default for the BFSI-correct refusal posture). The other five
add observability only — silent failures still don't crash chat, but
they now show up in HF Space logs so we can track failure rates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/faithfulness.py

@@ -251,7 +251,19 @@ Verify."""
         unsupported = list(data.get("unsupported_claims", []))
         return supported, unsupported
     except Exception as e:
-        # If judge fails, fail-open with a warning (don't block valid replies on infra hiccup)
+        # KI-001 — BFSI compliance posture: in production we FAIL CLOSED
+        # (block when the judge is unavailable) so an unsupported claim
+        # never leaks past Gate 4 just because NIM hiccupped. Set
+        # FAITHFULNESS_FAIL_CLOSED=0 in dev/smoke to revert to fail-open.
+        import logging
+        import os
+        logging.warning(
+            "faithfulness gate 4 judge failure (%s: %s)",
+            type(e).__name__, str(e)[:200],
+        )
+        fail_closed = os.environ.get("FAITHFULNESS_FAIL_CLOSED", "1") == "1"
+        if fail_closed:
+            return False, [f"judge_unavailable_failclosed: {type(e).__name__}"]
         return True, [f"judge_error_failopen: {type(e).__name__}"]
 
 

backend/orchestrator.py

@@ -145,8 +145,15 @@ async def handle_turn(
             translated_query = await translate_to_english(user_text)
             if translated_query and translated_query.strip() and translated_query != user_text:
                 user_text = translated_query  # use English for retrieval + reasoning
-        except Exception:
-            pass  # fall through with original; if Sarvam translator fails, DeepSeek can still try
+        except Exception as e:
+            # KI-004 — surface translator failures in HF Space logs. The
+            # brain will still try with the original Indic text, but with
+            # degraded quality. The log lets us tune the Sarvam fallback.
+            import logging
+            logging.warning(
+                "indic translator failed (session=%s lang=%s): %s: %s",
+                session_id, language, type(e).__name__, str(e)[:200],
+            )
 
     # 1b. SESSION-STATE-AWARE FACT-FIND
     # Load session state. If we're already in fact-find (awaiting an answer to a
@@ -304,10 +311,23 @@ async def handle_turn(
                     "health_conditions": session.profile.health_conditions,
                 }
                 await upsert_profile_chunk(session_id or "anonymous", profile_dict_for_chunk)
-            except Exception:
-                pass  # chunk upsert failure must not block the chat
-    except Exception:
-        pass  # extraction failure must never block the chat
+            except Exception as e:
+                # KI-005 — log profile-chunk upsert failures so we can see
+                # when Chroma is locking or schema-drifting. The chat still
+                # ships; subsequent turns just won't see the latest profile.
+                import logging
+                logging.warning(
+                    "profile-chunk upsert failed (session=%s): %s: %s",
+                    session_id, type(e).__name__, str(e)[:200],
+                )
+    except Exception as e:
+        # KI-006 — log profile-extraction failures (extractor LLM down,
+        # malformed model output, etc.). The chat ships unaffected.
+        import logging
+        logging.warning(
+            "profile extractor failed (session=%s): %s: %s",
+            session_id, type(e).__name__, str(e)[:200],
+        )
 
     # 2. Retrieve — pass session_id so the user's profile chunk (stored in
     # Chroma at POST /api/profile time) gets boosted to the top of the

backend/session_state.py

@@ -64,8 +64,14 @@ class SessionState:
             tmp = target.with_suffix(".json.tmp")
             tmp.write_text(json.dumps(payload, indent=2))
             tmp.replace(target)
-        except Exception:
-            pass  # never let a disk hiccup kill the request
+        except Exception as e:
+            # KI-002 — Log silent failures so HF Space logs reveal them.
+            # Don't crash the request — disk hiccups shouldn't kill chat.
+            import logging
+            logging.warning(
+                "session_state flush failed for %s: %s: %s",
+                self.session_id, type(e).__name__, str(e)[:200],
+            )
 
     def set_awaiting(self, question_id: Optional[str]) -> None:
         self.awaiting_question_id = question_id
@@ -111,7 +117,15 @@ def _load_from_disk(session_id: str) -> Optional[SessionState]:
             free_form_session=bool(raw.get("free_form_session", False)),
             last_touched=float(raw.get("last_touched", time.time())),
         )
-    except Exception:
+    except Exception as e:
+        # KI-003 — Log schema-drift / corrupt-JSON failures. The user will
+        # get a fresh session either way, but the log lets us detect when
+        # the Profile dataclass evolves in a way that breaks old sessions.
+        import logging
+        logging.warning(
+            "session_state load_from_disk failed for %s: %s: %s",
+            session_id, type(e).__name__, str(e)[:200],
+        )
         return None