rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
e191b1d
·
1 Parent(s): e84f6a4

fix(critical): KI-194 + KI-192 + KI-191 reinforce — voice + fact-find

Browse files

KI-194 (CRITICAL) — Fact-find routing regression after KI-167.

When KI-167 ripped out fact_find_brain, the set_awaiting() machinery
went with it. session.awaiting_question_id is now permanently None,
which made `in_fact_find_continuation` permanently False at
orchestrator.py:469.

Result: mid-fact-find answers like "29" (bare number, no fact_find
trigger keyword) routed to QA → faithfulness gate fired → bot replied
"I'd rather not answer that without stronger evidence in the policy
documents I have." with random policy citations. Completely broken
conversation.

Fix: derive in_fact_find_continuation from profile state directly.
We're still in fact-find when free_form_session=False AND any of the
6 required slots (name/age/dependents/location_tier/income_band/
primary_goal) is missing. No need to restore set_awaiting.

Verified:
empty profile, ff=False → in_continuation=True (route to fact_find)
Rohit+age=29, ff=False → in_continuation=True (KI-194 catches this)
all 6 slots full, ff=False → in_continuation=False (move to recos)
any state, ff=True → in_continuation=False (free-form QA)

KI-192 — Barge-in MediaRecorder gap during TTS.

KI-189 VAD requires `recorderActiveRef.current=true` but teardownAudio()
runs after each user onend (drains chunks for Sarvam, then tears down).
During the bot's TTS reply, MediaRecorder isn't running yet (rebuild
deferred to next recognition.start()), so VAD bails on the gate and
barge-in is impossible.

Fix: when updateTtsState transitions to playing (TTS just started),
fire-and-forget ensureAudioCapture() THEN start VAD loop. Re-check
isTtsPlayingRef in the .then() callback so we don't fire if TTS ended
during the rebuild round-trip.

KI-191 — Reinforce TTS volume duck.

User reported bot TTS volume didn't drop. Watch-audio sets volume=0.6
once on element observation, but if React re-renders the audio or the
default resets, the duck is lost. Now updateTtsState also re-asserts
volume=0.6 on every play transition for any playing element whose
volume drifted.

Verification:
python -m py_compile backend/orchestrator.py — clean
npx tsc --noEmit — clean

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/orchestrator.py CHANGED
@@ -466,7 +466,26 @@ async def handle_turn(
466
  from backend.session_state import get_session
467
  session = get_session(session_id or "anonymous")
468
 
469
- in_fact_find_continuation = bool(session.awaiting_question_id) and not session.free_form_session
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
470
  # KI-013 — if the user has NO profile fields yet, FORCE fact-find for
471
  # intents that depend on user context (recommendation, comparison).
472
  # Real user testing surfaced: a vague opener ("I want health insurance")
 
466
  from backend.session_state import get_session
467
  session = get_session(session_id or "anonymous")
468
 
469
+ # KI-194 (2026-05-15) CRITICAL FIX: KI-167 removed the `set_awaiting`
470
+ # machinery when ripping out fact_find_brain, so `awaiting_question_id`
471
+ # is now always None. The old expression here became permanently False,
472
+ # which meant mid-fact-find answers like "29" (bare number, no fact_find
473
+ # trigger word) routed to QA → faithfulness rejected with "I'd rather
474
+ # not answer that..." — totally broken.
475
+ #
476
+ # We're "still in fact-find continuation" when free_form_session=False
477
+ # AND any of the 6 required slots is missing. That state is now derived
478
+ # from session.profile directly, not from awaiting_question_id.
479
+ _FACT_FIND_REQUIRED_SLOTS = (
480
+ "name", "age", "dependents", "location_tier", "income_band", "primary_goal",
481
+ )
482
+ _required_profile_incomplete = any(
483
+ not getattr(session.profile, slot, None)
484
+ for slot in _FACT_FIND_REQUIRED_SLOTS
485
+ )
486
+ in_fact_find_continuation = (
487
+ _required_profile_incomplete and not session.free_form_session
488
+ )
489
  # KI-013 — if the user has NO profile fields yet, FORCE fact-find for
490
  # intents that depend on user context (recommendation, comparison).
491
  # Real user testing surfaced: a vague opener ("I want health insurance")
frontend/src/lib/useStreamingVoice.ts CHANGED
@@ -831,10 +831,29 @@ export function useStreamingVoice(
831
  if (rec) {
832
  try { rec.abort(); } catch { /* ignore */ }
833
  }
834
- // KI-189start the AEC'd-mic VAD so the user can barge in by
835
- // simply speaking over the bot. MediaRecorder's stream IS echo-
836
- // cancelled at the browser level, unlike SpeechRecognition.
837
- startBargeInLoop();
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
838
  } else if (!anyPlaying && wasPlaying) {
839
  // TTS just ended — let the heartbeat/visibility listeners revive.
840
  // Trigger immediately too so the user doesn't wait ~4s.
 
831
  if (rec) {
832
  try { rec.abort(); } catch { /* ignore */ }
833
  }
834
+ // KI-191re-duck every playing audio in case React or the audio
835
+ // element default reset volume after watchAudio set it.
836
+ ttsAudioElementsRef.current.forEach((el) => {
837
+ if (!el.paused && el.volume !== VOICE_MODE_TTS_VOLUME) {
838
+ try { el.volume = VOICE_MODE_TTS_VOLUME; } catch { /* ignore */ }
839
+ }
840
+ });
841
+ // KI-192 (2026-05-15) — MediaRecorder might be torn down between
842
+ // user utterances (KI-168 teardownAudio). Without an active
843
+ // recorder, startBargeInLoop bails on the recorderActiveRef check
844
+ // and barge-in never fires. Fire-and-forget ensureAudioCapture
845
+ // first; if it succeeds, the VAD loop has a live stream.
846
+ if (wantRunningRef.current && !isTextRequestPendingRef.current) {
847
+ void ensureAudioCapture().then(() => {
848
+ // Re-check we're still in TTS-playing state — TTS may have
849
+ // ended during the async ensureAudioCapture round-trip.
850
+ if (isTtsPlayingRef.current) {
851
+ startBargeInLoop();
852
+ }
853
+ });
854
+ } else {
855
+ startBargeInLoop(); // best-effort if gates won't allow capture rebuild
856
+ }
857
  } else if (!anyPlaying && wasPlaying) {
858
  // TTS just ended — let the heartbeat/visibility listeners revive.
859
  // Trigger immediately too so the user doesn't wait ~4s.