Spaces:
Sleeping
fix(critical): KI-194 + KI-192 + KI-191 reinforce — voice + fact-find
Browse filesKI-194 (CRITICAL) — Fact-find routing regression after KI-167.
When KI-167 ripped out fact_find_brain, the set_awaiting() machinery
went with it. session.awaiting_question_id is now permanently None,
which made `in_fact_find_continuation` permanently False at
orchestrator.py:469.
Result: mid-fact-find answers like "29" (bare number, no fact_find
trigger keyword) routed to QA → faithfulness gate fired → bot replied
"I'd rather not answer that without stronger evidence in the policy
documents I have." with random policy citations. Completely broken
conversation.
Fix: derive in_fact_find_continuation from profile state directly.
We're still in fact-find when free_form_session=False AND any of the
6 required slots (name/age/dependents/location_tier/income_band/
primary_goal) is missing. No need to restore set_awaiting.
Verified:
empty profile, ff=False → in_continuation=True (route to fact_find)
Rohit+age=29, ff=False → in_continuation=True (KI-194 catches this)
all 6 slots full, ff=False → in_continuation=False (move to recos)
any state, ff=True → in_continuation=False (free-form QA)
KI-192 — Barge-in MediaRecorder gap during TTS.
KI-189 VAD requires `recorderActiveRef.current=true` but teardownAudio()
runs after each user onend (drains chunks for Sarvam, then tears down).
During the bot's TTS reply, MediaRecorder isn't running yet (rebuild
deferred to next recognition.start()), so VAD bails on the gate and
barge-in is impossible.
Fix: when updateTtsState transitions to playing (TTS just started),
fire-and-forget ensureAudioCapture() THEN start VAD loop. Re-check
isTtsPlayingRef in the .then() callback so we don't fire if TTS ended
during the rebuild round-trip.
KI-191 — Reinforce TTS volume duck.
User reported bot TTS volume didn't drop. Watch-audio sets volume=0.6
once on element observation, but if React re-renders the audio or the
default resets, the duck is lost. Now updateTtsState also re-asserts
volume=0.6 on every play transition for any playing element whose
volume drifted.
Verification:
python -m py_compile backend/orchestrator.py — clean
npx tsc --noEmit — clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- backend/orchestrator.py +20 -1
- frontend/src/lib/useStreamingVoice.ts +23 -4
|
@@ -466,7 +466,26 @@ async def handle_turn(
|
|
| 466 |
from backend.session_state import get_session
|
| 467 |
session = get_session(session_id or "anonymous")
|
| 468 |
|
| 469 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 470 |
# KI-013 — if the user has NO profile fields yet, FORCE fact-find for
|
| 471 |
# intents that depend on user context (recommendation, comparison).
|
| 472 |
# Real user testing surfaced: a vague opener ("I want health insurance")
|
|
|
|
| 466 |
from backend.session_state import get_session
|
| 467 |
session = get_session(session_id or "anonymous")
|
| 468 |
|
| 469 |
+
# KI-194 (2026-05-15) — CRITICAL FIX: KI-167 removed the `set_awaiting`
|
| 470 |
+
# machinery when ripping out fact_find_brain, so `awaiting_question_id`
|
| 471 |
+
# is now always None. The old expression here became permanently False,
|
| 472 |
+
# which meant mid-fact-find answers like "29" (bare number, no fact_find
|
| 473 |
+
# trigger word) routed to QA → faithfulness rejected with "I'd rather
|
| 474 |
+
# not answer that..." — totally broken.
|
| 475 |
+
#
|
| 476 |
+
# We're "still in fact-find continuation" when free_form_session=False
|
| 477 |
+
# AND any of the 6 required slots is missing. That state is now derived
|
| 478 |
+
# from session.profile directly, not from awaiting_question_id.
|
| 479 |
+
_FACT_FIND_REQUIRED_SLOTS = (
|
| 480 |
+
"name", "age", "dependents", "location_tier", "income_band", "primary_goal",
|
| 481 |
+
)
|
| 482 |
+
_required_profile_incomplete = any(
|
| 483 |
+
not getattr(session.profile, slot, None)
|
| 484 |
+
for slot in _FACT_FIND_REQUIRED_SLOTS
|
| 485 |
+
)
|
| 486 |
+
in_fact_find_continuation = (
|
| 487 |
+
_required_profile_incomplete and not session.free_form_session
|
| 488 |
+
)
|
| 489 |
# KI-013 — if the user has NO profile fields yet, FORCE fact-find for
|
| 490 |
# intents that depend on user context (recommendation, comparison).
|
| 491 |
# Real user testing surfaced: a vague opener ("I want health insurance")
|
|
@@ -831,10 +831,29 @@ export function useStreamingVoice(
|
|
| 831 |
if (rec) {
|
| 832 |
try { rec.abort(); } catch { /* ignore */ }
|
| 833 |
}
|
| 834 |
-
// KI-
|
| 835 |
-
//
|
| 836 |
-
|
| 837 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 838 |
} else if (!anyPlaying && wasPlaying) {
|
| 839 |
// TTS just ended — let the heartbeat/visibility listeners revive.
|
| 840 |
// Trigger immediately too so the user doesn't wait ~4s.
|
|
|
|
| 831 |
if (rec) {
|
| 832 |
try { rec.abort(); } catch { /* ignore */ }
|
| 833 |
}
|
| 834 |
+
// KI-191 — re-duck every playing audio in case React or the audio
|
| 835 |
+
// element default reset volume after watchAudio set it.
|
| 836 |
+
ttsAudioElementsRef.current.forEach((el) => {
|
| 837 |
+
if (!el.paused && el.volume !== VOICE_MODE_TTS_VOLUME) {
|
| 838 |
+
try { el.volume = VOICE_MODE_TTS_VOLUME; } catch { /* ignore */ }
|
| 839 |
+
}
|
| 840 |
+
});
|
| 841 |
+
// KI-192 (2026-05-15) — MediaRecorder might be torn down between
|
| 842 |
+
// user utterances (KI-168 teardownAudio). Without an active
|
| 843 |
+
// recorder, startBargeInLoop bails on the recorderActiveRef check
|
| 844 |
+
// and barge-in never fires. Fire-and-forget ensureAudioCapture
|
| 845 |
+
// first; if it succeeds, the VAD loop has a live stream.
|
| 846 |
+
if (wantRunningRef.current && !isTextRequestPendingRef.current) {
|
| 847 |
+
void ensureAudioCapture().then(() => {
|
| 848 |
+
// Re-check we're still in TTS-playing state — TTS may have
|
| 849 |
+
// ended during the async ensureAudioCapture round-trip.
|
| 850 |
+
if (isTtsPlayingRef.current) {
|
| 851 |
+
startBargeInLoop();
|
| 852 |
+
}
|
| 853 |
+
});
|
| 854 |
+
} else {
|
| 855 |
+
startBargeInLoop(); // best-effort if gates won't allow capture rebuild
|
| 856 |
+
}
|
| 857 |
} else if (!anyPlaying && wasPlaying) {
|
| 858 |
// TTS just ended — let the heartbeat/visibility listeners revive.
|
| 859 |
// Trigger immediately too so the user doesn't wait ~4s.
|