Spaces:
Sleeping
fix: KI-036/037/038 — greeting flow, strict paraphrase verifier, waiting dots
Browse filesThree user-reported UX bugs from voice-mode testing, all fixed together.
KI-036 (orchestrator + paraphraser) — Strict paraphrase end-of-string
check. Previous verifier checked `"?" in paraphrase` (anywhere), letting
the dependents-slot paraphrase emit "...have their own insurance
options.?" — period AND question mark at end. Now require
`paraphrase.rstrip().endswith("?")` AND reject malformed sequences:
".?", "?.", "??", "!?", "?!", ",?". On rejection, fall back to canonical
question text (no UX regression).
KI-037 (orchestrator) — Greeting + intent-restart detection. Previous
behavior: ANY message arriving while `session.awaiting_question_id` was
set got parsed as an attempted answer. User opening the bot with "Hi, I
am looking to buy an insurance policy." after a prior partial fact-find
got "Sorry, I didn't catch that. Let me ask again — Who else needs
cover..." — which feels broken because the user intended a fresh start.
Fix:
• Pure-greeting detector — input that, after stripping punctuation, is
only words from {"hi", "hello", "hey", "namaste", "yo", "hola"}.
• Intent-restart detector — phrases like "i am looking", "want to buy",
"i need health insurance", "help me find", "first time".
• If either fires AND awaiting_question_id is set: clear the slot,
clear _reask_counts, recompute in_fact_find_continuation. Lets the
fact-find flow re-enter cleanly.
• Pure greeting on empty profile gets a WARM WELCOME message
(brain_tag "greeting::welcome") with an offer to start a 2-min
profile OR answer a specific question — NOT immediate age grilling.
KI-038 (frontend) — Visible waiting indicator. ThinkingDots now takes a
`phase` prop and renders different labels:
• "Hearing you…" — STT is transcribing the user's voice (~1-2s gap
that was previously invisible — user finished speaking, nothing on
screen until the user-bubble landed)
• "Thinking…" — brain in flight (existing behavior; was unlabeled)
• "Speaking…" — TTS playing (informational)
State machine: setVoicePhase("transcribing") before postTranscribe,
flip to "thinking" after pushUser, clear in finally{}. Wired across
Live mode, push-to-talk, and the typed-send flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- README.md +180 -96
- backend/orchestrator.py +62 -0
- backend/question_paraphraser.py +11 -2
- frontend/src/app/page.tsx +35 -8
|
@@ -126,107 +126,191 @@ A **voice-first conversational advisor** that:
|
|
| 126 |
|
| 127 |
## 3. Two parallel flows
|
| 128 |
|
| 129 |
-
The bot is two flows running together — the customer's experience and the technology underneath. They
|
| 130 |
|
| 131 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
|
| 133 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
``
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 226 |
|
| 227 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 228 |
|
| 229 |
-
|
|
|
|
| 230 |
|
| 231 |
```
|
| 232 |
┌──────────────────────────────────────────────────────────────────────────┐
|
|
|
|
| 126 |
|
| 127 |
## 3. Two parallel flows
|
| 128 |
|
| 129 |
+
The bot is two flows running together — the customer's experience and the technology underneath. They run in lockstep; the table below pairs each step of the user journey with the technology underneath. On narrow screens both columns are scrollable; on a desktop GitHub view they sit side-by-side.
|
| 130 |
|
| 131 |
+
<table>
|
| 132 |
+
<tr>
|
| 133 |
+
<th width="50%">3A · Customer / Process Flow</th>
|
| 134 |
+
<th width="50%">3B · Technology Flow (same turn, underneath)</th>
|
| 135 |
+
</tr>
|
| 136 |
+
<tr><td>
|
| 137 |
|
| 138 |
+
**Step 1 — Land on the bot**
|
| 139 |
+
- Sees: chat panel + Marketplace · Premium · Profile · Admin tabs
|
| 140 |
+
- Suggested questions in the chat box
|
| 141 |
+
- "Voice on — just speak" pill (Live mode, default ON) with green dot
|
| 142 |
+
- "🎤 Push-to-talk" button as labeled fallback
|
| 143 |
+
- "Voice reply" toggle controls whether the bot speaks back
|
| 144 |
|
| 145 |
+
</td><td>
|
| 146 |
+
|
| 147 |
+
**Tech 1 — Page load**
|
| 148 |
+
- Next.js 14 SSR ships HTML in ~200 ms
|
| 149 |
+
- React hydrates; useEffect fetches `/api/health`, `/api/coverage`, `/api/profile/completeness`
|
| 150 |
+
- `localStorage` rehydrates `messages[]` + `sessionId` if returning user
|
| 151 |
+
- `useLiveConversation` opens the persistent mic stream
|
| 152 |
+
|
| 153 |
+
</td></tr>
|
| 154 |
+
|
| 155 |
+
<tr><td>
|
| 156 |
+
|
| 157 |
+
**Step 2 — Ask the first question (voice or text)**
|
| 158 |
+
- User just talks — VAD detects speech start/end automatically
|
| 159 |
+
- *"Suggest a health insurance plan for me"*
|
| 160 |
+
- Bot recognises fact-find intent. The orchestrator picks the next slot from a 9-question graph; an LLM paraphraser rewrites the canonical question in a warmer voice each session (verified to still target the same slot before sending)
|
| 161 |
+
- Slots in order: age → dependents → income → existing cover → primary goal → location → parents (conditional) → health → budget
|
| 162 |
+
|
| 163 |
+
</td><td>
|
| 164 |
+
|
| 165 |
+
**Tech 2 — User submits**
|
| 166 |
+
- Voice path: `MediaRecorder` blob → `POST /api/transcribe` → Sarvam Saarika v2.5 STT → text
|
| 167 |
+
- Text path: direct `POST /api/chat`
|
| 168 |
+
- Payload: `user_text`, `session_id`, `chat_history[]`, `return_audio`, `tts_language_code`, `view_context { active_view, active_policy_id, filters }`
|
| 169 |
+
|
| 170 |
+
**Tech 3 — Orchestrator entry, `handle_turn()`**
|
| 171 |
+
- `classify_intent(user_text)` → `fact_find / qa / comparison / recommendation`
|
| 172 |
+
- `detect_language(user_text)` → `english / indic`
|
| 173 |
+
- Indic cascade: if indic, Sarvam-M translates → English for reasoning, response translated back
|
| 174 |
+
|
| 175 |
+
**Tech 4 — Fact-find branch**
|
| 176 |
+
- If profile empty AND intent ∈ {fact_find, recommendation, comparison}: pick next slot, paraphrase via `question_paraphraser` + verifier, emit question, return early
|
| 177 |
+
- Else: continue to retrieval
|
| 178 |
+
|
| 179 |
+
</td></tr>
|
| 180 |
+
|
| 181 |
+
<tr><td>
|
| 182 |
+
|
| 183 |
+
**Step 3 — Profile-driven personalisation unlocks**
|
| 184 |
+
- Completeness bar reaches ≥ 60% → "personalised scores unlocked"
|
| 185 |
+
- Marketplace tab shows per-policy A-F grades **re-computed for this user's specific situation**
|
| 186 |
+
- Chat references your profile inline ("at 32 with 1 dependent…")
|
| 187 |
+
|
| 188 |
+
</td><td>
|
| 189 |
+
|
| 190 |
+
**Tech 5 — Profile update extraction (free-form turns)**
|
| 191 |
+
- `extract_profile_updates(user_text, session.profile)` via the fast-brain chain (Nemotron 30B primary)
|
| 192 |
+
- Validation: enum / type / bounds checks; drop on failure
|
| 193 |
+
- Apply via `session.update_profile_field()`
|
| 194 |
+
- `upsert_profile_chunk()` re-embeds profile → Chroma so this turn's retrieval sees the fresh state
|
| 195 |
+
|
| 196 |
+
</td></tr>
|
| 197 |
+
|
| 198 |
+
<tr><td>
|
| 199 |
+
|
| 200 |
+
**Step 4 — Free-form questions with citations**
|
| 201 |
+
- *"What's the room rent cap on Care Supreme?"*
|
| 202 |
+
- *"Compare cataract waiting in HDFC Optima vs ICICI Elevate"*
|
| 203 |
+
- Every factual claim gets `[Source: <policy>, p.<page>]` citation
|
| 204 |
+
- Click a citation → opens the policy detail modal
|
| 205 |
+
|
| 206 |
+
</td><td>
|
| 207 |
+
|
| 208 |
+
**Tech 6 — Retrieval, `retrieve(query, session_id)`**
|
| 209 |
+
- LRU cache (key = normalised query + filters) — cache hit skips Voyage + Chroma
|
| 210 |
+
- BGE-small embeds user text → 384-d vector
|
| 211 |
+
- Chroma cosine search, `top_k=5`
|
| 212 |
+
- Profile chunk (`doc_type='profile'`) boosted to top
|
| 213 |
+
- Second pass on regulatory chunks if query mentions IRDAI / Section
|
| 214 |
+
- Returns `list[RetrievedChunk(policy_id, page_start, page_end, text, source_url, score)]`
|
| 215 |
+
|
| 216 |
+
</td></tr>
|
| 217 |
+
|
| 218 |
+
<tr><td>
|
| 219 |
+
|
| 220 |
+
**Step 5 — Conversational profile updates (mid-chat)**
|
| 221 |
+
- User says: *"I was just diagnosed with diabetes"*
|
| 222 |
+
- Bot silently extracts → updates `session.profile` → re-upserts profile chunk → completeness bar ticks up → scores refresh
|
| 223 |
+
- No form-filling required mid-conversation
|
| 224 |
+
|
| 225 |
+
</td><td>
|
| 226 |
+
|
| 227 |
+
**Tech 7 — Brain selection, `pick_brain(intent, language)`**
|
| 228 |
+
- `intent ∈ {comparison, recommendation}` → BRAIN_CHAIN (Qwen 80B primary, 50/50 with Groq Llama-3.3)
|
| 229 |
+
- `intent ∈ {qa, fact_find}` → FAST_BRAIN_CHAIN (Nemotron Nano 30B primary)
|
| 230 |
+
- Per-call rotation across two providers' independent rate caps
|
| 231 |
+
|
| 232 |
+
**Tech 8 — System prompt construction, `build_messages()`**
|
| 233 |
+
- `[System: ADVISOR_PROMPT + USER PROFILE block + USER IS LOOKING AT (view_context) block]`
|
| 234 |
+
- `[Assistant/User: last 5 turns of chat_history]`
|
| 235 |
+
- `[User: USER QUESTION + RETRIEVED POLICY CLAUSES + reply instructions]`
|
| 236 |
+
|
| 237 |
+
**Tech 9 — Brain LLM call → reply text**
|
| 238 |
+
- `NimChainLLM.chat(...)` with cumulative chain budget (brain 35 s, fast-brain 22 s)
|
| 239 |
+
- `strip_think_tags()` removes `<think>…</think>` reasoning
|
| 240 |
+
- Capture `brain_model_actual` for non-circular judge selection
|
| 241 |
+
|
| 242 |
+
</td></tr>
|
| 243 |
+
|
| 244 |
+
<tr><td>
|
| 245 |
+
|
| 246 |
+
**Step 6 — View-aware grounding (the copilot effect)**
|
| 247 |
+
- User opens a policy detail modal
|
| 248 |
+
- Asks *"What's the waiting period on this?"*
|
| 249 |
+
- Bot resolves *"this"* → answers without re-stating the policy name
|
| 250 |
+
|
| 251 |
+
</td><td>
|
| 252 |
+
|
| 253 |
+
**Tech 10 — Faithfulness gates (`backend/faithfulness.py`)**
|
| 254 |
+
- Gate 1 — retrieval floor: top score ≥ 0.30
|
| 255 |
+
- Gate 2 — citation integrity: every cited policy was retrieved
|
| 256 |
+
- Gate 3 — numeric grounding: every ₹/%/days/months/years in the reply appears in retrieved chunks
|
| 257 |
+
- Gate 4 — JUDGE_CHAIN (Mistral Large 3 675B primary, different family from brain)
|
| 258 |
+
- All 4 pass → reply ships
|
| 259 |
+
- Any fail (non-Gate-1) → cross-check retry with the judge model as rescue brain → tagged `crosscheck-rescued-by-judge`
|
| 260 |
+
- Still fails → safe refusal + log to `logs/hallucinations.jsonl`
|
| 261 |
+
|
| 262 |
+
</td></tr>
|
| 263 |
+
|
| 264 |
+
<tr><td>
|
| 265 |
+
|
| 266 |
+
**Step 7 — Refusal as a feature**
|
| 267 |
+
- User: *"Does this policy cover space-tourism injuries?"*
|
| 268 |
+
- Bot: *"I'd rather not answer that without stronger evidence in the policy documents I have."*
|
| 269 |
+
- The SAFE failure mode in BFSI is refuse > mis-cite
|
| 270 |
+
|
| 271 |
+
</td><td>
|
| 272 |
+
|
| 273 |
+
**Tech 11 — Indic cascade (only if user spoke Hinglish)**
|
| 274 |
+
- Sarvam-M translates English reply → Hinglish
|
| 275 |
+
- Gate A — regex: digits / citations / currency preserved
|
| 276 |
+
- Gate B — Mistral Large 3 675B LLM-judge for semantic preservation
|
| 277 |
+
- Gate C — back-translation cosine ≥ 0.80
|
| 278 |
+
- Any gate fails → fall back to the English reply
|
| 279 |
+
|
| 280 |
+
</td></tr>
|
| 281 |
+
|
| 282 |
+
<tr><td>
|
| 283 |
+
|
| 284 |
+
**Step 8 — Hindi / Hinglish flow**
|
| 285 |
+
- User switches the UI language toggle, or just speaks Hinglish
|
| 286 |
+
- *"Care Supreme mein PED ka waiting period kya hai?"*
|
| 287 |
+
- Bot responds in Hinglish with citations preserved
|
| 288 |
+
|
| 289 |
+
</td><td>
|
| 290 |
+
|
| 291 |
+
**Tech 12 — TTS synthesis (if `return_audio=true`)**
|
| 292 |
+
- `tts_preprocess()` expands acronyms (PED → pre-existing disease) + strips markdown
|
| 293 |
+
- Sarvam Bulbul v2 synthesises → base64 WAV
|
| 294 |
+
- Bundled into the `ChatResponse` alongside `reply_text`
|
| 295 |
+
|
| 296 |
+
</td></tr>
|
| 297 |
+
|
| 298 |
+
<tr><td>
|
| 299 |
+
|
| 300 |
+
**Step 9 — Persistent chat across sessions**
|
| 301 |
+
- Close the tab, come back tomorrow: chat history + profile restored
|
| 302 |
+
- Sessions persisted to disk on the backend; `localStorage` on the frontend
|
| 303 |
+
|
| 304 |
+
</td><td>
|
| 305 |
|
| 306 |
+
**Tech 13 — Response delivered**
|
| 307 |
+
- `ChatResponse { reply_text, citations[], audio_base64, brain_used, profile_updates, faithfulness_passed, blocked }`
|
| 308 |
+
- Frontend renders text → plays the in-DOM `<audio controls>` (autoplay-on-mount; barge-in pauses it) → updates `profileCompleteness`
|
| 309 |
+
- `localStorage` persists chat history; `session_state._flush()` writes the on-disk session JSON
|
| 310 |
+
- `log_turn()` writes to `logs/turns.jsonl`
|
| 311 |
|
| 312 |
+
</td></tr>
|
| 313 |
+
</table>
|
| 314 |
|
| 315 |
```
|
| 316 |
┌──────────────────────────────────────────────────────────────────────────┐
|
|
@@ -231,6 +231,68 @@ async def handle_turn(
|
|
| 231 |
and session.profile.dependents is None
|
| 232 |
and session.profile.income_band is None
|
| 233 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 234 |
treat_as_fact_find = should_route_to_fact_find(
|
| 235 |
intent,
|
| 236 |
profile_is_empty=profile_is_empty,
|
|
|
|
| 231 |
and session.profile.dependents is None
|
| 232 |
and session.profile.income_band is None
|
| 233 |
)
|
| 234 |
+
|
| 235 |
+
# KI-037 (2026-05-14) — greeting + intent-restart detection. The previous
|
| 236 |
+
# behavior was: any user message arriving while session.awaiting_question_id
|
| 237 |
+
# was set got parsed as an attempted answer. A user opening the bot with
|
| 238 |
+
# "Hi, I am looking to buy an insurance policy." would get this parsed as
|
| 239 |
+
# a (failed) answer to the dependents slot (because earlier turns had
|
| 240 |
+
# advanced the slot pointer) → "Sorry, I didn't catch that. Let me ask
|
| 241 |
+
# again — Who else needs cover..." which feels broken because the user
|
| 242 |
+
# intended a fresh start.
|
| 243 |
+
#
|
| 244 |
+
# Fix: if the user's message is a pure greeting or contains explicit
|
| 245 |
+
# restart intent, CLEAR any stale awaiting slot + reset re-ask counts
|
| 246 |
+
# so the fact-find re-enters cleanly.
|
| 247 |
+
_RESTART_INTENT_PHRASES = (
|
| 248 |
+
"i am looking", "i'm looking", "i want to buy", "want to buy",
|
| 249 |
+
"looking to buy", "i am here to", "i'm here to",
|
| 250 |
+
"i need health insurance", "i need insurance",
|
| 251 |
+
"help me find", "first time", "buy health insurance",
|
| 252 |
+
)
|
| 253 |
+
_PURE_GREETING_WORDS = {
|
| 254 |
+
"hi", "hello", "hey", "namaste", "yo", "hola",
|
| 255 |
+
}
|
| 256 |
+
_user_text_lc = user_text.lower().strip()
|
| 257 |
+
_stripped_user = "".join(c for c in _user_text_lc if c.isalnum() or c.isspace()).strip()
|
| 258 |
+
_is_pure_greeting = bool(_stripped_user) and all(
|
| 259 |
+
w in _PURE_GREETING_WORDS for w in _stripped_user.split()
|
| 260 |
+
)
|
| 261 |
+
_is_intent_restart = any(p in _user_text_lc for p in _RESTART_INTENT_PHRASES)
|
| 262 |
+
if (_is_pure_greeting or _is_intent_restart) and session.awaiting_question_id:
|
| 263 |
+
session.set_awaiting(None)
|
| 264 |
+
if hasattr(session, "_reask_counts"):
|
| 265 |
+
session._reask_counts.clear()
|
| 266 |
+
# Recompute the continuation flag now that we've cleared the slot.
|
| 267 |
+
in_fact_find_continuation = False
|
| 268 |
+
|
| 269 |
+
# KI-037 — pure greeting on empty profile gets a warm welcome, NOT an
|
| 270 |
+
# immediate age grilling. The bot can answer "Hi" with "Hi!" + offer
|
| 271 |
+
# before asking for a profile. User feedback: bot grilling for age on
|
| 272 |
+
# the very first "Hello" feels mechanical and unwelcoming.
|
| 273 |
+
if _is_pure_greeting and profile_is_empty and not session.free_form_session:
|
| 274 |
+
reply = (
|
| 275 |
+
"Hi! I'm an AI advisor for health insurance in India — I've read 208 policy "
|
| 276 |
+
"documents from 19 insurers plus the IRDAI master circulars, and I can "
|
| 277 |
+
"answer questions about coverage, waiting periods, exclusions, or "
|
| 278 |
+
"compare policies side-by-side.\n\n"
|
| 279 |
+
"Want me to build a 2-minute profile so I can grade each policy *for you* "
|
| 280 |
+
"specifically, or do you have a question in mind?"
|
| 281 |
+
)
|
| 282 |
+
return TurnResult(
|
| 283 |
+
reply_text=reply,
|
| 284 |
+
citations=[],
|
| 285 |
+
retrieved_chunk_ids=[],
|
| 286 |
+
brain_used="greeting::welcome",
|
| 287 |
+
intent="fact_find",
|
| 288 |
+
language=language,
|
| 289 |
+
latency_ms=int((time.time() - t0) * 1000),
|
| 290 |
+
raw_reply=reply,
|
| 291 |
+
faithfulness_passed=True,
|
| 292 |
+
blocked=False,
|
| 293 |
+
profile_updates={},
|
| 294 |
+
)
|
| 295 |
+
|
| 296 |
treat_as_fact_find = should_route_to_fact_find(
|
| 297 |
intent,
|
| 298 |
profile_is_empty=profile_is_empty,
|
|
@@ -170,8 +170,17 @@ async def paraphrase_question(
|
|
| 170 |
slot_id, claimed_slot)
|
| 171 |
_PARAPHRASE_CACHE[cache_key] = None
|
| 172 |
return None
|
| 173 |
-
|
| 174 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
_PARAPHRASE_CACHE[cache_key] = None
|
| 176 |
return None
|
| 177 |
if not (30 <= len(paraphrase) <= 500):
|
|
|
|
| 170 |
slot_id, claimed_slot)
|
| 171 |
_PARAPHRASE_CACHE[cache_key] = None
|
| 172 |
return None
|
| 173 |
+
# KI-036 (2026-05-14) — strict end-position "?" check + reject malformed
|
| 174 |
+
# punctuation patterns. Previous lenient `"?" in paraphrase` let outputs
|
| 175 |
+
# like "...have their own insurance options.?" through, producing
|
| 176 |
+
# awkward double-punctuation in production.
|
| 177 |
+
para_stripped = paraphrase.rstrip()
|
| 178 |
+
if not para_stripped.endswith("?"):
|
| 179 |
+
logging.info("paraphraser doesn't end with '?' slot=%s — falling back", slot_id)
|
| 180 |
+
_PARAPHRASE_CACHE[cache_key] = None
|
| 181 |
+
return None
|
| 182 |
+
if any(bad in para_stripped for bad in (".?", "?.", "??", "!?", "?!", ",?")):
|
| 183 |
+
logging.info("paraphraser malformed punctuation slot=%s — falling back", slot_id)
|
| 184 |
_PARAPHRASE_CACHE[cache_key] = None
|
| 185 |
return None
|
| 186 |
if not (30 <= len(paraphrase) <= 500):
|
|
@@ -52,6 +52,12 @@ export default function Page() {
|
|
| 52 |
const [messages, setMessages] = useState<DisplayMessage[]>([]);
|
| 53 |
const [input, setInput] = useState("");
|
| 54 |
const [busy, setBusy] = useState(false);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
const [recording, setRecording] = useState(false);
|
| 56 |
const [returnAudio, setReturnAudio] = useState(true);
|
| 57 |
const [ttsLang, setTtsLang] = useState<"en-IN" | "hi-IN">("en-IN");
|
|
@@ -258,6 +264,7 @@ export default function Page() {
|
|
| 258 |
async function send(text: string) {
|
| 259 |
if (!text.trim() || busy) return;
|
| 260 |
setBusy(true);
|
|
|
|
| 261 |
setInput("");
|
| 262 |
|
| 263 |
// Bug C — if the user is asking us to retry, resubmit the previous user
|
|
@@ -352,6 +359,7 @@ export default function Page() {
|
|
| 352 |
} finally {
|
| 353 |
setUploadStatus(null);
|
| 354 |
setBusy(false);
|
|
|
|
| 355 |
}
|
| 356 |
}
|
| 357 |
|
|
@@ -360,11 +368,13 @@ export default function Page() {
|
|
| 360 |
useEffect(() => {
|
| 361 |
liveOnUtteranceRef.current = async (blob, abort) => {
|
| 362 |
try {
|
|
|
|
| 363 |
const transcribed = await postTranscribe(blob, ttsLang, abort.signal);
|
| 364 |
const text = (transcribed.text || "").trim();
|
| 365 |
-
if (text.length < 2) return;
|
| 366 |
|
| 367 |
pushUser(text);
|
|
|
|
| 368 |
const history: ChatMessage[] = messages.map((m) => ({ role: m.role, content: m.content }));
|
| 369 |
const active_view: "chat" | "marketplace" | "profile" | "premium" | "policy_detail" =
|
| 370 |
openPolicy ? "policy_detail" :
|
|
@@ -399,9 +409,11 @@ export default function Page() {
|
|
| 399 |
// can pause it. See comment in the typed-send branch.
|
| 400 |
} catch (e: unknown) {
|
| 401 |
const name = (e as { name?: string })?.name;
|
| 402 |
-
if (name === "AbortError") return; //
|
| 403 |
// eslint-disable-next-line no-console
|
| 404 |
console.error("[live mode] turn failed:", e);
|
|
|
|
|
|
|
| 405 |
}
|
| 406 |
};
|
| 407 |
}, [messages, sessionId, ttsLang, openPolicy, showMarketplace, showProfile, showPremium]);
|
|
@@ -437,14 +449,20 @@ export default function Page() {
|
|
| 437 |
const maybeResumeLive = () => { if (userPrefersLive) live.setLive(true); };
|
| 438 |
if (blob.size < 1000) { maybeResumeLive(); return; }
|
| 439 |
setBusy(true);
|
|
|
|
| 440 |
try {
|
| 441 |
const { text } = await postTranscribe(blob, ttsLang);
|
| 442 |
-
if (text && text.trim())
|
| 443 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 444 |
} catch (e: unknown) {
|
| 445 |
pushAssistant(`Sorry — transcribe error: ${e instanceof Error ? e.message : String(e)}`);
|
| 446 |
} finally {
|
| 447 |
setBusy(false);
|
|
|
|
| 448 |
maybeResumeLive();
|
| 449 |
}
|
| 450 |
};
|
|
@@ -698,7 +716,7 @@ export default function Page() {
|
|
| 698 |
</div>
|
| 699 |
<div ref={scrollRef} className="flex-1 overflow-y-auto scrollbar-thin space-y-4 mb-4 pr-1">
|
| 700 |
{messages.map((m) => <Message key={m.id} m={m} />)}
|
| 701 |
-
{busy && <ThinkingDots />}
|
| 702 |
</div>
|
| 703 |
</>
|
| 704 |
)}
|
|
@@ -1632,15 +1650,24 @@ function ScorecardCard({ sc }: { sc: ScorecardResponse }) {
|
|
| 1632 |
);
|
| 1633 |
}
|
| 1634 |
|
| 1635 |
-
function ThinkingDots() {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1636 |
return (
|
| 1637 |
<div className="flex justify-start">
|
| 1638 |
-
<div className="bg-[var(--card)] border border-[var(--border)] rounded-2xl px-4 py-3">
|
| 1639 |
<div className="flex gap-1.5">
|
| 1640 |
{[0, 1, 2].map((i) => (
|
| 1641 |
-
<span key={i} className="w-2 h-2 rounded-full bg-[var(--
|
| 1642 |
))}
|
| 1643 |
</div>
|
|
|
|
| 1644 |
</div>
|
| 1645 |
</div>
|
| 1646 |
);
|
|
|
|
| 52 |
const [messages, setMessages] = useState<DisplayMessage[]>([]);
|
| 53 |
const [input, setInput] = useState("");
|
| 54 |
const [busy, setBusy] = useState(false);
|
| 55 |
+
// KI-038 (2026-05-14) — voicePhase exposes the otherwise-invisible
|
| 56 |
+
// "between recording and showing the user bubble" Sarvam-STT gap (~1-2s)
|
| 57 |
+
// and the "between user bubble and bot reply" /api/chat gap (~5-15s).
|
| 58 |
+
// null = idle / typed flow, "transcribing" = STT in flight, "thinking" =
|
| 59 |
+
// brain in flight, "speaking" = TTS playing (informational).
|
| 60 |
+
const [voicePhase, setVoicePhase] = useState<null | "transcribing" | "thinking" | "speaking">(null);
|
| 61 |
const [recording, setRecording] = useState(false);
|
| 62 |
const [returnAudio, setReturnAudio] = useState(true);
|
| 63 |
const [ttsLang, setTtsLang] = useState<"en-IN" | "hi-IN">("en-IN");
|
|
|
|
| 264 |
async function send(text: string) {
|
| 265 |
if (!text.trim() || busy) return;
|
| 266 |
setBusy(true);
|
| 267 |
+
setVoicePhase("thinking"); // KI-038 — show "..." while waiting on brain
|
| 268 |
setInput("");
|
| 269 |
|
| 270 |
// Bug C — if the user is asking us to retry, resubmit the previous user
|
|
|
|
| 359 |
} finally {
|
| 360 |
setUploadStatus(null);
|
| 361 |
setBusy(false);
|
| 362 |
+
setVoicePhase(null);
|
| 363 |
}
|
| 364 |
}
|
| 365 |
|
|
|
|
| 368 |
useEffect(() => {
|
| 369 |
liveOnUtteranceRef.current = async (blob, abort) => {
|
| 370 |
try {
|
| 371 |
+
setVoicePhase("transcribing"); // KI-038 — show indicator while STT runs
|
| 372 |
const transcribed = await postTranscribe(blob, ttsLang, abort.signal);
|
| 373 |
const text = (transcribed.text || "").trim();
|
| 374 |
+
if (text.length < 2) { setVoicePhase(null); return; }
|
| 375 |
|
| 376 |
pushUser(text);
|
| 377 |
+
setVoicePhase("thinking"); // KI-038 — STT done; brain in flight now
|
| 378 |
const history: ChatMessage[] = messages.map((m) => ({ role: m.role, content: m.content }));
|
| 379 |
const active_view: "chat" | "marketplace" | "profile" | "premium" | "policy_detail" =
|
| 380 |
openPolicy ? "policy_detail" :
|
|
|
|
| 409 |
// can pause it. See comment in the typed-send branch.
|
| 410 |
} catch (e: unknown) {
|
| 411 |
const name = (e as { name?: string })?.name;
|
| 412 |
+
if (name === "AbortError") { setVoicePhase(null); return; } // barge-in
|
| 413 |
// eslint-disable-next-line no-console
|
| 414 |
console.error("[live mode] turn failed:", e);
|
| 415 |
+
} finally {
|
| 416 |
+
setVoicePhase(null); // KI-038 — clear indicator once turn lands
|
| 417 |
}
|
| 418 |
};
|
| 419 |
}, [messages, sessionId, ttsLang, openPolicy, showMarketplace, showProfile, showPremium]);
|
|
|
|
| 449 |
const maybeResumeLive = () => { if (userPrefersLive) live.setLive(true); };
|
| 450 |
if (blob.size < 1000) { maybeResumeLive(); return; }
|
| 451 |
setBusy(true);
|
| 452 |
+
setVoicePhase("transcribing"); // KI-038 — STT in flight on PTT
|
| 453 |
try {
|
| 454 |
const { text } = await postTranscribe(blob, ttsLang);
|
| 455 |
+
if (text && text.trim()) {
|
| 456 |
+
// send() flips voicePhase to "thinking" itself; no need to set here
|
| 457 |
+
await send(text);
|
| 458 |
+
} else {
|
| 459 |
+
pushAssistant("Sorry, I couldn't hear that clearly. Please try again.");
|
| 460 |
+
}
|
| 461 |
} catch (e: unknown) {
|
| 462 |
pushAssistant(`Sorry — transcribe error: ${e instanceof Error ? e.message : String(e)}`);
|
| 463 |
} finally {
|
| 464 |
setBusy(false);
|
| 465 |
+
setVoicePhase(null);
|
| 466 |
maybeResumeLive();
|
| 467 |
}
|
| 468 |
};
|
|
|
|
| 716 |
</div>
|
| 717 |
<div ref={scrollRef} className="flex-1 overflow-y-auto scrollbar-thin space-y-4 mb-4 pr-1">
|
| 718 |
{messages.map((m) => <Message key={m.id} m={m} />)}
|
| 719 |
+
{(busy || voicePhase) && <ThinkingDots phase={voicePhase} />}
|
| 720 |
</div>
|
| 721 |
</>
|
| 722 |
)}
|
|
|
|
| 1650 |
);
|
| 1651 |
}
|
| 1652 |
|
| 1653 |
+
function ThinkingDots({ phase }: { phase?: null | "transcribing" | "thinking" | "speaking" } = {}) {
|
| 1654 |
+
// KI-038 — phase-labeled status, not just an opaque blob of dots. Users
|
| 1655 |
+
// need to know whether the bot is hearing them ("transcribing"), thinking
|
| 1656 |
+
// about the answer ("thinking"), or about to speak ("speaking").
|
| 1657 |
+
const label = phase === "transcribing"
|
| 1658 |
+
? "Hearing you…"
|
| 1659 |
+
: phase === "speaking"
|
| 1660 |
+
? "Speaking…"
|
| 1661 |
+
: "Thinking…";
|
| 1662 |
return (
|
| 1663 |
<div className="flex justify-start">
|
| 1664 |
+
<div className="bg-[var(--card)] border border-[var(--border)] rounded-2xl px-4 py-3 flex items-center gap-2.5">
|
| 1665 |
<div className="flex gap-1.5">
|
| 1666 |
{[0, 1, 2].map((i) => (
|
| 1667 |
+
<span key={i} className="w-2 h-2 rounded-full bg-[var(--primary)] opacity-70" style={{ animation: "fade-up 1.2s ease-in-out infinite", animationDelay: `${i * 0.2}s` }} />
|
| 1668 |
))}
|
| 1669 |
</div>
|
| 1670 |
+
<span className="text-xs text-[var(--muted-foreground)] font-medium">{label}</span>
|
| 1671 |
</div>
|
| 1672 |
</div>
|
| 1673 |
);
|