rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
ef90cd6
·
1 Parent(s): 438d1fa

fix(ui): KI-257 — restructure chat input: master Voice toggle + 3-button row + hold-SPACE

Browse files

Per user redesign: simplify the chat input to just textarea + Send by
default, with voice controls hidden behind a single "Enable voice"
toggle. When voice is enabled, two equal options appear: Live (BETA)
and Push-to-talk. Push-to-talk works in TWO modes:
(a) click mic to start, click Stop to submit (classic)
(b) hold SPACE to start, release SPACE to submit (like command line)

Both PTT modes share the same startRecording() entrypoint which already
calls interruptBotAudio("ptt-start") — so barge-in (interrupting the
bot's TTS mid-sentence) works cleanly on both click AND SPACE-hold paths.
This is the reliable barge-in path; Live voice barge-in remains in BETA.

Removed from the UI per user request (functionality preserved as defaults
behind the scenes):
- Paperclip / PDF upload icon (textarea only; drag-drop still works via
the existing fileInputRef handler if needed in future)
- Voice reply checkbox (returnAudio state now const, default true)
- Lang dropdown EN/हिन्दी (ttsLang state now const, default en-IN —
Sarvam STT auto-detects user input language; LLM mirrors via
SYSTEM_PROMPT RULE 8 Indic mirroring)
- Header EN/हिं language toggle (same rationale)
- "Enter to send · 🎤 to push-to-talk · 📎 PDF" hint text

Added:
- Master "Enable voice" pill (gray when off, emerald when on). Persisted
via localStorage.insurance_voice_master.
- Live (BETA) pill ONLY visible when voiceMasterOn. Same KI-256 confirm
dialog when enabling. Persists via insurance_live_pref.
- Push-to-talk button ONLY visible when voiceMasterOn. Always present
in both PTT modes.
- Helper text: "or hold SPACE to talk · release to submit" / "Listening…
release SPACE to submit" / "Listening… click Stop to submit".
- Hold-SPACE key handler: keydown when textarea NOT focused starts PTT;
keyup stops + submits. Uses refs to avoid stale closure on inline
startRecording/stopRecording.
- spaceHoldActive state drives the "ready to take audio" emerald-700
ring on the PTT button while SPACE is held.

When master Voice toggle goes OFF, userPrefersLive is force-set to false
so the Live mic is fully released (no background recording).

Frontend-only change. Backend contract (return_audio + tts_language_code)
unchanged — defaults are sent on every /api/chat call as before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. frontend/src/app/page.tsx +162 -95
frontend/src/app/page.tsx CHANGED
@@ -62,8 +62,22 @@ export default function Page() {
62
  // brain in flight, "speaking" = TTS playing (informational).
63
  const [voicePhase, setVoicePhase] = useState<null | "transcribing" | "thinking" | "speaking">(null);
64
  const [recording, setRecording] = useState(false);
65
- const [returnAudio, setReturnAudio] = useState(true);
66
- const [ttsLang, setTtsLang] = useState<"en-IN" | "hi-IN">("en-IN");
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  // Visual UI language — same source as ttsLang so the toggle controls both
68
  const uiLang: UILang = ttsLang === "hi-IN" ? "hi" : "en";
69
  const t = (key: StringKey, vars?: Record<string, string | number>) => translate(uiLang, key, vars);
@@ -429,6 +443,25 @@ export default function Page() {
429
  // eslint-disable-next-line react-hooks/exhaustive-deps
430
  }, [userPrefersLive]);
431
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
432
  // V3 FIX 2 + FIX 3 — Hardened TTS interrupt cleanup. When the bot is
433
  // mid-sentence and the user starts speaking / toggles voice off / clicks
434
  // PTT, we need to:
@@ -1070,6 +1103,56 @@ export default function Page() {
1070
  }
1071
  function stopRecording() { mediaRecorderRef.current?.stop(); }
1072
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1073
  async function handleFile(ev: React.ChangeEvent<HTMLInputElement>) {
1074
  const f = ev.target.files?.[0];
1075
  if (!f) return;
@@ -1257,14 +1340,9 @@ export default function Page() {
1257
  </div>
1258
  </div>
1259
  </button>
1260
- {/* UI language toggle flips visual chrome + voice TTS together */}
1261
- <button
1262
- onClick={() => setTtsLang(ttsLang === "en-IN" ? "hi-IN" : "en-IN")}
1263
- className="text-xs font-semibold px-2 py-1 rounded-md border border-[var(--border)] bg-[var(--card)] hover:border-[var(--primary)] hover:text-[var(--primary)]"
1264
- title={ttsLang === "en-IN" ? "Switch to Hindi" : "अंग्रेज़ी में बदलें"}
1265
- >
1266
- {ttsLang === "en-IN" ? "EN · हिं" : "हिं · EN"}
1267
- </button>
1268
  </div>
1269
  </div>
1270
  </header>
@@ -1383,49 +1461,6 @@ export default function Page() {
1383
  className="flex-1 resize-none bg-transparent outline-none text-sm sm:text-base px-2 py-2 min-h-[40px] max-h-32"
1384
  disabled={busy}
1385
  />
1386
- <input
1387
- ref={fileInputRef}
1388
- type="file"
1389
- accept="application/pdf"
1390
- onChange={handleFile}
1391
- className="hidden"
1392
- />
1393
- <button
1394
- type="button"
1395
- onClick={() => fileInputRef.current?.click()}
1396
- disabled={busy || !!uploadStatus}
1397
- title="Upload your own policy PDF"
1398
- className="shrink-0 w-11 h-11 rounded-xl flex items-center justify-center bg-[var(--muted)] hover:bg-[var(--border)] disabled:opacity-40 transition"
1399
- >
1400
- <UploadIcon />
1401
- </button>
1402
- {/* KI-029 (2026-05-14) — push-to-talk button is now LABELED and
1403
- visually highlighted when Voice is OFF, so the user can find
1404
- it. Was just a 40px icon indistinguishable from the upload
1405
- icon; user reported "no push to talk button" despite it
1406
- technically existing. */}
1407
- <button
1408
- type="button"
1409
- onClick={recording ? stopRecording : startRecording}
1410
- disabled={busy && !recording}
1411
- className={`shrink-0 h-11 px-3 rounded-xl flex items-center gap-1.5 transition-all text-sm font-medium ${
1412
- recording
1413
- ? "bg-[var(--error)] text-white animate-record-pulse"
1414
- : userPrefersLive
1415
- ? "bg-[var(--muted)] hover:bg-[var(--border)] text-[var(--foreground)]"
1416
- : "bg-emerald-600 hover:bg-emerald-700 text-white shadow-md ring-2 ring-emerald-300 dark:ring-emerald-700"
1417
- } disabled:opacity-40`}
1418
- title={recording
1419
- ? "Recording… click to stop, or stay silent for 2s"
1420
- : userPrefersLive
1421
- ? "Push-to-talk (one turn — Live resumes after)"
1422
- : "Push-to-talk — click and speak; Voice will stay off until you click the red dot"}
1423
- >
1424
- {recording ? <StopIcon /> : <MicIcon />}
1425
- <span className="hidden sm:inline">
1426
- {recording ? "Stop" : "Push-to-talk"}
1427
- </span>
1428
- </button>
1429
  <button
1430
  type="button"
1431
  onClick={() => send(input)}
@@ -1435,34 +1470,35 @@ export default function Page() {
1435
  Send
1436
  </button>
1437
  </div>
1438
- {/* V4 FIX 1 — live PTT interim transcript directly under the mic
1439
- row, shown in gray italic. Updates ~5/sec via the throttled
1440
- pttInterim state. Hidden when not recording or when there's
1441
- no partial yet, so we don't render an empty 1-line strip. */}
1442
- {recording && pttInterim && (
1443
- <div
1444
- className="mt-1 px-2 text-xs italic text-[var(--muted-foreground)] leading-snug truncate"
1445
- aria-live="polite"
1446
- aria-atomic="true"
1447
- title={pttInterim}
1448
- >
1449
- {pttInterim}
1450
- </div>
1451
- )}
1452
  <div className="flex items-center justify-between gap-3 mt-2 pt-2 px-2 text-xs text-[var(--muted-foreground)]">
1453
- <div className="flex items-center gap-3">
1454
- {/* KI-028 — Clickable Live toggle. Green = always-on listening,
1455
- red = off (user explicitly disabled). PTT works in BOTH states:
1456
- in green it temporarily suspends then resumes; in red it does
1457
- its one turn and leaves Live off so the user keeps using PTT
1458
- until they click back to green. */}
 
 
 
 
 
 
 
 
 
 
 
 
 
1459
  {!live.micPermissionDenied ? (
1460
  <button
1461
  type="button"
1462
  onClick={() => {
1463
- // KI-256 — show a one-time confirm when enabling live voice
1464
- // so users know it's beta + may cut them off / echo.
1465
- // Once they accept, persist a flag so we don't nag again.
1466
  if (!userPrefersLive) {
1467
  const seen = (typeof window !== "undefined")
1468
  ? window.localStorage.getItem("insurance_live_beta_ack") === "1"
@@ -1482,14 +1518,14 @@ export default function Page() {
1482
  }
1483
  setUserPrefersLive((p) => !p);
1484
  }}
1485
- className={`flex items-center gap-1.5 px-2 py-0.5 rounded-full border text-xs font-medium transition cursor-pointer ${
1486
  userPrefersLive
1487
  ? "border-amber-300 bg-amber-50 text-amber-700 hover:bg-amber-100 dark:border-amber-700 dark:bg-amber-900/30 dark:text-amber-300"
1488
  : "border-[var(--border)] text-[var(--muted-foreground)] hover:border-[var(--primary)] hover:text-[var(--primary)]"
1489
  }`}
1490
  title={userPrefersLive
1491
- ? "Always-on voice is BETA — may cut you off / echo. PTT (🎤) is more reliable."
1492
- : "Always-on voice (BETA — unstable). Prefer 🎤 push-to-talk for reliable input."}
1493
  >
1494
  <span className={`inline-block w-2 h-2 rounded-full ${
1495
  userPrefersLive
@@ -1497,27 +1533,58 @@ export default function Page() {
1497
  : "bg-gray-400"
1498
  }`} />
1499
  {userPrefersLive
1500
- ? (live.recording ? "Listening… (BETA)" : "Voice on · BETA — unstable")
1501
- : "Voice off · Always-on (BETA)"}
1502
  </button>
1503
  ) : (
1504
- <span className="text-rose-500 text-xs" title="Allow mic in your browser site settings, or use the 🎤 push-to-talk button">
1505
- 🔇 Mic blocked — use 🎤 to speak, or type below
1506
  </span>
1507
  )}
1508
- <label className="flex items-center gap-1.5 cursor-pointer" title="Bot replies with both text and audio">
1509
- <input type="checkbox" checked={returnAudio} onChange={(e) => setReturnAudio(e.target.checked)} className="w-3.5 h-3.5 accent-[var(--primary)]" /> Voice reply
1510
- </label>
1511
- <label className="flex items-center gap-1.5">
1512
- Lang:
1513
- <select value={ttsLang} onChange={(e) => setTtsLang(e.target.value as "en-IN" | "hi-IN")} className="bg-transparent border border-[var(--border)] rounded px-1.5 py-0.5">
1514
- <option value="en-IN">English</option>
1515
- <option value="hi-IN">हिन्दी</option>
1516
- </select>
1517
- </label>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1518
  </div>
1519
- <div className="hidden sm:block">Enter to send · 🎤 to push-to-talk · 📎 PDF</div>
1520
- </div>
 
 
 
 
 
 
 
 
 
 
 
1521
  </div>
1522
  </main>
1523
 
 
62
  // brain in flight, "speaking" = TTS playing (informational).
63
  const [voicePhase, setVoicePhase] = useState<null | "transcribing" | "thinking" | "speaking">(null);
64
  const [recording, setRecording] = useState(false);
65
+ // KI-257 voice reply + tts language UI toggles removed per user request.
66
+ // returnAudio stays true (bot always replies with audio); ttsLang stays
67
+ // en-IN by default (Sarvam STT auto-detects user's input language; LLM
68
+ // mirrors via SYSTEM_PROMPT RULE 8 Indic mirroring). Underlying state
69
+ // preserved so backend contract is unchanged.
70
+ const [returnAudio] = useState(true);
71
+ const [ttsLang] = useState<"en-IN" | "hi-IN">("en-IN");
72
+ // KI-257 — master Voice toggle. When OFF, the chat input shows only
73
+ // textarea + Send. When ON, reveals Live (BETA) option, Push-to-talk
74
+ // button. Hold-SPACE-to-talk is always active while Voice is on
75
+ // (no separate toggle) so the user can just press space whenever the
76
+ // textarea isn't focused. Persisted via localStorage.
77
+ const [voiceMasterOn, setVoiceMasterOn] = useState(false);
78
+ // KI-257 — true while the user is holding SPACE (drives the visual
79
+ // "ready to take audio" indicator near the PTT button).
80
+ const [spaceHoldActive, setSpaceHoldActive] = useState(false);
81
  // Visual UI language — same source as ttsLang so the toggle controls both
82
  const uiLang: UILang = ttsLang === "hi-IN" ? "hi" : "en";
83
  const t = (key: StringKey, vars?: Record<string, string | number>) => translate(uiLang, key, vars);
 
443
  // eslint-disable-next-line react-hooks/exhaustive-deps
444
  }, [userPrefersLive]);
445
 
446
+ // KI-257 — restore master Voice pref from localStorage on first mount.
447
+ // Master toggle off by default. When master goes OFF we also force
448
+ // userPrefersLive=false so the Live always-on path is suspended.
449
+ useEffect(() => {
450
+ if (typeof window === "undefined") return;
451
+ try {
452
+ const m = localStorage.getItem("insurance_voice_master") === "on";
453
+ if (m) setVoiceMasterOn(true);
454
+ } catch { /* ignore */ }
455
+ }, []);
456
+ useEffect(() => {
457
+ if (typeof window === "undefined") return;
458
+ localStorage.setItem("insurance_voice_master", voiceMasterOn ? "on" : "off");
459
+ if (!voiceMasterOn && userPrefersLive) {
460
+ setUserPrefersLive(false);
461
+ }
462
+ // eslint-disable-next-line react-hooks/exhaustive-deps
463
+ }, [voiceMasterOn]);
464
+
465
  // V3 FIX 2 + FIX 3 — Hardened TTS interrupt cleanup. When the bot is
466
  // mid-sentence and the user starts speaking / toggles voice off / clicks
467
  // PTT, we need to:
 
1103
  }
1104
  function stopRecording() { mediaRecorderRef.current?.stop(); }
1105
 
1106
+ // KI-257 — Hold-SPACE-to-talk. When Voice is on AND the user is NOT
1107
+ // focused on any text input, holding SPACE acts like Push-to-talk:
1108
+ // press starts recording, release stops + submits. Refs avoid stale
1109
+ // closures since startRecording/stopRecording are inline functions
1110
+ // recreated each render.
1111
+ const startRecordingRef = useRef<(() => Promise<void>) | null>(null);
1112
+ const stopRecordingRef = useRef<(() => void) | null>(null);
1113
+ useEffect(() => {
1114
+ startRecordingRef.current = startRecording;
1115
+ stopRecordingRef.current = stopRecording;
1116
+ });
1117
+ useEffect(() => {
1118
+ if (!voiceMasterOn) return;
1119
+ if (typeof window === "undefined") return;
1120
+ const isInputFocused = () => {
1121
+ const ae = document.activeElement as HTMLElement | null;
1122
+ if (!ae) return false;
1123
+ const tag = ae.tagName;
1124
+ if (tag === "INPUT" || tag === "TEXTAREA") return true;
1125
+ if (ae.isContentEditable) return true;
1126
+ return false;
1127
+ };
1128
+ const onKeyDown = (e: KeyboardEvent) => {
1129
+ if (e.key !== " " && e.code !== "Space") return;
1130
+ if (e.repeat) return;
1131
+ if (isInputFocused()) return;
1132
+ if (e.metaKey || e.ctrlKey || e.altKey || e.shiftKey) return;
1133
+ e.preventDefault();
1134
+ if (recording || busy) return;
1135
+ setSpaceHoldActive(true);
1136
+ const sr = startRecordingRef.current;
1137
+ if (sr) void sr();
1138
+ };
1139
+ const onKeyUp = (e: KeyboardEvent) => {
1140
+ if (e.key !== " " && e.code !== "Space") return;
1141
+ if (!spaceHoldActive && !recording) return;
1142
+ e.preventDefault();
1143
+ setSpaceHoldActive(false);
1144
+ const sp = stopRecordingRef.current;
1145
+ if (sp && recording) sp();
1146
+ };
1147
+ window.addEventListener("keydown", onKeyDown);
1148
+ window.addEventListener("keyup", onKeyUp);
1149
+ return () => {
1150
+ window.removeEventListener("keydown", onKeyDown);
1151
+ window.removeEventListener("keyup", onKeyUp);
1152
+ };
1153
+ // eslint-disable-next-line react-hooks/exhaustive-deps
1154
+ }, [voiceMasterOn, recording, busy, spaceHoldActive]);
1155
+
1156
  async function handleFile(ev: React.ChangeEvent<HTMLInputElement>) {
1157
  const f = ev.target.files?.[0];
1158
  if (!f) return;
 
1340
  </div>
1341
  </div>
1342
  </button>
1343
+ {/* KI-257header EN/HI toggle removed per user request.
1344
+ Sarvam STT auto-detects user language; LLM mirrors via
1345
+ SYSTEM_PROMPT RULE 8. ttsLang stays en-IN by default. */}
 
 
 
 
 
1346
  </div>
1347
  </div>
1348
  </header>
 
1461
  className="flex-1 resize-none bg-transparent outline-none text-sm sm:text-base px-2 py-2 min-h-[40px] max-h-32"
1462
  disabled={busy}
1463
  />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1464
  <button
1465
  type="button"
1466
  onClick={() => send(input)}
 
1470
  Send
1471
  </button>
1472
  </div>
1473
+
1474
+ {/* KI-257 Voice control row. Master "Enable voice" toggle on
1475
+ the left; helper text "Enter to submit chat" on the right.
1476
+ When Voice is ON the Live + Push-to-talk sub-row appears
1477
+ below this. */}
 
 
 
 
 
 
 
 
 
1478
  <div className="flex items-center justify-between gap-3 mt-2 pt-2 px-2 text-xs text-[var(--muted-foreground)]">
1479
+ <button
1480
+ type="button"
1481
+ onClick={() => setVoiceMasterOn((v) => !v)}
1482
+ className={`flex items-center gap-2 px-3 py-1 rounded-full border text-xs font-medium transition cursor-pointer ${
1483
+ voiceMasterOn
1484
+ ? "border-emerald-300 bg-emerald-50 text-emerald-700 hover:bg-emerald-100 dark:border-emerald-700 dark:bg-emerald-900/30 dark:text-emerald-300"
1485
+ : "border-[var(--border)] text-[var(--muted-foreground)] hover:border-[var(--primary)] hover:text-[var(--primary)]"
1486
+ }`}
1487
+ title={voiceMasterOn ? "Voice is on. Click to disable voice." : "Enable voice (shows Live and Push-to-talk options)"}
1488
+ >
1489
+ <span className={`inline-block w-2 h-2 rounded-full ${voiceMasterOn ? "bg-emerald-500 animate-pulse" : "bg-gray-400"}`} />
1490
+ {voiceMasterOn ? "Voice ON" : "Enable voice"}
1491
+ </button>
1492
+ <div className="hidden sm:block">Enter to submit chat</div>
1493
+ </div>
1494
+
1495
+ {voiceMasterOn && (
1496
+ <div className="mt-2 px-2 flex flex-wrap items-center gap-2">
1497
+ {/* Live (BETA) toggle — gated behind the confirm dialog */}
1498
  {!live.micPermissionDenied ? (
1499
  <button
1500
  type="button"
1501
  onClick={() => {
 
 
 
1502
  if (!userPrefersLive) {
1503
  const seen = (typeof window !== "undefined")
1504
  ? window.localStorage.getItem("insurance_live_beta_ack") === "1"
 
1518
  }
1519
  setUserPrefersLive((p) => !p);
1520
  }}
1521
+ className={`flex items-center gap-1.5 px-3 py-1 rounded-full border text-xs font-medium transition cursor-pointer ${
1522
  userPrefersLive
1523
  ? "border-amber-300 bg-amber-50 text-amber-700 hover:bg-amber-100 dark:border-amber-700 dark:bg-amber-900/30 dark:text-amber-300"
1524
  : "border-[var(--border)] text-[var(--muted-foreground)] hover:border-[var(--primary)] hover:text-[var(--primary)]"
1525
  }`}
1526
  title={userPrefersLive
1527
+ ? "Live voice is BETA — may cut you off / echo. Click to turn off."
1528
+ : "Enable Live always-on voice (BETA — unstable). Prefer Push-to-talk for reliable input."}
1529
  >
1530
  <span className={`inline-block w-2 h-2 rounded-full ${
1531
  userPrefersLive
 
1533
  : "bg-gray-400"
1534
  }`} />
1535
  {userPrefersLive
1536
+ ? (live.recording ? "Listening… (BETA)" : "Live ON · BETA")
1537
+ : "Live (BETA — unstable)"}
1538
  </button>
1539
  ) : (
1540
+ <span className="text-rose-500 text-xs" title="Allow mic in your browser site settings, or use Push-to-talk">
1541
+ 🔇 Mic blocked
1542
  </span>
1543
  )}
1544
+
1545
+ {/* Push-to-talk button click OR hold SPACE */}
1546
+ <button
1547
+ type="button"
1548
+ onClick={recording ? stopRecording : startRecording}
1549
+ disabled={busy && !recording}
1550
+ className={`h-9 px-3 rounded-full flex items-center gap-1.5 text-xs font-medium transition-all ${
1551
+ recording
1552
+ ? "bg-[var(--error)] text-white animate-record-pulse"
1553
+ : spaceHoldActive
1554
+ ? "bg-emerald-700 text-white ring-2 ring-emerald-300"
1555
+ : "bg-emerald-600 hover:bg-emerald-700 text-white shadow-md ring-2 ring-emerald-300 dark:ring-emerald-700"
1556
+ } disabled:opacity-40`}
1557
+ title={recording
1558
+ ? "Recording… click to stop and submit (or release SPACE)"
1559
+ : "Push-to-talk: click to start, click again to stop. Or hold SPACE."}
1560
+ >
1561
+ {recording ? <StopIcon /> : <MicIcon />}
1562
+ <span>{recording ? "Stop & send" : "Push-to-talk"}</span>
1563
+ </button>
1564
+
1565
+ {/* Hold SPACE helper — shows "ready to take audio" when
1566
+ user holds SPACE without yet recording. */}
1567
+ <span className="text-xs text-[var(--muted-foreground)] italic">
1568
+ {recording && spaceHoldActive
1569
+ ? "Listening… release SPACE to submit"
1570
+ : recording
1571
+ ? "Listening… click Stop to submit"
1572
+ : "or hold SPACE to talk · release to submit"}
1573
+ </span>
1574
  </div>
1575
+ )}
1576
+
1577
+ {/* PTT interim transcript — visible only while actually recording */}
1578
+ {voiceMasterOn && recording && pttInterim && (
1579
+ <div
1580
+ className="mt-1 px-2 text-xs italic text-[var(--muted-foreground)] leading-snug truncate"
1581
+ aria-live="polite"
1582
+ aria-atomic="true"
1583
+ title={pttInterim}
1584
+ >
1585
+ {pttInterim}
1586
+ </div>
1587
+ )}
1588
  </div>
1589
  </main>
1590