AgentNewTwo commited on
Commit
f705b22
·
1 Parent(s): f937854

Stream guarded stable audio prefixes

Browse files
Files changed (2) hide show
  1. README.md +21 -12
  2. live_app.py +337 -111
README.md CHANGED
@@ -28,11 +28,11 @@ RoleForge is a private feasibility lab for directing a fictional voice NPC. This
28
 
29
  ## Current gate
30
 
31
- This phase tests a quality-first free-Space approximation to live conversation:
32
 
33
  1. Wear headphones and press **Start 120-second live session**; approve microphone access when prompted.
34
  2. Wait for **Listening**, then speak naturally without pressing Stop.
35
- 3. A short pause ends the visitor turn; the model generates a reply and then plays reference-decoded speech.
36
  4. Speaking clearly during playback interrupts it and begins a new visitor turn.
37
 
38
  The app retains LFM's multimodal conversation state for up to six turns inside one bounded 120-second listening
@@ -43,9 +43,11 @@ This is interruptible turn-taking rather than mathematically simultaneous model
43
  speech can cancel an active reply before the new utterance is processed. A browser AudioWorklet captures continuous
44
  raw PCM and a same-origin WebSocket carries input and output packets. Gradio still owns controls, diagnostics, and
45
  the bounded ZeroGPU call, but it no longer owns the microphone or audio playback. This avoids both the native
46
- Record/Waiting state machine and HLS buffering. The server buffers each short answer through LFM's known-clean
47
- whole-response decoder, divides the result into 160 ms packets, and the browser schedules them sample-exactly through
48
- WebAudio with an adaptive 180 ms lead, a bounded server buffer,
 
 
49
  and immediate playback flush on barge-in. Playback epochs reset at reply boundaries so gaps between separate turns
50
  are not counted as underruns. Normal session completion stops microphone capture but lets already scheduled reply
51
  audio drain before closing WebAudio; manual stop and barge-in remain immediate. Microphone worklet packets are 4096 samples to reduce queue
@@ -54,20 +56,27 @@ cancellations.
54
 
55
  The system turn is exactly LiquidAI's required interleaved-mode instruction: `Respond with interleaved text and
56
  audio.` A short, public user-turn instruction pins the assistant's name to Lyra without assigning or inferring any
57
- visitor identity. Protected facts, trust scores, and private director instructions remain outside model context.
 
 
58
  Generated text is labeled `Model response text` until a physical test proves it matches the
59
  spoken assistant response. Audio-code handling follows LiquidAI's reference implementation: retain the complete code
60
  sequence and remove only a confirmed terminal all-2048 end-of-audio frame. The obsolete incremental Mimi decoder and
61
  its GPU warm-up are removed. A 600 ms pause closes the visitor turn. Model output runs until its audio EOS or the
62
- bounded 768-token generation cap so decoded text cannot outrun audible speech on longer replies, while the
63
- whole-response reference decoder preserves voice quality. Diagnostics report token/EOS handling, microphone signal quality/drops, playback
64
- underruns, startup/generation/retry/decode timing, and an inaudible stable-prefix equivalence comparison after the
65
- first eligible reply.
66
 
67
- On every turn, live generation rejects false assistant identities before playback. After the first turn it also rejects
 
68
  generic assistant openings and openings that closely match any earlier session response. Two bounded regeneration
69
  attempts use progressively broader text sampling; a remaining false-identity output fails closed rather than becoming
70
- audible. Diagnostics distinguish identity, generic-opening, history-repeat, and narration regenerations without
 
 
 
 
71
  changing the official system prompt or audio-code path.
72
 
73
  ## Privacy and safety
 
28
 
29
  ## Current gate
30
 
31
+ This phase tests a guarded low-latency free-Space approximation to live conversation:
32
 
33
  1. Wear headphones and press **Start 120-second live session**; approve microphone access when prompted.
34
  2. Wait for **Listening**, then speak naturally without pressing Stop.
35
+ 3. A short pause ends the visitor turn; playback begins once the opening guard and first stable audio prefix are ready.
36
  4. Speaking clearly during playback interrupts it and begins a new visitor turn.
37
 
38
  The app retains LFM's multimodal conversation state for up to six turns inside one bounded 120-second listening
 
43
  speech can cancel an active reply before the new utterance is processed. A browser AudioWorklet captures continuous
44
  raw PCM and a same-origin WebSocket carries input and output packets. Gradio still owns controls, diagnostics, and
45
  the bounded ZeroGPU call, but it no longer owns the microphone or audio playback. This avoids both the native
46
+ Record/Waiting state machine and HLS buffering. During generation, the server decodes growing audio-code prefixes,
47
+ holds back one unstable 80 ms frame, and emits only samples proven stable by the preceding physical A/B test. At EOS it
48
+ decodes the clean whole response once and emits only the not-yet-played reference tail. A prefix-decode exception falls
49
+ back automatically to the clean whole-response path. PCM is divided into packets of at most 160 ms and the browser
50
+ schedules them sample-exactly through WebAudio with an adaptive 180 ms lead, a bounded server buffer,
51
  and immediate playback flush on barge-in. Playback epochs reset at reply boundaries so gaps between separate turns
52
  are not counted as underruns. Normal session completion stops microphone capture but lets already scheduled reply
53
  audio drain before closing WebAudio; manual stop and barge-in remain immediate. Microphone worklet packets are 4096 samples to reduce queue
 
56
 
57
  The system turn is exactly LiquidAI's required interleaved-mode instruction: `Respond with interleaved text and
58
  audio.` A short, public user-turn instruction pins the assistant's name to Lyra without assigning or inferring any
59
+ fixed visitor identity. It states that personal names heard in visitor audio belong to the speaker, forbids Lyra from
60
+ adopting them, and asks for direct replies without opening fillers such as Ah, Um, or Hmm. Protected facts, trust scores,
61
+ and private director instructions remain outside model context.
62
  Generated text is labeled `Model response text` until a physical test proves it matches the
63
  spoken assistant response. Audio-code handling follows LiquidAI's reference implementation: retain the complete code
64
  sequence and remove only a confirmed terminal all-2048 end-of-audio frame. The obsolete incremental Mimi decoder and
65
  its GPU warm-up are removed. A 600 ms pause closes the visitor turn. Model output runs until its audio EOS or the
66
+ bounded 768-token generation cap so decoded text cannot outrun audible speech on longer replies. Diagnostics report
67
+ token/EOS handling, microphone signal quality/drops, playback underruns, startup/generation/retry/decode timing,
68
+ prefix-stream first-audio time, checkpoints, fallback/late-guard status, emitted duration, overhead, and live prefix/reference
69
+ waveform equivalence.
70
 
71
+ On every turn, live generation rejects false assistant identities—including attempts to “go by” a visitor's name—and
72
+ narrated openings before playback. The first attempt also rejects filler openings. After the first turn it rejects
73
  generic assistant openings and openings that closely match any earlier session response. Two bounded regeneration
74
  attempts use progressively broader text sampling; a remaining false-identity output fails closed rather than becoming
75
+ audible. Prefix playback waits for a complete opening sentence or a 96-character guard window. A protected marker,
76
+ false identity, or narrated role error discovered after streaming begins immediately aborts and flushes that reply.
77
+ The completed clean decode is compared with the emitted prefix; a waveform mismatch disables prefix streaming for
78
+ the remainder of that session and returns subsequent turns to whole-response playback.
79
+ Diagnostics distinguish identity, filler, generic-opening, history-repeat, and narration regenerations without
80
  changing the official system prompt or audio-code path.
81
 
82
  ## Privacy and safety
live_app.py CHANGED
@@ -47,14 +47,17 @@ SPEECH_RMS = 0.012
47
  BARGE_IN_RMS = 0.035
48
  BARGE_IN_CONFIRM_SECONDS = 0.22
49
  BARGE_IN_GRACE_SECONDS = 0.3
50
- STREAM_DIAGNOSTIC_CHUNK_FRAMES = 24
51
- STREAM_DIAGNOSTIC_STABLE_LAG_FRAMES = 1
52
- STREAM_DIAGNOSTIC_MAX_CHECKPOINTS = 4
53
- STREAM_DIAGNOSTIC_LATEST_START_SECONDS = 165.0
54
  TRANSPORT_VERSION = "webaudio-websocket-pcm-v1"
55
  INTERLEAVED_SYSTEM_PROMPT = "Respond with interleaved text and audio."
56
  ASSISTANT_NAME = "Lyra"
57
- ASSISTANT_IDENTITY_CONTEXT = "Your name is Lyra. Always identify yourself only as Lyra."
 
 
 
 
58
  AUDIO_EOS_TOKEN = 2048
59
 
60
  DIRECTOR_CUES = {
@@ -203,12 +206,20 @@ def generic_assistant_opening(value):
203
 
204
 
205
  def false_assistant_identity(value):
206
- opening = " ".join(normalized_words(value).split()[:24])
207
  if not opening:
208
  return False
209
- if "my name is" in opening and "my name is lyra" not in opening:
 
210
  return True
211
- if "you can call me" in opening and "you can call me lyra" not in opening:
 
 
 
 
 
 
 
212
  return True
213
  wrong_identity_markers = (
214
  "im qwen",
@@ -226,6 +237,18 @@ def false_assistant_identity(value):
226
  return "lyra" not in opening and any(marker in opening for marker in wrong_identity_markers)
227
 
228
 
 
 
 
 
 
 
 
 
 
 
 
 
229
  @dataclass
230
  class TurnDetector:
231
  speaking: bool = False
@@ -287,11 +310,11 @@ def prefix_stream_diagnostic(processor, audio_codes, reference_pcm):
287
  decode_seconds = 0.0
288
 
289
  for checkpoint in range(
290
- STREAM_DIAGNOSTIC_CHUNK_FRAMES,
291
  total_frames,
292
- STREAM_DIAGNOSTIC_CHUNK_FRAMES,
293
  ):
294
- if len(checkpoints) >= STREAM_DIAGNOSTIC_MAX_CHECKPOINTS:
295
  break
296
  started = time.perf_counter()
297
  prefix_pcm = (
@@ -305,7 +328,7 @@ def prefix_stream_diagnostic(processor, audio_codes, reference_pcm):
305
  decode_seconds += time.perf_counter() - started
306
  stable_samples = min(
307
  prefix_pcm.size,
308
- max(0, checkpoint - STREAM_DIAGNOSTIC_STABLE_LAG_FRAMES) * AUDIO_FRAME_SAMPLES,
309
  reference.size,
310
  )
311
  if stable_samples > emitted_samples:
@@ -355,6 +378,38 @@ def prefix_stream_diagnostic(processor, audio_codes, reference_pcm):
355
  }
356
 
357
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
358
  _model_cache = {}
359
  _manager = multiprocessing.Manager()
360
  _live_sessions = _manager.dict()
@@ -466,6 +521,7 @@ def session_metrics(phase, started, turns, counters, first_reply, interruptions,
466
  "duplicate_regenerations": playback.get("history_repeat_regenerations", 0),
467
  "generic_opening_regenerations": playback.get("generic_opening_regenerations", 0),
468
  "identity_regenerations": playback.get("identity_regenerations", 0),
 
469
  "total_regenerations": playback.get("regenerations", 0),
470
  "narration_regenerations": playback.get("style_regenerations", 0),
471
  "unsafe_outputs_suppressed": playback.get("unsafe_suppressed", 0),
@@ -549,9 +605,23 @@ def session_metrics(phase, started, turns, counters, first_reply, interruptions,
549
  "stream_diagnostic_max_abs_error": None
550
  if playback.get("stream_diagnostic_max_abs_error") is None
551
  else round(playback["stream_diagnostic_max_abs_error"], 6),
552
- "stream_diagnostic_audible": False,
553
- "stream_diagnostic_chunk_frames": STREAM_DIAGNOSTIC_CHUNK_FRAMES,
554
- "stream_diagnostic_stable_lag_frames": STREAM_DIAGNOSTIC_STABLE_LAG_FRAMES,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
555
  "reply_packets_emitted": playback["packets"],
556
  "reply_audio_emitted_seconds": round(playback["samples"] / OUTPUT_SAMPLE_RATE, 3),
557
  "reply_packet_target_seconds": round(OUTPUT_PACKET_SAMPLES / OUTPUT_SAMPLE_RATE, 3),
@@ -562,9 +632,9 @@ def session_metrics(phase, started, turns, counters, first_reply, interruptions,
562
  "transport": TRANSPORT_VERSION,
563
  "model_contract": "official exact interleaved system prompt",
564
  "assistant_identity": ASSISTANT_NAME,
565
- "identity_grounding": "public text attached to each visitor turn",
566
- "audio_delivery": "buffered whole-response reference decode",
567
- "mode": "interruptible quality-first LFM2.5-Audio",
568
  },
569
  indent=2,
570
  )
@@ -611,6 +681,7 @@ def run_live_session(scene, transport_id):
611
  "history_repeat_regenerations": 0,
612
  "generic_opening_regenerations": 0,
613
  "identity_regenerations": 0,
 
614
  "style_regenerations": 0,
615
  "style_suppressed": 0,
616
  "identity_suppressed": 0,
@@ -639,7 +710,7 @@ def run_live_session(scene, transport_id):
639
  "last_false_identity": None,
640
  "last_history_repeat_score": None,
641
  "legacy_filter_would_drop": None,
642
- "stream_diagnostic_status": "pending",
643
  "stream_diagnostic_checkpoints": 0,
644
  "stream_diagnostic_compared_seconds": None,
645
  "stream_diagnostic_decode_seconds": None,
@@ -647,9 +718,15 @@ def run_live_session(scene, transport_id):
647
  "stream_diagnostic_rms_ratio": None,
648
  "stream_diagnostic_snr_db": None,
649
  "stream_diagnostic_max_abs_error": None,
 
 
 
 
 
 
 
650
  }
651
- stream_diagnostic_codes = None
652
- stream_diagnostic_reference = None
653
 
654
  def emit_audio_packet(packet, response_started):
655
  nonlocal first_reply
@@ -663,6 +740,42 @@ def run_live_session(scene, transport_id):
663
  first_reply = time.perf_counter() - response_started
664
  return True
665
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
666
  ready_event.set()
667
  queue_control(output_queue, {"type": "phase", "phase": "listening"})
668
  live_started = time.perf_counter()
@@ -723,6 +836,18 @@ def run_live_session(scene, transport_id):
723
  yield "**Lyra is thinking.** You can speak again to cancel this reply.", gr.skip(), gr.skip()
724
 
725
  response_started = time.perf_counter()
 
 
 
 
 
 
 
 
 
 
 
 
726
  interrupted = False
727
  history_texts = [
728
  entry["response"]
@@ -735,6 +860,8 @@ def run_live_session(scene, transport_id):
735
  legacy_filter_would_drop = 0
736
  attempts_used = 0
737
  retry_generation_seconds = 0.0
 
 
738
  for attempt in range(MAX_GENERATION_ATTEMPTS):
739
  attempts_used = attempt + 1
740
  text_tokens = []
@@ -746,6 +873,17 @@ def run_live_session(scene, transport_id):
746
  attempt_started = time.perf_counter()
747
  attempt_first_token_seconds = None
748
  attempt_first_audio_token_seconds = None
 
 
 
 
 
 
 
 
 
 
 
749
 
750
  with torch.inference_mode():
751
  generation = model.generate_interleaved(
@@ -793,11 +931,28 @@ def run_live_session(scene, transport_id):
793
  text_tokens.append(token)
794
  modalities.append(LFMModality.TEXT)
795
  current_text = processor.text.decode(torch.cat(text_tokens)).removesuffix("<|text_end|>").strip()
796
- if attempt < MAX_GENERATION_ATTEMPTS - 1:
797
- if false_assistant_identity(current_text):
 
 
 
 
 
 
 
 
 
798
  opening_guard_reason = "identity"
799
  generation.close()
800
  break
 
 
 
 
 
 
 
 
801
  if guard_active and generic_assistant_opening(current_text):
802
  opening_guard_reason = "generic"
803
  generation.close()
@@ -817,8 +972,83 @@ def run_live_session(scene, transport_id):
817
  modalities.append(LFMModality.AUDIO_OUT)
818
  if torch.any(token == AUDIO_EOS_TOKEN).item():
819
  legacy_filter_would_drop += 1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
820
 
821
- attempt_seconds = time.perf_counter() - attempt_started
 
 
822
  playback["generation_seconds_last_attempt"] = attempt_seconds
823
  playback["first_token_seconds_last_attempt"] = attempt_first_token_seconds
824
  playback["first_audio_token_seconds_last_attempt"] = attempt_first_audio_token_seconds
@@ -832,8 +1062,12 @@ def run_live_session(scene, transport_id):
832
  playback["regenerations"] += 1
833
  if opening_guard_reason == "identity":
834
  playback["identity_regenerations"] += 1
 
 
835
  elif opening_guard_reason == "generic":
836
  playback["generic_opening_regenerations"] += 1
 
 
837
  else:
838
  playback["history_repeat_regenerations"] += 1
839
  print(f"RoleForge rejected opening reason={opening_guard_reason}; regenerating.")
@@ -843,7 +1077,11 @@ def run_live_session(scene, transport_id):
843
  if text_tokens
844
  else ""
845
  )
846
- if dialogue_style_violation(candidate_text) and attempt < MAX_GENERATION_ATTEMPTS - 1:
 
 
 
 
847
  retry_generation_seconds += attempt_seconds
848
  playback["regenerations"] += 1
849
  playback["style_regenerations"] += 1
@@ -851,7 +1089,10 @@ def run_live_session(scene, transport_id):
851
  continue
852
  break
853
 
854
- turn_generation_seconds = time.perf_counter() - response_started
 
 
 
855
  playback["generation_seconds_last_turn"] = turn_generation_seconds
856
  playback["retry_generation_seconds_last_turn"] = retry_generation_seconds
857
  model_seconds += turn_generation_seconds
@@ -879,6 +1120,10 @@ def run_live_session(scene, transport_id):
879
  identity_violation = false_assistant_identity(response_text)
880
  style_violation = dialogue_style_violation(response_text)
881
  reference_pcm = np.empty(0, dtype=np.float32)
 
 
 
 
882
  if (
883
  not interrupted
884
  and playable_audio_tokens
@@ -894,9 +1139,30 @@ def run_live_session(scene, transport_id):
894
  turn_decode_seconds = time.perf_counter() - decode_started
895
  playback["reference_decode_seconds"] += turn_decode_seconds
896
  playback["reference_decode_seconds_last_turn"] = turn_decode_seconds
897
- if stream_diagnostic_codes is None and len(playable_audio_tokens) > STREAM_DIAGNOSTIC_CHUNK_FRAMES:
898
- stream_diagnostic_codes = audio_codes.detach().cpu()
899
- stream_diagnostic_reference = reference_pcm.copy()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
900
 
901
  if leaks:
902
  playback["unsafe_suppressed"] += 1
@@ -912,42 +1178,43 @@ def run_live_session(scene, transport_id):
912
  response_text = "[narrated model output suppressed]"
913
 
914
  if reference_pcm.size and not interrupted:
915
- queue_control(output_queue, {"type": "phase", "phase": "speaking"})
916
- yield "**Model is speaking.** Speak clearly to interrupt.", gr.skip(), gr.skip()
917
- playback_started = time.perf_counter()
918
- reference_int16 = (reference_pcm * 32767.0).astype(np.int16)
919
- for offset in range(0, reference_int16.size, OUTPUT_PACKET_SAMPLES):
920
- for pending_rate, pending_audio in drain_queue(input_queue):
921
- pending_audio = np.asarray(pending_audio, dtype=np.float32).reshape(-1)
922
- pending_duration = pending_audio.size / max(float(pending_rate), 1.0)
923
- pending_rms = (
924
- float(np.sqrt(np.mean(pending_audio * pending_audio))) if pending_audio.size else 0.0
925
  )
926
- pending_events = detector.push(
927
- pending_rms,
928
- pending_duration,
929
- assistant_speaking=True,
930
- assistant_elapsed=time.perf_counter() - playback_started,
 
 
 
 
 
 
931
  )
932
- if "speech_started" in pending_events:
933
- utterance.extend(pre_roll)
934
- pre_roll.clear()
935
- utterance_rate = pending_rate
936
- if detector.speaking or "turn_ready" in pending_events:
937
- if utterance_rate == pending_rate:
938
- utterance.append(pending_audio)
939
- else:
940
- pre_roll.append(pending_audio)
941
- if "interrupt" in pending_events:
942
- interrupted = True
943
- if interrupted:
944
- interruptions += 1
945
- flush_output(output_queue)
946
- break
947
- packet = reference_int16[offset : offset + OUTPUT_PACKET_SAMPLES]
948
- if not emit_audio_packet(packet, response_started):
949
- interrupted = True
950
- break
951
 
952
  repeat_score = history_repetition_score(response_text, history_texts)
953
  repeated = repeat_score >= 0.86
@@ -977,48 +1244,6 @@ def run_live_session(scene, transport_id):
977
  "repeated": repeated,
978
  }
979
  )
980
- if stream_diagnostic_codes is not None and playback["stream_diagnostic_status"] == "pending":
981
- if time.perf_counter() - started > STREAM_DIAGNOSTIC_LATEST_START_SECONDS:
982
- playback["stream_diagnostic_status"] = "skipped_deadline_guard"
983
- else:
984
- queue_control(output_queue, {"type": "phase", "phase": "calibrating"})
985
- yield (
986
- "**Lyra audio calibration.** Comparing inaudible stable-prefix audio with the clean reply…",
987
- render_transcript(entries),
988
- session_metrics(
989
- "diagnostics",
990
- started,
991
- turns,
992
- counters,
993
- first_reply,
994
- interruptions,
995
- model_seconds,
996
- playback,
997
- repeats,
998
- ),
999
- )
1000
- try:
1001
- with torch.inference_mode():
1002
- diagnostic = prefix_stream_diagnostic(
1003
- processor,
1004
- stream_diagnostic_codes.to(DEVICE),
1005
- stream_diagnostic_reference,
1006
- )
1007
- playback["stream_diagnostic_status"] = (
1008
- "completed" if diagnostic["attempted"] else "skipped_insufficient_checkpoints"
1009
- )
1010
- playback["stream_diagnostic_checkpoints"] = len(diagnostic["checkpoints"])
1011
- playback["stream_diagnostic_compared_seconds"] = diagnostic["compared_seconds"]
1012
- playback["stream_diagnostic_decode_seconds"] = diagnostic["decode_seconds"]
1013
- playback["stream_diagnostic_correlation"] = diagnostic["correlation"]
1014
- playback["stream_diagnostic_rms_ratio"] = diagnostic["rms_ratio"]
1015
- playback["stream_diagnostic_snr_db"] = diagnostic["snr_db"]
1016
- playback["stream_diagnostic_max_abs_error"] = diagnostic["max_abs_error"]
1017
- except Exception as exc:
1018
- playback["stream_diagnostic_status"] = f"failed_{type(exc).__name__}"
1019
- print(f"RoleForge stable-prefix diagnostic failed: {type(exc).__name__}")
1020
- stream_diagnostic_codes = None
1021
- stream_diagnostic_reference = None
1022
  if turns >= MAX_TURNS:
1023
  break
1024
  queue_control(output_queue, {"type": "phase", "phase": "listening"})
@@ -1031,8 +1256,8 @@ def run_live_session(scene, transport_id):
1031
  ready_event.clear()
1032
  queue_control(output_queue, {"type": "complete"})
1033
  stop_event.set()
1034
- if playback["stream_diagnostic_status"] == "pending":
1035
- playback["stream_diagnostic_status"] = "skipped_no_eligible_response"
1036
  final_status = "Session complete. Start a new live session to continue." if turns else "Session ended before a turn completed."
1037
  yield (
1038
  final_status,
@@ -1242,15 +1467,16 @@ with gr.Blocks(title="RoleForge Live Voice NPC", theme=gr.themes.Soft(), css=CSS
1242
  gr.HTML(
1243
  """
1244
  <div class="hero">
1245
- <div class="phase">QUALITY-FIRST LIVE LFM2.5-AUDIO</div>
1246
  <h1>🎭 RoleForge: Live Voice NPC Director</h1>
1247
- <div>Continuous WebAudio microphone, reference-quality PCM playback, bounded memory, and barge-in.</div>
1248
  </div>
1249
  """
1250
  )
1251
  gr.Markdown(
1252
- "This quality-first build buffers each short reply through LFM's whole-response decoder before playback. "
1253
- "The microphone remains open and playback can still be interrupted. **Wear headphones** to prevent echo-triggered interruption."
 
1254
  )
1255
 
1256
  with gr.Row():
 
47
  BARGE_IN_RMS = 0.035
48
  BARGE_IN_CONFIRM_SECONDS = 0.22
49
  BARGE_IN_GRACE_SECONDS = 0.3
50
+ PREFIX_STREAM_CHUNK_FRAMES = 24
51
+ PREFIX_STREAM_STABLE_LAG_FRAMES = 1
52
+ PREFIX_STREAM_GUARD_MIN_CHARS = 48
 
53
  TRANSPORT_VERSION = "webaudio-websocket-pcm-v1"
54
  INTERLEAVED_SYSTEM_PROMPT = "Respond with interleaved text and audio."
55
  ASSISTANT_NAME = "Lyra"
56
+ ASSISTANT_IDENTITY_CONTEXT = (
57
+ "Your name is Lyra and you must identify yourself only as Lyra. The audio is spoken by the visitor. "
58
+ "Any personal name stated in that audio belongs to the visitor unless they are directly addressing Lyra; "
59
+ "never adopt that name as your own. Reply directly without opening fillers such as Ah, Um, or Hmm."
60
+ )
61
  AUDIO_EOS_TOKEN = 2048
62
 
63
  DIRECTOR_CUES = {
 
206
 
207
 
208
  def false_assistant_identity(value):
209
+ opening = " ".join(normalized_words(value).split()[:48])
210
  if not opening:
211
  return False
212
+ stated_name = re.search(r"\bmy name is\s+([a-z][a-z0-9'-]*)", opening)
213
+ if stated_name and stated_name.group(1) != "lyra":
214
  return True
215
+ called_name = re.search(r"\byou can call me\s+([a-z][a-z0-9'-]*)", opening)
216
+ if called_name and called_name.group(1) != "lyra":
217
+ return True
218
+ adopted_name = re.search(
219
+ r"\b(?:go by|call me|refer to me as)\s+([a-z][a-z0-9'-]*)",
220
+ opening,
221
+ )
222
+ if adopted_name and adopted_name.group(1) != "lyra":
223
  return True
224
  wrong_identity_markers = (
225
  "im qwen",
 
237
  return "lyra" not in opening and any(marker in opening for marker in wrong_identity_markers)
238
 
239
 
240
+ def filler_opening(value):
241
+ opening = normalized_words(value)
242
+ return bool(re.match(r"^(?:ah+|um+|uh+|hmm+)\b", opening))
243
+
244
+
245
+ def prefix_stream_guard_ready(value):
246
+ compact = " ".join(value.split())
247
+ if len(compact) < PREFIX_STREAM_GUARD_MIN_CHARS:
248
+ return False
249
+ return bool(re.search(r"[.!?](?:[\"'’”)]*)\s*$", compact)) or len(compact) >= 96
250
+
251
+
252
  @dataclass
253
  class TurnDetector:
254
  speaking: bool = False
 
310
  decode_seconds = 0.0
311
 
312
  for checkpoint in range(
313
+ PREFIX_STREAM_CHUNK_FRAMES,
314
  total_frames,
315
+ PREFIX_STREAM_CHUNK_FRAMES,
316
  ):
317
+ if len(checkpoints) >= 4:
318
  break
319
  started = time.perf_counter()
320
  prefix_pcm = (
 
328
  decode_seconds += time.perf_counter() - started
329
  stable_samples = min(
330
  prefix_pcm.size,
331
+ max(0, checkpoint - PREFIX_STREAM_STABLE_LAG_FRAMES) * AUDIO_FRAME_SAMPLES,
332
  reference.size,
333
  )
334
  if stable_samples > emitted_samples:
 
378
  }
379
 
380
 
381
+ def compare_prefix_waveform(candidate_pcm, reference_pcm):
382
+ candidate = np.asarray(candidate_pcm, dtype=np.float32).reshape(-1)
383
+ reference = np.asarray(reference_pcm, dtype=np.float32).reshape(-1)
384
+ compared_samples = min(candidate.size, reference.size)
385
+ if not compared_samples:
386
+ return {
387
+ "compared_seconds": 0.0,
388
+ "correlation": None,
389
+ "rms_ratio": None,
390
+ "snr_db": None,
391
+ "max_abs_error": None,
392
+ }
393
+ candidate = candidate[:compared_samples].astype(np.float64)
394
+ reference = reference[:compared_samples].astype(np.float64)
395
+ error = candidate - reference
396
+ reference_rms = float(np.sqrt(np.mean(reference * reference)))
397
+ candidate_rms = float(np.sqrt(np.mean(candidate * candidate)))
398
+ error_rms = float(np.sqrt(np.mean(error * error)))
399
+ correlation = (
400
+ float(np.corrcoef(candidate, reference)[0, 1])
401
+ if np.std(candidate) > 1e-12 and np.std(reference) > 1e-12
402
+ else (1.0 if np.allclose(candidate, reference) else 0.0)
403
+ )
404
+ return {
405
+ "compared_seconds": compared_samples / OUTPUT_SAMPLE_RATE,
406
+ "correlation": correlation,
407
+ "rms_ratio": candidate_rms / max(reference_rms, 1e-12),
408
+ "snr_db": 20.0 * np.log10(reference_rms / max(error_rms, 1e-12)),
409
+ "max_abs_error": float(np.max(np.abs(error))),
410
+ }
411
+
412
+
413
  _model_cache = {}
414
  _manager = multiprocessing.Manager()
415
  _live_sessions = _manager.dict()
 
521
  "duplicate_regenerations": playback.get("history_repeat_regenerations", 0),
522
  "generic_opening_regenerations": playback.get("generic_opening_regenerations", 0),
523
  "identity_regenerations": playback.get("identity_regenerations", 0),
524
+ "filler_opening_regenerations": playback.get("filler_regenerations", 0),
525
  "total_regenerations": playback.get("regenerations", 0),
526
  "narration_regenerations": playback.get("style_regenerations", 0),
527
  "unsafe_outputs_suppressed": playback.get("unsafe_suppressed", 0),
 
605
  "stream_diagnostic_max_abs_error": None
606
  if playback.get("stream_diagnostic_max_abs_error") is None
607
  else round(playback["stream_diagnostic_max_abs_error"], 6),
608
+ "stream_diagnostic_audible": playback.get("stream_diagnostic_status")
609
+ == "completed_audible_prefix",
610
+ "stream_diagnostic_chunk_frames": PREFIX_STREAM_CHUNK_FRAMES,
611
+ "stream_diagnostic_stable_lag_frames": PREFIX_STREAM_STABLE_LAG_FRAMES,
612
+ "prefix_stream_status": playback.get("prefix_stream_status"),
613
+ "prefix_stream_fallbacks": playback.get("prefix_stream_fallbacks", 0),
614
+ "prefix_stream_late_guard_aborts": playback.get("prefix_stream_late_guard_aborts", 0),
615
+ "prefix_stream_first_audio_seconds": None
616
+ if playback.get("prefix_stream_first_audio_seconds") is None
617
+ else round(playback["prefix_stream_first_audio_seconds"], 3),
618
+ "prefix_stream_emitted_seconds": round(
619
+ playback.get("prefix_stream_emitted_seconds", 0.0), 3
620
+ ),
621
+ "prefix_stream_overhead_seconds_last_turn": round(
622
+ playback.get("prefix_stream_overhead_seconds_last_turn", 0.0), 3
623
+ ),
624
+ "prefix_stream_session_enabled": playback.get("prefix_stream_session_enabled", True),
625
  "reply_packets_emitted": playback["packets"],
626
  "reply_audio_emitted_seconds": round(playback["samples"] / OUTPUT_SAMPLE_RATE, 3),
627
  "reply_packet_target_seconds": round(OUTPUT_PACKET_SAMPLES / OUTPUT_SAMPLE_RATE, 3),
 
632
  "transport": TRANSPORT_VERSION,
633
  "model_contract": "official exact interleaved system prompt",
634
  "assistant_identity": ASSISTANT_NAME,
635
+ "identity_grounding": "Lyra-only public role boundary attached to each visitor turn",
636
+ "audio_delivery": "guarded held-tail stable-prefix stream with clean reference tail",
637
+ "mode": "interruptible low-latency LFM2.5-Audio",
638
  },
639
  indent=2,
640
  )
 
681
  "history_repeat_regenerations": 0,
682
  "generic_opening_regenerations": 0,
683
  "identity_regenerations": 0,
684
+ "filler_regenerations": 0,
685
  "style_regenerations": 0,
686
  "style_suppressed": 0,
687
  "identity_suppressed": 0,
 
710
  "last_false_identity": None,
711
  "last_history_repeat_score": None,
712
  "legacy_filter_would_drop": None,
713
+ "stream_diagnostic_status": "pending_audible_prefix",
714
  "stream_diagnostic_checkpoints": 0,
715
  "stream_diagnostic_compared_seconds": None,
716
  "stream_diagnostic_decode_seconds": None,
 
718
  "stream_diagnostic_rms_ratio": None,
719
  "stream_diagnostic_snr_db": None,
720
  "stream_diagnostic_max_abs_error": None,
721
+ "prefix_stream_status": "not_started",
722
+ "prefix_stream_fallbacks": 0,
723
+ "prefix_stream_late_guard_aborts": 0,
724
+ "prefix_stream_first_audio_seconds": None,
725
+ "prefix_stream_emitted_seconds": 0.0,
726
+ "prefix_stream_overhead_seconds_last_turn": 0.0,
727
+ "prefix_stream_session_enabled": True,
728
  }
729
+ prefix_stream_enabled = True
 
730
 
731
  def emit_audio_packet(packet, response_started):
732
  nonlocal first_reply
 
740
  first_reply = time.perf_counter() - response_started
741
  return True
742
 
743
+ def collect_stream_interrupt(playback_started):
744
+ nonlocal utterance_rate
745
+ for pending_rate, pending_audio in drain_queue(input_queue):
746
+ pending_audio = np.asarray(pending_audio, dtype=np.float32).reshape(-1)
747
+ pending_duration = pending_audio.size / max(float(pending_rate), 1.0)
748
+ pending_rms = (
749
+ float(np.sqrt(np.mean(pending_audio * pending_audio))) if pending_audio.size else 0.0
750
+ )
751
+ pending_events = detector.push(
752
+ pending_rms,
753
+ pending_duration,
754
+ assistant_speaking=True,
755
+ assistant_elapsed=time.perf_counter() - playback_started,
756
+ )
757
+ if "speech_started" in pending_events:
758
+ utterance.extend(pre_roll)
759
+ pre_roll.clear()
760
+ utterance_rate = pending_rate
761
+ if detector.speaking or "turn_ready" in pending_events:
762
+ if utterance_rate == pending_rate:
763
+ utterance.append(pending_audio)
764
+ else:
765
+ pre_roll.append(pending_audio)
766
+ if "interrupt" in pending_events:
767
+ return True
768
+ return False
769
+
770
+ def emit_pcm(pcm, response_started, playback_started):
771
+ pcm16 = (np.clip(np.asarray(pcm, dtype=np.float32), -1.0, 1.0) * 32767.0).astype(np.int16)
772
+ for offset in range(0, pcm16.size, OUTPUT_PACKET_SAMPLES):
773
+ if collect_stream_interrupt(playback_started):
774
+ return False
775
+ if not emit_audio_packet(pcm16[offset : offset + OUTPUT_PACKET_SAMPLES], response_started):
776
+ return False
777
+ return True
778
+
779
  ready_event.set()
780
  queue_control(output_queue, {"type": "phase", "phase": "listening"})
781
  live_started = time.perf_counter()
 
836
  yield "**Lyra is thinking.** You can speak again to cancel this reply.", gr.skip(), gr.skip()
837
 
838
  response_started = time.perf_counter()
839
+ playback["prefix_stream_status"] = "not_started"
840
+ playback["prefix_stream_first_audio_seconds"] = None
841
+ playback["prefix_stream_emitted_seconds"] = 0.0
842
+ playback["prefix_stream_overhead_seconds_last_turn"] = 0.0
843
+ playback["stream_diagnostic_status"] = "pending_audible_prefix"
844
+ playback["stream_diagnostic_checkpoints"] = 0
845
+ playback["stream_diagnostic_compared_seconds"] = None
846
+ playback["stream_diagnostic_decode_seconds"] = None
847
+ playback["stream_diagnostic_correlation"] = None
848
+ playback["stream_diagnostic_rms_ratio"] = None
849
+ playback["stream_diagnostic_snr_db"] = None
850
+ playback["stream_diagnostic_max_abs_error"] = None
851
  interrupted = False
852
  history_texts = [
853
  entry["response"]
 
860
  legacy_filter_would_drop = 0
861
  attempts_used = 0
862
  retry_generation_seconds = 0.0
863
+ stream_overhead_turn = 0.0
864
+ stream_guard_abort = False
865
  for attempt in range(MAX_GENERATION_ATTEMPTS):
866
  attempts_used = attempt + 1
867
  text_tokens = []
 
873
  attempt_started = time.perf_counter()
874
  attempt_first_token_seconds = None
875
  attempt_first_audio_token_seconds = None
876
+ attempt_stream_overhead = 0.0
877
+ current_text = ""
878
+ stream_emitted_samples = 0
879
+ stream_candidate_parts = []
880
+ stream_checkpoints = 0
881
+ stream_last_checkpoint_frames = 0
882
+ stream_decode_seconds = 0.0
883
+ stream_started = False
884
+ stream_playback_started = None
885
+ stream_decode_failed = False
886
+ stream_guard_abort = False
887
 
888
  with torch.inference_mode():
889
  generation = model.generate_interleaved(
 
931
  text_tokens.append(token)
932
  modalities.append(LFMModality.TEXT)
933
  current_text = processor.text.decode(torch.cat(text_tokens)).removesuffix("<|text_end|>").strip()
934
+ identity_error = false_assistant_identity(current_text)
935
+ style_error = dialogue_style_violation(current_text)
936
+ leak_error = bool(leak_markers(current_text))
937
+ if stream_started and (identity_error or style_error or leak_error):
938
+ stream_guard_abort = True
939
+ playback["prefix_stream_late_guard_aborts"] += 1
940
+ generation.close()
941
+ flush_output(output_queue)
942
+ break
943
+ if not stream_started and attempt < MAX_GENERATION_ATTEMPTS - 1:
944
+ if identity_error:
945
  opening_guard_reason = "identity"
946
  generation.close()
947
  break
948
+ if attempt == 0 and filler_opening(current_text):
949
+ opening_guard_reason = "filler"
950
+ generation.close()
951
+ break
952
+ if style_error:
953
+ opening_guard_reason = "style"
954
+ generation.close()
955
+ break
956
  if guard_active and generic_assistant_opening(current_text):
957
  opening_guard_reason = "generic"
958
  generation.close()
 
972
  modalities.append(LFMModality.AUDIO_OUT)
973
  if torch.any(token == AUDIO_EOS_TOKEN).item():
974
  legacy_filter_would_drop += 1
975
+ should_decode_prefix = (
976
+ prefix_stream_enabled
977
+ and not is_audio_eos(token)
978
+ and not stream_decode_failed
979
+ and len(audio_tokens) >= PREFIX_STREAM_CHUNK_FRAMES
980
+ and (
981
+ not stream_started
982
+ or len(audio_tokens) - stream_last_checkpoint_frames
983
+ >= PREFIX_STREAM_CHUNK_FRAMES
984
+ )
985
+ and prefix_stream_guard_ready(current_text)
986
+ and not false_assistant_identity(current_text)
987
+ and not dialogue_style_violation(current_text)
988
+ and not leak_markers(current_text)
989
+ and (attempt > 0 or not filler_opening(current_text))
990
+ )
991
+ if should_decode_prefix:
992
+ prefix_started = time.perf_counter()
993
+ stable_delta = None
994
+ stable_samples = stream_emitted_samples
995
+ try:
996
+ prefix_codes = torch.stack(audio_tokens, dim=1).unsqueeze(0)
997
+ prefix_pcm = np.clip(
998
+ processor.decode(prefix_codes)[0]
999
+ .detach()
1000
+ .float()
1001
+ .cpu()
1002
+ .numpy(),
1003
+ -1.0,
1004
+ 1.0,
1005
+ ).reshape(-1)
1006
+ stable_samples = min(
1007
+ prefix_pcm.size,
1008
+ max(0, len(audio_tokens) - PREFIX_STREAM_STABLE_LAG_FRAMES)
1009
+ * AUDIO_FRAME_SAMPLES,
1010
+ )
1011
+ if stable_samples > stream_emitted_samples:
1012
+ stable_delta = prefix_pcm[stream_emitted_samples:stable_samples].copy()
1013
+ except Exception as exc:
1014
+ stream_decode_failed = True
1015
+ playback["prefix_stream_fallbacks"] += 1
1016
+ print(f"RoleForge prefix decode fell back: {type(exc).__name__}")
1017
+ finally:
1018
+ decode_elapsed = time.perf_counter() - prefix_started
1019
+ stream_decode_seconds += decode_elapsed
1020
+ attempt_stream_overhead += decode_elapsed
1021
+ if stable_delta is not None:
1022
+ if not stream_started:
1023
+ stream_started = True
1024
+ stream_playback_started = time.perf_counter()
1025
+ playback["prefix_stream_first_audio_seconds"] = (
1026
+ stream_playback_started - response_started
1027
+ )
1028
+ queue_control(output_queue, {"type": "phase", "phase": "speaking"})
1029
+ emission_started = time.perf_counter()
1030
+ emitted = emit_pcm(
1031
+ stable_delta,
1032
+ response_started,
1033
+ stream_playback_started,
1034
+ )
1035
+ attempt_stream_overhead += time.perf_counter() - emission_started
1036
+ if not emitted:
1037
+ interrupted = True
1038
+ interruptions += 1
1039
+ generation.close()
1040
+ flush_output(output_queue)
1041
+ break
1042
+ stream_candidate_parts.append(stable_delta)
1043
+ stream_emitted_samples = stable_samples
1044
+ stream_checkpoints += 1
1045
+ stream_last_checkpoint_frames = len(audio_tokens)
1046
+ if interrupted or stream_guard_abort:
1047
+ break
1048
 
1049
+ attempt_wall_seconds = time.perf_counter() - attempt_started
1050
+ stream_overhead_turn += attempt_stream_overhead
1051
+ attempt_seconds = max(0.0, attempt_wall_seconds - attempt_stream_overhead)
1052
  playback["generation_seconds_last_attempt"] = attempt_seconds
1053
  playback["first_token_seconds_last_attempt"] = attempt_first_token_seconds
1054
  playback["first_audio_token_seconds_last_attempt"] = attempt_first_audio_token_seconds
 
1062
  playback["regenerations"] += 1
1063
  if opening_guard_reason == "identity":
1064
  playback["identity_regenerations"] += 1
1065
+ elif opening_guard_reason == "filler":
1066
+ playback["filler_regenerations"] += 1
1067
  elif opening_guard_reason == "generic":
1068
  playback["generic_opening_regenerations"] += 1
1069
+ elif opening_guard_reason == "style":
1070
+ playback["style_regenerations"] += 1
1071
  else:
1072
  playback["history_repeat_regenerations"] += 1
1073
  print(f"RoleForge rejected opening reason={opening_guard_reason}; regenerating.")
 
1077
  if text_tokens
1078
  else ""
1079
  )
1080
+ if (
1081
+ dialogue_style_violation(candidate_text)
1082
+ and attempt < MAX_GENERATION_ATTEMPTS - 1
1083
+ and not stream_started
1084
+ ):
1085
  retry_generation_seconds += attempt_seconds
1086
  playback["regenerations"] += 1
1087
  playback["style_regenerations"] += 1
 
1089
  continue
1090
  break
1091
 
1092
+ turn_generation_seconds = max(
1093
+ 0.0,
1094
+ time.perf_counter() - response_started - stream_overhead_turn,
1095
+ )
1096
  playback["generation_seconds_last_turn"] = turn_generation_seconds
1097
  playback["retry_generation_seconds_last_turn"] = retry_generation_seconds
1098
  model_seconds += turn_generation_seconds
 
1120
  identity_violation = false_assistant_identity(response_text)
1121
  style_violation = dialogue_style_violation(response_text)
1122
  reference_pcm = np.empty(0, dtype=np.float32)
1123
+ playback["prefix_stream_overhead_seconds_last_turn"] = stream_overhead_turn
1124
+ playback["stream_diagnostic_checkpoints"] = stream_checkpoints
1125
+ playback["stream_diagnostic_decode_seconds"] = stream_decode_seconds
1126
+ playback["prefix_stream_emitted_seconds"] = stream_emitted_samples / OUTPUT_SAMPLE_RATE
1127
  if (
1128
  not interrupted
1129
  and playable_audio_tokens
 
1139
  turn_decode_seconds = time.perf_counter() - decode_started
1140
  playback["reference_decode_seconds"] += turn_decode_seconds
1141
  playback["reference_decode_seconds_last_turn"] = turn_decode_seconds
1142
+ if stream_candidate_parts:
1143
+ streamed_candidate = np.concatenate(stream_candidate_parts)
1144
+ comparison = compare_prefix_waveform(streamed_candidate, reference_pcm)
1145
+ playback["stream_diagnostic_status"] = "completed_audible_prefix"
1146
+ playback["stream_diagnostic_compared_seconds"] = comparison["compared_seconds"]
1147
+ playback["stream_diagnostic_correlation"] = comparison["correlation"]
1148
+ playback["stream_diagnostic_rms_ratio"] = comparison["rms_ratio"]
1149
+ playback["stream_diagnostic_snr_db"] = comparison["snr_db"]
1150
+ playback["stream_diagnostic_max_abs_error"] = comparison["max_abs_error"]
1151
+ waveform_safe = (
1152
+ comparison["correlation"] is not None
1153
+ and comparison["correlation"] >= 0.9999
1154
+ and comparison["rms_ratio"] is not None
1155
+ and 0.999 <= comparison["rms_ratio"] <= 1.001
1156
+ and comparison["snr_db"] is not None
1157
+ and comparison["snr_db"] >= 60.0
1158
+ and comparison["max_abs_error"] is not None
1159
+ and comparison["max_abs_error"] <= 0.001
1160
+ )
1161
+ if not waveform_safe:
1162
+ prefix_stream_enabled = False
1163
+ playback["prefix_stream_session_enabled"] = False
1164
+ playback["prefix_stream_fallbacks"] += 1
1165
+ playback["stream_diagnostic_status"] = "disabled_after_waveform_mismatch"
1166
 
1167
  if leaks:
1168
  playback["unsafe_suppressed"] += 1
 
1178
  response_text = "[narrated model output suppressed]"
1179
 
1180
  if reference_pcm.size and not interrupted:
1181
+ if stream_started:
1182
+ playback["prefix_stream_status"] = (
1183
+ "disabled_after_waveform_mismatch"
1184
+ if not prefix_stream_enabled
1185
+ else (
1186
+ "completed_with_reference_tail_after_decode_fallback"
1187
+ if stream_decode_failed
1188
+ else "completed"
 
 
1189
  )
1190
+ )
1191
+ tail_pcm = reference_pcm[stream_emitted_samples:]
1192
+ playback_started = stream_playback_started
1193
+ else:
1194
+ playback["prefix_stream_status"] = (
1195
+ "fallback_whole_response_stream_disabled"
1196
+ if not prefix_stream_enabled
1197
+ else (
1198
+ "fallback_whole_response_decode_error"
1199
+ if stream_decode_failed
1200
+ else "fallback_whole_response_guard_or_short_reply"
1201
  )
1202
+ )
1203
+ playback["stream_diagnostic_status"] = playback["prefix_stream_status"]
1204
+ tail_pcm = reference_pcm
1205
+ playback_started = time.perf_counter()
1206
+ queue_control(output_queue, {"type": "phase", "phase": "speaking"})
1207
+ yield "**Model is speaking.** Speak clearly to interrupt.", gr.skip(), gr.skip()
1208
+ if tail_pcm.size and not emit_pcm(
1209
+ tail_pcm,
1210
+ response_started,
1211
+ playback_started,
1212
+ ):
1213
+ interrupted = True
1214
+ interruptions += 1
1215
+ flush_output(output_queue)
1216
+ elif stream_guard_abort:
1217
+ playback["prefix_stream_status"] = "aborted_late_guard"
 
 
 
1218
 
1219
  repeat_score = history_repetition_score(response_text, history_texts)
1220
  repeated = repeat_score >= 0.86
 
1244
  "repeated": repeated,
1245
  }
1246
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1247
  if turns >= MAX_TURNS:
1248
  break
1249
  queue_control(output_queue, {"type": "phase", "phase": "listening"})
 
1256
  ready_event.clear()
1257
  queue_control(output_queue, {"type": "complete"})
1258
  stop_event.set()
1259
+ if playback["stream_diagnostic_status"] == "pending_audible_prefix":
1260
+ playback["stream_diagnostic_status"] = "no_eligible_response"
1261
  final_status = "Session complete. Start a new live session to continue." if turns else "Session ended before a turn completed."
1262
  yield (
1263
  final_status,
 
1467
  gr.HTML(
1468
  """
1469
  <div class="hero">
1470
+ <div class="phase">LOW-LATENCY LIVE LFM2.5-AUDIO</div>
1471
  <h1>🎭 RoleForge: Live Voice NPC Director</h1>
1472
+ <div>Continuous WebAudio microphone, stable-prefix PCM streaming, bounded memory, and barge-in.</div>
1473
  </div>
1474
  """
1475
  )
1476
  gr.Markdown(
1477
+ "This build begins playback from guarded, proven-stable LFM prefix decodes and finishes with the clean reference tail. "
1478
+ "It automatically falls back to whole-response playback if prefix decoding fails. The microphone remains open and playback "
1479
+ "can be interrupted. **Wear headphones** to prevent echo-triggered interruption."
1480
  )
1481
 
1482
  with gr.Row():