Mike0021 commited on
Commit
d6ae3bc
·
verified ·
1 Parent(s): dc0b34f

Deploy static browser-speak demo

Browse files
Files changed (7) hide show
  1. README.md +19 -4
  2. app.js +2591 -0
  3. index.html +240 -17
  4. styles.css +590 -0
  5. workers/asr-worker.js +277 -0
  6. workers/llm-worker.js +140 -0
  7. workers/tts-worker.js +192 -0
README.md CHANGED
@@ -1,10 +1,25 @@
1
  ---
2
  title: Browser Speak
3
- emoji: 📉
4
- colorFrom: green
5
- colorTo: yellow
6
  sdk: static
 
7
  pinned: false
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: Browser Speak
 
 
 
3
  sdk: static
4
+ app_file: index.html
5
  pinned: false
6
+ license: mit
7
+ short_description: Client-side voice LLM with local STT and TTS.
8
  ---
9
 
10
+ # Browser Speak
11
+
12
+ Static browser demo for client-side voice interaction:
13
+
14
+ - VAD: `onnx-community/silero-vad`
15
+ - STT: `onnx-community/moonshine-base-ONNX`
16
+ - LLM: `HuggingFaceTB/SmolLM2-135M-Instruct`
17
+ - TTS: `onnx-community/Supertonic-TTS-ONNX`
18
+
19
+ Open the Space, click **Load models**, then **Start mic**. The first load downloads model assets into the browser cache; after that, inference runs in the tab with no server inference.
20
+
21
+ Known testing notes:
22
+
23
+ - Use Chrome or another browser with WebGPU/WASM support and microphone permission enabled.
24
+ - The app automatically falls back to WASM if hardware WebGPU is unavailable.
25
+ - Initial load can be large because STT, LLM, TTS, and voice embeddings are all browser-side assets.
app.js ADDED
@@ -0,0 +1,2591 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ const $ = (id) => document.getElementById(id);
2
+
3
+ const elements = {
4
+ loadButton: $("loadButton"),
5
+ micButton: $("micButton"),
6
+ stopButton: $("stopButton"),
7
+ suiteButton: $("suiteButton"),
8
+ benchmarkButton: $("benchmarkButton"),
9
+ chatBenchmarkButton: $("chatBenchmarkButton"),
10
+ ttsBenchmarkButton: $("ttsBenchmarkButton"),
11
+ loopbackButton: $("loopbackButton"),
12
+ bargeInButton: $("bargeInButton"),
13
+ micBenchmarkButton: $("micBenchmarkButton"),
14
+ micSeriesButton: $("micSeriesButton"),
15
+ copyResultsButton: $("copyResultsButton"),
16
+ clearResultsButton: $("clearResultsButton"),
17
+ clearLogButton: $("clearLogButton"),
18
+ deviceSelect: $("deviceSelect"),
19
+ runtimeStatus: $("runtimeStatus"),
20
+ runtimeDeviceStatus: $("runtimeDeviceStatus"),
21
+ runtimeDeviceDetail: $("runtimeDeviceDetail"),
22
+ llmModelSelect: $("llmModelSelect"),
23
+ asrModelSelect: $("asrModelSelect"),
24
+ voiceSelect: $("voiceSelect"),
25
+ ttsSteps: $("ttsSteps"),
26
+ ttsStepsValue: $("ttsStepsValue"),
27
+ vadSilence: $("vadSilence"),
28
+ vadSilenceValue: $("vadSilenceValue"),
29
+ partialToggle: $("partialToggle"),
30
+ micBadge: $("micBadge"),
31
+ audioBadge: $("audioBadge"),
32
+ partialTranscript: $("partialTranscript"),
33
+ finalTranscript: $("finalTranscript"),
34
+ llmOutput: $("llmOutput"),
35
+ eventLog: $("eventLog"),
36
+ benchmarkSummary: $("benchmarkSummary"),
37
+ resultsBody: $("resultsBody"),
38
+ micValidationCard: $("micValidationCard"),
39
+ micValidationStatus: $("micValidationStatus"),
40
+ micValidationDetail: $("micValidationDetail"),
41
+ micValidationProgressBar: $("micValidationProgressBar"),
42
+ vadCloseLatency: $("vadCloseLatency"),
43
+ asrLatency: $("asrLatency"),
44
+ firstTokenLatency: $("firstTokenLatency"),
45
+ firstTtsQueuedLatency: $("firstTtsQueuedLatency"),
46
+ ttsSynthLatency: $("ttsSynthLatency"),
47
+ firstAudioLatency: $("firstAudioLatency"),
48
+ speechToAudioLatency: $("speechToAudioLatency"),
49
+ decodeRate: $("decodeRate"),
50
+ tiles: {
51
+ vad: $("vadTile"),
52
+ asr: $("asrTile"),
53
+ llm: $("llmTile"),
54
+ tts: $("ttsTile"),
55
+ },
56
+ states: {
57
+ vad: $("vadState"),
58
+ asr: $("asrState"),
59
+ llm: $("llmState"),
60
+ tts: $("ttsState"),
61
+ },
62
+ };
63
+
64
+ const APP_IDENTITY_PROMPT = "What app is this?";
65
+ const APP_IDENTITY_ANSWER =
66
+ "This is a client-side browser voice assistant demo with local speech recognition, an LLM, and Supertonic TTS.";
67
+ const GENERIC_SYSTEM_PROMPT =
68
+ "You are the assistant inside a local browser voice demo. Reply in one short sentence under 18 words by default. Do not wrap responses in quotation marks.";
69
+ const IDENTITY_SYSTEM_PROMPT = `If asked what this demo, app, or application is, reply exactly: ${APP_IDENTITY_ANSWER} Do not shorten or paraphrase that identity answer. You are the assistant inside that local browser voice demo. Reply in one short sentence under 18 words by default. Do not wrap responses in quotation marks.`;
70
+ const SYSTEM_MESSAGES = [
71
+ {
72
+ role: "system",
73
+ content: GENERIC_SYSTEM_PROMPT,
74
+ },
75
+ ];
76
+ const IDENTITY_PRIMER_MESSAGES = [
77
+ {
78
+ role: "user",
79
+ content: APP_IDENTITY_PROMPT,
80
+ },
81
+ {
82
+ role: "assistant",
83
+ content: APP_IDENTITY_ANSWER,
84
+ },
85
+ ];
86
+ const APP_IDENTITY_QUALITY_RULES = [
87
+ { label: "client/browser/local", pattern: /\b(client(?:-side)?|browser|local)\b/i },
88
+ { label: "speech recognition", pattern: /\b(speech recognition|stt|transcrib(?:e|es|ing|er|ers|ed)?)\b/i },
89
+ { label: "LLM", pattern: /\b(llm|language model)\b/i },
90
+ { label: "TTS/Supertonic", pattern: /\b(tts|supertonic|speech synthesis|text-to-speech)\b/i },
91
+ ];
92
+ const WEBGPU_ONLY_LLMS = new Set([
93
+ "onnx-community/Qwen3-0.6B-ONNX",
94
+ "onnx-community/granite-4.0-350m-ONNX-web",
95
+ "HuggingFaceTB/SmolLM2-1.7B-Instruct",
96
+ ]);
97
+ const WEBGPU_ONLY_REASONS = new Map([
98
+ [
99
+ "onnx-community/granite-4.0-350m-ONNX-web",
100
+ "its q4 WASM external data is about 576 MB; keep it on WebGPU until it is measured there.",
101
+ ],
102
+ [
103
+ "onnx-community/Qwen3-0.6B-ONNX",
104
+ "local testing showed its q4 WASM load can exhaust the browser.",
105
+ ],
106
+ [
107
+ "HuggingFaceTB/SmolLM2-1.7B-Instruct",
108
+ "its browser assets are too large for this WASM fallback test environment.",
109
+ ],
110
+ ]);
111
+
112
+ const DEFAULT_VAD_SILENCE_MS = 480;
113
+ const MIN_VAD_SILENCE_MS = 200;
114
+ const MAX_VAD_SILENCE_MS = 800;
115
+ const DEFAULT_LOOPBACK_SPEED = 1.10;
116
+ const MIC_START_TIMEOUT_MS = 15000;
117
+ const DEFAULT_TTS_CHUNKING = Object.freeze({
118
+ firstSentenceMinChars: 5,
119
+ sentenceMinChars: 8,
120
+ firstClauseMinChars: 5,
121
+ clauseMinChars: 24,
122
+ firstTargetChars: 5,
123
+ targetChars: 36,
124
+ firstMinSpaceChars: 3,
125
+ minSpaceChars: 20,
126
+ });
127
+
128
+ const state = {
129
+ asrWorker: null,
130
+ llmWorker: null,
131
+ ttsWorker: null,
132
+ webgpuAvailable: null,
133
+ webgpuAdapterInfo: null,
134
+ webgpuAdapterFeatures: [],
135
+ webgpuSoftwareAdapter: false,
136
+ modelsLoaded: false,
137
+ modelsLoading: false,
138
+ micActive: false,
139
+ mediaStream: null,
140
+ audioContext: null,
141
+ micSource: null,
142
+ workletNode: null,
143
+ micMonitorGain: null,
144
+ playback: null,
145
+ messages: initialMessages(),
146
+ currentAssistant: "",
147
+ currentUserStartedAt: null,
148
+ lastSpeechEndAt: null,
149
+ lastVadCloseAt: null,
150
+ lastVadCloseDelayMs: null,
151
+ lastTranscriptAt: null,
152
+ awaitingFirstToken: false,
153
+ awaitingFirstAudio: false,
154
+ firstTtsChunkQueued: false,
155
+ ttsBuffer: "",
156
+ ttsSequence: 0,
157
+ loopbackRequestId: 0,
158
+ loopbackFeedTimers: new Set(),
159
+ ttsVoiceRequestId: 0,
160
+ ttsVoiceRequests: new Map(),
161
+ automationSynthesisRequestId: 0,
162
+ automationSynthesisRequests: new Map(),
163
+ activeTurnId: 0,
164
+ modelLoadStartedAt: null,
165
+ loadedStack: null,
166
+ inputSampleRate: null,
167
+ micInputStats: {
168
+ chunks: 0,
169
+ samples: 0,
170
+ peak: 0,
171
+ sumSquares: 0,
172
+ },
173
+ activeBenchmark: null,
174
+ benchmarkTimeout: null,
175
+ pendingPlaybackSchedules: 0,
176
+ suiteRunning: false,
177
+ micSeries: {
178
+ active: false,
179
+ target: 3,
180
+ completed: 0,
181
+ timer: 0,
182
+ },
183
+ ttsChunking: { ...DEFAULT_TTS_CHUNKING },
184
+ benchmarkResults: [],
185
+ };
186
+
187
+ class PlaybackQueue {
188
+ constructor() {
189
+ this.context = null;
190
+ this.nextTime = 0;
191
+ this.sources = new Set();
192
+ this.started = false;
193
+ }
194
+
195
+ async ensure() {
196
+ if (!this.context) {
197
+ this.context = new AudioContext();
198
+ }
199
+ if (this.context.state !== "running") {
200
+ await this.context.resume();
201
+ }
202
+ }
203
+
204
+ async unlock() {
205
+ await this.ensure();
206
+ const buffer = this.context.createBuffer(1, 1, 22050);
207
+ const source = this.context.createBufferSource();
208
+ const gain = this.context.createGain();
209
+ gain.gain.value = 0;
210
+ source.buffer = buffer;
211
+ source.connect(gain);
212
+ gain.connect(this.context.destination);
213
+ source.start();
214
+ }
215
+
216
+ async play(samples, sampleRate) {
217
+ await this.ensure();
218
+ const data = samples instanceof Float32Array ? samples : new Float32Array(samples);
219
+ const buffer = this.context.createBuffer(1, data.length, sampleRate);
220
+ buffer.copyToChannel(data, 0);
221
+ const source = this.context.createBufferSource();
222
+ source.buffer = buffer;
223
+ source.connect(this.context.destination);
224
+ source.onended = () => {
225
+ this.sources.delete(source);
226
+ if (this.sources.size === 0) {
227
+ this.started = false;
228
+ setAudioState("Audio idle", false);
229
+ }
230
+ };
231
+ const startAt = Math.max(this.context.currentTime + 0.015, this.nextTime);
232
+ const scheduledAt = this.context.currentTime;
233
+ source.start(startAt);
234
+ this.nextTime = startAt + buffer.duration;
235
+ this.sources.add(source);
236
+ if (!this.started) {
237
+ this.started = true;
238
+ setAudioState("Playing", true);
239
+ }
240
+ return {
241
+ startDelayMs: Math.max(0, startAt - scheduledAt) * 1000,
242
+ durationMs: buffer.duration * 1000,
243
+ endDelayMs: Math.max(0, this.nextTime - scheduledAt) * 1000,
244
+ };
245
+ }
246
+
247
+ stop() {
248
+ for (const source of this.sources) {
249
+ try {
250
+ source.stop();
251
+ } catch {
252
+ // Source may already have ended.
253
+ }
254
+ }
255
+ this.sources.clear();
256
+ if (this.context) {
257
+ this.nextTime = this.context.currentTime;
258
+ }
259
+ this.started = false;
260
+ setAudioState("Audio idle", false);
261
+ }
262
+
263
+ async close() {
264
+ this.stop();
265
+ if (this.context && this.context.state !== "closed") {
266
+ await this.context.close().catch(() => {});
267
+ }
268
+ this.context = null;
269
+ this.nextTime = 0;
270
+ this.started = false;
271
+ }
272
+ }
273
+
274
+ function setTile(key, text, mode = "idle") {
275
+ elements.tiles[key].dataset.state = mode;
276
+ elements.states[key].textContent = text;
277
+ }
278
+
279
+ function setAudioState(text, active) {
280
+ elements.audioBadge.textContent = text;
281
+ elements.audioBadge.classList.toggle("active", active);
282
+ }
283
+
284
+ function setMicState(text, active) {
285
+ elements.micBadge.textContent = text;
286
+ elements.micBadge.classList.toggle("active", active);
287
+ }
288
+
289
+ function formatGpuAdapter(adapter) {
290
+ if (!adapter) return "unknown adapter";
291
+ return (
292
+ [adapter.vendor, adapter.architecture, adapter.device, adapter.description]
293
+ .filter(Boolean)
294
+ .join(" / ") || "unknown adapter"
295
+ );
296
+ }
297
+
298
+ function isSoftwareWebGpuAdapter(adapter) {
299
+ const text = [adapter?.vendor, adapter?.architecture, adapter?.device, adapter?.description]
300
+ .filter(Boolean)
301
+ .join(" ")
302
+ .toLowerCase();
303
+ return /\b(swiftshader|llvmpipe|software rasterizer|software adapter|warp)\b/.test(text);
304
+ }
305
+
306
+ function updateRuntimeStatus() {
307
+ const requested = elements.deviceSelect.value;
308
+ let mode = "warn";
309
+ let status = "Checking runtime";
310
+ let detail = "Adapter probe pending.";
311
+
312
+ if (state.webgpuAvailable === false) {
313
+ mode = requested === "webgpu" ? "error" : "warn";
314
+ status = requested === "webgpu" ? "WebGPU unavailable" : "WASM fallback";
315
+ detail = "This browser did not expose a WebGPU adapter.";
316
+ } else if (state.webgpuAvailable === true && state.webgpuSoftwareAdapter) {
317
+ const adapter = formatGpuAdapter(state.webgpuAdapterInfo);
318
+ mode = "warn";
319
+ if (requested === "webgpu") {
320
+ status = "Software WebGPU selected";
321
+ detail = `${adapter}. Explicit WebGPU is allowed, but benchmark automation skips software adapters by default.`;
322
+ } else if (requested === "wasm") {
323
+ status = "WASM selected";
324
+ detail = `${adapter}. Auto also uses WASM because this is a software adapter.`;
325
+ } else {
326
+ status = "Auto: WASM fallback";
327
+ detail = `${adapter}. Hardware WebGPU is not exposed.`;
328
+ }
329
+ } else if (state.webgpuAvailable === true) {
330
+ const adapter = formatGpuAdapter(state.webgpuAdapterInfo);
331
+ mode = "ready";
332
+ status = requested === "wasm" ? "WASM selected" : requested === "webgpu" ? "WebGPU selected" : "Auto: WebGPU";
333
+ detail = adapter;
334
+ }
335
+
336
+ elements.runtimeStatus.dataset.state = mode;
337
+ elements.runtimeDeviceStatus.textContent = status;
338
+ elements.runtimeDeviceDetail.textContent = detail;
339
+ }
340
+
341
+ function formatMs(value) {
342
+ if (!Number.isFinite(value)) return "-";
343
+ if (value < 1000) return `${Math.round(value)} ms`;
344
+ return `${(value / 1000).toFixed(2)} s`;
345
+ }
346
+
347
+ function formatPercent(value) {
348
+ if (!Number.isFinite(value)) return "-";
349
+ return `${Math.round(value * 100)}%`;
350
+ }
351
+
352
+ function formatQuality(result) {
353
+ if (result.llmQualityPass == null) return "-";
354
+ const total = result.llmQualityTotal ?? APP_IDENTITY_QUALITY_RULES.length;
355
+ const score = result.llmQualityScore ?? result.llmQualityHits?.length;
356
+ const suffix = Number.isFinite(score) && Number.isFinite(total) ? ` ${score}/${total}` : "";
357
+ return `${result.llmQualityPass ? "pass" : "fail"}${suffix}`;
358
+ }
359
+
360
+ function currentVadSilenceMs() {
361
+ const value = Number(elements.vadSilence.value);
362
+ if (!Number.isFinite(value)) return DEFAULT_VAD_SILENCE_MS;
363
+ return Math.min(MAX_VAD_SILENCE_MS, Math.max(MIN_VAD_SILENCE_MS, Math.round(value)));
364
+ }
365
+
366
+ function updateVadSilenceLabel() {
367
+ elements.vadSilenceValue.textContent = String(currentVadSilenceMs());
368
+ }
369
+
370
+ function updateTtsStepsLabel() {
371
+ elements.ttsStepsValue.textContent = String(Number(elements.ttsSteps.value));
372
+ }
373
+
374
+ function configureAsrWorker() {
375
+ state.asrWorker?.postMessage({
376
+ type: "configure",
377
+ partial: elements.partialToggle.checked,
378
+ silenceMs: currentVadSilenceMs(),
379
+ });
380
+ }
381
+
382
+ function logEvent(message) {
383
+ const li = document.createElement("li");
384
+ li.textContent = `${new Date().toLocaleTimeString()} ${message}`;
385
+ elements.eventLog.prepend(li);
386
+ while (elements.eventLog.children.length > 80) {
387
+ elements.eventLog.lastElementChild.remove();
388
+ }
389
+ }
390
+
391
+ function setLoadButton(mode) {
392
+ const options = {
393
+ load: { icon: "↓", label: "Load models", disabled: false },
394
+ loading: { icon: "…", label: "Loading", disabled: true },
395
+ unload: { icon: "↻", label: "Unload models", disabled: false },
396
+ };
397
+ const option = options[mode] ?? options.load;
398
+ elements.loadButton.innerHTML = `<span class="button-icon" aria-hidden="true">${option.icon}</span> ${option.label}`;
399
+ elements.loadButton.disabled = option.disabled;
400
+ }
401
+
402
+ function setSessionControls(mode) {
403
+ const loaded = mode === "loaded";
404
+ const busy = mode === "loading";
405
+ setLoadButton(busy ? "loading" : loaded ? "unload" : "load");
406
+ elements.deviceSelect.disabled = busy || loaded;
407
+ elements.llmModelSelect.disabled = busy || loaded;
408
+ elements.asrModelSelect.disabled = busy || loaded;
409
+ elements.micButton.disabled = !loaded;
410
+ elements.stopButton.disabled = !loaded;
411
+ setBenchmarkControlsDisabled(!loaded || state.suiteRunning);
412
+ if (!loaded) {
413
+ elements.micButton.innerHTML = '<span class="button-icon" aria-hidden="true">●</span> Start mic';
414
+ }
415
+ updateMicValidationStatus();
416
+ }
417
+
418
+ function setBenchmarkControlsDisabled(disabled) {
419
+ const locked = disabled || state.micSeries.active;
420
+ elements.suiteButton.disabled = locked;
421
+ elements.benchmarkButton.disabled = locked;
422
+ elements.chatBenchmarkButton.disabled = locked;
423
+ elements.ttsBenchmarkButton.disabled = locked;
424
+ elements.loopbackButton.disabled = locked;
425
+ elements.bargeInButton.disabled = locked;
426
+ elements.micBenchmarkButton.disabled = locked;
427
+ elements.micSeriesButton.disabled = locked;
428
+ }
429
+
430
+ function resetMetrics() {
431
+ elements.vadCloseLatency.textContent = "-";
432
+ elements.asrLatency.textContent = "-";
433
+ elements.firstTokenLatency.textContent = "-";
434
+ elements.firstTtsQueuedLatency.textContent = "-";
435
+ elements.ttsSynthLatency.textContent = "-";
436
+ elements.firstAudioLatency.textContent = "-";
437
+ elements.speechToAudioLatency.textContent = "-";
438
+ elements.decodeRate.textContent = "-";
439
+ }
440
+
441
+ function resetConversationUi() {
442
+ elements.partialTranscript.textContent = "Waiting for speech.";
443
+ elements.finalTranscript.textContent = "";
444
+ elements.llmOutput.textContent = "Load the models, start the microphone, and speak naturally.";
445
+ }
446
+
447
+ function updateMicValidationStatus() {
448
+ const prompt = APP_IDENTITY_PROMPT;
449
+ const target = state.micSeries.target || 3;
450
+ const activeStackKey = state.loadedStack ? stackKey(currentBenchmarkStack()) : "";
451
+ const scopedRows = activeStackKey
452
+ ? state.benchmarkResults.filter((result) => stackKey(result.stack) === activeStackKey)
453
+ : state.benchmarkResults;
454
+ const completedMicRows = scopedRows.filter((result) => result.kind === "mic" && !result.error);
455
+ const completed = Math.min(target, completedMicRows.length);
456
+ const activeMic = state.activeBenchmark?.kind === "mic";
457
+ let mode = "idle";
458
+ let status = state.modelsLoaded
459
+ ? `${completed}/${target} real-mic rows collected`
460
+ : completed > 0
461
+ ? `${completed}/${target} real-mic rows in saved results`
462
+ : `Load models to collect ${target} rows`;
463
+ let detail =
464
+ completed > 0 && !state.modelsLoaded
465
+ ? "Load a stack to collect or compare current-stack rows."
466
+ : `Use the mic series and say: "${prompt}"`;
467
+ let progress = completed / target;
468
+
469
+ if (state.modelsLoading) {
470
+ mode = "warn";
471
+ status = "Models loading";
472
+ detail = "Real-mic validation starts after the local stack is ready.";
473
+ } else if (state.micSeries.active || activeMic) {
474
+ const run = state.activeBenchmark?.micSeriesRun ?? state.micSeries.completed + 1;
475
+ const runTarget = state.activeBenchmark?.micSeriesTarget ?? target;
476
+ mode = "active";
477
+ status = `Listening for row ${Math.min(run, runTarget)}/${runTarget}`;
478
+ detail = `Say: "${prompt}" after the mic badge shows Listening.`;
479
+ progress = Math.min(1, (state.micSeries.completed + (activeMic ? 0.35 : 0)) / runTarget);
480
+ } else if (state.modelsLoaded && completed >= target) {
481
+ const summary = benchmarkSummaryForRows(scopedRows, { scope: "current" });
482
+ mode = "ready";
483
+ status = `${completed}/${target} real-mic rows complete`;
484
+ detail = `Median WER ${formatPercent(summary.micMedianWer)}, speech end to audio ${formatMs(
485
+ summary.micMedianSpeechEndToFirstAudioMs,
486
+ )}.`;
487
+ progress = 1;
488
+ } else if (state.modelsLoaded) {
489
+ mode = "warn";
490
+ detail = `Click Run 3 real-mic series and say: "${prompt}" each time.`;
491
+ } else if (completed >= target) {
492
+ mode = "ready";
493
+ progress = 1;
494
+ } else if (completed > 0) {
495
+ mode = "warn";
496
+ }
497
+
498
+ elements.micValidationCard.dataset.state = mode;
499
+ elements.micValidationStatus.textContent = status;
500
+ elements.micValidationDetail.textContent = detail;
501
+ elements.micValidationProgressBar.style.width = `${Math.round(Math.max(0, Math.min(1, progress)) * 100)}%`;
502
+ }
503
+
504
+ function resetTurnState() {
505
+ state.messages = initialMessages();
506
+ state.currentAssistant = "";
507
+ state.currentUserStartedAt = null;
508
+ state.lastSpeechEndAt = null;
509
+ state.lastVadCloseAt = null;
510
+ state.lastVadCloseDelayMs = null;
511
+ state.lastTranscriptAt = null;
512
+ state.awaitingFirstToken = false;
513
+ state.awaitingFirstAudio = false;
514
+ state.firstTtsChunkQueued = false;
515
+ state.ttsBuffer = "";
516
+ state.ttsSequence = 0;
517
+ state.loopbackRequestId += 1;
518
+ clearLoopbackFeed();
519
+ state.activeBenchmark = null;
520
+ }
521
+
522
+ function resetMicInputStats() {
523
+ state.micInputStats = {
524
+ chunks: 0,
525
+ samples: 0,
526
+ peak: 0,
527
+ sumSquares: 0,
528
+ };
529
+ }
530
+
531
+ function updateMicInputStats(buffer) {
532
+ if (!buffer?.length) return;
533
+ let chunkPeak = 0;
534
+ let chunkSumSquares = 0;
535
+ for (let i = 0; i < buffer.length; i += 1) {
536
+ const sample = buffer[i];
537
+ const abs = Math.abs(sample);
538
+ if (abs > chunkPeak) chunkPeak = abs;
539
+ chunkSumSquares += sample * sample;
540
+ }
541
+ state.micInputStats.chunks += 1;
542
+ state.micInputStats.samples += buffer.length;
543
+ state.micInputStats.peak = Math.max(state.micInputStats.peak, chunkPeak);
544
+ state.micInputStats.sumSquares += chunkSumSquares;
545
+ if (state.activeBenchmark?.kind === "mic") {
546
+ state.activeBenchmark.micInputChunks = state.micInputStats.chunks;
547
+ state.activeBenchmark.micInputPeak = state.micInputStats.peak;
548
+ state.activeBenchmark.micInputRms = Math.sqrt(
549
+ state.micInputStats.sumSquares / Math.max(1, state.micInputStats.samples),
550
+ );
551
+ }
552
+ }
553
+
554
+ function terminateWorkers() {
555
+ state.asrWorker?.terminate();
556
+ state.llmWorker?.terminate();
557
+ state.ttsWorker?.terminate();
558
+ state.asrWorker = null;
559
+ state.llmWorker = null;
560
+ state.ttsWorker = null;
561
+ }
562
+
563
+ async function resetPipelineTiles() {
564
+ setTile("vad", "Idle", "idle");
565
+ setTile("asr", "Idle", "idle");
566
+ setTile("llm", "Idle", "idle");
567
+ setTile("tts", "Idle", "idle");
568
+ await supportsWebGPU();
569
+ if (!state.webgpuAvailable || state.webgpuSoftwareAdapter) {
570
+ const label = state.webgpuSoftwareAdapter ? "WASM auto" : "WASM only";
571
+ setTile("llm", label, "warn");
572
+ setTile("tts", label, "warn");
573
+ }
574
+ }
575
+
576
+ async function supportsWebGPU() {
577
+ if (state.webgpuAvailable !== null) {
578
+ updateRuntimeStatus();
579
+ return state.webgpuAvailable;
580
+ }
581
+ if (!navigator.gpu) {
582
+ state.webgpuAvailable = false;
583
+ state.webgpuAdapterInfo = null;
584
+ state.webgpuAdapterFeatures = [];
585
+ state.webgpuSoftwareAdapter = false;
586
+ updateRuntimeStatus();
587
+ return false;
588
+ }
589
+ try {
590
+ const adapter = await navigator.gpu.requestAdapter();
591
+ state.webgpuAvailable = Boolean(adapter);
592
+ state.webgpuAdapterInfo = await readGpuAdapterInfo(adapter);
593
+ state.webgpuAdapterFeatures = adapter?.features ? [...adapter.features].sort() : [];
594
+ state.webgpuSoftwareAdapter = isSoftwareWebGpuAdapter(state.webgpuAdapterInfo);
595
+ } catch {
596
+ state.webgpuAvailable = false;
597
+ state.webgpuAdapterInfo = null;
598
+ state.webgpuAdapterFeatures = [];
599
+ state.webgpuSoftwareAdapter = false;
600
+ }
601
+ updateRuntimeStatus();
602
+ return state.webgpuAvailable;
603
+ }
604
+
605
+ async function readGpuAdapterInfo(adapter) {
606
+ let info = adapter?.info;
607
+ if (!info && typeof adapter?.requestAdapterInfo === "function") {
608
+ info = await adapter.requestAdapterInfo().catch(() => null);
609
+ }
610
+ if (!info) return null;
611
+ return {
612
+ vendor: info.vendor || "",
613
+ architecture: info.architecture || "",
614
+ device: info.device || "",
615
+ description: info.description || "",
616
+ };
617
+ }
618
+
619
+ async function resolveDevice() {
620
+ const requested = elements.deviceSelect.value;
621
+ if (requested === "auto") {
622
+ await supportsWebGPU();
623
+ return state.webgpuAvailable && !state.webgpuSoftwareAdapter ? "webgpu" : "wasm";
624
+ }
625
+ return requested;
626
+ }
627
+
628
+ function createWorker(path, kind, handler) {
629
+ const worker = new Worker(new URL(path, import.meta.url), { type: "module" });
630
+ worker.addEventListener("message", handler);
631
+ worker.addEventListener("error", (event) => {
632
+ const message = `${path} error: ${event.message || "worker failed to load"}`;
633
+ logEvent(message);
634
+ window.dispatchEvent(new CustomEvent("model-error", { detail: { kind, message } }));
635
+ });
636
+ return worker;
637
+ }
638
+
639
+ async function loadModels({ ttsWarmup = true } = {}) {
640
+ if (state.modelsLoaded || state.modelsLoading) return;
641
+ state.modelsLoading = true;
642
+ setSessionControls("loading");
643
+ const device = await resolveDevice();
644
+ if (device === "webgpu" && !(await supportsWebGPU())) {
645
+ logEvent("WebGPU was requested but is not available in this browser.");
646
+ state.modelsLoading = false;
647
+ setSessionControls("unloaded");
648
+ await resetPipelineTiles();
649
+ return;
650
+ }
651
+ const stackError = selectedStackError(device);
652
+ if (stackError) {
653
+ logEvent(stackError);
654
+ state.modelsLoading = false;
655
+ setSessionControls("unloaded");
656
+ await resetPipelineTiles();
657
+ return;
658
+ }
659
+ state.playback ??= new PlaybackQueue();
660
+ const audioUnlock = state.playback.unlock().catch((error) => {
661
+ logEvent(`Audio unlock deferred: ${error.message}`);
662
+ });
663
+
664
+ setTile("vad", "Queued", "warn");
665
+ setTile("asr", "Queued", "warn");
666
+ setTile("llm", "Queued", "warn");
667
+ setTile("tts", "Queued", "warn");
668
+ logEvent(`Loading models on ${device.toUpperCase()}.`);
669
+ state.modelLoadStartedAt = performance.now();
670
+ state.loadedStack = {
671
+ device,
672
+ llm: elements.llmModelSelect.value,
673
+ asr: elements.asrModelSelect.value,
674
+ vad: "onnx-community/silero-vad",
675
+ tts: "onnx-community/Supertonic-TTS-ONNX",
676
+ transformers: "@huggingface/transformers@4.2.0",
677
+ modelLoadMs: null,
678
+ };
679
+
680
+ void audioUnlock;
681
+
682
+ try {
683
+ setTile("vad", "Loading", "warn");
684
+ setTile("asr", "Loading", "warn");
685
+ logEvent("Loading local VAD and STT models.");
686
+ state.asrWorker = createWorker("./workers/asr-worker.js", "asr", onAsrMessage);
687
+ const asrReady = waitForReady("asr");
688
+ state.asrWorker.postMessage({
689
+ type: "load",
690
+ model: elements.asrModelSelect.value,
691
+ device,
692
+ partial: elements.partialToggle.checked,
693
+ silenceMs: currentVadSilenceMs(),
694
+ });
695
+ await asrReady;
696
+
697
+ setTile("llm", "Loading", "warn");
698
+ logEvent("Loading local LLM.");
699
+ state.llmWorker = createWorker("./workers/llm-worker.js", "llm", onLlmMessage);
700
+ const llmReady = waitForReady("llm");
701
+ state.llmWorker.postMessage({
702
+ type: "load",
703
+ model: elements.llmModelSelect.value,
704
+ device,
705
+ });
706
+ await llmReady;
707
+
708
+ setTile("tts", "Loading", "warn");
709
+ logEvent("Loading local Supertonic TTS.");
710
+ state.ttsWorker = createWorker("./workers/tts-worker.js", "tts", onTtsMessage);
711
+ const ttsReady = waitForReady("tts");
712
+ state.ttsWorker.postMessage({
713
+ type: "load",
714
+ model: "onnx-community/Supertonic-TTS-ONNX",
715
+ device,
716
+ voice: elements.voiceSelect.value,
717
+ warmup: ttsWarmup,
718
+ });
719
+ await ttsReady;
720
+
721
+ const loadMs = performance.now() - state.modelLoadStartedAt;
722
+ state.loadedStack.modelLoadMs = loadMs;
723
+ state.modelsLoading = false;
724
+ state.modelsLoaded = true;
725
+ setSessionControls("loaded");
726
+ logEvent(`All local inference models are ready in ${formatMs(loadMs)}.`);
727
+ } catch (error) {
728
+ logEvent(`Model loading failed: ${error.message}`);
729
+ await unloadModels({ quiet: true });
730
+ }
731
+ }
732
+
733
+ async function unloadModels({ quiet = false } = {}) {
734
+ if (state.micActive) stopMic();
735
+ cancelMicSeries();
736
+ interruptForBargeIn();
737
+ cancelActiveBenchmark();
738
+ rejectTtsVoiceRequests(new Error("TTS worker unloaded."));
739
+ terminateWorkers();
740
+ state.modelsLoaded = false;
741
+ state.modelsLoading = false;
742
+ state.modelLoadStartedAt = null;
743
+ state.loadedStack = null;
744
+ state.suiteRunning = false;
745
+ resetTurnState();
746
+ resetMetrics();
747
+ resetConversationUi();
748
+ setMicState("Mic off", false);
749
+ setAudioState("Audio idle", false);
750
+ await state.playback?.close();
751
+ state.playback = null;
752
+ setSessionControls("unloaded");
753
+ await resetPipelineTiles();
754
+ if (!quiet) logEvent("Unloaded local inference models. Benchmark rows are preserved.");
755
+ }
756
+
757
+ function waitForReady(kind) {
758
+ return new Promise((resolve, reject) => {
759
+ const timeout = window.setTimeout(() => reject(new Error(`${kind} load timeout`)), 180000);
760
+ const handler = (event) => {
761
+ if (event.detail.kind !== kind) return;
762
+ window.clearTimeout(timeout);
763
+ window.removeEventListener("model-ready", handler);
764
+ window.removeEventListener("model-error", errorHandler);
765
+ resolve();
766
+ };
767
+ const errorHandler = (event) => {
768
+ if (event.detail.kind !== kind) return;
769
+ window.clearTimeout(timeout);
770
+ window.removeEventListener("model-ready", handler);
771
+ window.removeEventListener("model-error", errorHandler);
772
+ reject(new Error(event.detail.message));
773
+ };
774
+ window.addEventListener("model-ready", handler);
775
+ window.addEventListener("model-error", errorHandler);
776
+ });
777
+ }
778
+
779
+ async function startMic() {
780
+ if (!state.modelsLoaded || state.micActive) return;
781
+ state.playback?.unlock().catch(() => {});
782
+ state.mediaStream = await getUserMediaWithTimeout({
783
+ audio: {
784
+ channelCount: 1,
785
+ echoCancellation: true,
786
+ noiseSuppression: true,
787
+ autoGainControl: true,
788
+ },
789
+ });
790
+ state.audioContext = new AudioContext();
791
+ state.inputSampleRate = state.audioContext.sampleRate;
792
+ if (state.activeBenchmark) {
793
+ state.activeBenchmark.inputSampleRate = state.audioContext.sampleRate;
794
+ if (state.activeBenchmark.stack?.environment) {
795
+ state.activeBenchmark.stack.environment.inputSampleRate = state.audioContext.sampleRate;
796
+ }
797
+ }
798
+ const workletUrl = URL.createObjectURL(
799
+ new Blob([audioWorkletSource()], { type: "text/javascript" }),
800
+ );
801
+ await state.audioContext.audioWorklet.addModule(workletUrl);
802
+ URL.revokeObjectURL(workletUrl);
803
+ state.micSource = state.audioContext.createMediaStreamSource(state.mediaStream);
804
+ state.workletNode = new AudioWorkletNode(state.audioContext, "pcm-capture");
805
+ state.micMonitorGain = state.audioContext.createGain();
806
+ state.micMonitorGain.gain.value = 0;
807
+ state.workletNode.port.onmessage = (event) => {
808
+ if (!state.asrWorker || !state.micActive) return;
809
+ const buffer = event.data;
810
+ updateMicInputStats(buffer);
811
+ state.asrWorker.postMessage(
812
+ {
813
+ type: "audio",
814
+ buffer,
815
+ sampleRate: state.audioContext.sampleRate,
816
+ },
817
+ [buffer.buffer],
818
+ );
819
+ };
820
+ state.micSource.connect(state.workletNode);
821
+ state.workletNode.connect(state.micMonitorGain);
822
+ state.micMonitorGain.connect(state.audioContext.destination);
823
+ state.micActive = true;
824
+ elements.micButton.innerHTML = '<span class="button-icon" aria-hidden="true">●</span> Stop mic';
825
+ setMicState("Listening", true);
826
+ setTile("vad", "Listening", "active");
827
+ logEvent("Microphone capture started.");
828
+ }
829
+
830
+ async function getUserMediaWithTimeout(constraints, timeoutMs = MIC_START_TIMEOUT_MS) {
831
+ let timeoutId = 0;
832
+ let timedOut = false;
833
+ const mediaPromise = navigator.mediaDevices.getUserMedia(constraints).then((stream) => {
834
+ if (timedOut) {
835
+ for (const track of stream.getTracks()) track.stop();
836
+ return null;
837
+ }
838
+ return stream;
839
+ });
840
+ mediaPromise.catch(() => {});
841
+ const timeoutPromise = new Promise((resolve) => {
842
+ timeoutId = window.setTimeout(() => {
843
+ timedOut = true;
844
+ resolve(null);
845
+ }, timeoutMs);
846
+ });
847
+ const stream = await Promise.race([mediaPromise, timeoutPromise]);
848
+ window.clearTimeout(timeoutId);
849
+ if (!stream) throw new Error(`Microphone start timed out after ${(timeoutMs / 1000).toFixed(0)} seconds.`);
850
+ return stream;
851
+ }
852
+
853
+ function stopMic() {
854
+ if (!state.micActive) return;
855
+ state.micActive = false;
856
+ state.workletNode?.disconnect();
857
+ state.micMonitorGain?.disconnect();
858
+ state.micSource?.disconnect();
859
+ for (const track of state.mediaStream?.getTracks() ?? []) {
860
+ track.stop();
861
+ }
862
+ state.audioContext?.close();
863
+ state.audioContext = null;
864
+ state.inputSampleRate = null;
865
+ state.mediaStream = null;
866
+ state.workletNode = null;
867
+ state.micSource = null;
868
+ state.micMonitorGain = null;
869
+ elements.micButton.innerHTML = '<span class="button-icon" aria-hidden="true">●</span> Start mic';
870
+ setMicState("Mic off", false);
871
+ setTile("vad", "Ready", "ready");
872
+ logEvent("Microphone capture stopped.");
873
+ }
874
+
875
+ function audioWorkletSource() {
876
+ return `
877
+ class PcmCaptureProcessor extends AudioWorkletProcessor {
878
+ constructor() {
879
+ super();
880
+ this.size = 512;
881
+ this.buffer = new Float32Array(this.size);
882
+ this.offset = 0;
883
+ }
884
+
885
+ process(inputs) {
886
+ const channel = inputs[0] && inputs[0][0];
887
+ if (!channel) return true;
888
+ for (let i = 0; i < channel.length; i += 1) {
889
+ this.buffer[this.offset++] = channel[i];
890
+ if (this.offset === this.size) {
891
+ const out = this.buffer;
892
+ this.port.postMessage(out, [out.buffer]);
893
+ this.buffer = new Float32Array(this.size);
894
+ this.offset = 0;
895
+ }
896
+ }
897
+ return true;
898
+ }
899
+ }
900
+ registerProcessor("pcm-capture", PcmCaptureProcessor);
901
+ `;
902
+ }
903
+
904
+ function shouldAcceptAsrSpeechEvent() {
905
+ return state.micActive || state.activeBenchmark?.kind === "loopback";
906
+ }
907
+
908
+ function shouldAcceptAsrTranscript() {
909
+ if (state.micActive || state.activeBenchmark?.kind === "loopback") return true;
910
+ if (state.activeBenchmark?.kind === "mic" && !state.activeBenchmark.transcript) return true;
911
+ return false;
912
+ }
913
+
914
+ function onAsrMessage(event) {
915
+ const message = event.data;
916
+ if (message.type === "status") {
917
+ handleStatus("asr", message);
918
+ if (message.scope === "vad") handleStatus("vad", message);
919
+ return;
920
+ }
921
+ if (message.type === "ready") {
922
+ setTile("vad", "Ready", "ready");
923
+ setTile("asr", "Ready", "ready");
924
+ window.dispatchEvent(new CustomEvent("model-ready", { detail: { kind: "asr" } }));
925
+ return;
926
+ }
927
+ if (message.type === "error") {
928
+ setTile("asr", "Error", "error");
929
+ window.dispatchEvent(
930
+ new CustomEvent("model-error", { detail: { kind: "asr", message: message.message } }),
931
+ );
932
+ return;
933
+ }
934
+ if (message.type === "speechstart") {
935
+ if (!shouldAcceptAsrSpeechEvent()) return;
936
+ handleSpeechStart();
937
+ return;
938
+ }
939
+ if (message.type === "speechend") {
940
+ if (!shouldAcceptAsrSpeechEvent()) return;
941
+ const now = performance.now();
942
+ const closeDelay = Number.isFinite(message.trailingSilenceMs) ? message.trailingSilenceMs : 0;
943
+ state.lastVadCloseAt = now;
944
+ state.lastVadCloseDelayMs = closeDelay;
945
+ state.lastSpeechEndAt = now - closeDelay;
946
+ elements.vadCloseLatency.textContent = formatMs(closeDelay);
947
+ if (state.activeBenchmark) state.activeBenchmark.vadCloseDelayMs = closeDelay;
948
+ elements.partialTranscript.textContent = "Transcribing...";
949
+ setTile("vad", "Transcribing", "warn");
950
+ setTile("asr", "Transcribing", "active");
951
+ return;
952
+ }
953
+ if (message.type === "partial") {
954
+ if (!shouldAcceptAsrSpeechEvent()) return;
955
+ if (message.text?.trim()) {
956
+ elements.partialTranscript.textContent = message.text.trim();
957
+ }
958
+ return;
959
+ }
960
+ if (message.type === "transcript") {
961
+ if (!shouldAcceptAsrTranscript()) return;
962
+ const text = message.text.trim();
963
+ const shouldStopMicAfterTranscript =
964
+ state.activeBenchmark?.kind === "mic" && state.activeBenchmark.stopMicAfterTranscript === true;
965
+ state.lastTranscriptAt = performance.now();
966
+ elements.asrLatency.textContent = formatMs(state.lastTranscriptAt - state.lastSpeechEndAt);
967
+ if (state.activeBenchmark) {
968
+ state.activeBenchmark.transcript = text;
969
+ state.activeBenchmark.asrMs = state.lastTranscriptAt - state.lastSpeechEndAt;
970
+ updateTranscriptQuality(state.activeBenchmark, text);
971
+ }
972
+ setTile("vad", state.micActive ? "Listening" : "Ready", state.micActive ? "active" : "ready");
973
+ setTile("asr", "Ready", "ready");
974
+ if (!text) {
975
+ elements.partialTranscript.textContent = "No speech recognized.";
976
+ failActiveBenchmark("No speech recognized.");
977
+ return;
978
+ }
979
+ if (
980
+ state.activeBenchmark?.kind === "mic" &&
981
+ state.activeBenchmark.requireExactTranscript === true &&
982
+ state.activeBenchmark.sttWer !== 0
983
+ ) {
984
+ elements.partialTranscript.textContent = `Heard: "${text}"`;
985
+ logEvent(`Ignoring non-matching scripted transcript: "${text}"`);
986
+ return;
987
+ }
988
+ elements.partialTranscript.textContent = "";
989
+ elements.finalTranscript.textContent = text;
990
+ logEvent(`Transcript: "${text}"`);
991
+ generateResponse(text);
992
+ if (shouldStopMicAfterTranscript) stopMic();
993
+ }
994
+ }
995
+
996
+ function onLlmMessage(event) {
997
+ const message = event.data;
998
+ if (message.type === "status") {
999
+ handleStatus("llm", message);
1000
+ return;
1001
+ }
1002
+ if (message.type === "ready") {
1003
+ setTile("llm", "Ready", "ready");
1004
+ window.dispatchEvent(new CustomEvent("model-ready", { detail: { kind: "llm" } }));
1005
+ return;
1006
+ }
1007
+ if (message.type === "error") {
1008
+ setTile("llm", "Error", "error");
1009
+ window.dispatchEvent(
1010
+ new CustomEvent("model-error", { detail: { kind: "llm", message: message.message } }),
1011
+ );
1012
+ return;
1013
+ }
1014
+ if (message.type === "start") {
1015
+ if (message.turnId !== state.activeTurnId) return;
1016
+ state.currentAssistant = "";
1017
+ state.ttsBuffer = "";
1018
+ state.awaitingFirstToken = true;
1019
+ state.awaitingFirstAudio = true;
1020
+ state.firstTtsChunkQueued = false;
1021
+ elements.llmOutput.textContent = "";
1022
+ elements.firstTokenLatency.textContent = "-";
1023
+ elements.firstTtsQueuedLatency.textContent = "-";
1024
+ elements.ttsSynthLatency.textContent = "-";
1025
+ elements.firstAudioLatency.textContent = "-";
1026
+ elements.speechToAudioLatency.textContent = "-";
1027
+ elements.decodeRate.textContent = "-";
1028
+ setTile("llm", "Generating", "active");
1029
+ return;
1030
+ }
1031
+ if (message.type === "token") {
1032
+ if (message.turnId !== state.activeTurnId) return;
1033
+ if (state.awaitingFirstToken) {
1034
+ state.awaitingFirstToken = false;
1035
+ const latency = performance.now() - state.lastTranscriptAt;
1036
+ elements.firstTokenLatency.textContent = formatMs(latency);
1037
+ if (state.activeBenchmark) state.activeBenchmark.firstTokenMs = latency;
1038
+ }
1039
+ state.currentAssistant += message.text;
1040
+ elements.llmOutput.textContent = state.currentAssistant;
1041
+ if (message.tps) elements.decodeRate.textContent = `${message.tps.toFixed(1)} tok/s`;
1042
+ bufferTts(message.text);
1043
+ return;
1044
+ }
1045
+ if (message.type === "complete") {
1046
+ if (message.turnId !== state.activeTurnId) return;
1047
+ flushTts(true);
1048
+ setTile("llm", "Ready", "ready");
1049
+ const cleanAssistant = cleanAssistantResponse(state.currentAssistant);
1050
+ if (cleanAssistant) {
1051
+ state.currentAssistant = cleanAssistant;
1052
+ elements.llmOutput.textContent = cleanAssistant;
1053
+ state.messages.push({ role: "assistant", content: cleanAssistant });
1054
+ trimConversation();
1055
+ }
1056
+ if (state.activeBenchmark) {
1057
+ state.activeBenchmark.output = cleanAssistant;
1058
+ updateOutputQuality(state.activeBenchmark, cleanAssistant);
1059
+ state.activeBenchmark.decodeRate = elements.decodeRate.textContent;
1060
+ state.activeBenchmark.llmDoneAt = performance.now();
1061
+ state.activeBenchmark.llmCompleteMs = state.activeBenchmark.llmDoneAt - state.lastTranscriptAt;
1062
+ window.setTimeout(() => finalizeBenchmarkIfIdle(), 1200);
1063
+ }
1064
+ logEvent("LLM response complete.");
1065
+ }
1066
+ }
1067
+
1068
+ function onTtsMessage(event) {
1069
+ const message = event.data;
1070
+ if (message.type === "status") {
1071
+ handleStatus("tts", message);
1072
+ return;
1073
+ }
1074
+ if (message.type === "ready") {
1075
+ setTile("tts", "Ready", "ready");
1076
+ window.dispatchEvent(new CustomEvent("model-ready", { detail: { kind: "tts" } }));
1077
+ return;
1078
+ }
1079
+ if (message.type === "error") {
1080
+ setTile("tts", "Error", "error");
1081
+ rejectTtsVoiceRequests(new Error(message.message));
1082
+ window.dispatchEvent(
1083
+ new CustomEvent("model-error", { detail: { kind: "tts", message: message.message } }),
1084
+ );
1085
+ return;
1086
+ }
1087
+ if (message.type === "voice-ready") {
1088
+ const request = state.ttsVoiceRequests.get(message.requestId);
1089
+ if (request) {
1090
+ window.clearTimeout(request.timeout);
1091
+ state.ttsVoiceRequests.delete(message.requestId);
1092
+ request.resolve();
1093
+ }
1094
+ return;
1095
+ }
1096
+ if (message.type === "voice-error") {
1097
+ const request = state.ttsVoiceRequests.get(message.requestId);
1098
+ if (request) {
1099
+ window.clearTimeout(request.timeout);
1100
+ state.ttsVoiceRequests.delete(message.requestId);
1101
+ request.reject(new Error(message.message));
1102
+ return;
1103
+ }
1104
+ logEvent(`Voice ${message.voice} preload failed: ${message.message}`);
1105
+ return;
1106
+ }
1107
+ if (message.type === "audio") {
1108
+ if (message.turnId !== state.activeTurnId) return;
1109
+ let recordedFirstAudio = false;
1110
+ if (state.awaitingFirstAudio) {
1111
+ recordedFirstAudio = true;
1112
+ state.awaitingFirstAudio = false;
1113
+ const now = performance.now();
1114
+ const latency = now - state.lastTranscriptAt;
1115
+ const speechEndToAudioMs = Number.isFinite(state.lastSpeechEndAt)
1116
+ ? now - state.lastSpeechEndAt
1117
+ : null;
1118
+ elements.firstAudioLatency.textContent = formatMs(latency);
1119
+ elements.ttsSynthLatency.textContent = formatMs(message.synthesisMs);
1120
+ elements.speechToAudioLatency.textContent = formatMs(speechEndToAudioMs);
1121
+ if (state.activeBenchmark) {
1122
+ state.activeBenchmark.firstAudioMs = latency;
1123
+ state.activeBenchmark.speechEndToFirstAudioMs =
1124
+ state.activeBenchmark.asrMs != null ? speechEndToAudioMs : null;
1125
+ state.activeBenchmark.firstTtsSynthesisMs = message.synthesisMs ?? null;
1126
+ state.activeBenchmark.firstTtsRoundTripMs = Number.isFinite(message.enqueuedAt)
1127
+ ? now - message.enqueuedAt
1128
+ : null;
1129
+ state.activeBenchmark.firstTtsChars = message.text?.length ?? null;
1130
+ state.activeBenchmark.firstTtsText = message.text ?? "";
1131
+ }
1132
+ }
1133
+ if (state.activeBenchmark) {
1134
+ state.activeBenchmark.ttsChunkCount += 1;
1135
+ }
1136
+ state.pendingPlaybackSchedules += 1;
1137
+ state.playback
1138
+ .play(message.audio, message.sampleRate)
1139
+ .then((playback) => {
1140
+ if (message.turnId !== state.activeTurnId) return;
1141
+ if (state.activeBenchmark && playback) {
1142
+ const now = performance.now();
1143
+ const audioEndMs = now + (playback.endDelayMs ?? playback.durationMs ?? 0) - state.lastTranscriptAt;
1144
+ state.activeBenchmark.audioEndMs = Math.max(
1145
+ state.activeBenchmark.audioEndMs ?? 0,
1146
+ audioEndMs,
1147
+ );
1148
+ state.activeBenchmark.speechEndToAudioEndMs = Number.isFinite(state.lastSpeechEndAt)
1149
+ ? Math.max(
1150
+ state.activeBenchmark.speechEndToAudioEndMs ?? 0,
1151
+ now + (playback.endDelayMs ?? playback.durationMs ?? 0) - state.lastSpeechEndAt,
1152
+ )
1153
+ : null;
1154
+ }
1155
+ if (recordedFirstAudio && state.activeBenchmark) {
1156
+ state.activeBenchmark.firstTtsPlaybackDelayMs = playback?.startDelayMs ?? null;
1157
+ }
1158
+ if (state.activeBenchmark?.llmDoneAt) window.setTimeout(() => finalizeBenchmarkIfIdle(), 0);
1159
+ })
1160
+ .catch((error) => logEvent(`Playback failed: ${error.message}`))
1161
+ .finally(() => {
1162
+ state.pendingPlaybackSchedules = Math.max(0, state.pendingPlaybackSchedules - 1);
1163
+ if (state.activeBenchmark?.llmDoneAt) window.setTimeout(() => finalizeBenchmarkIfIdle(), 0);
1164
+ });
1165
+ setTile("tts", "Speaking", "active");
1166
+ return;
1167
+ }
1168
+ if (message.type === "synthetic-audio") {
1169
+ if (state.automationSynthesisRequests.has(message.requestId)) {
1170
+ const request = state.automationSynthesisRequests.get(message.requestId);
1171
+ request.audio = new Float32Array(message.audio);
1172
+ request.sampleRate = message.sampleRate;
1173
+ request.synthesisMs = message.synthesisMs ?? null;
1174
+ return;
1175
+ }
1176
+ if (message.requestId !== state.loopbackRequestId) return;
1177
+ if (state.activeBenchmark) state.activeBenchmark.loopbackTtsSynthesisMs = message.synthesisMs ?? null;
1178
+ feedLoopbackAudio(message.audio, message.sampleRate);
1179
+ return;
1180
+ }
1181
+ if (message.type === "done") {
1182
+ if (state.automationSynthesisRequests.has(message.requestId)) {
1183
+ const request = state.automationSynthesisRequests.get(message.requestId);
1184
+ state.automationSynthesisRequests.delete(message.requestId);
1185
+ setTile("tts", "Ready", "ready");
1186
+ if (!request.audio) {
1187
+ request.reject(new Error("Automation synthesis finished without audio."));
1188
+ return;
1189
+ }
1190
+ request.resolve({
1191
+ text: request.text,
1192
+ voice: request.voice,
1193
+ steps: request.steps,
1194
+ speed: request.speed,
1195
+ sampleRate: request.sampleRate,
1196
+ synthesisMs: request.synthesisMs,
1197
+ audio: Array.from(request.audio),
1198
+ });
1199
+ return;
1200
+ }
1201
+ if (message.requestId === state.loopbackRequestId) {
1202
+ setTile("tts", "Ready", "ready");
1203
+ if (state.activeBenchmark?.llmDoneAt) window.setTimeout(() => finalizeBenchmarkIfIdle(), 0);
1204
+ return;
1205
+ }
1206
+ if (message.turnId !== state.activeTurnId) return;
1207
+ setTile("tts", "Ready", "ready");
1208
+ if (state.activeBenchmark?.llmDoneAt) window.setTimeout(() => finalizeBenchmarkIfIdle(), 0);
1209
+ }
1210
+ }
1211
+
1212
+ function handleStatus(kind, message) {
1213
+ const label = message.message ?? "Working";
1214
+ const stateMode = message.mode ?? (label.toLowerCase().includes("load") ? "warn" : "active");
1215
+ setTile(kind, label, stateMode);
1216
+ if (message.detail) logEvent(message.detail);
1217
+ }
1218
+
1219
+ function currentBenchmarkStack() {
1220
+ return {
1221
+ ...state.loadedStack,
1222
+ voice: elements.voiceSelect.value,
1223
+ ttsSteps: Number(elements.ttsSteps.value),
1224
+ vadSilenceMs: currentVadSilenceMs(),
1225
+ partialAsr: elements.partialToggle.checked,
1226
+ ttsChunking: { ...state.ttsChunking },
1227
+ environment: runtimeEnvironment(),
1228
+ };
1229
+ }
1230
+
1231
+ function selectedStackError(device) {
1232
+ const llm = elements.llmModelSelect.value;
1233
+ if (device !== "webgpu" && WEBGPU_ONLY_LLMS.has(llm)) {
1234
+ const reason = WEBGPU_ONLY_REASONS.get(llm) ?? "it is too large for the WASM fallback path.";
1235
+ return `${shortModel(llm)} is disabled on WASM fallback because ${reason} Select WebGPU on a supported browser for this candidate.`;
1236
+ }
1237
+ return "";
1238
+ }
1239
+
1240
+ function updateTranscriptQuality(benchmark, transcript) {
1241
+ if (!benchmark.referenceText) return;
1242
+ benchmark.sttWer = errorRate(words(benchmark.referenceText), words(transcript));
1243
+ benchmark.sttCer = errorRate(chars(benchmark.referenceText), chars(transcript));
1244
+ }
1245
+
1246
+ function updateOutputQuality(benchmark, output) {
1247
+ if (!isAppIdentityQuestion(benchmark.prompt) && !isAppIdentityQuestion(benchmark.transcript)) return;
1248
+ const hits = APP_IDENTITY_QUALITY_RULES
1249
+ .filter((rule) => rule.pattern.test(output))
1250
+ .map((rule) => rule.label);
1251
+ const missing = APP_IDENTITY_QUALITY_RULES
1252
+ .filter((rule) => !rule.pattern.test(output))
1253
+ .map((rule) => rule.label);
1254
+ benchmark.llmQualityHits = hits;
1255
+ benchmark.llmQualityMissing = missing;
1256
+ benchmark.llmQualityScore = hits.length;
1257
+ benchmark.llmQualityTotal = APP_IDENTITY_QUALITY_RULES.length;
1258
+ benchmark.llmQualityPass = missing.length === 0;
1259
+ }
1260
+
1261
+ function isAppIdentityQuestion(text) {
1262
+ const normalized = normalizeForQuality(text);
1263
+ return /^(what's|what is|what app is|what demo is|what application is) this$/.test(normalized);
1264
+ }
1265
+
1266
+ function words(text) {
1267
+ return normalizeForQuality(text).split(" ").filter(Boolean);
1268
+ }
1269
+
1270
+ function chars(text) {
1271
+ return normalizeForQuality(text).replace(/\s+/g, "").split("");
1272
+ }
1273
+
1274
+ function normalizeForQuality(text) {
1275
+ return String(text ?? "")
1276
+ .toLowerCase()
1277
+ .replace(/[^a-z0-9' ]+/g, " ")
1278
+ .replace(/\s+/g, " ")
1279
+ .trim();
1280
+ }
1281
+
1282
+ function errorRate(reference, hypothesis) {
1283
+ if (reference.length === 0) return hypothesis.length === 0 ? 0 : 1;
1284
+ return editDistance(reference, hypothesis) / reference.length;
1285
+ }
1286
+
1287
+ function editDistance(reference, hypothesis) {
1288
+ let previous = Array.from({ length: hypothesis.length + 1 }, (_, index) => index);
1289
+ for (let i = 1; i <= reference.length; i += 1) {
1290
+ const current = [i];
1291
+ for (let j = 1; j <= hypothesis.length; j += 1) {
1292
+ const substitutionCost = reference[i - 1] === hypothesis[j - 1] ? 0 : 1;
1293
+ current[j] = Math.min(
1294
+ previous[j] + 1,
1295
+ current[j - 1] + 1,
1296
+ previous[j - 1] + substitutionCost,
1297
+ );
1298
+ }
1299
+ previous = current;
1300
+ }
1301
+ return previous[hypothesis.length];
1302
+ }
1303
+
1304
+ function runtimeEnvironment() {
1305
+ return {
1306
+ page: location.href,
1307
+ secureContext: window.isSecureContext,
1308
+ userAgent: navigator.userAgent,
1309
+ platform: navigator.platform || "",
1310
+ language: navigator.language || "",
1311
+ hardwareConcurrency: navigator.hardwareConcurrency ?? null,
1312
+ deviceMemoryGb: navigator.deviceMemory ?? null,
1313
+ webgpuAvailable: state.webgpuAvailable,
1314
+ webgpuAdapter: state.webgpuAdapterInfo,
1315
+ webgpuFeatures: state.webgpuAdapterFeatures,
1316
+ webgpuSoftwareAdapter: state.webgpuSoftwareAdapter,
1317
+ inputSampleRate: state.inputSampleRate,
1318
+ };
1319
+ }
1320
+
1321
+ function startBenchmark(kind, prompt, { referenceText = "", timeoutMs = 120000 } = {}) {
1322
+ clearBenchmarkTimeout();
1323
+ if (kind === "mic") resetMicInputStats();
1324
+ state.activeBenchmark = {
1325
+ id: state.benchmarkResults.length + 1,
1326
+ kind,
1327
+ startedAt: new Date().toISOString(),
1328
+ stack: currentBenchmarkStack(),
1329
+ prompt,
1330
+ referenceText: referenceText || (kind === "loopback" ? prompt : ""),
1331
+ micSeriesRun: kind === "mic" && state.micSeries.active ? state.micSeries.completed + 1 : null,
1332
+ micSeriesTarget: kind === "mic" && state.micSeries.active ? state.micSeries.target : null,
1333
+ transcript: "",
1334
+ output: "",
1335
+ asrMs: null,
1336
+ sttWer: null,
1337
+ sttCer: null,
1338
+ vadCloseDelayMs: null,
1339
+ firstTokenMs: null,
1340
+ firstTtsQueuedMs: null,
1341
+ firstTtsSynthesisMs: null,
1342
+ firstTtsRoundTripMs: null,
1343
+ firstTtsPlaybackDelayMs: null,
1344
+ firstTtsChars: null,
1345
+ firstTtsText: "",
1346
+ firstTtsBoundaryKind: "",
1347
+ firstTtsWordBoundarySafe: null,
1348
+ ttsChunkCount: 0,
1349
+ audioEndMs: null,
1350
+ speechEndToAudioEndMs: null,
1351
+ loopbackTtsSynthesisMs: null,
1352
+ inputSampleRate: state.inputSampleRate,
1353
+ micInputChunks: null,
1354
+ micInputPeak: null,
1355
+ micInputRms: null,
1356
+ loopbackSampleRate: null,
1357
+ bargeInMs: null,
1358
+ bargeInPass: null,
1359
+ firstAudioMs: null,
1360
+ speechEndToFirstAudioMs: null,
1361
+ decodeRate: "-",
1362
+ llmCompleteMs: null,
1363
+ llmQualityPass: null,
1364
+ llmQualityScore: null,
1365
+ llmQualityTotal: APP_IDENTITY_QUALITY_RULES.length,
1366
+ llmQualityHits: [],
1367
+ llmQualityMissing: [],
1368
+ error: "",
1369
+ llmDoneAt: null,
1370
+ };
1371
+ state.benchmarkTimeout = window.setTimeout(() => {
1372
+ failActiveBenchmark(`${kind} benchmark timed out after ${(timeoutMs / 1000).toFixed(0)} seconds.`);
1373
+ }, timeoutMs);
1374
+ updateMicValidationStatus();
1375
+ }
1376
+
1377
+ function cancelActiveBenchmark(message) {
1378
+ if (!state.activeBenchmark) return;
1379
+ clearBenchmarkTimeout();
1380
+ state.activeBenchmark = null;
1381
+ updateMicValidationStatus();
1382
+ if (message) logEvent(message);
1383
+ }
1384
+
1385
+ function clearBenchmarkTimeout() {
1386
+ if (!state.benchmarkTimeout) return;
1387
+ window.clearTimeout(state.benchmarkTimeout);
1388
+ state.benchmarkTimeout = null;
1389
+ }
1390
+
1391
+ function updateMicSeriesButton() {
1392
+ if (!state.micSeries.active) {
1393
+ elements.micSeriesButton.textContent = "Run 3 real-mic series";
1394
+ updateMicValidationStatus();
1395
+ return;
1396
+ }
1397
+ const currentRun = Math.min(state.micSeries.completed + 1, state.micSeries.target);
1398
+ elements.micSeriesButton.textContent = `Real mic ${currentRun}/${state.micSeries.target}`;
1399
+ updateMicValidationStatus();
1400
+ }
1401
+
1402
+ function cancelMicSeries(message = "") {
1403
+ const wasActive = state.micSeries.active || Boolean(state.micSeries.timer);
1404
+ if (state.micSeries.timer) {
1405
+ window.clearTimeout(state.micSeries.timer);
1406
+ state.micSeries.timer = 0;
1407
+ }
1408
+ state.micSeries.active = false;
1409
+ state.micSeries.completed = 0;
1410
+ updateMicSeriesButton();
1411
+ if (state.modelsLoaded && !state.suiteRunning) setBenchmarkControlsDisabled(false);
1412
+ updateMicValidationStatus();
1413
+ if (message && wasActive) logEvent(message);
1414
+ }
1415
+
1416
+ function failActiveBenchmark(message) {
1417
+ if (!state.activeBenchmark) return;
1418
+ clearBenchmarkTimeout();
1419
+ clearLoopbackFeed();
1420
+ state.activeBenchmark.error = message;
1421
+ state.activeBenchmark.output ||= message;
1422
+ state.activeBenchmark.decodeRate = elements.decodeRate.textContent;
1423
+ state.activeBenchmark.firstAudioMs ??= parseMetricMs(elements.firstAudioLatency.textContent);
1424
+ state.activeBenchmark.speechEndToFirstAudioMs ??= parseMetricMs(elements.speechToAudioLatency.textContent);
1425
+ if (state.activeBenchmark.kind === "mic") {
1426
+ state.activeBenchmark.micInputChunks = state.micInputStats.chunks;
1427
+ state.activeBenchmark.micInputPeak = state.micInputStats.peak;
1428
+ state.activeBenchmark.micInputRms =
1429
+ state.micInputStats.samples > 0
1430
+ ? Math.sqrt(state.micInputStats.sumSquares / state.micInputStats.samples)
1431
+ : null;
1432
+ }
1433
+ const result = { ...state.activeBenchmark };
1434
+ state.benchmarkResults.unshift(result);
1435
+ state.activeBenchmark = null;
1436
+ renderBenchmarkResults();
1437
+ logEvent(message);
1438
+ handleBenchmarkRowAdded(result);
1439
+ updateMicValidationStatus();
1440
+ }
1441
+
1442
+ function completeActiveBenchmark() {
1443
+ if (!state.activeBenchmark) return;
1444
+ clearBenchmarkTimeout();
1445
+ const result = { ...state.activeBenchmark };
1446
+ state.benchmarkResults.unshift(result);
1447
+ state.activeBenchmark = null;
1448
+ renderBenchmarkResults();
1449
+ handleBenchmarkRowAdded(result);
1450
+ updateMicValidationStatus();
1451
+ }
1452
+
1453
+ function handleBenchmarkRowAdded(result) {
1454
+ if (result.kind !== "mic" || !state.micSeries.active) return;
1455
+ if (result.error) {
1456
+ cancelMicSeries("Real-mic series stopped after a failed run.");
1457
+ return;
1458
+ }
1459
+ state.micSeries.completed += 1;
1460
+ updateMicSeriesButton();
1461
+ if (state.micSeries.completed >= state.micSeries.target) {
1462
+ cancelMicSeries("Real-mic series complete.");
1463
+ return;
1464
+ }
1465
+ const nextRun = state.micSeries.completed + 1;
1466
+ const prompt = APP_IDENTITY_PROMPT;
1467
+ elements.partialTranscript.textContent = `Say: "${prompt}" (${nextRun}/${state.micSeries.target})`;
1468
+ logEvent(`Ready for real-mic run ${nextRun}/${state.micSeries.target}.`);
1469
+ state.micSeries.timer = window.setTimeout(() => {
1470
+ state.micSeries.timer = 0;
1471
+ if (!state.micSeries.active) return;
1472
+ runMicBenchmark({ series: true })
1473
+ .then((started) => {
1474
+ if (!started) cancelMicSeries("Real-mic series stopped before the next run.");
1475
+ })
1476
+ .catch((error) => {
1477
+ cancelMicSeries(`Real-mic series stopped: ${error.message}`);
1478
+ });
1479
+ }, 900);
1480
+ }
1481
+
1482
+ function finalizeBenchmarkIfIdle() {
1483
+ if (!state.activeBenchmark) return;
1484
+ if (!Number.isFinite(state.activeBenchmark.llmDoneAt)) return;
1485
+ const pendingAudio = state.activeBenchmark.firstAudioMs == null && elements.firstAudioLatency.textContent === "-";
1486
+ const ttsBusy = elements.states.tts.textContent === "Synthesizing" || elements.states.tts.textContent === "Speaking";
1487
+ if (
1488
+ state.pendingPlaybackSchedules > 0 ||
1489
+ (pendingAudio && (ttsBusy || performance.now() - state.activeBenchmark.llmDoneAt < 8000))
1490
+ ) {
1491
+ window.setTimeout(() => finalizeBenchmarkIfIdle(), state.pendingPlaybackSchedules > 0 ? 100 : 1000);
1492
+ return;
1493
+ }
1494
+ clearBenchmarkTimeout();
1495
+ state.activeBenchmark.firstAudioMs = parseMetricMs(elements.firstAudioLatency.textContent);
1496
+ state.activeBenchmark.decodeRate = elements.decodeRate.textContent;
1497
+ completeActiveBenchmark();
1498
+ }
1499
+
1500
+ function parseMetricMs(text) {
1501
+ if (!text || text === "-") return null;
1502
+ if (text.endsWith(" ms")) return Number(text.replace(" ms", ""));
1503
+ if (text.endsWith(" s")) return Number(text.replace(" s", "")) * 1000;
1504
+ return null;
1505
+ }
1506
+
1507
+ function renderBenchmarkResults() {
1508
+ elements.resultsBody.replaceChildren();
1509
+ elements.copyResultsButton.disabled = state.benchmarkResults.length === 0;
1510
+ elements.clearResultsButton.disabled = state.benchmarkResults.length === 0;
1511
+ renderBenchmarkSummary();
1512
+ if (state.benchmarkResults.length === 0) {
1513
+ const row = document.createElement("tr");
1514
+ const cell = document.createElement("td");
1515
+ cell.colSpan = 15;
1516
+ cell.textContent = "No benchmark runs yet.";
1517
+ row.append(cell);
1518
+ elements.resultsBody.append(row);
1519
+ return;
1520
+ }
1521
+ for (const result of state.benchmarkResults) {
1522
+ const row = document.createElement("tr");
1523
+ const stack = result.stack ?? {};
1524
+ const runLabel = result.micSeriesTarget
1525
+ ? `${result.kind} ${result.micSeriesRun}/${result.micSeriesTarget}`
1526
+ : result.kind;
1527
+ const values = [
1528
+ `${result.id}. ${runLabel}`,
1529
+ `${shortModel(stack.llm)} / ${stack.device?.toUpperCase() ?? "-"} / ${stack.voice ?? "-"} / ${stack.ttsSteps ?? "-"} steps`,
1530
+ formatMs(result.asrMs),
1531
+ formatPercent(result.sttWer),
1532
+ formatMs(result.vadCloseDelayMs),
1533
+ formatMs(result.firstTokenMs),
1534
+ formatMs(result.firstTtsQueuedMs),
1535
+ formatMs(result.firstTtsSynthesisMs),
1536
+ formatMs(result.firstAudioMs),
1537
+ formatMs(result.speechEndToFirstAudioMs),
1538
+ formatMs(result.audioEndMs),
1539
+ result.decodeRate ?? "-",
1540
+ formatQuality(result),
1541
+ result.transcript || result.prompt || "-",
1542
+ result.error || result.output || "-",
1543
+ ];
1544
+ for (const value of values) {
1545
+ const cell = document.createElement("td");
1546
+ cell.textContent = value;
1547
+ row.append(cell);
1548
+ }
1549
+ elements.resultsBody.append(row);
1550
+ }
1551
+ }
1552
+
1553
+ function renderBenchmarkSummary() {
1554
+ const summary = displayBenchmarkSummary();
1555
+ if (summary.totalRuns === 0 && summary.allRuns === 0) {
1556
+ elements.benchmarkSummary.textContent = "No benchmark runs yet.";
1557
+ updateMicValidationStatus();
1558
+ return;
1559
+ }
1560
+ const items = [
1561
+ ["Runs", `${summary.totalRuns} ${summary.scope}`],
1562
+ ["Real mic", `${summary.micRuns}/3${summary.micMedianWer == null ? "" : `, WER ${formatPercent(summary.micMedianWer)}`}`],
1563
+ ["Mic end -> audio", formatMs(summary.micMedianSpeechEndToFirstAudioMs)],
1564
+ ["Loopback WER", summary.loopbackMedianWer == null ? "-" : formatPercent(summary.loopbackMedianWer)],
1565
+ ["Loopback end -> audio", formatMs(summary.loopbackMedianSpeechEndToFirstAudioMs)],
1566
+ ["Barge-in", `${summary.bargeInPasses}/${summary.bargeInRuns} pass`],
1567
+ ];
1568
+ elements.benchmarkSummary.replaceChildren(
1569
+ ...items.map(([label, value]) => {
1570
+ const item = document.createElement("div");
1571
+ item.className = "summary-item";
1572
+ const labelEl = document.createElement("span");
1573
+ labelEl.textContent = label;
1574
+ const valueEl = document.createElement("strong");
1575
+ valueEl.textContent = value;
1576
+ item.append(labelEl, valueEl);
1577
+ return item;
1578
+ }),
1579
+ );
1580
+ updateMicValidationStatus();
1581
+ }
1582
+
1583
+ function displayBenchmarkSummary() {
1584
+ const allSummary = benchmarkSummaryForRows(state.benchmarkResults, { scope: "all" });
1585
+ if (!state.loadedStack) return allSummary;
1586
+ const activeKey = stackKey(currentBenchmarkStack());
1587
+ const currentRows = state.benchmarkResults.filter((result) => stackKey(result.stack) === activeKey);
1588
+ return {
1589
+ ...benchmarkSummaryForRows(currentRows, {
1590
+ scope: "current",
1591
+ stackKey: activeKey,
1592
+ stackLabel: stackLabel(currentBenchmarkStack()),
1593
+ }),
1594
+ allRuns: allSummary.totalRuns,
1595
+ };
1596
+ }
1597
+
1598
+ function exportBenchmarkSummary() {
1599
+ const activeKey = state.loadedStack ? stackKey(currentBenchmarkStack()) : null;
1600
+ const currentRows = activeKey
1601
+ ? state.benchmarkResults.filter((result) => stackKey(result.stack) === activeKey)
1602
+ : [];
1603
+ return {
1604
+ all: benchmarkSummaryForRows(state.benchmarkResults, { scope: "all" }),
1605
+ current: activeKey
1606
+ ? benchmarkSummaryForRows(currentRows, {
1607
+ scope: "current",
1608
+ stackKey: activeKey,
1609
+ stackLabel: stackLabel(currentBenchmarkStack()),
1610
+ })
1611
+ : null,
1612
+ byStack: benchmarkSummariesByStack(),
1613
+ };
1614
+ }
1615
+
1616
+ function benchmarkSummariesByStack() {
1617
+ const grouped = new Map();
1618
+ for (const result of state.benchmarkResults) {
1619
+ const key = stackKey(result.stack);
1620
+ if (!grouped.has(key)) grouped.set(key, []);
1621
+ grouped.get(key).push(result);
1622
+ }
1623
+ return [...grouped.entries()].map(([key, rows]) =>
1624
+ benchmarkSummaryForRows(rows, {
1625
+ scope: "stack",
1626
+ stackKey: key,
1627
+ stackLabel: stackLabel(rows[0]?.stack),
1628
+ }),
1629
+ );
1630
+ }
1631
+
1632
+ function benchmarkSummaryForRows(results, { scope, stackKey: key = "", stackLabel: label = "" } = {}) {
1633
+ const completed = results.filter((result) => !result.error);
1634
+ const ttsRows = completed.filter((result) => result.kind === "tts");
1635
+ const identityRows = completed.filter((result) => result.kind === "identity");
1636
+ const chatRows = completed.filter((result) => result.kind === "chat");
1637
+ const micRows = completed.filter((result) => result.kind === "mic");
1638
+ const loopbackRows = completed.filter((result) => result.kind === "loopback");
1639
+ const bargeInRows = results.filter((result) => result.kind === "barge-in");
1640
+ return {
1641
+ scope,
1642
+ stackKey: key,
1643
+ stackLabel: label,
1644
+ totalRuns: results.length,
1645
+ completedRuns: completed.length,
1646
+ ttsRuns: ttsRows.length,
1647
+ ttsMedianFirstAudioMs: median(ttsRows.map((result) => result.firstAudioMs)),
1648
+ ttsMedianSynthesisMs: median(ttsRows.map((result) => result.firstTtsSynthesisMs)),
1649
+ ttsMedianAudioEndMs: median(ttsRows.map((result) => result.audioEndMs)),
1650
+ identityRuns: identityRows.length,
1651
+ identityMedianFirstTokenMs: median(identityRows.map((result) => result.firstTokenMs)),
1652
+ identityMedianFirstAudioMs: median(identityRows.map((result) => result.firstAudioMs)),
1653
+ identityPasses: identityRows.filter((result) => result.llmQualityPass).length,
1654
+ chatRuns: chatRows.length,
1655
+ chatMedianFirstTokenMs: median(chatRows.map((result) => result.firstTokenMs)),
1656
+ chatMedianFirstAudioMs: median(chatRows.map((result) => result.firstAudioMs)),
1657
+ micRuns: micRows.length,
1658
+ micTargetRuns: 3,
1659
+ micMedianWer: median(micRows.map((result) => result.sttWer)),
1660
+ micMedianSpeechEndToFirstAudioMs: median(micRows.map((result) => result.speechEndToFirstAudioMs)),
1661
+ micMedianSpeechEndToAudioEndMs: median(micRows.map((result) => result.speechEndToAudioEndMs)),
1662
+ loopbackRuns: loopbackRows.length,
1663
+ loopbackMedianWer: median(loopbackRows.map((result) => result.sttWer)),
1664
+ loopbackMedianSpeechEndToFirstAudioMs: median(
1665
+ loopbackRows.map((result) => result.speechEndToFirstAudioMs),
1666
+ ),
1667
+ loopbackMedianSpeechEndToAudioEndMs: median(
1668
+ loopbackRows.map((result) => result.speechEndToAudioEndMs),
1669
+ ),
1670
+ bargeInRuns: bargeInRows.length,
1671
+ bargeInPasses: bargeInRows.filter((result) => result.bargeInPass).length,
1672
+ };
1673
+ }
1674
+
1675
+ function stackKey(stack = {}) {
1676
+ return [
1677
+ stack.device ?? "",
1678
+ stack.llm ?? "",
1679
+ stack.asr ?? "",
1680
+ stack.voice ?? "",
1681
+ stack.ttsSteps ?? "",
1682
+ stack.vadSilenceMs ?? "",
1683
+ stack.partialAsr ? "partial" : "final",
1684
+ ttsChunkingKey(stack.ttsChunking),
1685
+ ].join("|");
1686
+ }
1687
+
1688
+ function stackLabel(stack = {}) {
1689
+ const parts = [
1690
+ shortModel(stack.llm),
1691
+ stack.device?.toUpperCase() ?? "-",
1692
+ shortModel(stack.asr),
1693
+ stack.voice ?? "-",
1694
+ `${stack.ttsSteps ?? "-"} steps`,
1695
+ ];
1696
+ const chunking = ttsChunkingLabel(stack.ttsChunking);
1697
+ if (chunking) parts.push(chunking);
1698
+ return parts.join(" / ");
1699
+ }
1700
+
1701
+ function ttsChunkingKey(chunking = DEFAULT_TTS_CHUNKING) {
1702
+ const normalized = { ...DEFAULT_TTS_CHUNKING, ...(chunking ?? {}) };
1703
+ return Object.keys(DEFAULT_TTS_CHUNKING)
1704
+ .map((key) => `${key}:${normalized[key]}`)
1705
+ .join(",");
1706
+ }
1707
+
1708
+ function ttsChunkingLabel(chunking = DEFAULT_TTS_CHUNKING) {
1709
+ const normalized = { ...DEFAULT_TTS_CHUNKING, ...(chunking ?? {}) };
1710
+ if (ttsChunkingKey(normalized) === ttsChunkingKey(DEFAULT_TTS_CHUNKING)) return "";
1711
+ return `chunk ${normalized.firstTargetChars}/${normalized.targetChars}`;
1712
+ }
1713
+
1714
+ function median(values) {
1715
+ const finite = values.filter(Number.isFinite).sort((a, b) => a - b);
1716
+ if (finite.length === 0) return null;
1717
+ const middle = Math.floor(finite.length / 2);
1718
+ if (finite.length % 2 === 1) return finite[middle];
1719
+ return (finite[middle - 1] + finite[middle]) / 2;
1720
+ }
1721
+
1722
+ function shortModel(modelId = "") {
1723
+ return modelId.split("/").at(-1)?.replace(/-ONNX$/, "") ?? modelId;
1724
+ }
1725
+
1726
+ function clearLoopbackFeed() {
1727
+ for (const timer of state.loopbackFeedTimers) {
1728
+ window.clearTimeout(timer);
1729
+ }
1730
+ state.loopbackFeedTimers.clear();
1731
+ }
1732
+
1733
+ function scheduleLoopbackChunk(callback, delayMs) {
1734
+ const timer = window.setTimeout(() => {
1735
+ state.loopbackFeedTimers.delete(timer);
1736
+ callback();
1737
+ }, delayMs);
1738
+ state.loopbackFeedTimers.add(timer);
1739
+ }
1740
+
1741
+ function handleSpeechStart() {
1742
+ state.currentUserStartedAt = performance.now();
1743
+ elements.partialTranscript.textContent = "Speech detected.";
1744
+ setTile("vad", "Speech", "active");
1745
+ interruptForBargeIn({
1746
+ preserveLoopbackFeed: state.activeBenchmark?.kind === "loopback",
1747
+ logInterruption: true,
1748
+ });
1749
+ }
1750
+
1751
+ function interruptForBargeIn({ preserveLoopbackFeed = false, cancelTts = true, logInterruption = false } = {}) {
1752
+ const llmBusy = elements.states.llm.textContent === "Generating";
1753
+ const ttsBusy = elements.states.tts.textContent === "Synthesizing" || elements.states.tts.textContent === "Speaking";
1754
+ state.activeTurnId += 1;
1755
+ state.ttsBuffer = "";
1756
+ state.awaitingFirstToken = false;
1757
+ state.awaitingFirstAudio = false;
1758
+ state.firstTtsChunkQueued = false;
1759
+ state.pendingPlaybackSchedules = 0;
1760
+ if (!preserveLoopbackFeed) clearLoopbackFeed();
1761
+ state.playback?.stop();
1762
+ if (cancelTts) state.ttsWorker?.postMessage({ type: "cancel" });
1763
+ state.llmWorker?.postMessage({ type: "interrupt" });
1764
+ if (state.modelsLoaded && llmBusy) setTile("llm", "Ready", "ready");
1765
+ if (state.modelsLoaded && cancelTts && ttsBusy) setTile("tts", "Ready", "ready");
1766
+ if (logInterruption && (llmBusy || ttsBusy)) logEvent("Barge-in interrupted generation or playback.");
1767
+ setAudioState("Audio idle", false);
1768
+ }
1769
+
1770
+ function stopAll() {
1771
+ state.suiteRunning = false;
1772
+ cancelMicSeries();
1773
+ interruptForBargeIn();
1774
+ cancelActiveBenchmark();
1775
+ setTile("llm", state.modelsLoaded ? "Ready" : "Idle", state.modelsLoaded ? "ready" : "idle");
1776
+ setTile("tts", state.modelsLoaded ? "Ready" : "Idle", state.modelsLoaded ? "ready" : "idle");
1777
+ if (state.modelsLoaded) setBenchmarkControlsDisabled(false);
1778
+ logEvent("Stopped generation and playback.");
1779
+ }
1780
+
1781
+ function generateResponse(text) {
1782
+ state.activeTurnId += 1;
1783
+ state.messages.push({ role: "user", content: text });
1784
+ trimConversation();
1785
+ const promptMessages = promptMessagesForTurn(text);
1786
+ state.lastTranscriptAt = performance.now();
1787
+ state.llmWorker.postMessage({
1788
+ type: "generate",
1789
+ turnId: state.activeTurnId,
1790
+ messages: promptMessages,
1791
+ });
1792
+ }
1793
+
1794
+ function trimConversation() {
1795
+ const pinned = state.messages.slice(0, SYSTEM_MESSAGES.length);
1796
+ const rest = state.messages.slice(SYSTEM_MESSAGES.length);
1797
+ state.messages = [...pinned, ...rest.slice(-8)];
1798
+ }
1799
+
1800
+ function promptMessagesForTurn(text) {
1801
+ if (!isAppIdentityQuestion(text)) return state.messages;
1802
+ const rest = state.messages.slice(SYSTEM_MESSAGES.length);
1803
+ return [
1804
+ {
1805
+ role: "system",
1806
+ content: IDENTITY_SYSTEM_PROMPT,
1807
+ },
1808
+ ...IDENTITY_PRIMER_MESSAGES,
1809
+ ...rest,
1810
+ ];
1811
+ }
1812
+
1813
+ function initialMessages() {
1814
+ return SYSTEM_MESSAGES.map((message) => ({ ...message }));
1815
+ }
1816
+
1817
+ function resetConversationHistory() {
1818
+ state.messages = initialMessages();
1819
+ }
1820
+
1821
+ function bufferTts(text) {
1822
+ state.ttsBuffer += text;
1823
+ const chunks = extractSpeakableChunks(state.ttsBuffer, false);
1824
+ for (const chunk of chunks.ready) {
1825
+ enqueueTts(chunk);
1826
+ }
1827
+ state.ttsBuffer = chunks.remainder;
1828
+ }
1829
+
1830
+ function flushTts(final = false) {
1831
+ const chunks = extractSpeakableChunks(state.ttsBuffer, final);
1832
+ for (const chunk of chunks.ready) {
1833
+ enqueueTts(chunk);
1834
+ }
1835
+ state.ttsBuffer = chunks.remainder;
1836
+ }
1837
+
1838
+ function extractSpeakableChunks(buffer, final) {
1839
+ const ready = [];
1840
+ let text = buffer.replace(/\s+/g, " ");
1841
+ while (text.length > 0) {
1842
+ const boundary = findBoundary(text, final);
1843
+ if (!boundary) break;
1844
+ const chunk = text.slice(0, boundary.index).trim();
1845
+ text = text.slice(boundary.index).trimStart();
1846
+ if (chunk.length > 0) {
1847
+ ready.push({
1848
+ text: chunk,
1849
+ boundaryKind: boundary.kind,
1850
+ wordBoundarySafe: boundary.wordBoundarySafe,
1851
+ });
1852
+ }
1853
+ }
1854
+ return { ready, remainder: text };
1855
+ }
1856
+
1857
+ function findBoundary(text, final) {
1858
+ const firstChunk = state.awaitingFirstAudio && !state.firstTtsChunkQueued;
1859
+ const chunking = state.ttsChunking;
1860
+ const sentence = text.search(/[.!?]\s/);
1861
+ if (sentence >= (firstChunk ? chunking.firstSentenceMinChars : chunking.sentenceMinChars)) {
1862
+ return boundaryAt(text, sentence + 1, "sentence");
1863
+ }
1864
+ const clause = text.search(/[,;:]\s/);
1865
+ if (clause >= (firstChunk ? chunking.firstClauseMinChars : chunking.clauseMinChars)) {
1866
+ return boundaryAt(text, clause + 1, "clause");
1867
+ }
1868
+ const targetLength = firstChunk ? chunking.firstTargetChars : chunking.targetChars;
1869
+ const minSpace = firstChunk ? chunking.firstMinSpaceChars : chunking.minSpaceChars;
1870
+ if (text.length >= targetLength) {
1871
+ const space = text.lastIndexOf(" ", targetLength);
1872
+ if (space > minSpace) return boundaryAt(text, space, "space-before-target");
1873
+ const maxForward = targetLength * 2;
1874
+ const nextSpace = text.indexOf(" ", targetLength);
1875
+ if (nextSpace > minSpace && nextSpace <= maxForward) return boundaryAt(text, nextSpace, "space-after-target");
1876
+ if (firstChunk && text.length < maxForward) return null;
1877
+ return boundaryAt(text, targetLength, "hard-limit");
1878
+ }
1879
+ if (final && text.trim().length > 0) return boundaryAt(text, text.length, "final");
1880
+ return null;
1881
+ }
1882
+
1883
+ function boundaryAt(text, index, kind) {
1884
+ return {
1885
+ index,
1886
+ kind,
1887
+ wordBoundarySafe: isWordBoundarySafe(text, index),
1888
+ };
1889
+ }
1890
+
1891
+ function isWordBoundarySafe(text, index) {
1892
+ if (index >= text.length) return true;
1893
+ const before = text[index - 1] ?? "";
1894
+ const after = text[index] ?? "";
1895
+ if (/\s/.test(after)) return true;
1896
+ if (/[.!?,;:]/.test(before)) return true;
1897
+ return !(isWordLikeChar(before) && isWordLikeChar(after));
1898
+ }
1899
+
1900
+ function isWordLikeChar(char) {
1901
+ return /[A-Za-z0-9']/.test(char);
1902
+ }
1903
+
1904
+ function enqueueTts(chunk) {
1905
+ const voice = elements.voiceSelect.value;
1906
+ const steps = Number(elements.ttsSteps.value);
1907
+ const sequence = ++state.ttsSequence;
1908
+ const chunkInfo = normalizeTtsChunk(chunk);
1909
+ const speakableText = chunkInfo.text.trim().replace(/^["'“”]+|["'“”]+$/g, "");
1910
+ if (!speakableText) return;
1911
+ const enqueuedAt = performance.now();
1912
+ if (!state.firstTtsChunkQueued) {
1913
+ state.firstTtsChunkQueued = true;
1914
+ const queuedMs = Number.isFinite(state.lastTranscriptAt) ? enqueuedAt - state.lastTranscriptAt : null;
1915
+ elements.firstTtsQueuedLatency.textContent = formatMs(queuedMs);
1916
+ if (state.activeBenchmark) {
1917
+ state.activeBenchmark.firstTtsQueuedMs = queuedMs;
1918
+ state.activeBenchmark.firstTtsText = speakableText;
1919
+ state.activeBenchmark.firstTtsChars = speakableText.length;
1920
+ state.activeBenchmark.firstTtsBoundaryKind = chunkInfo.boundaryKind;
1921
+ state.activeBenchmark.firstTtsWordBoundarySafe = chunkInfo.wordBoundarySafe;
1922
+ }
1923
+ }
1924
+ state.ttsWorker.postMessage({
1925
+ type: "speak",
1926
+ turnId: state.activeTurnId,
1927
+ sequence,
1928
+ enqueuedAt,
1929
+ text: speakableText,
1930
+ voice,
1931
+ steps,
1932
+ speed: 1.08,
1933
+ });
1934
+ }
1935
+
1936
+ function normalizeTtsChunk(chunk) {
1937
+ if (typeof chunk === "string") {
1938
+ return {
1939
+ text: chunk,
1940
+ boundaryKind: "direct",
1941
+ wordBoundarySafe: true,
1942
+ };
1943
+ }
1944
+ return {
1945
+ text: chunk?.text ?? "",
1946
+ boundaryKind: chunk?.boundaryKind ?? "",
1947
+ wordBoundarySafe: chunk?.wordBoundarySafe ?? null,
1948
+ };
1949
+ }
1950
+
1951
+ function cleanAssistantResponse(text) {
1952
+ return text.trim().replace(/^["'“”]+|["'“”]+$/g, "").trim();
1953
+ }
1954
+
1955
+ function runBenchmark() {
1956
+ if (!state.modelsLoaded || !canStartBenchmark()) return false;
1957
+ state.playback?.unlock().catch(() => {});
1958
+ interruptForBargeIn();
1959
+ resetConversationHistory();
1960
+ state.lastSpeechEndAt = null;
1961
+ state.lastVadCloseAt = null;
1962
+ state.lastVadCloseDelayMs = null;
1963
+ resetMetrics();
1964
+ const prompt = APP_IDENTITY_PROMPT;
1965
+ startBenchmark("identity", prompt);
1966
+ logEvent("Running identity benchmark.");
1967
+ elements.finalTranscript.textContent = prompt;
1968
+ generateResponse(prompt);
1969
+ return true;
1970
+ }
1971
+
1972
+ function runChatBenchmark() {
1973
+ if (!state.modelsLoaded || !canStartBenchmark()) return false;
1974
+ state.playback?.unlock().catch(() => {});
1975
+ interruptForBargeIn();
1976
+ resetConversationHistory();
1977
+ state.lastSpeechEndAt = null;
1978
+ state.lastVadCloseAt = null;
1979
+ state.lastVadCloseDelayMs = null;
1980
+ resetMetrics();
1981
+ const prompt = "Greet the user in one short sentence.";
1982
+ startBenchmark("chat", prompt);
1983
+ logEvent("Running chat benchmark.");
1984
+ elements.finalTranscript.textContent = prompt;
1985
+ generateResponse(prompt);
1986
+ return true;
1987
+ }
1988
+
1989
+ function runTtsBenchmark() {
1990
+ if (!state.modelsLoaded || !canStartBenchmark()) return false;
1991
+ state.playback?.unlock().catch(() => {});
1992
+ const ttsIdle = elements.states.tts.textContent === "Ready" || elements.states.tts.textContent === "Idle";
1993
+ interruptForBargeIn({ cancelTts: !ttsIdle });
1994
+ state.lastSpeechEndAt = null;
1995
+ state.lastVadCloseAt = null;
1996
+ state.lastVadCloseDelayMs = null;
1997
+ resetMetrics();
1998
+ const text = "This is a short local speech benchmark.";
1999
+ startBenchmark("tts", text);
2000
+ state.lastTranscriptAt = performance.now();
2001
+ state.awaitingFirstAudio = true;
2002
+ state.firstTtsChunkQueued = false;
2003
+ state.ttsBuffer = "";
2004
+ state.currentAssistant = text;
2005
+ elements.partialTranscript.textContent = "";
2006
+ elements.finalTranscript.textContent = "TTS benchmark";
2007
+ elements.llmOutput.textContent = text;
2008
+ state.activeBenchmark.output = text;
2009
+ logEvent("Running TTS benchmark.");
2010
+ enqueueTts(text);
2011
+ state.activeBenchmark.llmDoneAt = performance.now();
2012
+ state.activeBenchmark.llmCompleteMs = 0;
2013
+ window.setTimeout(() => finalizeBenchmarkIfIdle(), 0);
2014
+ return true;
2015
+ }
2016
+
2017
+ function runLoopbackBenchmark(options = {}) {
2018
+ if (!state.modelsLoaded || !canStartBenchmark()) return false;
2019
+ state.playback?.unlock().catch(() => {});
2020
+ interruptForBargeIn();
2021
+ resetConversationHistory();
2022
+ state.lastSpeechEndAt = null;
2023
+ state.lastVadCloseAt = null;
2024
+ state.lastVadCloseDelayMs = null;
2025
+ state.loopbackRequestId += 1;
2026
+ const text = APP_IDENTITY_PROMPT;
2027
+ startBenchmark("loopback", text);
2028
+ elements.partialTranscript.textContent = "Synthesizing local loopback speech...";
2029
+ elements.finalTranscript.textContent = "";
2030
+ elements.llmOutput.textContent = "";
2031
+ resetMetrics();
2032
+ const speed = Number.isFinite(options?.speed) ? options.speed : DEFAULT_LOOPBACK_SPEED;
2033
+ state.activeBenchmark.loopbackSpeed = speed;
2034
+ logEvent(`Running voice loopback benchmark: "${text}"`);
2035
+ state.ttsWorker.postMessage({
2036
+ type: "synthesize",
2037
+ requestId: state.loopbackRequestId,
2038
+ text,
2039
+ voice: elements.voiceSelect.value,
2040
+ steps: 2,
2041
+ speed,
2042
+ });
2043
+ return true;
2044
+ }
2045
+
2046
+ function runBargeInBenchmark() {
2047
+ if (!state.modelsLoaded || !canStartBenchmark()) return false;
2048
+ state.playback?.unlock().catch(() => {});
2049
+ interruptForBargeIn();
2050
+ state.lastSpeechEndAt = null;
2051
+ state.lastVadCloseAt = null;
2052
+ state.lastVadCloseDelayMs = null;
2053
+ resetMetrics();
2054
+ const text = "This is a deliberately long local speech benchmark for interrupting audio playback.";
2055
+ startBenchmark("barge-in", "Synthetic speech start during TTS");
2056
+ state.lastTranscriptAt = performance.now();
2057
+ state.awaitingFirstAudio = true;
2058
+ state.firstTtsChunkQueued = false;
2059
+ state.ttsBuffer = "";
2060
+ state.currentAssistant = text;
2061
+ elements.partialTranscript.textContent = "";
2062
+ elements.finalTranscript.textContent = "Barge-in check";
2063
+ elements.llmOutput.textContent = text;
2064
+ state.activeBenchmark.output = "Waiting to interrupt in-flight TTS.";
2065
+ state.activeBenchmark.llmDoneAt = performance.now();
2066
+ state.activeBenchmark.llmCompleteMs = 0;
2067
+ logEvent("Running barge-in check.");
2068
+ enqueueTts(text);
2069
+ const startedAt = performance.now();
2070
+ window.setTimeout(() => {
2071
+ if (state.activeBenchmark?.kind !== "barge-in") return;
2072
+ state.activeBenchmark.bargeInMs = performance.now() - startedAt;
2073
+ handleSpeechStart();
2074
+ window.setTimeout(() => {
2075
+ if (state.activeBenchmark?.kind !== "barge-in") return;
2076
+ state.activeBenchmark.firstAudioMs ??= parseMetricMs(elements.firstAudioLatency.textContent);
2077
+ state.activeBenchmark.decodeRate = elements.decodeRate.textContent;
2078
+ state.activeBenchmark.bargeInPass = state.activeBenchmark.firstAudioMs == null;
2079
+ state.activeBenchmark.output = state.activeBenchmark.bargeInPass
2080
+ ? "Barge-in cancelled in-flight TTS before playback."
2081
+ : "Barge-in did not prevent first audio.";
2082
+ if (!state.activeBenchmark.bargeInPass) state.activeBenchmark.error = state.activeBenchmark.output;
2083
+ completeActiveBenchmark();
2084
+ }, 2200);
2085
+ }, 250);
2086
+ return true;
2087
+ }
2088
+
2089
+ async function runMicBenchmark(options = {}) {
2090
+ const series = options?.series === true;
2091
+ if (!state.modelsLoaded) return false;
2092
+ if (!canStartBenchmark({ allowMicSeries: series })) return false;
2093
+ state.playback?.unlock().catch(() => {});
2094
+ interruptForBargeIn();
2095
+ state.lastSpeechEndAt = null;
2096
+ state.lastVadCloseAt = null;
2097
+ state.lastVadCloseDelayMs = null;
2098
+ resetMetrics();
2099
+ const prompt = APP_IDENTITY_PROMPT;
2100
+ const runLabel = series ? ` (${state.micSeries.completed + 1}/${state.micSeries.target})` : "";
2101
+ elements.finalTranscript.textContent = prompt;
2102
+ elements.llmOutput.textContent = "";
2103
+ elements.partialTranscript.textContent = `Say: "${prompt}"${runLabel}`;
2104
+ startBenchmark("mic", prompt, { referenceText: prompt, timeoutMs: options?.timeoutMs ?? 120000 });
2105
+ state.activeBenchmark.stopMicAfterTranscript = options?.stopMicAfterTranscript === true;
2106
+ state.activeBenchmark.requireExactTranscript = options?.requireExactTranscript === true;
2107
+ updateMicValidationStatus();
2108
+ logEvent(`Starting real-mic benchmark${runLabel}: "${prompt}"`);
2109
+ try {
2110
+ if (!state.micActive) await startMic();
2111
+ logEvent(`Benchmarking real-mic turn: "${prompt}"`);
2112
+ return true;
2113
+ } catch (error) {
2114
+ cancelActiveBenchmark();
2115
+ if (series) cancelMicSeries();
2116
+ logEvent(`Microphone benchmark failed: ${error.message}`);
2117
+ return false;
2118
+ }
2119
+ }
2120
+
2121
+ async function runMicSeriesBenchmark() {
2122
+ if (!state.modelsLoaded) return false;
2123
+ if (!canStartBenchmark()) return false;
2124
+ state.micSeries.active = true;
2125
+ state.micSeries.completed = 0;
2126
+ state.micSeries.target = 3;
2127
+ updateMicSeriesButton();
2128
+ setBenchmarkControlsDisabled(false);
2129
+ logEvent("Starting 3-run real-mic series.");
2130
+ const started = await runMicBenchmark({ series: true });
2131
+ if (!started) cancelMicSeries();
2132
+ return started;
2133
+ }
2134
+
2135
+ function canStartBenchmark({ allowMicSeries = false } = {}) {
2136
+ if (state.activeBenchmark) {
2137
+ logEvent("A benchmark is already running.");
2138
+ return false;
2139
+ }
2140
+ if (state.micSeries.active && !allowMicSeries) {
2141
+ logEvent("A real-mic series is already running.");
2142
+ return false;
2143
+ }
2144
+ return true;
2145
+ }
2146
+
2147
+ async function runBenchmarkSuite() {
2148
+ if (!state.modelsLoaded || state.suiteRunning) return;
2149
+ if (!canStartBenchmark()) return;
2150
+ state.suiteRunning = true;
2151
+ setBenchmarkControlsDisabled(true);
2152
+ logEvent("Running benchmark suite for the current stack.");
2153
+ try {
2154
+ await runSuiteStep("TTS", runTtsBenchmark);
2155
+ await runSuiteStep("barge-in", runBargeInBenchmark);
2156
+ await runSuiteStep("identity", runBenchmark);
2157
+ await runSuiteStep("chat", runChatBenchmark);
2158
+ await runSuiteStep("loopback", runLoopbackBenchmark);
2159
+ logEvent("Benchmark suite complete.");
2160
+ } catch (error) {
2161
+ logEvent(`Benchmark suite stopped: ${error.message}`);
2162
+ } finally {
2163
+ state.suiteRunning = false;
2164
+ if (state.modelsLoaded) setBenchmarkControlsDisabled(false);
2165
+ }
2166
+ }
2167
+
2168
+ async function runSuiteStep(label, runner) {
2169
+ const previousCount = state.benchmarkResults.length;
2170
+ if (runner() === false) throw new Error(`${label} benchmark did not start.`);
2171
+ await waitForBenchmarkRow(previousCount, label);
2172
+ }
2173
+
2174
+ function waitForBenchmarkRow(previousCount, label) {
2175
+ return new Promise((resolve, reject) => {
2176
+ const startedAt = performance.now();
2177
+ const timer = window.setInterval(() => {
2178
+ if (state.benchmarkResults.length > previousCount) {
2179
+ window.clearInterval(timer);
2180
+ resolve();
2181
+ return;
2182
+ }
2183
+ if (!state.suiteRunning) {
2184
+ window.clearInterval(timer);
2185
+ reject(new Error(`${label} benchmark was cancelled.`));
2186
+ return;
2187
+ }
2188
+ if (performance.now() - startedAt > 135000) {
2189
+ window.clearInterval(timer);
2190
+ reject(new Error(`${label} benchmark did not finish.`));
2191
+ }
2192
+ }, 250);
2193
+ });
2194
+ }
2195
+
2196
+ function feedLoopbackAudio(audio, sampleRate) {
2197
+ clearLoopbackFeed();
2198
+ const requestId = state.loopbackRequestId;
2199
+ const samples = audio instanceof Float32Array ? audio : new Float32Array(audio);
2200
+ const chunkSize = Math.max(512, Math.floor(sampleRate * 0.032));
2201
+ if (state.activeBenchmark) state.activeBenchmark.loopbackSampleRate = sampleRate;
2202
+ elements.partialTranscript.textContent = "Feeding synthesized speech through VAD...";
2203
+ let delayMs = 0;
2204
+ const postChunk = (chunk) => {
2205
+ scheduleLoopbackChunk(() => {
2206
+ if (requestId !== state.loopbackRequestId || !state.asrWorker) return;
2207
+ state.asrWorker.postMessage({ type: "audio", buffer: chunk, sampleRate }, [chunk.buffer]);
2208
+ }, delayMs);
2209
+ delayMs += (chunk.length / sampleRate) * 1000;
2210
+ };
2211
+ for (let offset = 0; offset < samples.length; offset += chunkSize) {
2212
+ const chunk = samples.slice(offset, Math.min(offset + chunkSize, samples.length));
2213
+ postChunk(chunk);
2214
+ }
2215
+ const silenceSeconds = (currentVadSilenceMs() + 220) / 1000;
2216
+ const silenceChunks = Math.ceil(sampleRate * silenceSeconds / chunkSize);
2217
+ for (let i = 0; i < silenceChunks; i += 1) {
2218
+ const silence = new Float32Array(chunkSize);
2219
+ postChunk(silence);
2220
+ }
2221
+ scheduleLoopbackChunk(() => {
2222
+ if (requestId !== state.loopbackRequestId || !state.asrWorker) return;
2223
+ state.asrWorker.postMessage({ type: "flush" });
2224
+ }, delayMs + 1000);
2225
+ }
2226
+
2227
+ function isLocalAutomationHost() {
2228
+ return ["localhost", "127.0.0.1", "::1", ""].includes(location.hostname);
2229
+ }
2230
+
2231
+ function automationSnapshot() {
2232
+ return {
2233
+ modelsLoaded: state.modelsLoaded,
2234
+ modelsLoading: state.modelsLoading,
2235
+ micActive: state.micActive,
2236
+ micInputStats: {
2237
+ chunks: state.micInputStats.chunks,
2238
+ samples: state.micInputStats.samples,
2239
+ peak: state.micInputStats.peak,
2240
+ rms:
2241
+ state.micInputStats.samples > 0
2242
+ ? Math.sqrt(state.micInputStats.sumSquares / state.micInputStats.samples)
2243
+ : null,
2244
+ },
2245
+ activeBenchmark: state.activeBenchmark ? { ...state.activeBenchmark } : null,
2246
+ suiteRunning: state.suiteRunning,
2247
+ micSeries: { ...state.micSeries, timer: Boolean(state.micSeries.timer) },
2248
+ ttsChunking: { ...state.ttsChunking },
2249
+ stack: state.loadedStack ? currentBenchmarkStack() : null,
2250
+ summary: exportBenchmarkSummary(),
2251
+ results: [...state.benchmarkResults],
2252
+ events: [...elements.eventLog.children].map((item) => item.textContent),
2253
+ };
2254
+ }
2255
+
2256
+ async function automationWebGpuInfo() {
2257
+ await supportsWebGPU();
2258
+ return {
2259
+ available: state.webgpuAvailable,
2260
+ adapter: state.webgpuAdapterInfo,
2261
+ features: state.webgpuAdapterFeatures,
2262
+ softwareAdapter: state.webgpuSoftwareAdapter,
2263
+ userAgent: navigator.userAgent,
2264
+ platform: navigator.platform || "",
2265
+ hardwareConcurrency: navigator.hardwareConcurrency ?? null,
2266
+ deviceMemoryGb: navigator.deviceMemory ?? null,
2267
+ };
2268
+ }
2269
+
2270
+ function applyAutomationStack(options = {}) {
2271
+ if (state.modelsLoaded || state.modelsLoading) {
2272
+ throw new Error("Unload models before changing the benchmark stack.");
2273
+ }
2274
+ if (options.device) elements.deviceSelect.value = options.device;
2275
+ if (options.llm) elements.llmModelSelect.value = options.llm;
2276
+ if (options.asr) elements.asrModelSelect.value = options.asr;
2277
+ applyRuntimeOptions(options);
2278
+ }
2279
+
2280
+ function normalizedTtsChunking(options = {}) {
2281
+ const next = { ...state.ttsChunking };
2282
+ for (const key of Object.keys(DEFAULT_TTS_CHUNKING)) {
2283
+ if (options[key] == null) continue;
2284
+ const value = Number(options[key]);
2285
+ if (!Number.isFinite(value)) continue;
2286
+ next[key] = Math.max(1, Math.min(160, Math.round(value)));
2287
+ }
2288
+ return next;
2289
+ }
2290
+
2291
+ function hasTtsChunkingOption(options = {}) {
2292
+ return Object.keys(DEFAULT_TTS_CHUNKING).some((key) => options[key] != null);
2293
+ }
2294
+
2295
+ function applyRuntimeOptions(options = {}) {
2296
+ if (options.voice) {
2297
+ elements.voiceSelect.value = options.voice;
2298
+ if (state.modelsLoaded || state.modelsLoading) {
2299
+ state.ttsWorker?.postMessage({ type: "preload-voice", voice: elements.voiceSelect.value });
2300
+ }
2301
+ }
2302
+ if (options.ttsSteps != null) {
2303
+ elements.ttsSteps.value = String(options.ttsSteps);
2304
+ updateTtsStepsLabel();
2305
+ }
2306
+ if (options.vadSilenceMs != null) {
2307
+ elements.vadSilence.value = String(options.vadSilenceMs);
2308
+ updateVadSilenceLabel();
2309
+ }
2310
+ if (options.partialAsr != null) {
2311
+ elements.partialToggle.checked = Boolean(options.partialAsr);
2312
+ }
2313
+ if (options.ttsChunking || hasTtsChunkingOption(options)) {
2314
+ state.ttsChunking = normalizedTtsChunking(options.ttsChunking ?? options);
2315
+ }
2316
+ if (state.modelsLoaded || state.modelsLoading) configureAsrWorker();
2317
+ }
2318
+
2319
+ function preloadTtsVoice(voice = elements.voiceSelect.value, { timeoutMs = 30000 } = {}) {
2320
+ if (!state.ttsWorker) return Promise.reject(new Error("TTS worker is not loaded."));
2321
+ const requestId = ++state.ttsVoiceRequestId;
2322
+ return new Promise((resolve, reject) => {
2323
+ const timeout = window.setTimeout(() => {
2324
+ state.ttsVoiceRequests.delete(requestId);
2325
+ reject(new Error(`Voice ${voice} preload timed out.`));
2326
+ }, timeoutMs);
2327
+ state.ttsVoiceRequests.set(requestId, { resolve, reject, timeout });
2328
+ state.ttsWorker.postMessage({ type: "preload-voice", voice, requestId });
2329
+ });
2330
+ }
2331
+
2332
+ function rejectTtsVoiceRequests(error) {
2333
+ for (const request of state.ttsVoiceRequests.values()) {
2334
+ window.clearTimeout(request.timeout);
2335
+ request.reject(error);
2336
+ }
2337
+ state.ttsVoiceRequests.clear();
2338
+ }
2339
+
2340
+ function waitForAutomationCondition(predicate, timeoutMs, label) {
2341
+ return new Promise((resolve, reject) => {
2342
+ const startedAt = performance.now();
2343
+ const timer = window.setInterval(() => {
2344
+ if (predicate()) {
2345
+ window.clearInterval(timer);
2346
+ resolve();
2347
+ return;
2348
+ }
2349
+ if (performance.now() - startedAt > timeoutMs) {
2350
+ window.clearInterval(timer);
2351
+ reject(new Error(`${label} timed out after ${(timeoutMs / 1000).toFixed(0)} seconds.`));
2352
+ }
2353
+ }, 250);
2354
+ });
2355
+ }
2356
+
2357
+ function benchmarkRowsAddedSince(previousCount) {
2358
+ return state.benchmarkResults.slice(0, Math.max(0, state.benchmarkResults.length - previousCount));
2359
+ }
2360
+
2361
+ function synthesizeAutomationAudio({
2362
+ text = APP_IDENTITY_PROMPT,
2363
+ voice = elements.voiceSelect.value,
2364
+ steps = 2,
2365
+ speed = 1.05,
2366
+ timeoutMs = 60000,
2367
+ } = {}) {
2368
+ if (!state.modelsLoaded || !state.ttsWorker) {
2369
+ return Promise.reject(new Error("Load models before synthesizing automation audio."));
2370
+ }
2371
+ const requestId = `automation-${++state.automationSynthesisRequestId}`;
2372
+ return new Promise((resolve, reject) => {
2373
+ const timeout = window.setTimeout(() => {
2374
+ state.automationSynthesisRequests.delete(requestId);
2375
+ reject(new Error("Automation synthesis timed out."));
2376
+ }, timeoutMs);
2377
+ state.automationSynthesisRequests.set(requestId, {
2378
+ text,
2379
+ voice,
2380
+ steps,
2381
+ speed,
2382
+ resolve: (result) => {
2383
+ window.clearTimeout(timeout);
2384
+ resolve(result);
2385
+ },
2386
+ reject: (error) => {
2387
+ window.clearTimeout(timeout);
2388
+ reject(error);
2389
+ },
2390
+ });
2391
+ state.ttsWorker.postMessage({
2392
+ type: "synthesize",
2393
+ requestId,
2394
+ text,
2395
+ voice,
2396
+ steps,
2397
+ speed,
2398
+ });
2399
+ });
2400
+ }
2401
+
2402
+ async function runSingleAutomationBenchmark(runner, { timeoutMs = 150000, ...runnerOptions } = {}) {
2403
+ const previousCount = state.benchmarkResults.length;
2404
+ const started = await runner({ ...runnerOptions, timeoutMs: Math.max(1000, timeoutMs - 5000) });
2405
+ if (started === false) throw new Error("Benchmark did not start.");
2406
+ await waitForAutomationCondition(
2407
+ () => state.benchmarkResults.length > previousCount,
2408
+ timeoutMs,
2409
+ "Benchmark",
2410
+ );
2411
+ return automationSnapshot();
2412
+ }
2413
+
2414
+ async function runAutomationMicSeries({ timeoutMs = 420000 } = {}) {
2415
+ const previousCount = state.benchmarkResults.length;
2416
+ const started = await runMicSeriesBenchmark();
2417
+ if (started === false) throw new Error("Real-mic series did not start.");
2418
+ await waitForAutomationCondition(
2419
+ () => !state.micSeries.active && !state.activeBenchmark,
2420
+ timeoutMs,
2421
+ "Real-mic series",
2422
+ );
2423
+ const rows = benchmarkRowsAddedSince(previousCount);
2424
+ const target = rows.find((result) => result.kind === "mic")?.micSeriesTarget ?? 3;
2425
+ const micRows = rows.filter((result) => result.kind === "mic" && result.micSeriesTarget === target);
2426
+ if (micRows.length < target || micRows.some((result) => result.error)) {
2427
+ throw new Error(`Real-mic series produced ${micRows.length}/${target} completed rows.`);
2428
+ }
2429
+ return automationSnapshot();
2430
+ }
2431
+
2432
+ function installAutomationApi() {
2433
+ if (!isLocalAutomationHost()) return;
2434
+ window.browserSpeakBench = {
2435
+ version: 1,
2436
+ state: automationSnapshot,
2437
+ exportResults() {
2438
+ return {
2439
+ summary: exportBenchmarkSummary(),
2440
+ results: [...state.benchmarkResults],
2441
+ };
2442
+ },
2443
+ webgpuInfo: automationWebGpuInfo,
2444
+ setRuntimeOptions(options = {}) {
2445
+ applyRuntimeOptions(options);
2446
+ return automationSnapshot();
2447
+ },
2448
+ async preloadVoice(options = {}) {
2449
+ await preloadTtsVoice(options.voice ?? elements.voiceSelect.value, {
2450
+ timeoutMs: options.timeoutMs ?? 30000,
2451
+ });
2452
+ return automationSnapshot();
2453
+ },
2454
+ async loadStack(options = {}) {
2455
+ applyAutomationStack(options);
2456
+ await loadModels({ ttsWarmup: options.ttsWarmup !== false });
2457
+ if (!state.modelsLoaded) throw new Error("Model load did not complete.");
2458
+ return automationSnapshot();
2459
+ },
2460
+ async unload() {
2461
+ await unloadModels();
2462
+ return automationSnapshot();
2463
+ },
2464
+ runSuite: async () => {
2465
+ await runBenchmarkSuite();
2466
+ return automationSnapshot();
2467
+ },
2468
+ runIdentity: (options) => runSingleAutomationBenchmark(runBenchmark, options),
2469
+ runChat: (options) => runSingleAutomationBenchmark(runChatBenchmark, options),
2470
+ runTts: (options) => runSingleAutomationBenchmark(runTtsBenchmark, options),
2471
+ runLoopback: (options) => runSingleAutomationBenchmark(runLoopbackBenchmark, options),
2472
+ runBargeIn: (options) => runSingleAutomationBenchmark(runBargeInBenchmark, options),
2473
+ runMic: (options) => runSingleAutomationBenchmark(runMicBenchmark, options),
2474
+ runMicSeries: runAutomationMicSeries,
2475
+ synthesizeAudio: synthesizeAutomationAudio,
2476
+ waitForRows(count, { timeoutMs = 180000 } = {}) {
2477
+ return waitForAutomationCondition(
2478
+ () => state.benchmarkResults.length >= count,
2479
+ timeoutMs,
2480
+ `${count} benchmark rows`,
2481
+ ).then(automationSnapshot);
2482
+ },
2483
+ clearResults() {
2484
+ cancelMicSeries();
2485
+ clearBenchmarkTimeout();
2486
+ state.benchmarkResults = [];
2487
+ state.activeBenchmark = null;
2488
+ renderBenchmarkResults();
2489
+ return automationSnapshot();
2490
+ },
2491
+ stop() {
2492
+ stopAll();
2493
+ return automationSnapshot();
2494
+ },
2495
+ stopMic() {
2496
+ stopMic();
2497
+ return automationSnapshot();
2498
+ },
2499
+ };
2500
+ }
2501
+
2502
+ elements.loadButton.addEventListener("click", () => {
2503
+ const action = state.modelsLoaded ? unloadModels() : loadModels();
2504
+ action.catch((error) => logEvent(`Model control failed: ${error.message}`));
2505
+ });
2506
+ elements.micButton.addEventListener("click", () => {
2507
+ if (state.micActive) stopMic();
2508
+ else startMic().catch((error) => logEvent(`Microphone failed: ${error.message}`));
2509
+ });
2510
+ elements.stopButton.addEventListener("click", stopAll);
2511
+ elements.suiteButton.addEventListener("click", runBenchmarkSuite);
2512
+ elements.benchmarkButton.addEventListener("click", runBenchmark);
2513
+ elements.chatBenchmarkButton.addEventListener("click", runChatBenchmark);
2514
+ elements.ttsBenchmarkButton.addEventListener("click", runTtsBenchmark);
2515
+ elements.loopbackButton.addEventListener("click", runLoopbackBenchmark);
2516
+ elements.bargeInButton.addEventListener("click", runBargeInBenchmark);
2517
+ elements.micBenchmarkButton.addEventListener("click", runMicBenchmark);
2518
+ elements.micSeriesButton.addEventListener("click", runMicSeriesBenchmark);
2519
+ elements.copyResultsButton.addEventListener("click", () => {
2520
+ const payload = JSON.stringify(
2521
+ {
2522
+ summary: exportBenchmarkSummary(),
2523
+ results: state.benchmarkResults,
2524
+ },
2525
+ null,
2526
+ 2,
2527
+ );
2528
+ const write = navigator.clipboard?.writeText(payload);
2529
+ if (write) {
2530
+ write.then(() => logEvent("Benchmark JSON copied.")).catch(() => logEvent(payload));
2531
+ } else {
2532
+ logEvent(payload);
2533
+ }
2534
+ });
2535
+ elements.clearResultsButton.addEventListener("click", () => {
2536
+ cancelMicSeries();
2537
+ clearBenchmarkTimeout();
2538
+ state.benchmarkResults = [];
2539
+ state.activeBenchmark = null;
2540
+ renderBenchmarkResults();
2541
+ });
2542
+ elements.clearLogButton.addEventListener("click", () => {
2543
+ elements.eventLog.replaceChildren();
2544
+ });
2545
+ elements.deviceSelect.addEventListener("change", () => {
2546
+ updateRuntimeStatus();
2547
+ if (!state.modelsLoaded && !state.modelsLoading) {
2548
+ resetPipelineTiles().catch((error) => logEvent(`Runtime status failed: ${error.message}`));
2549
+ }
2550
+ });
2551
+ elements.ttsSteps.addEventListener("input", updateTtsStepsLabel);
2552
+ elements.voiceSelect.addEventListener("change", () => {
2553
+ if (!state.modelsLoaded) return;
2554
+ preloadTtsVoice().catch((error) => logEvent(`Voice preload failed: ${error.message}`));
2555
+ });
2556
+ elements.partialToggle.addEventListener("change", () => {
2557
+ configureAsrWorker();
2558
+ });
2559
+ elements.vadSilence.addEventListener("input", () => {
2560
+ updateVadSilenceLabel();
2561
+ configureAsrWorker();
2562
+ });
2563
+
2564
+ updateTtsStepsLabel();
2565
+ updateVadSilenceLabel();
2566
+ updateMicValidationStatus();
2567
+ installAutomationApi();
2568
+ initRuntimeSupport();
2569
+
2570
+ async function initRuntimeSupport() {
2571
+ if (!window.isSecureContext) {
2572
+ logEvent("Use http://localhost or HTTPS so the browser will allow microphone access.");
2573
+ }
2574
+ await supportsWebGPU();
2575
+ if (!state.webgpuAvailable || state.webgpuSoftwareAdapter) {
2576
+ if (!state.modelsLoaded && !state.modelsLoading) {
2577
+ const label = state.webgpuSoftwareAdapter ? "WASM auto" : "WASM only";
2578
+ setTile("llm", label, "warn");
2579
+ setTile("tts", label, "warn");
2580
+ }
2581
+ if (state.webgpuSoftwareAdapter) {
2582
+ logEvent(
2583
+ `Software WebGPU adapter detected (${formatGpuAdapter(
2584
+ state.webgpuAdapterInfo,
2585
+ )}); Auto uses WASM fallback.`,
2586
+ );
2587
+ } else {
2588
+ logEvent("WebGPU is unavailable; the demo can fall back to WASM but latency will be higher.");
2589
+ }
2590
+ }
2591
+ }
index.html CHANGED
@@ -1,19 +1,242 @@
1
  <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  </html>
 
1
  <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
6
+ <title>Local Voice LLM</title>
7
+ <link rel="icon" href="data:," />
8
+ <link rel="stylesheet" href="./styles.css" />
9
+ </head>
10
+ <body>
11
+ <main class="shell">
12
+ <section class="topbar" aria-label="Session controls">
13
+ <div>
14
+ <h1>Local Voice LLM</h1>
15
+ <p>Client-side VAD, speech recognition, language model, and Supertonic speech synthesis.</p>
16
+ </div>
17
+ <div class="actions">
18
+ <button id="loadButton" type="button">
19
+ <span class="button-icon" aria-hidden="true">↓</span>
20
+ Load models
21
+ </button>
22
+ <button id="micButton" type="button" disabled>
23
+ <span class="button-icon" aria-hidden="true">●</span>
24
+ Start mic
25
+ </button>
26
+ <button id="stopButton" type="button" disabled title="Stop generation and audio playback">
27
+ <span class="button-icon" aria-hidden="true">■</span>
28
+ Stop
29
+ </button>
30
+ </div>
31
+ </section>
32
+
33
+ <section class="status-grid" aria-label="Model and pipeline state">
34
+ <article class="status-tile" data-state="idle" id="vadTile">
35
+ <span>VAD</span>
36
+ <strong id="vadState">Idle</strong>
37
+ </article>
38
+ <article class="status-tile" data-state="idle" id="asrTile">
39
+ <span>STT</span>
40
+ <strong id="asrState">Idle</strong>
41
+ </article>
42
+ <article class="status-tile" data-state="idle" id="llmTile">
43
+ <span>LLM</span>
44
+ <strong id="llmState">Idle</strong>
45
+ </article>
46
+ <article class="status-tile" data-state="idle" id="ttsTile">
47
+ <span>TTS</span>
48
+ <strong id="ttsState">Idle</strong>
49
+ </article>
50
+ </section>
51
+
52
+ <section class="workspace">
53
+ <div class="conversation" aria-label="Conversation">
54
+ <div class="transcript-panel">
55
+ <div class="panel-head">
56
+ <h2>Input</h2>
57
+ <span id="micBadge" class="badge">Mic off</span>
58
+ </div>
59
+ <div id="partialTranscript" class="partial">Waiting for speech.</div>
60
+ <div id="finalTranscript" class="final"></div>
61
+ </div>
62
+
63
+ <div class="response-panel">
64
+ <div class="panel-head">
65
+ <h2>Assistant</h2>
66
+ <span id="audioBadge" class="badge">Audio idle</span>
67
+ </div>
68
+ <div id="llmOutput" class="output">Load the models, start the microphone, and speak naturally.</div>
69
+ </div>
70
+ </div>
71
+
72
+ <aside class="side">
73
+ <section class="settings">
74
+ <h2>Runtime</h2>
75
+ <label>
76
+ Device
77
+ <select id="deviceSelect">
78
+ <option value="auto" selected>Auto</option>
79
+ <option value="webgpu">WebGPU</option>
80
+ <option value="wasm">WASM</option>
81
+ </select>
82
+ </label>
83
+ <div class="runtime-status" id="runtimeStatus" data-state="checking" aria-live="polite">
84
+ <span id="runtimeDeviceStatus">Checking runtime</span>
85
+ <small id="runtimeDeviceDetail">Adapter probe pending.</small>
86
+ </div>
87
+ <label>
88
+ LLM
89
+ <select id="llmModelSelect">
90
+ <option value="HuggingFaceTB/SmolLM2-135M-Instruct" selected>SmolLM2 135M Instruct</option>
91
+ <option value="onnx-community/SmolLM2-360M-Instruct-ONNX">SmolLM2 360M Instruct</option>
92
+ <option value="onnx-community/granite-4.0-350m-ONNX-web">Granite 4.0 350M (WebGPU)</option>
93
+ <option value="onnx-community/Qwen3-0.6B-ONNX">Qwen3 0.6B (WebGPU)</option>
94
+ <option value="HuggingFaceTB/SmolLM2-1.7B-Instruct">SmolLM2 1.7B (WebGPU)</option>
95
+ </select>
96
+ </label>
97
+ <label>
98
+ STT
99
+ <select id="asrModelSelect">
100
+ <option value="onnx-community/moonshine-base-ONNX" selected>Moonshine Base</option>
101
+ <option value="onnx-community/moonshine-tiny-ONNX">Moonshine Tiny</option>
102
+ <option value="onnx-community/whisper-tiny.en">Whisper Tiny English</option>
103
+ </select>
104
+ </label>
105
+ <label>
106
+ TTS voice
107
+ <select id="voiceSelect">
108
+ <option value="F1">F1</option>
109
+ <option value="F2" selected>F2</option>
110
+ <option value="M1">M1</option>
111
+ <option value="M2">M2</option>
112
+ </select>
113
+ </label>
114
+ <label>
115
+ <span class="range-label">
116
+ TTS steps
117
+ <output id="ttsStepsValue" for="ttsSteps">2</output>
118
+ </span>
119
+ <input id="ttsSteps" type="range" min="2" max="8" value="2" />
120
+ </label>
121
+ <label>
122
+ <span class="range-label">
123
+ VAD silence
124
+ <span><output id="vadSilenceValue" for="vadSilence">480</output> ms</span>
125
+ </span>
126
+ <input id="vadSilence" type="range" min="200" max="800" step="20" value="480" />
127
+ </label>
128
+ <label class="check">
129
+ <input id="partialToggle" type="checkbox" checked />
130
+ Partial ASR previews
131
+ </label>
132
+ </section>
133
+
134
+ <section class="metrics">
135
+ <h2>Latency</h2>
136
+ <dl>
137
+ <div>
138
+ <dt>VAD close delay</dt>
139
+ <dd id="vadCloseLatency">-</dd>
140
+ </div>
141
+ <div>
142
+ <dt>Speech end → transcript</dt>
143
+ <dd id="asrLatency">-</dd>
144
+ </div>
145
+ <div>
146
+ <dt>Transcript → first token</dt>
147
+ <dd id="firstTokenLatency">-</dd>
148
+ </div>
149
+ <div>
150
+ <dt>Transcript → TTS queued</dt>
151
+ <dd id="firstTtsQueuedLatency">-</dd>
152
+ </div>
153
+ <div>
154
+ <dt>First TTS synth</dt>
155
+ <dd id="ttsSynthLatency">-</dd>
156
+ </div>
157
+ <div>
158
+ <dt>Transcript → first audio</dt>
159
+ <dd id="firstAudioLatency">-</dd>
160
+ </div>
161
+ <div>
162
+ <dt>Speech end → first audio</dt>
163
+ <dd id="speechToAudioLatency">-</dd>
164
+ </div>
165
+ <div>
166
+ <dt>LLM decode</dt>
167
+ <dd id="decodeRate">-</dd>
168
+ </div>
169
+ </dl>
170
+ <div id="micValidationCard" class="mic-validation" data-state="idle" aria-live="polite">
171
+ <span>Real mic validation</span>
172
+ <strong id="micValidationStatus">Load models to collect 3 rows</strong>
173
+ <small id="micValidationDetail">Use the mic series and say: "What app is this?"</small>
174
+ <div class="mic-progress" aria-hidden="true">
175
+ <span id="micValidationProgressBar"></span>
176
+ </div>
177
+ </div>
178
+ <button id="suiteButton" type="button" disabled>Run benchmark suite</button>
179
+ <button id="benchmarkButton" type="button" disabled>Run identity benchmark</button>
180
+ <button id="chatBenchmarkButton" type="button" disabled>Run chat benchmark</button>
181
+ <button id="ttsBenchmarkButton" type="button" disabled>Run TTS benchmark</button>
182
+ <button id="loopbackButton" type="button" disabled>Run voice loopback</button>
183
+ <button id="bargeInButton" type="button" disabled>Run barge-in check</button>
184
+ <button id="micBenchmarkButton" type="button" disabled>Benchmark real mic</button>
185
+ <button id="micSeriesButton" type="button" disabled>Run 3 real-mic series</button>
186
+ </section>
187
+ </aside>
188
+ </section>
189
+
190
+ <section class="events" aria-label="Event log">
191
+ <div class="panel-head">
192
+ <h2>Events</h2>
193
+ <button id="clearLogButton" type="button">Clear</button>
194
+ </div>
195
+ <ol id="eventLog"></ol>
196
+ </section>
197
+
198
+ <section class="bench-results" aria-label="Benchmark results">
199
+ <div class="panel-head">
200
+ <h2>Benchmarks</h2>
201
+ <div class="mini-actions">
202
+ <button id="copyResultsButton" type="button" disabled>Copy JSON</button>
203
+ <button id="clearResultsButton" type="button" disabled>Clear</button>
204
+ </div>
205
+ </div>
206
+ <div id="benchmarkSummary" class="benchmark-summary" aria-live="polite">
207
+ No benchmark runs yet.
208
+ </div>
209
+ <div class="table-wrap">
210
+ <table>
211
+ <thead>
212
+ <tr>
213
+ <th>Run</th>
214
+ <th>Stack</th>
215
+ <th>ASR</th>
216
+ <th>STT WER</th>
217
+ <th>VAD close</th>
218
+ <th>1st token</th>
219
+ <th>TTS queued</th>
220
+ <th>TTS synth</th>
221
+ <th>1st audio</th>
222
+ <th>End → audio</th>
223
+ <th>Audio done</th>
224
+ <th>Decode</th>
225
+ <th>LLM OK</th>
226
+ <th>Transcript</th>
227
+ <th>Output</th>
228
+ </tr>
229
+ </thead>
230
+ <tbody id="resultsBody">
231
+ <tr>
232
+ <td colspan="15">No benchmark runs yet.</td>
233
+ </tr>
234
+ </tbody>
235
+ </table>
236
+ </div>
237
+ </section>
238
+ </main>
239
+
240
+ <script type="module" src="./app.js"></script>
241
+ </body>
242
  </html>
styles.css ADDED
@@ -0,0 +1,590 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ :root {
2
+ color-scheme: dark;
3
+ --bg: #101114;
4
+ --panel: #191c20;
5
+ --panel-2: #20242a;
6
+ --line: #313741;
7
+ --text: #f4f5f7;
8
+ --muted: #aab2bd;
9
+ --green: #45c486;
10
+ --yellow: #f4bd4f;
11
+ --red: #ee6b6e;
12
+ --blue: #70a7ff;
13
+ --accent: #f1d270;
14
+ font-family:
15
+ Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
16
+ }
17
+
18
+ * {
19
+ box-sizing: border-box;
20
+ }
21
+
22
+ body {
23
+ margin: 0;
24
+ min-width: 320px;
25
+ background:
26
+ linear-gradient(180deg, rgba(255, 255, 255, 0.035), transparent 280px),
27
+ var(--bg);
28
+ color: var(--text);
29
+ }
30
+
31
+ button,
32
+ select,
33
+ input {
34
+ font: inherit;
35
+ }
36
+
37
+ button,
38
+ select {
39
+ border: 1px solid var(--line);
40
+ border-radius: 8px;
41
+ background: #252a31;
42
+ color: var(--text);
43
+ }
44
+
45
+ button {
46
+ min-height: 40px;
47
+ padding: 0 14px;
48
+ cursor: pointer;
49
+ display: inline-flex;
50
+ align-items: center;
51
+ justify-content: center;
52
+ gap: 8px;
53
+ }
54
+
55
+ button:hover:not(:disabled) {
56
+ border-color: #5d6877;
57
+ background: #2d343d;
58
+ }
59
+
60
+ button:disabled {
61
+ cursor: not-allowed;
62
+ opacity: 0.5;
63
+ }
64
+
65
+ select {
66
+ width: 100%;
67
+ min-height: 38px;
68
+ padding: 0 10px;
69
+ }
70
+
71
+ .shell {
72
+ width: min(1440px, 100%);
73
+ margin: 0 auto;
74
+ padding: 24px;
75
+ }
76
+
77
+ .topbar {
78
+ display: flex;
79
+ align-items: flex-start;
80
+ justify-content: space-between;
81
+ gap: 20px;
82
+ padding-bottom: 20px;
83
+ }
84
+
85
+ h1,
86
+ h2,
87
+ p {
88
+ margin: 0;
89
+ }
90
+
91
+ h1 {
92
+ font-size: clamp(30px, 4vw, 56px);
93
+ line-height: 0.95;
94
+ letter-spacing: 0;
95
+ }
96
+
97
+ h2 {
98
+ font-size: 14px;
99
+ text-transform: uppercase;
100
+ letter-spacing: 0;
101
+ color: var(--muted);
102
+ }
103
+
104
+ .topbar p {
105
+ max-width: 760px;
106
+ margin-top: 10px;
107
+ color: var(--muted);
108
+ line-height: 1.5;
109
+ }
110
+
111
+ .actions {
112
+ display: flex;
113
+ flex-wrap: wrap;
114
+ justify-content: flex-end;
115
+ gap: 10px;
116
+ }
117
+
118
+ .button-icon {
119
+ width: 18px;
120
+ text-align: center;
121
+ color: var(--accent);
122
+ }
123
+
124
+ .status-grid {
125
+ display: grid;
126
+ grid-template-columns: repeat(4, minmax(0, 1fr));
127
+ gap: 10px;
128
+ margin-bottom: 14px;
129
+ }
130
+
131
+ .status-tile {
132
+ border: 1px solid var(--line);
133
+ border-radius: 8px;
134
+ background: var(--panel);
135
+ min-height: 76px;
136
+ padding: 14px;
137
+ }
138
+
139
+ .status-tile span {
140
+ display: block;
141
+ color: var(--muted);
142
+ font-size: 12px;
143
+ margin-bottom: 8px;
144
+ }
145
+
146
+ .status-tile strong {
147
+ font-size: 16px;
148
+ font-weight: 650;
149
+ }
150
+
151
+ .status-tile[data-state="ready"] {
152
+ border-color: color-mix(in srgb, var(--green), var(--line) 50%);
153
+ }
154
+
155
+ .status-tile[data-state="active"] {
156
+ border-color: color-mix(in srgb, var(--blue), var(--line) 30%);
157
+ background: #1b2430;
158
+ }
159
+
160
+ .status-tile[data-state="warn"] {
161
+ border-color: color-mix(in srgb, var(--yellow), var(--line) 35%);
162
+ }
163
+
164
+ .status-tile[data-state="error"] {
165
+ border-color: color-mix(in srgb, var(--red), var(--line) 35%);
166
+ }
167
+
168
+ .workspace {
169
+ display: grid;
170
+ grid-template-columns: minmax(0, 1fr) 360px;
171
+ gap: 14px;
172
+ }
173
+
174
+ .conversation {
175
+ display: grid;
176
+ gap: 14px;
177
+ min-width: 0;
178
+ }
179
+
180
+ .transcript-panel,
181
+ .response-panel,
182
+ .settings,
183
+ .metrics,
184
+ .events,
185
+ .bench-results {
186
+ border: 1px solid var(--line);
187
+ border-radius: 8px;
188
+ background: var(--panel);
189
+ }
190
+
191
+ .transcript-panel,
192
+ .response-panel {
193
+ min-height: 280px;
194
+ padding: 16px;
195
+ }
196
+
197
+ .panel-head {
198
+ display: flex;
199
+ align-items: center;
200
+ justify-content: space-between;
201
+ gap: 12px;
202
+ margin-bottom: 14px;
203
+ }
204
+
205
+ .badge {
206
+ min-width: 82px;
207
+ text-align: center;
208
+ border: 1px solid var(--line);
209
+ border-radius: 999px;
210
+ padding: 5px 10px;
211
+ color: var(--muted);
212
+ font-size: 12px;
213
+ white-space: nowrap;
214
+ }
215
+
216
+ .badge.active {
217
+ color: var(--text);
218
+ border-color: color-mix(in srgb, var(--green), var(--line) 40%);
219
+ }
220
+
221
+ .partial {
222
+ min-height: 86px;
223
+ color: var(--muted);
224
+ line-height: 1.55;
225
+ padding-bottom: 14px;
226
+ border-bottom: 1px solid var(--line);
227
+ }
228
+
229
+ .final,
230
+ .output {
231
+ margin-top: 14px;
232
+ font-size: clamp(18px, 2vw, 25px);
233
+ line-height: 1.45;
234
+ white-space: pre-wrap;
235
+ overflow-wrap: anywhere;
236
+ }
237
+
238
+ .output {
239
+ min-height: 190px;
240
+ color: #f8f1dc;
241
+ }
242
+
243
+ .side {
244
+ display: grid;
245
+ gap: 14px;
246
+ align-content: start;
247
+ }
248
+
249
+ .settings,
250
+ .metrics,
251
+ .events,
252
+ .bench-results {
253
+ padding: 16px;
254
+ }
255
+
256
+ .settings {
257
+ display: grid;
258
+ gap: 12px;
259
+ }
260
+
261
+ .settings label {
262
+ display: grid;
263
+ gap: 7px;
264
+ min-width: 0;
265
+ color: var(--muted);
266
+ font-size: 13px;
267
+ }
268
+
269
+ .settings input[type="range"] {
270
+ width: 100%;
271
+ max-width: 100%;
272
+ min-width: 0;
273
+ }
274
+
275
+ .runtime-status {
276
+ display: grid;
277
+ gap: 3px;
278
+ min-height: 44px;
279
+ border-top: 1px solid var(--line);
280
+ border-bottom: 1px solid var(--line);
281
+ padding: 10px 0;
282
+ color: var(--muted);
283
+ font-size: 13px;
284
+ overflow-wrap: anywhere;
285
+ }
286
+
287
+ .runtime-status span {
288
+ color: var(--text);
289
+ font-weight: 650;
290
+ }
291
+
292
+ .runtime-status small {
293
+ color: var(--muted);
294
+ line-height: 1.35;
295
+ }
296
+
297
+ .runtime-status[data-state="ready"] span {
298
+ color: var(--green);
299
+ }
300
+
301
+ .runtime-status[data-state="warn"] span {
302
+ color: var(--yellow);
303
+ }
304
+
305
+ .runtime-status[data-state="error"] span {
306
+ color: var(--red);
307
+ }
308
+
309
+ .range-label {
310
+ display: flex;
311
+ align-items: center;
312
+ justify-content: space-between;
313
+ gap: 12px;
314
+ }
315
+
316
+ .range-label span {
317
+ color: var(--text);
318
+ font-variant-numeric: tabular-nums;
319
+ }
320
+
321
+ .settings .check {
322
+ grid-template-columns: auto 1fr;
323
+ align-items: center;
324
+ color: var(--text);
325
+ }
326
+
327
+ .metrics dl {
328
+ margin: 0;
329
+ display: grid;
330
+ gap: 10px;
331
+ }
332
+
333
+ .metrics dl div {
334
+ display: flex;
335
+ justify-content: space-between;
336
+ gap: 16px;
337
+ border-bottom: 1px solid var(--line);
338
+ padding-bottom: 9px;
339
+ }
340
+
341
+ .metrics dt {
342
+ color: var(--muted);
343
+ }
344
+
345
+ .metrics dd {
346
+ margin: 0;
347
+ font-variant-numeric: tabular-nums;
348
+ }
349
+
350
+ .mic-validation {
351
+ display: grid;
352
+ gap: 5px;
353
+ margin-top: 14px;
354
+ border: 1px solid var(--line);
355
+ border-radius: 8px;
356
+ background: var(--panel-2);
357
+ padding: 12px;
358
+ overflow-wrap: anywhere;
359
+ }
360
+
361
+ .mic-validation span {
362
+ color: var(--muted);
363
+ font-size: 12px;
364
+ text-transform: uppercase;
365
+ }
366
+
367
+ .mic-validation strong {
368
+ color: var(--text);
369
+ font-size: 15px;
370
+ font-weight: 650;
371
+ }
372
+
373
+ .mic-validation small {
374
+ color: var(--muted);
375
+ line-height: 1.35;
376
+ }
377
+
378
+ .mic-validation[data-state="active"] {
379
+ border-color: color-mix(in srgb, var(--blue), var(--line) 35%);
380
+ }
381
+
382
+ .mic-validation[data-state="active"] strong {
383
+ color: var(--blue);
384
+ }
385
+
386
+ .mic-validation[data-state="ready"] {
387
+ border-color: color-mix(in srgb, var(--green), var(--line) 45%);
388
+ }
389
+
390
+ .mic-validation[data-state="ready"] strong {
391
+ color: var(--green);
392
+ }
393
+
394
+ .mic-validation[data-state="warn"] {
395
+ border-color: color-mix(in srgb, var(--yellow), var(--line) 40%);
396
+ }
397
+
398
+ .mic-validation[data-state="warn"] strong {
399
+ color: var(--yellow);
400
+ }
401
+
402
+ .mic-progress {
403
+ height: 6px;
404
+ overflow: hidden;
405
+ border-radius: 999px;
406
+ background: #11151a;
407
+ }
408
+
409
+ .mic-progress span {
410
+ display: block;
411
+ width: 0%;
412
+ height: 100%;
413
+ background: var(--green);
414
+ transition: width 160ms ease;
415
+ }
416
+
417
+ #suiteButton,
418
+ #benchmarkButton,
419
+ #ttsBenchmarkButton,
420
+ #loopbackButton,
421
+ #bargeInButton,
422
+ #micBenchmarkButton,
423
+ #micSeriesButton {
424
+ width: 100%;
425
+ margin-top: 14px;
426
+ }
427
+
428
+ #benchmarkButton,
429
+ #ttsBenchmarkButton,
430
+ #loopbackButton,
431
+ #bargeInButton,
432
+ #micBenchmarkButton,
433
+ #micSeriesButton {
434
+ margin-top: 8px;
435
+ }
436
+
437
+ .events {
438
+ margin-top: 14px;
439
+ }
440
+
441
+ .events .panel-head button,
442
+ .mini-actions button {
443
+ min-height: 30px;
444
+ padding: 0 10px;
445
+ font-size: 13px;
446
+ }
447
+
448
+ .bench-results {
449
+ margin-top: 14px;
450
+ }
451
+
452
+ .mini-actions {
453
+ display: flex;
454
+ gap: 8px;
455
+ }
456
+
457
+ .benchmark-summary {
458
+ display: grid;
459
+ grid-template-columns: repeat(6, minmax(0, 1fr));
460
+ gap: 8px;
461
+ margin-bottom: 12px;
462
+ color: var(--muted);
463
+ font-size: 13px;
464
+ }
465
+
466
+ .summary-item {
467
+ min-height: 58px;
468
+ border: 1px solid var(--line);
469
+ border-radius: 8px;
470
+ background: var(--panel-2);
471
+ padding: 10px;
472
+ }
473
+
474
+ .summary-item span,
475
+ .summary-item strong {
476
+ display: block;
477
+ }
478
+
479
+ .summary-item span {
480
+ margin-bottom: 6px;
481
+ }
482
+
483
+ .summary-item strong {
484
+ color: var(--text);
485
+ font-size: 15px;
486
+ font-weight: 650;
487
+ font-variant-numeric: tabular-nums;
488
+ }
489
+
490
+ .table-wrap {
491
+ overflow-x: auto;
492
+ }
493
+
494
+ table {
495
+ width: 100%;
496
+ border-collapse: collapse;
497
+ min-width: 1500px;
498
+ font-size: 13px;
499
+ }
500
+
501
+ th,
502
+ td {
503
+ border-bottom: 1px solid var(--line);
504
+ padding: 9px 10px;
505
+ text-align: left;
506
+ vertical-align: top;
507
+ }
508
+
509
+ th {
510
+ color: var(--muted);
511
+ font-weight: 650;
512
+ text-transform: uppercase;
513
+ letter-spacing: 0;
514
+ white-space: nowrap;
515
+ }
516
+
517
+ td {
518
+ color: #dce3ec;
519
+ }
520
+
521
+ td:nth-child(13),
522
+ td:nth-child(14) {
523
+ max-width: 260px;
524
+ overflow-wrap: anywhere;
525
+ }
526
+
527
+ #eventLog {
528
+ height: 170px;
529
+ margin: 0;
530
+ padding-left: 22px;
531
+ overflow: auto;
532
+ color: var(--muted);
533
+ line-height: 1.45;
534
+ }
535
+
536
+ #eventLog li {
537
+ padding: 3px 0;
538
+ }
539
+
540
+ @media (max-width: 980px) {
541
+ .shell {
542
+ padding: 16px;
543
+ }
544
+
545
+ .topbar,
546
+ .workspace {
547
+ grid-template-columns: 1fr;
548
+ display: grid;
549
+ }
550
+
551
+ .actions {
552
+ justify-content: stretch;
553
+ }
554
+
555
+ .actions button {
556
+ flex: 1 1 150px;
557
+ }
558
+
559
+ .status-grid {
560
+ grid-template-columns: repeat(2, minmax(0, 1fr));
561
+ }
562
+
563
+ .benchmark-summary {
564
+ grid-template-columns: repeat(3, minmax(0, 1fr));
565
+ }
566
+ }
567
+
568
+ @media (max-width: 560px) {
569
+ .actions {
570
+ display: grid;
571
+ grid-template-columns: 1fr;
572
+ }
573
+
574
+ .actions button {
575
+ width: 100%;
576
+ }
577
+
578
+ .status-grid {
579
+ grid-template-columns: 1fr;
580
+ }
581
+
582
+ .benchmark-summary {
583
+ grid-template-columns: 1fr;
584
+ }
585
+
586
+ .transcript-panel,
587
+ .response-panel {
588
+ min-height: 230px;
589
+ }
590
+ }
workers/asr-worker.js ADDED
@@ -0,0 +1,277 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { AutoModel, Tensor, env, pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.2.0";
2
+
3
+ env.allowLocalModels = false;
4
+ env.useBrowserCache = true;
5
+
6
+ const SAMPLE_RATE = 16000;
7
+ const SPEECH_THRESHOLD = 0.3;
8
+ const EXIT_THRESHOLD = 0.1;
9
+ const DEFAULT_SILENCE_DURATION_MS = 480;
10
+ const MIN_SILENCE_DURATION_MS = 200;
11
+ const MAX_SILENCE_DURATION_MS = 800;
12
+ const SPEECH_PAD_SAMPLES = 80 * (SAMPLE_RATE / 1000);
13
+ const MIN_SPEECH_DURATION_SAMPLES = 250 * (SAMPLE_RATE / 1000);
14
+ const MAX_BUFFER_DURATION = 30;
15
+ const NEW_BUFFER_SIZE = 512;
16
+ const MAX_NUM_PREV_BUFFERS = Math.ceil(SPEECH_PAD_SAMPLES / NEW_BUFFER_SIZE);
17
+ const PARTIAL_INTERVAL_MS = 1600;
18
+
19
+ let vadModel = null;
20
+ let transcriber = null;
21
+ let device = "wasm";
22
+ let inputQueue = new Float32Array(0);
23
+ let vadChain = Promise.resolve();
24
+ let asrChain = Promise.resolve();
25
+ let vadState = new Tensor("float32", new Float32Array(2 * 1 * 128), [2, 1, 128]);
26
+ let srTensor = new Tensor("int64", [SAMPLE_RATE], []);
27
+ let isRecording = false;
28
+ let bufferPointer = 0;
29
+ let postSpeechSamples = 0;
30
+ let previousBuffers = [];
31
+ let partialEnabled = true;
32
+ let minSilenceDurationSamples = silenceDurationSamples(DEFAULT_SILENCE_DURATION_MS);
33
+ let partialBusy = false;
34
+ let lastPartialAt = 0;
35
+ let utteranceStartedAt = 0;
36
+ const recordingBuffer = new Float32Array(MAX_BUFFER_DURATION * SAMPLE_RATE);
37
+
38
+ self.onmessage = async (event) => {
39
+ const message = event.data;
40
+ try {
41
+ if (message.type === "load") {
42
+ await load(message);
43
+ } else if (message.type === "configure") {
44
+ configure(message);
45
+ } else if (message.type === "audio") {
46
+ ingestAudio(message.buffer, message.sampleRate);
47
+ } else if (message.type === "flush") {
48
+ await flushRecording();
49
+ }
50
+ } catch (error) {
51
+ self.postMessage({ type: "error", message: error.message ?? String(error) });
52
+ }
53
+ };
54
+
55
+ async function load({ model, device: requestedDevice, partial, silenceMs }) {
56
+ device = requestedDevice;
57
+ configure({ partial, silenceMs });
58
+ self.postMessage({ type: "status", scope: "vad", message: "Loading", mode: "warn" });
59
+ vadModel = await AutoModel.from_pretrained("onnx-community/silero-vad", {
60
+ config: { model_type: "custom" },
61
+ dtype: "fp32",
62
+ progress_callback: reportProgress("VAD"),
63
+ });
64
+
65
+ self.postMessage({ type: "status", message: "Loading", mode: "warn" });
66
+ const dtype =
67
+ model.includes("moonshine")
68
+ ? {
69
+ encoder_model: device === "webgpu" ? "fp32" : "fp32",
70
+ decoder_model_merged: "q4",
71
+ }
72
+ : device === "webgpu"
73
+ ? {
74
+ encoder_model: "fp32",
75
+ decoder_model_merged: "q4",
76
+ }
77
+ : {
78
+ encoder_model: "fp32",
79
+ decoder_model_merged: "q4",
80
+ };
81
+
82
+ transcriber = await pipeline("automatic-speech-recognition", model, {
83
+ device,
84
+ dtype,
85
+ progress_callback: reportProgress("STT"),
86
+ });
87
+
88
+ self.postMessage({ type: "status", message: "Warming", mode: "warn" });
89
+ await transcribeBuffer(new Float32Array(SAMPLE_RATE), { warmup: true });
90
+ self.postMessage({ type: "ready" });
91
+ }
92
+
93
+ function reportProgress(label) {
94
+ return (progress) => {
95
+ if (progress.status === "progress") {
96
+ const pct = Number.isFinite(progress.progress) ? ` ${progress.progress.toFixed(0)}%` : "";
97
+ self.postMessage({ type: "status", message: `${label}${pct}`, mode: "warn" });
98
+ }
99
+ };
100
+ }
101
+
102
+ function ingestAudio(buffer, sourceRate) {
103
+ const resampled = resampleTo16k(buffer, sourceRate);
104
+ inputQueue = concat(inputQueue, resampled);
105
+ while (inputQueue.length >= NEW_BUFFER_SIZE) {
106
+ const chunk = inputQueue.slice(0, NEW_BUFFER_SIZE);
107
+ inputQueue = inputQueue.slice(NEW_BUFFER_SIZE);
108
+ void handleVadChunk(chunk);
109
+ }
110
+ }
111
+
112
+ async function handleVadChunk(buffer) {
113
+ const wasRecording = isRecording;
114
+ const speech = await vad(buffer);
115
+
116
+ if (!wasRecording && !speech) {
117
+ if (previousBuffers.length >= MAX_NUM_PREV_BUFFERS) previousBuffers.shift();
118
+ previousBuffers.push(buffer);
119
+ return;
120
+ }
121
+
122
+ const remaining = recordingBuffer.length - bufferPointer;
123
+ if (buffer.length >= remaining) {
124
+ recordingBuffer.set(buffer.subarray(0, remaining), bufferPointer);
125
+ bufferPointer += remaining;
126
+ dispatchForTranscription(buffer.subarray(remaining));
127
+ return;
128
+ }
129
+
130
+ recordingBuffer.set(buffer, bufferPointer);
131
+ bufferPointer += buffer.length;
132
+
133
+ if (speech) {
134
+ if (!isRecording) {
135
+ utteranceStartedAt = performance.now();
136
+ self.postMessage({ type: "speechstart" });
137
+ }
138
+ isRecording = true;
139
+ postSpeechSamples = 0;
140
+ maybePartial();
141
+ return;
142
+ }
143
+
144
+ postSpeechSamples += buffer.length;
145
+ if (postSpeechSamples < minSilenceDurationSamples) return;
146
+
147
+ if (bufferPointer < MIN_SPEECH_DURATION_SAMPLES) {
148
+ reset();
149
+ return;
150
+ }
151
+
152
+ self.postMessage({
153
+ type: "speechend",
154
+ trailingSilenceMs: sampleDurationMs(postSpeechSamples),
155
+ });
156
+ dispatchForTranscription();
157
+ }
158
+
159
+ async function vad(buffer) {
160
+ const input = new Tensor("float32", buffer, [1, buffer.length]);
161
+ const result = await (vadChain = vadChain.then(() =>
162
+ vadModel({ input, sr: srTensor, state: vadState }),
163
+ ));
164
+ vadState = result.stateN;
165
+ const probability = result.output.data[0];
166
+ return probability > SPEECH_THRESHOLD || (isRecording && probability >= EXIT_THRESHOLD);
167
+ }
168
+
169
+ function maybePartial() {
170
+ if (!partialEnabled || partialBusy) return;
171
+ const now = performance.now();
172
+ if (now - lastPartialAt < PARTIAL_INTERVAL_MS || bufferPointer < SAMPLE_RATE) return;
173
+ partialBusy = true;
174
+ lastPartialAt = now;
175
+ const buffer = paddedRecordingBuffer();
176
+ transcribeBuffer(buffer, { partial: true })
177
+ .then((text) => {
178
+ if (text.trim()) self.postMessage({ type: "partial", text });
179
+ })
180
+ .finally(() => {
181
+ partialBusy = false;
182
+ });
183
+ }
184
+
185
+ function dispatchForTranscription(overflow) {
186
+ const buffer = paddedRecordingBuffer();
187
+ transcribeBuffer(buffer, { partial: false }).then((text) => {
188
+ self.postMessage({
189
+ type: "transcript",
190
+ text,
191
+ durationMs: performance.now() - utteranceStartedAt,
192
+ });
193
+ });
194
+
195
+ if (overflow?.length) {
196
+ recordingBuffer.set(overflow, 0);
197
+ }
198
+ reset(overflow?.length ?? 0);
199
+ }
200
+
201
+ async function flushRecording() {
202
+ await (vadChain = vadChain.then(() => Promise.resolve()));
203
+ if (!isRecording || bufferPointer < MIN_SPEECH_DURATION_SAMPLES) return;
204
+ self.postMessage({
205
+ type: "speechend",
206
+ trailingSilenceMs: sampleDurationMs(postSpeechSamples),
207
+ forced: true,
208
+ });
209
+ dispatchForTranscription();
210
+ }
211
+
212
+ function paddedRecordingBuffer() {
213
+ const current = recordingBuffer.slice(0, Math.min(bufferPointer + SPEECH_PAD_SAMPLES, recordingBuffer.length));
214
+ const prevLength = previousBuffers.reduce((sum, item) => sum + item.length, 0);
215
+ const padded = new Float32Array(prevLength + current.length);
216
+ let offset = 0;
217
+ for (const prev of previousBuffers) {
218
+ padded.set(prev, offset);
219
+ offset += prev.length;
220
+ }
221
+ padded.set(current, offset);
222
+ return padded;
223
+ }
224
+
225
+ async function transcribeBuffer(buffer, { warmup = false } = {}) {
226
+ const output = await (asrChain = asrChain.then(() => transcriber(buffer)));
227
+ if (warmup) return "";
228
+ return output.text ?? "";
229
+ }
230
+
231
+ function reset(offset = 0) {
232
+ recordingBuffer.fill(0, offset);
233
+ bufferPointer = offset;
234
+ isRecording = false;
235
+ postSpeechSamples = 0;
236
+ previousBuffers = [];
237
+ lastPartialAt = 0;
238
+ }
239
+
240
+ function configure({ partial, silenceMs } = {}) {
241
+ if (typeof partial === "boolean") partialEnabled = partial;
242
+ if (silenceMs != null) minSilenceDurationSamples = silenceDurationSamples(silenceMs);
243
+ }
244
+
245
+ function silenceDurationSamples(value) {
246
+ const numericValue = Number(value);
247
+ const ms = Number.isFinite(numericValue) ? numericValue : DEFAULT_SILENCE_DURATION_MS;
248
+ const clampedMs = Math.min(MAX_SILENCE_DURATION_MS, Math.max(MIN_SILENCE_DURATION_MS, ms));
249
+ return Math.round(clampedMs * (SAMPLE_RATE / 1000));
250
+ }
251
+
252
+ function sampleDurationMs(samples) {
253
+ return (samples / SAMPLE_RATE) * 1000;
254
+ }
255
+
256
+ function resampleTo16k(input, sourceRate) {
257
+ if (sourceRate === SAMPLE_RATE) return input;
258
+ const ratio = sourceRate / SAMPLE_RATE;
259
+ const length = Math.floor(input.length / ratio);
260
+ const output = new Float32Array(length);
261
+ for (let i = 0; i < length; i += 1) {
262
+ const position = i * ratio;
263
+ const left = Math.floor(position);
264
+ const right = Math.min(left + 1, input.length - 1);
265
+ const weight = position - left;
266
+ output[i] = input[left] * (1 - weight) + input[right] * weight;
267
+ }
268
+ return output;
269
+ }
270
+
271
+ function concat(left, right) {
272
+ if (left.length === 0) return right;
273
+ const out = new Float32Array(left.length + right.length);
274
+ out.set(left, 0);
275
+ out.set(right, left.length);
276
+ return out;
277
+ }
workers/llm-worker.js ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import {
2
+ AutoModelForCausalLM,
3
+ AutoTokenizer,
4
+ InterruptableStoppingCriteria,
5
+ TextStreamer,
6
+ env,
7
+ } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.2.0";
8
+
9
+ env.allowLocalModels = false;
10
+ env.useBrowserCache = true;
11
+
12
+ let tokenizer = null;
13
+ let model = null;
14
+ let modelId = "";
15
+ let stoppingCriteria = new InterruptableStoppingCriteria();
16
+ let currentTurnId = 0;
17
+
18
+ self.onmessage = async (event) => {
19
+ const message = event.data;
20
+ try {
21
+ if (message.type === "load") {
22
+ await load(message);
23
+ } else if (message.type === "generate") {
24
+ currentTurnId = message.turnId;
25
+ await generate(message.messages, message.turnId, {
26
+ sentenceLimit: message.sentenceLimit ?? (message.stopAfterFirstSentence === false ? 0 : 1),
27
+ });
28
+ } else if (message.type === "interrupt") {
29
+ stoppingCriteria.interrupt();
30
+ }
31
+ } catch (error) {
32
+ self.postMessage({ type: "error", message: error.message ?? String(error) });
33
+ }
34
+ };
35
+
36
+ async function load({ model: requestedModelId, device }) {
37
+ modelId = requestedModelId;
38
+ self.postMessage({ type: "status", message: "Loading", mode: "warn" });
39
+ tokenizer = await AutoTokenizer.from_pretrained(modelId, {
40
+ progress_callback: reportProgress("LLM tokenizer"),
41
+ });
42
+ model = await AutoModelForCausalLM.from_pretrained(modelId, {
43
+ device,
44
+ dtype: device === "webgpu" ? "q4f16" : "q4",
45
+ progress_callback: reportProgress("LLM"),
46
+ });
47
+ self.postMessage({ type: "status", message: "Warming", mode: "warn" });
48
+ const inputs = tokenizer("hello");
49
+ await model.generate({ ...inputs, max_new_tokens: 1 });
50
+ self.postMessage({ type: "ready" });
51
+ }
52
+
53
+ function reportProgress(label) {
54
+ return (progress) => {
55
+ if (progress.status === "progress") {
56
+ const pct = Number.isFinite(progress.progress) ? ` ${progress.progress.toFixed(0)}%` : "";
57
+ self.postMessage({ type: "status", message: `${label}${pct}`, mode: "warn" });
58
+ }
59
+ };
60
+ }
61
+
62
+ async function generate(messages, turnId, { sentenceLimit = 1 } = {}) {
63
+ stoppingCriteria = new InterruptableStoppingCriteria();
64
+ const promptMessages = normalizeMessages(messages);
65
+ const inputs = tokenizer.apply_chat_template(promptMessages, {
66
+ add_generation_prompt: true,
67
+ enable_thinking: false,
68
+ return_dict: true,
69
+ });
70
+
71
+ let firstTokenAt = 0;
72
+ let tokenCount = 0;
73
+ let tps = 0;
74
+ let decodedText = "";
75
+ let emittedChars = 0;
76
+ let sentenceStopped = false;
77
+ const startedAt = performance.now();
78
+
79
+ const streamer = new TextStreamer(tokenizer, {
80
+ skip_prompt: true,
81
+ skip_special_tokens: true,
82
+ token_callback_function: () => {
83
+ tokenCount += 1;
84
+ firstTokenAt ||= performance.now();
85
+ const elapsed = performance.now() - startedAt;
86
+ if (elapsed > 0) tps = (tokenCount / elapsed) * 1000;
87
+ },
88
+ callback_function: (text) => {
89
+ if (turnId !== currentTurnId || sentenceStopped) return;
90
+ decodedText += text;
91
+ const boundary = sentenceLimit > 0 ? sentenceBoundary(decodedText, sentenceLimit) : -1;
92
+ const emitUntil = boundary > 0 ? boundary : decodedText.length;
93
+ const emitText = decodedText.slice(emittedChars, emitUntil);
94
+ emittedChars = emitUntil;
95
+ if (emitText) {
96
+ self.postMessage({ type: "token", turnId, text: emitText, tps, tokenCount });
97
+ }
98
+ if (boundary > 0) {
99
+ sentenceStopped = true;
100
+ stoppingCriteria.interrupt();
101
+ }
102
+ },
103
+ });
104
+
105
+ self.postMessage({ type: "start", turnId });
106
+ await model.generate({
107
+ ...inputs,
108
+ max_new_tokens: maxNewTokens(),
109
+ do_sample: false,
110
+ repetition_penalty: 1.08,
111
+ streamer,
112
+ stopping_criteria: stoppingCriteria,
113
+ });
114
+ self.postMessage({ type: "complete", turnId, tokenCount, firstTokenMs: firstTokenAt - startedAt });
115
+ }
116
+
117
+ function maxNewTokens() {
118
+ if (modelId.includes("Qwen3")) return 40;
119
+ if (modelId.includes("135M")) return 64;
120
+ return 48;
121
+ }
122
+
123
+ function sentenceBoundary(text, targetCount = 1) {
124
+ const normalized = text.replace(/\s+/g, " ").trim();
125
+ if (normalized.length < 12) return -1;
126
+ const regex = /[.!?]["')\]]?(?:\s|$)/g;
127
+ let match = null;
128
+ let count = 0;
129
+ while ((match = regex.exec(text))) {
130
+ count += 1;
131
+ if (count >= targetCount) return match.index + match[0].trimEnd().length;
132
+ }
133
+ return -1;
134
+ }
135
+
136
+ function normalizeMessages(messages) {
137
+ if (typeof tokenizer.apply_chat_template === "function") return messages;
138
+ const content = messages.map((message) => `${message.role}: ${message.content}`).join("\n");
139
+ return [{ role: "user", content }];
140
+ }
workers/tts-worker.js ADDED
@@ -0,0 +1,192 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { env, pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.2.0";
2
+
3
+ env.allowLocalModels = false;
4
+ env.useBrowserCache = true;
5
+
6
+ let synthesizer = null;
7
+ let modelId = "";
8
+ let cancelEpoch = 0;
9
+ let queue = Promise.resolve();
10
+ const voiceEmbeddings = new Map();
11
+ const voicePromises = new Map();
12
+ const VOICES = ["F1", "F2", "M1", "M2"];
13
+
14
+ self.onmessage = (event) => {
15
+ const message = event.data;
16
+ if (message.type === "load") {
17
+ queue = queue.then(() => load(message)).catch(postError);
18
+ } else if (message.type === "preload-voice") {
19
+ queue = queue.then(() => preloadVoice(message)).catch(postError);
20
+ } else if (message.type === "speak") {
21
+ const item = { ...message, cancelEpoch };
22
+ queue = queue.then(() => speak(item)).catch(postError);
23
+ } else if (message.type === "synthesize") {
24
+ const item = { ...message, cancelEpoch };
25
+ queue = queue.then(() => synthesize(item)).catch(postError);
26
+ } else if (message.type === "cancel") {
27
+ cancelEpoch += 1;
28
+ }
29
+ };
30
+
31
+ async function load({ model, device, voice = "F2", warmup = true }) {
32
+ modelId = model;
33
+ voiceEmbeddings.clear();
34
+ voicePromises.clear();
35
+ self.postMessage({
36
+ type: "status",
37
+ message: "Loading",
38
+ mode: "warn",
39
+ detail: "Loading Supertonic pipeline.",
40
+ });
41
+ synthesizer = await pipeline("text-to-speech", model, {
42
+ device,
43
+ dtype: "fp32",
44
+ progress_callback: reportProgress("TTS"),
45
+ });
46
+ self.postMessage({
47
+ type: "status",
48
+ message: `Voice ${voice}`,
49
+ mode: "warn",
50
+ detail: `Supertonic pipeline loaded; loading ${voice} voice embedding.`,
51
+ });
52
+ const speaker = await ensureVoice(model, voice);
53
+ if (warmup) {
54
+ self.postMessage({
55
+ type: "status",
56
+ message: "Warming",
57
+ mode: "warn",
58
+ detail: "Warming Supertonic with the selected voice.",
59
+ });
60
+ await synthesizer("Ready.", {
61
+ speaker_embeddings: speaker,
62
+ num_inference_steps: 2,
63
+ speed: 1.08,
64
+ });
65
+ }
66
+ self.postMessage({ type: "status", message: "Ready", mode: "ready", detail: "Supertonic is ready." });
67
+ self.postMessage({ type: "ready" });
68
+ preloadVoices(model, VOICES.filter((candidate) => candidate !== voice));
69
+ }
70
+
71
+ async function preloadVoice({ voice = "F2", requestId = null }) {
72
+ try {
73
+ await ensureVoice(modelId, voice);
74
+ self.postMessage({ type: "voice-ready", voice, requestId });
75
+ } catch (error) {
76
+ self.postMessage({
77
+ type: "voice-error",
78
+ voice,
79
+ requestId,
80
+ message: error.message ?? String(error),
81
+ });
82
+ }
83
+ }
84
+
85
+ function preloadVoices(model, voices) {
86
+ for (const voice of voices) {
87
+ ensureVoice(model, voice).catch((error) => {
88
+ self.postMessage({
89
+ type: "voice-error",
90
+ voice,
91
+ requestId: null,
92
+ message: error.message ?? String(error),
93
+ });
94
+ });
95
+ }
96
+ }
97
+
98
+ function reportProgress(label) {
99
+ return (progress) => {
100
+ if (progress.status === "progress") {
101
+ const pct = Number.isFinite(progress.progress) ? ` ${progress.progress.toFixed(0)}%` : "";
102
+ self.postMessage({ type: "status", message: `${label}${pct}`, mode: "warn" });
103
+ }
104
+ };
105
+ }
106
+
107
+ async function speak({ text, voice, steps, speed, turnId, sequence, enqueuedAt, cancelEpoch: itemEpoch }) {
108
+ if (!synthesizer || !text.trim()) return;
109
+ if (itemEpoch !== cancelEpoch) return;
110
+ self.postMessage({ type: "status", message: "Synthesizing", mode: "active" });
111
+ const speaker = await ensureVoice(modelId, voice);
112
+ if (itemEpoch !== cancelEpoch) return;
113
+ const startedAt = performance.now();
114
+ const output = await synthesizer(text.trim(), {
115
+ speaker_embeddings: speaker,
116
+ num_inference_steps: steps,
117
+ speed,
118
+ });
119
+ if (itemEpoch !== cancelEpoch) return;
120
+ self.postMessage(
121
+ {
122
+ type: "audio",
123
+ turnId,
124
+ sequence,
125
+ enqueuedAt,
126
+ text,
127
+ audio: output.audio,
128
+ sampleRate: output.sampling_rate,
129
+ synthesisMs: performance.now() - startedAt,
130
+ },
131
+ [output.audio.buffer],
132
+ );
133
+ self.postMessage({ type: "done", turnId, sequence });
134
+ }
135
+
136
+ async function synthesize({ text, voice, steps, speed, requestId, cancelEpoch: itemEpoch }) {
137
+ if (!synthesizer || !text.trim()) return;
138
+ if (itemEpoch !== cancelEpoch) return;
139
+ self.postMessage({ type: "status", message: "Loopback", mode: "active" });
140
+ const speaker = await ensureVoice(modelId, voice);
141
+ if (itemEpoch !== cancelEpoch) return;
142
+ const startedAt = performance.now();
143
+ const output = await synthesizer(text.trim(), {
144
+ speaker_embeddings: speaker,
145
+ num_inference_steps: steps,
146
+ speed,
147
+ });
148
+ if (itemEpoch !== cancelEpoch) return;
149
+ self.postMessage(
150
+ {
151
+ type: "synthetic-audio",
152
+ requestId,
153
+ audio: output.audio,
154
+ sampleRate: output.sampling_rate,
155
+ synthesisMs: performance.now() - startedAt,
156
+ },
157
+ [output.audio.buffer],
158
+ );
159
+ self.postMessage({ type: "done", requestId });
160
+ }
161
+
162
+ function voiceUrl(model, voice) {
163
+ return `https://huggingface.co/${model}/resolve/main/voices/${voice}.bin`;
164
+ }
165
+
166
+ async function loadVoice(model, voice) {
167
+ const response = await fetch(voiceUrl(model, voice), { cache: "no-store" });
168
+ if (!response.ok) {
169
+ throw new Error(`Failed to load ${voice} voice embedding: ${response.status}`);
170
+ }
171
+ voiceEmbeddings.set(voice, new Float32Array(await response.arrayBuffer()));
172
+ return voiceEmbeddings.get(voice);
173
+ }
174
+
175
+ async function ensureVoice(model, voice) {
176
+ if (!model) throw new Error("TTS model has not loaded.");
177
+ const embedding = voiceEmbeddings.get(voice);
178
+ if (embedding) return embedding;
179
+ if (!voicePromises.has(voice)) {
180
+ voicePromises.set(
181
+ voice,
182
+ loadVoice(model, voice).finally(() => {
183
+ voicePromises.delete(voice);
184
+ }),
185
+ );
186
+ }
187
+ return voicePromises.get(voice);
188
+ }
189
+
190
+ function postError(error) {
191
+ self.postMessage({ type: "error", message: error.message ?? String(error) });
192
+ }