Spaces:
Configuration error
Configuration error
Show live microphone input level
Browse files- README.md +5 -5
- app.js +26 -0
- index.html +14 -0
- styles.css +32 -0
README.md
CHANGED
|
@@ -52,7 +52,7 @@ The Latency panel includes validation cards for the two external evidence gates:
|
|
| 52 |
| Role | Default | Why |
|
| 53 |
| --- | --- | --- |
|
| 54 |
| VAD | `onnx-community/silero-vad` | Small ONNX model used in the Transformers.js Moonshine web demo; fast enough to run on 512-sample chunks and drive turn-taking. The app defaults to 480 ms of trailing silence, configurable from 200-800 ms. The fp32 ONNX file is about 2.2 MB. |
|
| 55 |
-
| STT | `onnx-community/moonshine-base-ONNX` | Moonshine Base is the default balanced STT after the current fake-mic series reached 3/3 exact rows, 0% median WER, 1.
|
| 56 |
| LLM | `HuggingFaceTB/SmolLM2-135M-Instruct` | The fastest instruct model found for this stack. It is tagged for Transformers.js and includes ONNX q4/q4f16 files; q4 WASM is about 182 MB and q4f16 WebGPU is about 118 MB. In headless WASM it cut first-token latency by roughly 2-3x versus SmolLM2 360M. A tiny pinned identity example keeps the default 135M stack passing the LLM OK gate without switching to the much slower 360M. |
|
| 57 |
| TTS | `onnx-community/Supertonic-TTS-ONNX` | Transformers.js packaging of `Supertone/supertonic`, using local ONNX inference and Supertonic voice embeddings. The demo defaults to voice F2 and 2 inference steps because F2 is still faster in current full-stack fake-mic and default-suite runs; M2 was slightly faster in isolated TTS but slower in loopback and fake-mic full-stack validation, so it remains a candidate for real-mic/WebGPU validation rather than the default. The selected voice is loaded before the app reports ready, and the other voices are preloaded in the background. The repo is about 263 MB. |
|
| 58 |
|
|
@@ -71,7 +71,7 @@ Relevant sources:
|
|
| 71 |
|
| 72 |
## Benchmark Plan
|
| 73 |
|
| 74 |
-
Use **Run benchmark suite** after loading models to execute the current stack's TTS, barge-in, identity, chat, and voice-loopback benchmarks in sequence. Use **Benchmark real mic** for one real microphone row, **Run 3 real-mic series** to collect the three real-mic repetitions expected by the summary cards, or **Run evidence capture** to collect the hardware WebGPU row when available and then the full real-mic series. The real-mic paths prompt the user to say "What app is this?" so each row captures real-mic STT WER/CER, end-to-end latency, and the identity-answer LLM quality gate. The Latency panel includes a **Real mic validation** card that shows the exact phrase, the current row during a series, and current-stack progress toward the 3-row target. **Run TTS benchmark** isolates Supertonic latency for the current voice and inference-step setting, **Run barge-in check** verifies that the speech-start interruption path cancels in-flight TTS before stale audio can play, **Run identity benchmark** checks the same strict "What app is this?" quality gate without the microphone, **Run chat benchmark** measures a normal non-identity text turn without the identity primer, and **Run voice loopback** synthesizes a local Supertonic utterance and feeds it through the same VAD/STT path when a test environment has no microphone. Each run is appended to the **Benchmarks** table with the selected stack, prompt/transcript, response, latency metrics, loopback or real-mic STT word error rate, and an identity-answer LLM quality gate for the "What app is this?" prompt; failed or timed-out runs are kept as rows with an `error` field so candidate comparisons do not hide unstable stacks. The summary cards focus on the currently loaded stack when models are loaded, otherwise all rows. **Copy JSON** exports `{ summary, results }`, where `summary` contains `all`, `current`, and `byStack` aggregate objects with TTS, identity, chat, real-mic, loopback, and barge-in medians for comparing candidate stacks.
|
| 75 |
|
| 76 |
Microphone startup is capped at 15 seconds. If browser permissions, fake media devices, or audio worklet startup hang, the active mic benchmark is cancelled and the event log records the failure instead of leaving a silent pending run. Loopback audio is also followed by an explicit ASR flush after the synthesized prompt and trailing silence have been fed, so sticky VAD/STT candidates fail or finish as rows instead of waiting for the global benchmark timeout.
|
| 77 |
|
|
@@ -169,16 +169,16 @@ Measured on this workspace with Chrome 145 headless, x86_64, 4 vCPU, Transformer
|
|
| 169 |
| Current default barge-in suite row | N/A | N/A | Supertonic F2, 2 steps | N/A | N/A | N/A | N/A | N/A | 0 ms | cancelled | cancelled | N/A | N/A |
|
| 170 |
| Text benchmark | N/A | SmolLM2 360M q4 WASM | 2 | N/A | N/A | fail 3/4 | N/A | 9.56 s | 13.43 s | 1.31 s | 14.75 s | N/A | 0.5 tok/s |
|
| 171 |
| Current default voice loopback suite row | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 1.80 s | 0% | pass 4/4 | 480 ms | 4.14 s | 4.14 s | 545 ms | 4.69 s | 6.49 s | 1.5 tok/s |
|
| 172 |
-
| 3-run loopback stability series | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps |
|
| 173 |
| M2 loopback validation | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic M2, 2 steps | 2.33 s | 0% | pass 4/4 | 480 ms | 6.64 s | 8.48 s | 1.74 s | 10.23 s | 12.56 s | 0.9 tok/s |
|
| 174 |
-
| Fake microphone scripted series | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 1.
|
| 175 |
| Whisper Tiny fake microphone validation | Whisper Tiny English fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 4.54 s | 0% | pass 4/4 | 480 ms | 4.70 s | 5.80 s | 603 ms | 6.40 s | 10.94 s | 1.5 tok/s |
|
| 176 |
| M2 fake microphone validation | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic M2, 2 steps | 2.37 s | 0% | pass 4/4 | 480 ms | 6.64 s | 7.84 s | 1.43 s | 9.27 s | 11.64 s | 1.1 tok/s |
|
| 177 |
| Previous voice loopback | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F1, 2 steps | 1.65 s | 25% | pass 4/4 | 480 ms | 3.74 s | 4.81 s | 773 ms | 5.58 s | 7.24 s | 1.6 tok/s |
|
| 178 |
| Voice loopback | Moonshine Tiny fp32/q4 | SmolLM2 135M q4 WASM | 2 | 1.08 s | 25% | pass 4/4 | 480 ms | 3.72 s | 4.84 s | 951 ms | 5.79 s | 6.88 s | 1.4 tok/s |
|
| 179 |
| Voice loopback | Whisper Tiny English fp32/q4 | SmolLM2 135M q4 WASM | 2 | 5.62 s | 0% | pass 4/4 | 480 ms | 4.63 s | 6.63 s | 902 ms | 7.53 s | 13.16 s | 1.3 tok/s |
|
| 180 |
|
| 181 |
-
The first-clause TTS threshold reduced the earlier SmolLM2 360M text benchmark's first-audio time from 45.84 s to 12.25 s in headless WASM by moving TTS queueing from 44.43 s after transcript to 11.01 s. Switching the default LLM to SmolLM2 135M, splitting generic and identity prompts, and using a word-boundary-safe first chunk reduced normal chat first audio to 4.34 s in the earlier default suite. The first response chunk now targets roughly 5 characters, keeps a minimum safe space boundary, and searches forward to the next word boundary before falling back to a hard character cut; this fixed the rejected early failure that produced "I'm read", keeps the current chat first chunk as "I'm ready", and lets identity turns start on the safe first-word chunk "This". Identity turns use a stricter system prompt plus tiny pinned examples, and repeated identity prompts now ignore prior identity history, keeping the fake-mic prompt at 187 input tokens across all three scripted rows instead of growing turn by turn. The identity-intent detector also tolerates narrow ASR near-misses such as "browser dome" for "browser demo" and "identifies" for "identify"; the original transcript and WER are still preserved. The latest fake-mic series measured 4.
|
| 182 |
|
| 183 |
## Known Limitations
|
| 184 |
|
|
|
|
| 52 |
| Role | Default | Why |
|
| 53 |
| --- | --- | --- |
|
| 54 |
| VAD | `onnx-community/silero-vad` | Small ONNX model used in the Transformers.js Moonshine web demo; fast enough to run on 512-sample chunks and drive turn-taking. The app defaults to 480 ms of trailing silence, configurable from 200-800 ms. The fp32 ONNX file is about 2.2 MB. |
|
| 55 |
+
| STT | `onnx-community/moonshine-base-ONNX` | Moonshine Base is the default balanced STT after the current fake-mic series reached 3/3 exact rows, 0% median WER, 1.66 s median ASR, and 7.21 s median speech-end-to-audio. Moonshine Tiny remains selectable as the low-latency experiment, but it failed the current exact fake-mic gate by repeatedly hearing variants like "What happens this?" and timing out. Whisper Tiny English is selectable as the higher-accuracy fallback; it reached 3/3 and 0% WER in fake-mic validation, but its 4.54 s median ASR raised speech-end-to-audio to 10.94 s. The demo uses fp32 encoder + q4 merged decoder because the q8 WASM path failed on Transformers.js 4.2.0 in local verification. |
|
| 56 |
| LLM | `HuggingFaceTB/SmolLM2-135M-Instruct` | The fastest instruct model found for this stack. It is tagged for Transformers.js and includes ONNX q4/q4f16 files; q4 WASM is about 182 MB and q4f16 WebGPU is about 118 MB. In headless WASM it cut first-token latency by roughly 2-3x versus SmolLM2 360M. A tiny pinned identity example keeps the default 135M stack passing the LLM OK gate without switching to the much slower 360M. |
|
| 57 |
| TTS | `onnx-community/Supertonic-TTS-ONNX` | Transformers.js packaging of `Supertone/supertonic`, using local ONNX inference and Supertonic voice embeddings. The demo defaults to voice F2 and 2 inference steps because F2 is still faster in current full-stack fake-mic and default-suite runs; M2 was slightly faster in isolated TTS but slower in loopback and fake-mic full-stack validation, so it remains a candidate for real-mic/WebGPU validation rather than the default. The selected voice is loaded before the app reports ready, and the other voices are preloaded in the background. The repo is about 263 MB. |
|
| 58 |
|
|
|
|
| 71 |
|
| 72 |
## Benchmark Plan
|
| 73 |
|
| 74 |
+
Use **Run benchmark suite** after loading models to execute the current stack's TTS, barge-in, identity, chat, and voice-loopback benchmarks in sequence. Use **Benchmark real mic** for one real microphone row, **Run 3 real-mic series** to collect the three real-mic repetitions expected by the summary cards, or **Run evidence capture** to collect the hardware WebGPU row when available and then the full real-mic series. The real-mic paths prompt the user to say "What app is this?" so each row captures real-mic STT WER/CER, end-to-end latency, and the identity-answer LLM quality gate. The Input panel includes a live microphone input-level meter sourced from the same audio worklet chunks that feed VAD/STT, so manual testers can confirm that the browser is receiving audio before waiting on recognition. The Latency panel includes a **Real mic validation** card that shows the exact phrase, the current row during a series, and current-stack progress toward the 3-row target. **Run TTS benchmark** isolates Supertonic latency for the current voice and inference-step setting, **Run barge-in check** verifies that the speech-start interruption path cancels in-flight TTS before stale audio can play, **Run identity benchmark** checks the same strict "What app is this?" quality gate without the microphone, **Run chat benchmark** measures a normal non-identity text turn without the identity primer, and **Run voice loopback** synthesizes a local Supertonic utterance and feeds it through the same VAD/STT path when a test environment has no microphone. Each run is appended to the **Benchmarks** table with the selected stack, prompt/transcript, response, latency metrics, loopback or real-mic STT word error rate, and an identity-answer LLM quality gate for the "What app is this?" prompt; failed or timed-out runs are kept as rows with an `error` field so candidate comparisons do not hide unstable stacks. The summary cards focus on the currently loaded stack when models are loaded, otherwise all rows. **Copy JSON** exports `{ summary, results }`, where `summary` contains `all`, `current`, and `byStack` aggregate objects with TTS, identity, chat, real-mic, loopback, and barge-in medians for comparing candidate stacks.
|
| 75 |
|
| 76 |
Microphone startup is capped at 15 seconds. If browser permissions, fake media devices, or audio worklet startup hang, the active mic benchmark is cancelled and the event log records the failure instead of leaving a silent pending run. Loopback audio is also followed by an explicit ASR flush after the synthesized prompt and trailing silence have been fed, so sticky VAD/STT candidates fail or finish as rows instead of waiting for the global benchmark timeout.
|
| 77 |
|
|
|
|
| 169 |
| Current default barge-in suite row | N/A | N/A | Supertonic F2, 2 steps | N/A | N/A | N/A | N/A | N/A | 0 ms | cancelled | cancelled | N/A | N/A |
|
| 170 |
| Text benchmark | N/A | SmolLM2 360M q4 WASM | 2 | N/A | N/A | fail 3/4 | N/A | 9.56 s | 13.43 s | 1.31 s | 14.75 s | N/A | 0.5 tok/s |
|
| 171 |
| Current default voice loopback suite row | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 1.80 s | 0% | pass 4/4 | 480 ms | 4.14 s | 4.14 s | 545 ms | 4.69 s | 6.49 s | 1.5 tok/s |
|
| 172 |
+
| 3-run loopback stability series | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 1.92 s | 0% median, 3/3 exact | pass 3/3 | 480 ms | 5.13 s | 5.14 s | 610 ms | 5.70 s | 7.57 s | 1.4 tok/s |
|
| 173 |
| M2 loopback validation | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic M2, 2 steps | 2.33 s | 0% | pass 4/4 | 480 ms | 6.64 s | 8.48 s | 1.74 s | 10.23 s | 12.56 s | 0.9 tok/s |
|
| 174 |
+
| Fake microphone scripted series | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 1.66 s | 0% | pass 3/3 rows | 480 ms | 4.98 s | 4.98 s | 599 ms | 5.50 s | 7.21 s | 1.4 tok/s |
|
| 175 |
| Whisper Tiny fake microphone validation | Whisper Tiny English fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F2, 2 steps | 4.54 s | 0% | pass 4/4 | 480 ms | 4.70 s | 5.80 s | 603 ms | 6.40 s | 10.94 s | 1.5 tok/s |
|
| 176 |
| M2 fake microphone validation | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic M2, 2 steps | 2.37 s | 0% | pass 4/4 | 480 ms | 6.64 s | 7.84 s | 1.43 s | 9.27 s | 11.64 s | 1.1 tok/s |
|
| 177 |
| Previous voice loopback | Moonshine Base fp32/q4 | SmolLM2 135M q4 WASM | Supertonic F1, 2 steps | 1.65 s | 25% | pass 4/4 | 480 ms | 3.74 s | 4.81 s | 773 ms | 5.58 s | 7.24 s | 1.6 tok/s |
|
| 178 |
| Voice loopback | Moonshine Tiny fp32/q4 | SmolLM2 135M q4 WASM | 2 | 1.08 s | 25% | pass 4/4 | 480 ms | 3.72 s | 4.84 s | 951 ms | 5.79 s | 6.88 s | 1.4 tok/s |
|
| 179 |
| Voice loopback | Whisper Tiny English fp32/q4 | SmolLM2 135M q4 WASM | 2 | 5.62 s | 0% | pass 4/4 | 480 ms | 4.63 s | 6.63 s | 902 ms | 7.53 s | 13.16 s | 1.3 tok/s |
|
| 180 |
|
| 181 |
+
The first-clause TTS threshold reduced the earlier SmolLM2 360M text benchmark's first-audio time from 45.84 s to 12.25 s in headless WASM by moving TTS queueing from 44.43 s after transcript to 11.01 s. Switching the default LLM to SmolLM2 135M, splitting generic and identity prompts, and using a word-boundary-safe first chunk reduced normal chat first audio to 4.34 s in the earlier default suite. The first response chunk now targets roughly 5 characters, keeps a minimum safe space boundary, and searches forward to the next word boundary before falling back to a hard character cut; this fixed the rejected early failure that produced "I'm read", keeps the current chat first chunk as "I'm ready", and lets identity turns start on the safe first-word chunk "This". Identity turns use a stricter system prompt plus tiny pinned examples, and repeated identity prompts now ignore prior identity history, keeping the fake-mic prompt at 187 input tokens across all three scripted rows instead of growing turn by turn. The identity-intent detector also tolerates narrow ASR near-misses such as "browser dome" for "browser demo" and "identifies" for "identify"; the original transcript and WER are still preserved. The latest fake-mic series measured 4.98 s median transcript-to-first-token and 7.21 s median speech-end-to-first-audio. A current-code 360M text run improved the identity answer from 2/4 to 3/4 concepts, but first audio was still 14.75 s and the answer omitted the LLM component, so 135M remains the default latency/quality choice. Qwen3 0.6B terminated the local headless browser during WASM model load before it reached benchmark-ready state, so the app now blocks Qwen3 and SmolLM2 1.7B on WASM fallback and requires WebGPU for those candidates. The one-load TTS sweep confirms 2-step Supertonic is the right default for latency; 8 steps roughly doubled first-audio time, and M2 at 2 steps was the fastest isolated voice/step setting at 2.00 s versus F2 at 2.10 s. The app still defaults to F2 because the latest F2 fake-mic series reached 7.21 s median speech-end-to-audio, while M2's repeated fake-mic validation reached 0% WER but slower 11.64 s speech-end-to-audio and 23.42 s speech-end-to-audio-done; M2 also measured slower in loopback, so real-mic and WebGPU validation are still needed before reconsidering the default. The TTS worker now loads the selected voice before reporting ready, then preloads the remaining voices in the background; the latest hosted client-side smoke measured a 1.65 s TTS first-audio row with all benchmark-phase network requests already settled. The current default benchmark suite was verified with actual workers and produced TTS, barge-in, identity, chat, and loopback rows before re-enabling controls. The barge-in check verified that a synthetic speech-start event cancels in-flight TTS before stale audio plays and returns the TTS tile to Ready. After repeated loopback checks, the synthetic loopback stability gate now uses the prompt "Identify this browser demo.", 1.00x prompt speed, and a short silence preroll; the latest targeted run completed 3/3 rows, reached 3/3 exact transcripts with 0% median WER, passed the identity answer gate on all 3 rows, and reached 7.57 s median speech-end-to-audio. The previous "Please identify this browser demo." prompt passed the identity gate but intermittently dropped the low-value opening word, pushing WER to 20%; the shorter "What app is this?", "What application is this?", "What is this app?", and "What demo is this?" synthetic prompts also intermittently dropped opening words or confused "application" with "cation/location", so they remain stress options rather than the default loopback gate. A partial-ASR-off loopback comparison was slightly faster at 5.94 s median speech-end-to-audio, but it dropped to 2/3 exact transcripts with "Wap is this.", so partial previews remain enabled by default. The loopback gate is still treated as a synthetic stability signal, not a substitute for real microphone validation. Normal response TTS remains at 1.08x. The fake microphone scripted series reached 3/3 completed rows with 0% median WER, 1.66 s median ASR, 7.21 s median speech-end-to-audio, and 19.30 s median speech-end-to-audio-done through the browser capture path; because Chrome loops fake WAV input, the harness ignores non-matching partial prompt captures until the reference phrase is heard, then stops the mic so the response can finish. A stale-ASR guard now ignores queued speech events after scripted fake-mic capture stops; the M2 regression run completed 3/3 rows after previously timing out from those stale interruptions. The live input-level meter uses the same microphone worklet chunks as VAD/STT, so it helps distinguish permission/device silence from VAD or transcription delay during real-mic testing. The loopback benchmark now feeds synthesized audio at real-time chunk intervals, uses a 480 ms VAD close delay after a 280 ms default split the prompt into separate words, and posts an explicit ASR flush after trailing silence so sticky candidates finish. Moonshine Base remains the default STT because it is much faster than Whisper Tiny English while less error-prone than Tiny: Whisper Tiny English also reached 0% WER in fake mic, but raised median ASR to 4.54 s and speech-end-to-audio to 10.94 s, while Moonshine Tiny failed the exact fake-mic gate before completing one row. The scripted mic benchmark still uses "What app is this?" as its reference phrase so real-mic rows can report WER/CER against the short spoken phrase, and the 3-run mic series keeps the same microphone open while collecting the repeated rows. The latest full-stack model loads completed in about 15-35 seconds cold in this environment, with a same-profile fake-mic reload at about 10.5 seconds. A real WebGPU browser should be benchmarked next; this headless environment exposes only the SwiftShader software adapter.
|
| 182 |
|
| 183 |
## Known Limitations
|
| 184 |
|
app.js
CHANGED
|
@@ -32,6 +32,9 @@ const elements = {
|
|
| 32 |
vadSilenceValue: $("vadSilenceValue"),
|
| 33 |
partialToggle: $("partialToggle"),
|
| 34 |
micBadge: $("micBadge"),
|
|
|
|
|
|
|
|
|
|
| 35 |
audioBadge: $("audioBadge"),
|
| 36 |
partialTranscript: $("partialTranscript"),
|
| 37 |
finalTranscript: $("finalTranscript"),
|
|
@@ -148,6 +151,7 @@ const HARDWARE_WEBGPU_EVIDENCE_STACK = Object.freeze({
|
|
| 148 |
});
|
| 149 |
const MIC_START_TIMEOUT_MS = 15000;
|
| 150 |
const WEBGPU_ADAPTER_TIMEOUT_MS = 5000;
|
|
|
|
| 151 |
const DEFAULT_TTS_CHUNKING = Object.freeze({
|
| 152 |
firstSentenceMinChars: 5,
|
| 153 |
sentenceMinChars: 8,
|
|
@@ -212,6 +216,8 @@ const state = {
|
|
| 212 |
peak: 0,
|
| 213 |
sumSquares: 0,
|
| 214 |
},
|
|
|
|
|
|
|
| 215 |
activeBenchmark: null,
|
| 216 |
benchmarkTimeout: null,
|
| 217 |
pendingPlaybackSchedules: 0,
|
|
@@ -518,6 +524,15 @@ function resetMetrics() {
|
|
| 518 |
elements.decodeRate.textContent = "-";
|
| 519 |
}
|
| 520 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 521 |
function resetConversationUi() {
|
| 522 |
elements.partialTranscript.textContent = "Waiting for speech.";
|
| 523 |
elements.finalTranscript.textContent = "";
|
|
@@ -683,6 +698,8 @@ function resetMicInputStats() {
|
|
| 683 |
peak: 0,
|
| 684 |
sumSquares: 0,
|
| 685 |
};
|
|
|
|
|
|
|
| 686 |
}
|
| 687 |
|
| 688 |
function updateMicInputStats(buffer) {
|
|
@@ -699,6 +716,13 @@ function updateMicInputStats(buffer) {
|
|
| 699 |
state.micInputStats.samples += buffer.length;
|
| 700 |
state.micInputStats.peak = Math.max(state.micInputStats.peak, chunkPeak);
|
| 701 |
state.micInputStats.sumSquares += chunkSumSquares;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 702 |
if (state.activeBenchmark?.kind === "mic") {
|
| 703 |
state.activeBenchmark.micInputChunks = state.micInputStats.chunks;
|
| 704 |
state.activeBenchmark.micInputPeak = state.micInputStats.peak;
|
|
@@ -1063,6 +1087,7 @@ function stopMic() {
|
|
| 1063 |
state.workletNode = null;
|
| 1064 |
state.micSource = null;
|
| 1065 |
state.micMonitorGain = null;
|
|
|
|
| 1066 |
elements.micButton.innerHTML = '<span class="button-icon" aria-hidden="true">●</span> Start mic';
|
| 1067 |
setMicState("Mic off", false);
|
| 1068 |
setTile("vad", "Ready", "ready");
|
|
@@ -2865,6 +2890,7 @@ function automationSnapshot() {
|
|
| 2865 |
chunks: state.micInputStats.chunks,
|
| 2866 |
samples: state.micInputStats.samples,
|
| 2867 |
peak: state.micInputStats.peak,
|
|
|
|
| 2868 |
rms:
|
| 2869 |
state.micInputStats.samples > 0
|
| 2870 |
? Math.sqrt(state.micInputStats.sumSquares / state.micInputStats.samples)
|
|
|
|
| 32 |
vadSilenceValue: $("vadSilenceValue"),
|
| 33 |
partialToggle: $("partialToggle"),
|
| 34 |
micBadge: $("micBadge"),
|
| 35 |
+
micLevelMeter: $("micLevelMeter"),
|
| 36 |
+
micLevelBar: $("micLevelBar"),
|
| 37 |
+
micLevelValue: $("micLevelValue"),
|
| 38 |
audioBadge: $("audioBadge"),
|
| 39 |
partialTranscript: $("partialTranscript"),
|
| 40 |
finalTranscript: $("finalTranscript"),
|
|
|
|
| 151 |
});
|
| 152 |
const MIC_START_TIMEOUT_MS = 15000;
|
| 153 |
const WEBGPU_ADAPTER_TIMEOUT_MS = 5000;
|
| 154 |
+
const MIC_LEVEL_UI_INTERVAL_MS = 80;
|
| 155 |
const DEFAULT_TTS_CHUNKING = Object.freeze({
|
| 156 |
firstSentenceMinChars: 5,
|
| 157 |
sentenceMinChars: 8,
|
|
|
|
| 216 |
peak: 0,
|
| 217 |
sumSquares: 0,
|
| 218 |
},
|
| 219 |
+
micLevel: 0,
|
| 220 |
+
lastMicLevelUiAt: 0,
|
| 221 |
activeBenchmark: null,
|
| 222 |
benchmarkTimeout: null,
|
| 223 |
pendingPlaybackSchedules: 0,
|
|
|
|
| 524 |
elements.decodeRate.textContent = "-";
|
| 525 |
}
|
| 526 |
|
| 527 |
+
function setMicInputLevel(level) {
|
| 528 |
+
const normalized = Math.max(0, Math.min(1, Number(level) || 0));
|
| 529 |
+
const percent = Math.round(normalized * 100);
|
| 530 |
+
state.micLevel = normalized;
|
| 531 |
+
elements.micLevelBar.style.width = `${percent}%`;
|
| 532 |
+
elements.micLevelMeter.setAttribute("aria-valuenow", String(percent));
|
| 533 |
+
elements.micLevelValue.textContent = `${percent}%`;
|
| 534 |
+
}
|
| 535 |
+
|
| 536 |
function resetConversationUi() {
|
| 537 |
elements.partialTranscript.textContent = "Waiting for speech.";
|
| 538 |
elements.finalTranscript.textContent = "";
|
|
|
|
| 698 |
peak: 0,
|
| 699 |
sumSquares: 0,
|
| 700 |
};
|
| 701 |
+
state.lastMicLevelUiAt = 0;
|
| 702 |
+
setMicInputLevel(0);
|
| 703 |
}
|
| 704 |
|
| 705 |
function updateMicInputStats(buffer) {
|
|
|
|
| 716 |
state.micInputStats.samples += buffer.length;
|
| 717 |
state.micInputStats.peak = Math.max(state.micInputStats.peak, chunkPeak);
|
| 718 |
state.micInputStats.sumSquares += chunkSumSquares;
|
| 719 |
+
const now = performance.now();
|
| 720 |
+
if (now - state.lastMicLevelUiAt >= MIC_LEVEL_UI_INTERVAL_MS) {
|
| 721 |
+
const chunkRms = Math.sqrt(chunkSumSquares / Math.max(1, buffer.length));
|
| 722 |
+
const displayLevel = Math.min(1, Math.max(chunkPeak, chunkRms * 3) * 4);
|
| 723 |
+
state.lastMicLevelUiAt = now;
|
| 724 |
+
setMicInputLevel(displayLevel);
|
| 725 |
+
}
|
| 726 |
if (state.activeBenchmark?.kind === "mic") {
|
| 727 |
state.activeBenchmark.micInputChunks = state.micInputStats.chunks;
|
| 728 |
state.activeBenchmark.micInputPeak = state.micInputStats.peak;
|
|
|
|
| 1087 |
state.workletNode = null;
|
| 1088 |
state.micSource = null;
|
| 1089 |
state.micMonitorGain = null;
|
| 1090 |
+
resetMicInputStats();
|
| 1091 |
elements.micButton.innerHTML = '<span class="button-icon" aria-hidden="true">●</span> Start mic';
|
| 1092 |
setMicState("Mic off", false);
|
| 1093 |
setTile("vad", "Ready", "ready");
|
|
|
|
| 2890 |
chunks: state.micInputStats.chunks,
|
| 2891 |
samples: state.micInputStats.samples,
|
| 2892 |
peak: state.micInputStats.peak,
|
| 2893 |
+
level: state.micLevel,
|
| 2894 |
rms:
|
| 2895 |
state.micInputStats.samples > 0
|
| 2896 |
? Math.sqrt(state.micInputStats.sumSquares / state.micInputStats.samples)
|
index.html
CHANGED
|
@@ -57,6 +57,20 @@
|
|
| 57 |
<span id="micBadge" class="badge">Mic off</span>
|
| 58 |
</div>
|
| 59 |
<div id="partialTranscript" class="partial">Waiting for speech.</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
<div id="finalTranscript" class="final"></div>
|
| 61 |
</div>
|
| 62 |
|
|
|
|
| 57 |
<span id="micBadge" class="badge">Mic off</span>
|
| 58 |
</div>
|
| 59 |
<div id="partialTranscript" class="partial">Waiting for speech.</div>
|
| 60 |
+
<div class="mic-level" aria-label="Microphone input level">
|
| 61 |
+
<span>Input level</span>
|
| 62 |
+
<div
|
| 63 |
+
id="micLevelMeter"
|
| 64 |
+
class="mic-level-meter"
|
| 65 |
+
role="meter"
|
| 66 |
+
aria-valuemin="0"
|
| 67 |
+
aria-valuemax="100"
|
| 68 |
+
aria-valuenow="0"
|
| 69 |
+
>
|
| 70 |
+
<span id="micLevelBar"></span>
|
| 71 |
+
</div>
|
| 72 |
+
<output id="micLevelValue">0%</output>
|
| 73 |
+
</div>
|
| 74 |
<div id="finalTranscript" class="final"></div>
|
| 75 |
</div>
|
| 76 |
|
styles.css
CHANGED
|
@@ -218,6 +218,38 @@ h2 {
|
|
| 218 |
border-color: color-mix(in srgb, var(--green), var(--line) 40%);
|
| 219 |
}
|
| 220 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 221 |
.partial {
|
| 222 |
min-height: 86px;
|
| 223 |
color: var(--muted);
|
|
|
|
| 218 |
border-color: color-mix(in srgb, var(--green), var(--line) 40%);
|
| 219 |
}
|
| 220 |
|
| 221 |
+
.mic-level {
|
| 222 |
+
display: grid;
|
| 223 |
+
grid-template-columns: auto minmax(120px, 1fr) 42px;
|
| 224 |
+
align-items: center;
|
| 225 |
+
gap: 10px;
|
| 226 |
+
min-height: 30px;
|
| 227 |
+
margin-top: 12px;
|
| 228 |
+
color: var(--muted);
|
| 229 |
+
font-size: 12px;
|
| 230 |
+
}
|
| 231 |
+
|
| 232 |
+
.mic-level-meter {
|
| 233 |
+
height: 7px;
|
| 234 |
+
overflow: hidden;
|
| 235 |
+
border-radius: 999px;
|
| 236 |
+
background: #11151a;
|
| 237 |
+
}
|
| 238 |
+
|
| 239 |
+
.mic-level-meter span {
|
| 240 |
+
display: block;
|
| 241 |
+
width: 0%;
|
| 242 |
+
height: 100%;
|
| 243 |
+
background: var(--blue);
|
| 244 |
+
transition: width 80ms linear;
|
| 245 |
+
}
|
| 246 |
+
|
| 247 |
+
.mic-level output {
|
| 248 |
+
text-align: right;
|
| 249 |
+
color: var(--text);
|
| 250 |
+
font-variant-numeric: tabular-nums;
|
| 251 |
+
}
|
| 252 |
+
|
| 253 |
.partial {
|
| 254 |
min-height: 86px;
|
| 255 |
color: var(--muted);
|