| # Demo and runtime contract |
|
|
| The demo accepts microphone or uploaded audio. In production a VAD should call |
| the endpoint model only after a pause; the silence slider simulates one such |
| checkpoint. The stateful controller has the following contract: |
|
|
| - `SPEAKING`: VAD has observed speech since the last decision. |
| - `HOLD`: a pause exists, but responding now would risk interruption. |
| - `END`: the calibrated classifier or maximum timeout allows a response. |
|
|
| The UI creates a fresh controller for each click, so it visualizes only a |
| single `HOLD` or `END` pause decision. `SPEAKING`, latching, and repeated |
| checkpoint behavior belong to the controller/replay API, not to a stateful UI |
| session. |
|
|
| Run locally: |
|
|
| ```bash |
| uv sync --extra demo |
| TURN_MODEL_PATH=artifacts/partial-shard-warmstart-lr3e4-5ep/model.onnx \ |
| uv run python app.py |
| ``` |
|
|
| Every ONNX file must have an adjacent `model_metadata.json` describing its |
| frontend, input names, output activation, selected threshold, and model version. |
| Development weights also display their training status and exact data scope in |
| the page banner and every result payload. The threshold slider initializes from |
| the exported calibrated threshold. |
| If the model is absent, the UI activates a conspicuous heuristic development |
| fallback. Those scores are useful only for testing the UI and must never be |
| reported as experiment results. |
|
|
| The packaged preview has 151,812 parameters, a four-second window, and stored |
| threshold `0.7410007715`. It is development-only: the acoustic baseline is |
| stronger on the current IID split, and no official-test or verified Hinglish |
| result exists. The model is also sensitive to appended silence (800 ms flips |
| 26/326 development decisions), so the slider demonstrates controller behavior; |
| it does not validate a production silence policy. |
|
|
| For continuous evaluation, produce a JSONL file with these fields at every VAD |
| pause checkpoint: |
|
|
| ```json |
| {"turn_id":"call-1/turn-2","timestamp_ms":840.0,"silence_ms":300.0,"endpoint_probability":0.18,"target_endpoint":false,"inference_ms":6.2} |
| ``` |
|
|
| The bundled smoke fixture exercises the exact serialized controller: |
|
|
| ```bash |
| uv run python scripts/replay_stream.py \ |
| --input data/collection/controller_replay_fixture.jsonl \ |
| --output reports/controller_replay_integration.jsonl \ |
| --metadata artifacts/partial-shard-warmstart-lr3e4-5ep/model_metadata.json \ |
| --evidence-scope synthetic_integration |
| ``` |
|
|
| That fixture is hand-authored integration data. Its probabilities and targets |
| are not ONNX outputs or human annotations, and its latency values are synthetic. |
| It validates policy binding, threshold relaxation, timeout, END latching, and |
| duplicate-emission prevention only—not model, conversation, latency, or |
| product quality. Use private, human-annotated VAD logs for real replay metrics. |
|
|
| Replay refuses an unbound default policy: provide exported metadata or an |
| explicit threshold. The metadata serializes all controller values. Once a turn |
| emits END, the controller remains latched until speech/reset and records later |
| callbacks with `emit_response=false`, so downstream code can never treat a |
| repeated pause callback as a second response edge. |
|
|