suvradeepp's picture
Publish Tiny Hinglish Turn Detector development preview
35d483e verified
|
Raw
History Blame Contribute Delete
3.17 kB
# Demo and runtime contract
The demo accepts microphone or uploaded audio. In production a VAD should call
the endpoint model only after a pause; the silence slider simulates one such
checkpoint. The stateful controller has the following contract:
- `SPEAKING`: VAD has observed speech since the last decision.
- `HOLD`: a pause exists, but responding now would risk interruption.
- `END`: the calibrated classifier or maximum timeout allows a response.
The UI creates a fresh controller for each click, so it visualizes only a
single `HOLD` or `END` pause decision. `SPEAKING`, latching, and repeated
checkpoint behavior belong to the controller/replay API, not to a stateful UI
session.
Run locally:
```bash
uv sync --extra demo
TURN_MODEL_PATH=artifacts/partial-shard-warmstart-lr3e4-5ep/model.onnx \
uv run python app.py
```
Every ONNX file must have an adjacent `model_metadata.json` describing its
frontend, input names, output activation, selected threshold, and model version.
Development weights also display their training status and exact data scope in
the page banner and every result payload. The threshold slider initializes from
the exported calibrated threshold.
If the model is absent, the UI activates a conspicuous heuristic development
fallback. Those scores are useful only for testing the UI and must never be
reported as experiment results.
The packaged preview has 151,812 parameters, a four-second window, and stored
threshold `0.7410007715`. It is development-only: the acoustic baseline is
stronger on the current IID split, and no official-test or verified Hinglish
result exists. The model is also sensitive to appended silence (800 ms flips
26/326 development decisions), so the slider demonstrates controller behavior;
it does not validate a production silence policy.
For continuous evaluation, produce a JSONL file with these fields at every VAD
pause checkpoint:
```json
{"turn_id":"call-1/turn-2","timestamp_ms":840.0,"silence_ms":300.0,"endpoint_probability":0.18,"target_endpoint":false,"inference_ms":6.2}
```
The bundled smoke fixture exercises the exact serialized controller:
```bash
uv run python scripts/replay_stream.py \
--input data/collection/controller_replay_fixture.jsonl \
--output reports/controller_replay_integration.jsonl \
--metadata artifacts/partial-shard-warmstart-lr3e4-5ep/model_metadata.json \
--evidence-scope synthetic_integration
```
That fixture is hand-authored integration data. Its probabilities and targets
are not ONNX outputs or human annotations, and its latency values are synthetic.
It validates policy binding, threshold relaxation, timeout, END latching, and
duplicate-emission prevention only—not model, conversation, latency, or
product quality. Use private, human-annotated VAD logs for real replay metrics.
Replay refuses an unbound default policy: provide exported metadata or an
explicit threshold. The metadata serializes all controller values. Once a turn
emits END, the controller remains latched until speech/reset and records later
callbacks with `emit_response=false`, so downstream code can never treat a
repeated pause callback as a second response edge.