suvradeepp's picture
Publish Tiny Hinglish Turn Detector development preview
35d483e verified
|
Raw
History Blame Contribute Delete
3.17 kB

Demo and runtime contract

The demo accepts microphone or uploaded audio. In production a VAD should call the endpoint model only after a pause; the silence slider simulates one such checkpoint. The stateful controller has the following contract:

  • SPEAKING: VAD has observed speech since the last decision.
  • HOLD: a pause exists, but responding now would risk interruption.
  • END: the calibrated classifier or maximum timeout allows a response.

The UI creates a fresh controller for each click, so it visualizes only a single HOLD or END pause decision. SPEAKING, latching, and repeated checkpoint behavior belong to the controller/replay API, not to a stateful UI session.

Run locally:

uv sync --extra demo
TURN_MODEL_PATH=artifacts/partial-shard-warmstart-lr3e4-5ep/model.onnx \
  uv run python app.py

Every ONNX file must have an adjacent model_metadata.json describing its frontend, input names, output activation, selected threshold, and model version. Development weights also display their training status and exact data scope in the page banner and every result payload. The threshold slider initializes from the exported calibrated threshold. If the model is absent, the UI activates a conspicuous heuristic development fallback. Those scores are useful only for testing the UI and must never be reported as experiment results.

The packaged preview has 151,812 parameters, a four-second window, and stored threshold 0.7410007715. It is development-only: the acoustic baseline is stronger on the current IID split, and no official-test or verified Hinglish result exists. The model is also sensitive to appended silence (800 ms flips 26/326 development decisions), so the slider demonstrates controller behavior; it does not validate a production silence policy.

For continuous evaluation, produce a JSONL file with these fields at every VAD pause checkpoint:

{"turn_id":"call-1/turn-2","timestamp_ms":840.0,"silence_ms":300.0,"endpoint_probability":0.18,"target_endpoint":false,"inference_ms":6.2}

The bundled smoke fixture exercises the exact serialized controller:

uv run python scripts/replay_stream.py \
  --input data/collection/controller_replay_fixture.jsonl \
  --output reports/controller_replay_integration.jsonl \
  --metadata artifacts/partial-shard-warmstart-lr3e4-5ep/model_metadata.json \
  --evidence-scope synthetic_integration

That fixture is hand-authored integration data. Its probabilities and targets are not ONNX outputs or human annotations, and its latency values are synthetic. It validates policy binding, threshold relaxation, timeout, END latching, and duplicate-emission prevention only—not model, conversation, latency, or product quality. Use private, human-annotated VAD logs for real replay metrics.

Replay refuses an unbound default policy: provide exported metadata or an explicit threshold. The metadata serializes all controller values. Once a turn emits END, the controller remains latched until speech/reset and records later callbacks with emit_response=false, so downstream code can never treat a repeated pause callback as a second response edge.