# Hinglish benchmark collection protocol

## Purpose

This benchmark measures whether a voice agent would interrupt Indian Hinglish
speakers at natural pauses. It is not a speaker-identification dataset. Do not
collect names, phone numbers, real addresses, order IDs, or other customer data.

## Consent and governance

Before recording, each adult participant must receive and affirm:

1. The recording purpose, expected duration, and examples of intended use.
2. That raw voice is biometric/personal data and participation is voluntary.
3. The chosen redistribution license—or an explicit statement that recordings
   remain private and only aggregate metrics/model updates will be published.
4. A deletion/contact process and retention period.

Store the signed consent record separately from audio. Use random speaker IDs in
all manifests. The `data/private/` directory is gitignored. This repository does
not provide legal advice; the collector remains responsible for applicable law
and organizational policy.

## Recording

- Target 30 speakers with varied regions, genders, devices, and acoustic spaces.
- Generate 30 balanced assignments per speaker with
  `scripts/generate_hinglish_protocol.py`.
- Record 16 kHz or higher, mono preferred, lossless WAV/FLAC.
- For `END`, speak the full request naturally and stop.
- For `HOLD`, speak the entire request and take a natural 0.3–2.0 second pause at
  `<PAUSE>` before continuing. Do not stop the recording at the marker.
- Repeat failed recordings; do not silently relabel them.

The generated split is speaker-disjoint. Never move recordings between splits
after looking at model predictions.

## Annotation

At each derived pause checkpoint, three annotators independently answer:

> If the agent responded now, would it feel like an interruption?

Options: `yes`, `no`, `uncertain`, plus a short optional reason. Store all votes.
The soft endpoint target is `no_votes / valid_votes`; preserve `uncertain` as
disagreement rather than coercing it into a confident binary label. Report raw
agreement and a chance-corrected statistic.

## Release gate

Before publishing any audio, confirm consent scope and license, remove accidental
PII, run duplicate checks, and manually listen to a stratified quality sample.
If redistribution rights are unclear, publish only the protocol, anonymized
metadata, aggregate metrics, and trained artifacts whose terms have been reviewed.