DATA_CARD.md

---
pretty_name: Tiny Hinglish Turn Detection Data Plan
language:
- hi
- en
task_categories:
- audio-classification
license: other
---

# Data card: Tiny Hinglish Turn Detection

This project uses two distinct data components:

1. an external multilingual Smart Turn corpus for representation training and
   benchmark compatibility; and
2. a proposed, consent-first Indian Hinglish hard-case benchmark for the actual
   deployment domain.

No upstream audio or locally recorded voice is distributed in this repository.
Only data-preparation code, small machine-readable reports, and the local prompt
assignment plan are included.

## Component A: Smart Turn v3.2

### Source and versions

| Role | Hugging Face dataset | Pinned revision | Catalog scope | Local state |
|---|---|---|---:|---|
| Train/development | [`pipecat-ai/smart-turn-data-v3.2-train`](https://huggingface.co/datasets/pipecat-ai/smart-turn-data-v3.2-train) | `e564e2ac567f774d1880aa1db6ce97afb8c519b7` | 270,946 rows, 83 Parquet shards, about 41.4 GB | 1 shard present and audited |
| Official test | `pipecat-ai/smart-turn-data-v3.2-test` | `0500378e8ed6d38e37b016e24d261e8e6c6a6859` | 31,527 rows, 10 shards, about 4.84 GB | Sealed; not downloaded or inspected |

Catalog sizes identify the expected snapshot; they are not local row-count
claims. The download scripts verify the exact shard count before declaring a
snapshot complete.

### Intended task and label schema

The normalized schema is one row per labeled audio example:

| Field | Type | Meaning |
|---|---|---|
| `record_id` | string | Upstream stable ID, or provenance-based fallback |
| `audio` | encoded bytes/path/array | Utterance or turn suffix audio |
| `endpoint` | required bool | `true = END`, `false = HOLD` |
| `midfiller` | nullable bool | Optional internal-filler auxiliary target |
| `endfiller` | nullable bool | Optional turn-final-filler auxiliary target |
| `synthetic` | nullable bool | Upstream synthetic-status tag |
| `language` | nullable string | Upstream language tag |
| `dataset` | nullable string | Upstream source/collection tag |
| `spoken_text` | nullable string | Transcript/text when supplied; never used at inference |
| `source_file`, `source_row` | provenance | Lazy resolution back to the raw row |

Nullable filler fields remain null. Converting null to `false` would inject
unobserved negative labels and bias the auxiliary tasks.

### Local audit scope

The only locally audited file is:

```text
data/raw/smart-turn-data-v3.2-train/data/train-00010-of-00083.parquet
```

Its evidence is stored in
[`reports/partial_shard_audit.json`](reports/partial_shard_audit.json).

| Property | Count/value |
|---|---:|
| Records | 3,265 |
| Valid records | 3,265 |
| Invalid records | 0 |
| Duration observed | 24,685.5167 seconds (6.857 hours) |
| Minimum / maximum duration | 0.36 / 29.56 seconds |
| Encoded bytes | 492,503,246 |
| Audio format | 3,265 FLAC |
| `endpoint=false` / `true` | 1,667 / 1,598 |
| `synthetic=false` / `true` | 553 / 2,712 |
| `midfiller=null` / `false` / `true` | 642 / 1,284 / 1,339 |
| `endfiller=null` / `false` / `true` | 642 / 1,718 / 905 |
| Unique language tags | 23 |
| Exact-audio duplicate hashes | 0 |
| Duplicate record IDs | 0 |
| Conflicting-label audio hashes | 0 |

Largest language counts are English 758, Spanish 206, French 161, Hindi 158,
and Dutch 153. The presence of Hindi and English rows does not establish
Hinglish code-switching within recordings.

The shard is about 83% synthetic. It may have been sharded by source or another
non-random rule; these distributions must not be extrapolated to the other 82
files.

### Validation and audio inspection

The audit is streaming and does not need to materialize the 41 GB corpus in
memory. For each row it:

1. normalizes known aliases without conflating false and missing values;
2. validates the required endpoint and audio fields;
3. inspects encoded audio for hash, bytes, format, rate, channels, frames,
   bit depth, and duration;
4. retains all errors and warnings in the manifest;
5. summarizes label/language/source/audio distributions and duplicate conflicts.

`--fail-on-error` still writes the manifest/report for diagnosis, then returns a
failure status if any row has hard errors.

### Leakage-aware grouping

The pipeline creates an edge between rows sharing any available:

- exact encoded-audio SHA-256;
- conversation/session/call identifier;
- speaker/user/actor identifier;
- TTS voice identifier;
- source recording or clip identifier;
- prompt/template/script/parent identifier;
- upstream record identifier;
- sufficiently long normalized transcript/prompt;
- audio basename when no content hash is available.

Metadata values are canonicalized and SHA-256 hashed before being written as
linkage keys. They are namespaced by source where appropriate. This reduces
identifier exposure but is pseudonymization, not guaranteed anonymization.

Union-find computes transitive components across all keys. A deterministic hash
of the component keys becomes `group_id`, independent of input row order. Split
assignment moves entire groups and validates that no group, audio hash, or
individual metadata key crosses boundaries.

The currently audited raw schema does not expose usable speaker, conversation,
session, or TTS-voice identifiers. As a result, all 3,265 base linkage groups
used by the IID split are singletons (largest group: one row; multi-row groups:
zero). The source stress split separately groups rows by 12 dataset values. The grouping logic
is ready for richer metadata, but the current artifact is only best-effort
row-disjoint and exact-duplicate-safe—not speaker- or voice-disjoint. Unknown
identity/template leakage remains possible, and a grouped bootstrap on this
shard is numerically a row bootstrap.

### Current split artifacts

The local-shard IID development split is deterministic with seed 42 and balances
the endpoint, language, synthetic, and source marginals:

| Split | Rows | END | HOLD | Synthetic | Human-tagged |
|---|---:|---:|---:|---:|---:|
| Train | 2,939 | 1,438 | 1,501 | 2,441 | 498 |
| Validation | 326 | 160 | 166 | 271 | 55 |

No crossing of the *observed* keys was detected; this does not close the missing
speaker/voice linkage gap. Evidence:
[`reports/partial_iid_split.json`](reports/partial_iid_split.json).

A source-held-out stress split has 2,617 train and 648 validation rows, also
with zero detected crossings. It is strongly confounded:

- train: 2,614 synthetic / 3 human-tagged, spanning 23 languages;
- validation: 98 synthetic / 550 human-tagged, only English and Spanish.

It measures joint collection-domain shift, not the independent causal effect of
an unseen source. Evidence:
[`reports/partial_source_holdout_split.json`](reports/partial_source_holdout_split.json).

### Current experiment-use boundary

The 326-row IID validation split was reused for checkpoint inspection, model
thresholds, and bounded follow-up experiments. Current model comparisons are
therefore adaptive development evidence, not an untouched holdout. Because all
IID groups are singletons, its grouped bootstrap is row bootstrap in effect.

On the 648-row source stress validation side, both the acoustic baseline and
TinyTCN rank near chance (AUROC 0.5539 and 0.5523). The comparison bootstrap has
only nine validation source groups and wide intervals. This supports a joint
domain-shift warning, not an unseen-source generalization claim.

The official-test catalog identifies 31,527 rows, but no official-test audio has
been downloaded, inspected, or evaluated. No test metric or Hinglish quality
claim exists.

### Known data limitations

- Only 1/83 train shards is locally characterized.
- The inspected shard is predominantly synthetic.
- The upstream label-generation and annotator-agreement process is not
  revalidated by this repository.
- Language tags do not measure code-switch position or Indian regional variety.
- Available source/domain fields are confounded with human/synthetic status and
  language.
- Exact hash checks do not catch acoustically identical audio with different
  encodings; future audits should add perceptual/audio fingerprints.
- Repeated short fillers are intentionally excluded from text grouping because
  grouping all “haan” examples would collapse unrelated turns; this leaves some
  template leakage risk.
- Transcripts can contain PII. Processed manifests are ignored by version
  control and should follow the same access/retention policy as raw data.
- Audio corruption checks validate decodability and headers, not semantic label
  correctness. A stratified listening audit is still required.

## Component B: proposed Shiprocket Hinglish benchmark

### Motivation

The target failures are natural Indian Hinglish holds: “haan…”, “matlab…”,
addresses and identifiers broken across pauses, self-corrections, enumeration,
and code-switching around logistics terms. Those behaviors cannot be certified
from aggregate upstream language tags.

### Planned composition

[`data/collection/assignments.jsonl`](data/collection/assignments.jsonl) contains
900 deterministic assignments generated with seed 20260823:

| Dimension | Planned count |
|---|---:|
| Speakers | 30 |
| Assignments per speaker | 30 |
| HOLD / END | 450 / 450 |
| Train | 21 speakers, 630 examples |
| Development | 4 speakers, 120 examples |
| Test | 5 speakers, 150 examples |

Scenarios cover address, COD, damaged parcels, delivery instructions, order ID,
phone, pickup, reschedule, return, support, and tracking. Minimal-pair prompts
change the pause location or completion while holding words/domain close. HOLD
recordings must continue naturally after a 0.3–2.0 second `<PAUSE>` marker; the
marker is an instruction and must not be spoken.

The assignments are not recordings. **Zero participant audio examples have been
collected, annotated, trained on, or evaluated.**

### Participant and acoustic sampling

Recruit adults across regions, gender identities, speech rates, and code-switch
styles, without turning demographic coverage into an identity-inference task.
Record across representative phone microphones, headsets, rooms, traffic/noise,
far-field conditions, and codecs. Store only the attributes participants consent
to disclose and only at a granularity justified by slice analysis.

### Annotation

At every derived pause checkpoint, three independent annotators answer:

> If the agent responded now, would it feel like an interruption?

Allowed votes are `yes`, `no`, and `uncertain`, with an optional short reason.
For a soft endpoint target, use `no_votes / valid_votes`. Preserve uncertain
votes and report disagreement rather than converting them into confident binary
labels. Report raw agreement and a chance-corrected statistic. Adjudication must
not use model predictions.

The speaker-disjoint split assignment is generated before model evaluation and
must not be changed after viewing errors.

### Consent, privacy, and governance

Before recording, each participant must receive and affirm:

- purpose, expected recording duration, and examples of use;
- that voice is biometric/personal data and participation is voluntary;
- whether audio, annotations, metadata, and derived weights may be redistributed;
- retention period, access controls, withdrawal/deletion contact and process;
- compensation and any intended commercial use.

Use random speaker IDs. Store identity/consent records separately from audio.
Never solicit or record real names, customer phone numbers, addresses, order IDs,
tracking IDs, or production support calls. Screen transcripts and a stratified
audio sample for accidental PII before any release.

Raw recordings belong under ignored private storage. If redistribution rights
are unclear, release only the protocol, non-identifying aggregate statistics,
and artifacts whose terms have been reviewed.

## Recommended full-data release checks

Before using either component in a public model claim:

1. verify all expected shards and pinned revisions;
2. audit 100% of rows and resolve hard errors;
3. report per-shard and full-corpus distributions to detect sharding effects;
4. add perceptual duplicate checks and a stratified manual listening sample;
5. freeze grouping and split policies;
6. report both IID and leave-one-source-out results with confounding warnings;
7. collect and lock the speaker-disjoint Hinglish test;
8. publish label definitions, agreement, exclusion counts, and slice denominators;
9. conduct PII, consent, retention, and redistribution reviews;
10. open the official upstream test once, after model and threshold freeze.

## Licensing

The inspected upstream dataset card did not state an explicit dataset license.
The `license: other` metadata on this card reflects mixed and unresolved data
provenance; it is not a grant of permission. Repository-authored code is
Apache-2.0, but that license does not apply automatically to upstream audio,
participant recordings, annotations, speaker likeness, or derived artifacts.

Confirm rights with the dataset owner and legal/privacy reviewers before
commercial use, redistribution, or public weight release.