Smart Turn Hinglish: side-by-side demo vs stock v3.2

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+examples/01_hesitation_INCOMPLETE.wav filter=lfs diff=lfs merge=lfs -text
+examples/02_tagparticle_COMPLETE.wav filter=lfs diff=lfs merge=lfs -text
+examples/03_hesitation_INCOMPLETE.wav filter=lfs diff=lfs merge=lfs -text
+examples/04_tagparticle_COMPLETE.wav filter=lfs diff=lfs merge=lfs -text
+examples/05_SAMEsentence_COMPLETE.wav filter=lfs diff=lfs merge=lfs -text
+examples/06_SAMEsentence_INCOMPLETE.wav filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,13 +1,70 @@
----
-title: Smart Turn Hinglish
-emoji: 🦀
-colorFrom: green
-colorTo: blue
-sdk: gradio
-sdk_version: 6.26.0
-python_version: '3.13'
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: Smart Turn Hinglish
+emoji: 🗣️
+colorFrom: indigo
+colorTo: green
+sdk: gradio
+sdk_version: 5.50.0
+app_file: app.py
+pinned: false
+license: bsd-2-clause
+---
+
+# Smart Turn — Hinglish
+
+Audio-only **turn detection** for Indian Hinglish, Indian-accented English and
+Indian filler words — compared live against stock
+[Smart Turn v3.2](https://huggingface.co/pipecat-ai/smart-turn-v3).
+
+**Complete** = the speaker has finished, a voice bot may reply.
+**Incomplete** = they are mid-thought; replying now interrupts them.
+
+## Why this exists
+
+Indian fillers split into two families that sound similar and label **opposite**:
+
+| Ends with | Label | Examples |
+|---|---|---|
+| hesitation marker | incomplete | matlab, toh, woh, arre, yaani |
+| tag particle | complete | na, haan, ji, achha, theek hai |
+
+Stock v3.2 has never heard them. On held-out Hinglish it scores **0.596 on tag
+particles** — barely above chance. This model scores **0.907**.
+
+## Results
+
+| | stock v3.2 | this model |
+|---|---|---|
+| Hinglish accuracy | 0.621 | **0.909** |
+| False-interruption rate | 46.9% | **7.0%** |
+| Tag-particle accuracy | 0.596 | **0.907** |
+| CPU p95 latency | 40.4 ms | 42.5 ms |
+| Params | 8M | 8M |
+
+False-interruption is the error that matters: the bot talks over a human and the
+user has to start again. A false *wait* just adds a short pause.
+
+On Pipecat's own multilingual test set, stock shows the same asymmetry —
+**22.4% false-interruption on Indic languages vs 8.4% elsewhere**, with false-wait
+identical (5.78% vs 5.90%).
+
+## Try the examples
+
+`05` and `06` are the same sentence in the same voice, one complete and one cut
+off mid-thought. Only the ending differs — which is the whole point: turn
+completion lives in the terminal prosody, not the words.
+
+## Limitations
+
+Training audio is synthetic (Sarvam Bulbul v3, 36 voices). The human eval set is
+60 hand-labelled FLEURS clips — read Hindi, not conversational, and containing no
+Indian-accented English. Full detail in the
+[GitHub repo](https://github.com/abhinav7289A/Hinglish-turn-detection).
+
+## Credits
+
+Built on [pipecat-ai/smart-turn](https://github.com/pipecat-ai/smart-turn)
+(BSD-2-Clause) and `openai/whisper-tiny`. `models/smart-turn-v3.2-cpu.onnx` is
+Pipecat's unmodified released checkpoint, redistributed here under BSD-2-Clause
+so the side-by-side comparison works without a network fetch. Hinglish speech
+synthesised with Sarvam Bulbul v3; evaluation clips from Google FLEURS.

app.py

@@ -0,0 +1,202 @@
+"""Gradio demo -- stock Smart Turn v3.2 vs our Hinglish model, side by side.
+
+The side-by-side IS the demo. Anyone evaluating this should see the difference
+in about five seconds without reading a table, so the layout is two probability
+bars and a one-line verdict, and the preloaded examples are chosen to land on
+the exact cases stock v3 gets wrong: a hesitation filler that should read as
+incomplete, a tag particle that should read as complete, and an Indian-English
+declarative with a non-falling terminal contour.
+
+Function over form -- CLAUDE.md lists over-polishing this as a known time sink.
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import gradio as gr
+import numpy as np
+
+from src.audio_utils import build_ort_session, extract_features, ort_predict
+from src.config import EVAL_MANIFEST, MODELS, RESULTS, SAMPLE_RATE, TRAIN_MANIFEST
+
+OURS = MODELS / "smart-turn-hinglish-int8.onnx"
+STOCK = MODELS / "smart-turn-v3.2-cpu.onnx"
+EXAMPLES_DIR = Path(__file__).resolve().parent / "examples"
+
+_sessions: dict[str, object] = {}
+
+
+def ensure_stock() -> None:
+    """Fetch stock v3.2 if it is not vendored.
+
+    Our own ONNX is committed, but stock's is gitignored (it is Pipecat's file,
+    not ours to redistribute in-repo). On a fresh Hugging Face Space the
+    comparison half of the demo would silently disappear, so pull it at startup.
+    """
+    if STOCK.exists():
+        return
+    try:
+        from huggingface_hub import hf_hub_download
+        MODELS.mkdir(parents=True, exist_ok=True)
+        src = hf_hub_download("pipecat-ai/smart-turn-v3", "smart-turn-v3.2-cpu.onnx")
+        STOCK.write_bytes(Path(src).read_bytes())
+        print(f"downloaded stock v3.2 -> {STOCK}")
+    except Exception as e:
+        print(f"could not fetch stock v3.2 ({e}); running without the comparison")
+
+
+ensure_stock()
+
+
+def session(path: Path):
+    key = str(path)
+    if key not in _sessions:
+        if not path.exists():
+            return None
+        _sessions[key] = build_ort_session(path)
+    return _sessions[key]
+
+
+def _threshold() -> float:
+    """Use the operating point chosen in Phase 7, not a bare 0.5."""
+    p = RESULTS / "benchmark.json"
+    if not p.exists():
+        return 0.5
+    try:
+        report = json.loads(p.read_text())
+        for run in ("e2_d", "e2_c", "e1"):
+            s = report.get(run, {}).get("splits", {}).get("human")
+            if s:
+                return float(s["chosen_threshold"]["threshold"])
+    except Exception:
+        pass
+    return 0.5
+
+
+THR = _threshold()
+
+
+def predict(audio) -> tuple[dict, dict, str]:
+    if audio is None:
+        return {}, {}, "Record or upload a clip to compare the two models."
+
+    sr, data = audio
+    data = np.asarray(data, dtype=np.float32)
+    if data.ndim > 1:
+        data = data.mean(axis=1)
+    peak = float(np.max(np.abs(data))) or 1.0
+    data = data / max(peak, 1.0) if peak > 1.0 else data
+    if data.dtype.kind in "iu" or peak > 1.5:
+        data = data / 32768.0
+    if sr != SAMPLE_RATE:
+        import librosa
+        data = librosa.resample(data, orig_sr=sr, target_sr=SAMPLE_RATE)
+
+    feats = extract_features(data)
+    ours_s, stock_s = session(OURS), session(STOCK)
+    p_ours = ort_predict(ours_s, feats) if ours_s else float("nan")
+    p_stock = ort_predict(stock_s, feats) if stock_s else float("nan")
+
+    ours_lbl = {"complete (bot may reply)": p_ours, "incomplete (keep listening)": 1 - p_ours}
+    stock_lbl = {"complete (bot may reply)": p_stock, "incomplete (keep listening)": 1 - p_stock}
+
+    def verdict(p, thr):
+        return "COMPLETE" if p > thr else "INCOMPLETE"
+
+    agree = verdict(p_ours, THR) == verdict(p_stock, 0.5)
+    note = (
+        f"**Ours (Hinglish):** {verdict(p_ours, THR)} — p={p_ours:.3f} "
+        f"(threshold {THR:.2f}, chosen to hold false-interruptions under 5%)\n\n"
+        f"**Stock Smart Turn v3.2:** {verdict(p_stock, 0.5)} — p={p_stock:.3f} "
+        f"(threshold 0.50)\n\n"
+        + ("The two models agree on this clip."
+           if agree else
+           "**They disagree.** This is the kind of clip the Hinglish training targets — "
+           "typically a hesitation marker (*matlab, toh, woh*) that should read as "
+           "incomplete, a tag particle (*na, haan, ji*) that should read as complete, "
+           "or an Indian-English declarative whose terminal pitch does not fall.")
+    )
+    return ours_lbl, stock_lbl, note
+
+
+def load_examples(n: int = 6) -> list[list]:
+    """Pick contrastive examples: hesitation, tag, and plain, both classes.
+
+    Prefers the committed `examples/` folder, which is what a deployed Space
+    has; falls back to the local manifests when running from a full checkout.
+    """
+    if EXAMPLES_DIR.exists():
+        wavs = sorted(EXAMPLES_DIR.glob("*.wav"))
+        if wavs:
+            return [[str(w)] for w in wavs[:n]]
+
+    rows: list[dict] = []
+    for manifest in (EVAL_MANIFEST, TRAIN_MANIFEST):
+        if manifest.exists():
+            rows += [json.loads(l) for l in
+                     manifest.read_text(encoding="utf-8").splitlines() if l.strip()]
+    wanted = [("hesitation", 0), ("tag", 1), ("none", 1), ("none", 0),
+              ("hesitation", 0), ("tag", 1)]
+    picked, seen = [], set()
+    for want in wanted:
+        for r in rows:
+            path = Path(r.get("path", ""))
+            if (r.get("filler_type"), r.get("label")) == want \
+                    and str(path) not in seen and path.exists():
+                picked.append([str(path)])
+                seen.add(str(path))
+                break
+    for r in rows:
+        if len(picked) >= n:
+            break
+        path = Path(r.get("path", ""))
+        if str(path) not in seen and path.exists():
+            picked.append([str(path)])
+            seen.add(str(path))
+    return picked[:n]
+
+
+def build() -> gr.Blocks:
+    missing = [p.name for p in (OURS, STOCK) if not p.exists()]
+    banner = (f"\n\n⚠️ Missing model file(s): {missing}. "
+              f"Run `python run.py export` first." if missing else "")
+
+    with gr.Blocks(title="Smart Turn Hinglish") as demo:
+        gr.Markdown(
+            "# Smart Turn — Hinglish\n"
+            "Audio-only turn detection tuned for Indian Hinglish, Indian-accented "
+            "English, and Indian filler words. Compared live against stock "
+            "Smart Turn v3.2.\n\n"
+            "**Complete** = the speaker has finished and a voice bot may reply. "
+            "**Incomplete** = they are mid-thought; replying now interrupts them."
+            + banner)
+
+        with gr.Row():
+            audio_in = gr.Audio(sources=["microphone", "upload"], type="numpy",
+                                label="Record or upload (16 kHz mono, ≤8 s used)")
+        btn = gr.Button("Compare", variant="primary")
+
+        with gr.Row():
+            ours_out = gr.Label(label="Ours — Hinglish fine-tuned", num_top_classes=2)
+            stock_out = gr.Label(label="Stock — Smart Turn v3.2", num_top_classes=2)
+        note_out = gr.Markdown()
+
+        btn.click(predict, [audio_in], [ours_out, stock_out, note_out])
+        audio_in.change(predict, [audio_in], [ours_out, stock_out, note_out])
+
+        examples = load_examples()
+        if examples:
+            gr.Examples(examples=examples, inputs=[audio_in],
+                        label="Examples — the filler/tag pairs are where the two differ")
+        gr.Markdown(
+            "---\n*Hesitation markers* (matlab, toh, woh, arre, yaani) signal "
+            "mid-thought → **incomplete**. *Tag particles* (na, haan, ji, theek hai) "
+            "hand over the turn → **complete**. They sound similar and label "
+            "oppositely, which is why a model that has never heard them gets both "
+            "directions wrong.")
+    return demo
+
+
+if __name__ == "__main__":
+    build().launch()

examples/01_hesitation_INCOMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7264ea004fd95fe41af9983c30dd860f30ccfafcb364f18d4974a862a0fae8d
+size 125120

examples/02_tagparticle_COMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1239f83fe4fe359b6c2b45857dd304a8b0b1499ab7f6ec09ca618223887c317
+size 170784

examples/03_hesitation_INCOMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:361370d08006dea5b0af2dd26877ba37b68e70a645eecc70689453345034d281
+size 146784

examples/04_tagparticle_COMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cddff0d07b9b5f77d0029f3fe9a54023dbe2be669de4fdc2f623e1ffa46d1e7
+size 163478

examples/05_SAMEsentence_COMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f8cf51621dfe3235d260ddf2a51adcb58019118bff71e0381b6113dacff0c8b
+size 165824

examples/06_SAMEsentence_INCOMPLETE.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e8cbffd88210426d8809b0a620b14d0097fec574592a3b6b3462d692b6d2783
+size 149940

examples/README.md

@@ -0,0 +1,12 @@
+# Demo examples
+
+Hesitation markers (*matlab, toh, woh, yaani*) mark mid-thought -> **incomplete**. Tag particles (*na, haan, ji, theek hai*) hand over the turn -> **complete**. They sound alike and label oppositely, which is exactly where stock Smart Turn v3.2 fails (0.596 accuracy on tags).
+
+Clips 05/06 are the **same sentence, same voice** — only the ending differs.
+
+- `01_hesitation_INCOMPLETE.wav` — label 0 (hesitation): Aapka KYC update pending hai yaani... woh last date kab tak hai submit karne ki?
+- `02_tagparticle_COMPLETE.wav` — label 1 (tag): Bhaiya metro card recharge karna hai but OTP nahi aa raha hai since morning, kya
+- `03_hesitation_INCOMPLETE.wav` — label 0 (hesitation): Bhaiya mera metro card recharge nahi ho raha hai toh... kya bolte hain, koi alte
+- `04_tagparticle_COMPLETE.wav` — label 1 (tag): Yaar vo courier wale ne phir se galat address pe delivery kar di, humein abhi ca
+- `05_SAMEsentence_COMPLETE.wav` — label 1 (hesitation): Geyser repair ke liye bill pay karna hai matlab... yaani, EMI option available h
+- `06_SAMEsentence_INCOMPLETE.wav` — label 0 (hesitation): Geyser repair ke liye bill pay karna hai matlab... yaani, EMI option available h

models/smart-turn-hinglish-int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf268d9b6b4a0a5d994cee14974ca5f576abf218a242d7f5532ad18ef1c8f52f
+size 8244451

models/smart-turn-v3.2-cpu.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bb026316b14a660486a75b1733cd3fbab8c2fd0314dc9af7be49f8cca967e4f
+size 8679182

requirements.txt

@@ -0,0 +1,12 @@
+# Hugging Face Space requirements -- runtime only.
+# Deliberately excludes torch, faster-whisper, silero-vad, datasets and
+# umap-learn: those are needed to BUILD the corpus and train, not to run
+# inference. The demo is pure ONNX Runtime.
+gradio>=5.0,<6.0
+onnxruntime>=1.18
+numpy>=1.26,<2.0
+librosa>=0.10
+soundfile>=0.12
+transformers>=4.40,<5.0
+huggingface_hub>=0.34,<1.0
+python-dotenv>=1.0

src/audio_utils.py

@@ -0,0 +1,169 @@
+"""The preprocessing contract.
+
+Reproduced from pipecat-ai/smart-turn `audio_utils.py` + `inference.py`.
+Every model in this project -- stock v3.2, E1, the E2 sweep, E3 -- sees audio
+through THIS module and nothing else. If these functions drift, no number in
+`results/` is comparable to anything.
+
+The order matters and is easy to get wrong:
+    1. left-zero-pad (or keep the LAST 8 s) to exactly 8 s
+    2. THEN hand to WhisperFeatureExtractor(chunk_length=8)
+Doing it the other way round right-pads, and a padding bug measurably hurt
+official v3.1 before it was fixed in v3.2.
+"""
+from __future__ import annotations
+
+from functools import lru_cache
+from pathlib import Path
+
+import numpy as np
+
+from .config import MAX_AUDIO_S, N_FRAMES, N_MELS, SAMPLE_RATE
+
+MAX_SAMPLES = MAX_AUDIO_S * SAMPLE_RATE
+
+
+def truncate_or_leftpad(audio: np.ndarray, n_seconds: int = MAX_AUDIO_S,
+                        sample_rate: int = SAMPLE_RATE) -> np.ndarray:
+    """Keep the last `n_seconds`, or left-pad with zeros up to it.
+
+    Verbatim behaviour of pipecat's `truncate_audio_to_last_n_seconds`.
+    Left padding is deliberate: the decision lives at the END of the clip, so
+    the terminal contour must always land at the same position in the window.
+    """
+    max_samples = n_seconds * sample_rate
+    if len(audio) > max_samples:
+        return audio[-max_samples:]
+    if len(audio) < max_samples:
+        return np.pad(audio, (max_samples - len(audio), 0),
+                      mode="constant", constant_values=0)
+    return audio
+
+
+@lru_cache(maxsize=1)
+def get_feature_extractor():
+    from transformers import WhisperFeatureExtractor
+    return WhisperFeatureExtractor(chunk_length=MAX_AUDIO_S)
+
+
+def extract_features(audio: np.ndarray, batched: bool = True) -> np.ndarray:
+    """float32 waveform at 16 kHz -> log-mel (1, 80, 800) float32.
+
+    `batched=False` returns (80, 800), for writing into a feature cache.
+    """
+    audio = np.asarray(audio, dtype=np.float32)
+    audio = truncate_or_leftpad(audio)
+    inputs = get_feature_extractor()(
+        audio,
+        sampling_rate=SAMPLE_RATE,
+        return_tensors="np",
+        padding="max_length",
+        max_length=MAX_SAMPLES,
+        truncation=True,
+        do_normalize=True,
+    )
+    feats = inputs.input_features.squeeze(0).astype(np.float32)
+    return feats[None, ...] if batched else feats
+
+
+def load_audio(path: str | Path, sr: int = SAMPLE_RATE) -> np.ndarray:
+    """Load any file to mono float32 at `sr`, peak-normalised into [-1, 1].
+
+    Manifests store repo-relative paths so they survive the trip to Kaggle and
+    the Hub. Resolve against the repo root as a fallback so callers work
+    regardless of the current working directory.
+    """
+    import librosa
+    p = Path(path)
+    if not p.exists() and not p.is_absolute():
+        from .config import ROOT
+        if (ROOT / p).exists():
+            p = ROOT / p
+    audio, _ = librosa.load(str(p), sr=sr, mono=True)
+    audio = audio.astype(np.float32)
+    peak = float(np.max(np.abs(audio))) if audio.size else 0.0
+    if peak > 1.0:
+        audio = audio / peak
+    return audio
+
+
+def decode_bytes(raw: bytes, sr: int = SAMPLE_RATE) -> np.ndarray:
+    """Decode an in-memory audio blob (FLAC/WAV from HF parquet) to mono 16 kHz."""
+    import io
+
+    import librosa
+    import soundfile as sf
+    data, src_sr = sf.read(io.BytesIO(raw), dtype="float32", always_2d=False)
+    if data.ndim > 1:
+        data = data.mean(axis=1)
+    if src_sr != sr:
+        data = librosa.resample(data, orig_sr=src_sr, target_sr=sr)
+    return np.ascontiguousarray(data, dtype=np.float32)
+
+
+def build_ort_session(onnx_path: str | Path):
+    """ONNX Runtime session with pipecat's exact options.
+
+    These options are part of the latency claim -- benchmarking under different
+    ones would not be comparable to their published 12 ms.
+    """
+    import onnxruntime as ort
+    so = ort.SessionOptions()
+    so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
+    so.inter_op_num_threads = 1
+    so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+    return ort.InferenceSession(str(onnx_path), sess_options=so,
+                                providers=["CPUExecutionProvider"])
+
+
+def build_fast_session(onnx_path: str | Path):
+    """Default threading -- for BULK SCORING ONLY.
+
+    `build_ort_session` deliberately pins ORT_SEQUENTIAL and one inter-op thread
+    because those options ARE Pipecat's published latency contract; benchmarking
+    under anything else would not be comparable. Accuracy scoring has no such
+    contract, so running thousands of clips single-threaded is pure waste.
+
+    Never use this for latency_bench.
+    """
+    import onnxruntime as ort
+    so = ort.SessionOptions()
+    so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+    return ort.InferenceSession(str(onnx_path), sess_options=so,
+                                providers=["CPUExecutionProvider"])
+
+
+def ort_predict_batch(session, feats: np.ndarray) -> np.ndarray:
+    """(B, 80, 800) -> (B,) probabilities. Uses the graph's dynamic batch axis."""
+    out = session.run(None, {"input_features": feats.astype(np.float32)})
+    return np.asarray(out[0]).reshape(-1)
+
+
+def ort_predict(session, feats: np.ndarray) -> float:
+    """Run a session on (1, 80, 800) features. Output is ALREADY sigmoided."""
+    out = session.run(None, {"input_features": feats.astype(np.float32)})
+    return float(np.asarray(out[0]).reshape(-1)[0])
+
+
+def self_test() -> None:
+    """Contract test. Cheap, and it catches the highest-risk bug in the project."""
+    short = np.ones(SAMPLE_RATE, dtype=np.float32) * 0.5      # 1 s
+    padded = truncate_or_leftpad(short)
+    assert padded.shape == (MAX_SAMPLES,), padded.shape
+    assert np.all(padded[: MAX_SAMPLES - SAMPLE_RATE] == 0), "must pad on the LEFT"
+    assert np.all(padded[MAX_SAMPLES - SAMPLE_RATE:] == 0.5), "signal must land at the END"
+
+    long = np.arange(12 * SAMPLE_RATE, dtype=np.float32)      # 12 s
+    kept = truncate_or_leftpad(long)
+    assert kept.shape == (MAX_SAMPLES,)
+    assert kept[-1] == long[-1], "must keep the LAST 8 s, not the first"
+
+    feats = extract_features(short)
+    assert feats.shape == (1, N_MELS, N_FRAMES), feats.shape
+    assert feats.dtype == np.float32
+    assert np.isfinite(feats).all()
+    print(f"audio_utils self-test OK  -> {feats.shape} {feats.dtype}")
+
+
+if __name__ == "__main__":
+    self_test()

src/config.py

@@ -0,0 +1,191 @@
+"""Single source of truth for the whole project.
+
+Every path, seed, budget and hyperparameter lives here. Nothing downstream
+hardcodes a constant; if a number matters, it is in this file.
+"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+from dotenv import load_dotenv
+
+ROOT = Path(__file__).resolve().parent.parent
+load_dotenv(ROOT / ".env")
+
+# ---------------------------------------------------------------- paths
+DATA = ROOT / "data"
+RAW = DATA / "raw"
+TEXT_DIR = DATA / "hinglish_text"
+AUDIO_DIR = DATA / "hinglish_audio"
+TRAIN_DIR = DATA / "hinglish_train"
+EVAL_DIR = DATA / "eval_human"
+NOISE_DIR = DATA / "noise"
+MODELS = ROOT / "models"
+RESULTS = ROOT / "results"
+EDA_DIR = RESULTS / "eda"
+FIG_DIR = RESULTS / "figures"
+NOTES = ROOT / "notes.md"
+
+for _d in (RAW, TEXT_DIR, AUDIO_DIR, TRAIN_DIR, EVAL_DIR, NOISE_DIR,
+           MODELS, RESULTS, EDA_DIR, FIG_DIR):
+    _d.mkdir(parents=True, exist_ok=True)
+
+
+def rel(p) -> str:
+    """Repo-relative POSIX path, for anything written into a manifest.
+
+    Manifests travel to Kaggle and to the HF Hub, so an absolute Windows path
+    with backslashes is useless on the other end. Everything that records a
+    clip location goes through this.
+    """
+    p = Path(p).resolve()
+    try:
+        return p.relative_to(ROOT).as_posix()
+    except ValueError:
+        return p.as_posix()
+
+
+UTTERANCES = TEXT_DIR / "utterances.jsonl"
+TRAIN_MANIFEST = TRAIN_DIR / "manifest.jsonl"
+EVAL_MANIFEST = EVAL_DIR / "manifest.jsonl"
+EVAL_FROZEN = EVAL_DIR / "FROZEN.json"
+TEST_SUBSET = RAW / "test_subset"
+
+# ---------------------------------------------------------------- audio contract
+# These four numbers ARE the Pipecat contract. Changing any of them silently
+# invalidates every comparison against stock Smart Turn v3.
+SAMPLE_RATE = 16_000
+MAX_AUDIO_S = 8
+N_MELS = 80
+N_FRAMES = 800          # 8 s * 100 frames/s
+ENCODER_POSITIONS = 400  # N_FRAMES // 2 (Whisper conv stride 2)
+
+SEED = 42
+
+# ---------------------------------------------------------------- upstream ids
+STOCK_REPO = "pipecat-ai/smart-turn-v3"
+STOCK_INT8 = "smart-turn-v3.2-cpu.onnx"
+STOCK_FP32 = "smart-turn-v3.2-gpu.onnx"
+BASE_MODEL = "openai/whisper-tiny"
+DS_TRAIN = "pipecat-ai/smart-turn-data-v3.2-train"
+DS_TEST = "pipecat-ai/smart-turn-data-v3.2-test"
+# Overridable from the environment so the Kaggle notebook can change them for a
+# subprocess (`!python -m src.prepare`) -- assigning to the module in the parent
+# process would have no effect there.
+N_TRAIN_SHARDS = int(os.getenv("N_TRAIN_SHARDS", "20"))   # of 83; ~65k rows, ~10 GB
+N_TOTAL_SHARDS = 83
+
+# ---------------------------------------------------------------- eval subset
+TEST_SUBSET_PER_LANG = 500   # cap per language
+TEST_SUBSET_TARGET = 6000    # ~ +/-1.3% at 95% CI
+
+# ---------------------------------------------------------------- budgets
+SARVAM_CHAR_LIMIT = 70_000       # HARD stop (spec: 83k affordable, stop at 70k)
+SARVAM_URL = "https://api.sarvam.ai/text-to-speech"
+SARVAM_MODEL = "bulbul:v3"
+SARVAM_VOICES = [
+    "shubh", "aditya", "ritu", "priya", "neha", "rahul", "pooja", "rohan",
+    "simran", "kavya", "amit", "dev", "ishita", "shreya", "ratan", "varun",
+    "manan", "sumit", "roopa", "kabir", "aayan", "ashutosh", "advait", "anand",
+    "tanya", "tarun", "sunny", "mani", "gokul", "vijay", "shruti", "suhani",
+    "mohit", "kavitha", "rehan", "soham", "rupali",
+]
+SARVAM_KEY = os.getenv("SARVAM_API_KEY", "")
+
+OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
+# deepseek-chat-v3-0324 is the only DeepSeek on OpenRouter with NO reasoning
+# capability at all -- its supported_parameters list has no `reasoning`,
+# `reasoning_effort` or `include_reasoning`. That matters: every v4/v3.2 variant
+# is a hybrid reasoner, and `reasoning={"exclude": True}` only HIDES the thinking,
+# it does not stop it. The model burns the whole completion budget thinking and
+# returns content=null with finish_reason=length. A model that cannot think
+# cannot do that to us.
+#
+# Costs ~$0.20 for the full corpus instead of ~$0.03. Irrelevant next to the
+# wall clock; buy the reliability.
+OPENROUTER_MODEL = os.getenv("OPENROUTER_MODEL", "deepseek/deepseek-chat-v3-0324")
+OPENROUTER_KEY = os.getenv("OPENROUTER_API_KEY", "")
+
+HF_USER = os.getenv("HF_USER", "")
+HF_CORPUS_REPO = f"{HF_USER}/hinglish-turn-detection" if HF_USER else ""
+HF_EVAL_REPO = f"{HF_USER}/hinglish-turn-eval" if HF_USER else ""
+
+# ---------------------------------------------------------------- G3: filler taxonomy
+# The sharpest differentiator: these two classes sound similar and label OPPOSITE.
+HESITATION_FILLERS = [       # mid-thought -> label 0 (incomplete)
+    "matlab", "toh", "woh", "arre", "yaani", "kya bolte hain",
+    "iska matlab", "aisa hai ki", "kaise bolun",
+]
+TAG_PARTICLES = [            # terminal tag -> label 1 (complete)
+    "na", "haan", "ji", "achha", "theek hai", "hai na", "bas", "hi na",
+]
+
+REGISTERS = ["hinglish", "indian_english", "hindi_en_nouns"]
+REGISTER_MIX = {"hinglish": 0.50, "indian_english": 0.30, "hindi_en_nouns": 0.20}
+DOMAINS = ["booking", "complaint", "food_delivery", "casual",
+           "bank_telecom", "directions"]
+
+N_UTTERANCES = 2000
+N_HESITATION = 400
+N_TAG = 300
+
+# ---------------------------------------------------------------- text gen
+# Small batches on purpose. Providers cap completion length well below whatever
+# `max_tokens` we request (OpenInference truncated us at ~1,700 tokens), and 25
+# utterances of JSON needs ~2,000+. Asking for 8 keeps every response inside any
+# plausible cap; the extra calls are parallel and cost fractions of a cent.
+GEN_BATCH = 8            # utterances per LLM request
+GEN_WORKERS = 20
+GEN_MAX_TOKENS = 4000
+DUP_JACCARD = 0.80       # char-3-gram near-duplicate threshold
+
+# ---------------------------------------------------------------- segmentation
+TRAIL_SILENCE_MS = (200, 400)   # appended after every truncation
+DURATION_JS_MAX = 0.15          # class duration histograms must overlap
+
+# Word-boundary alignment decides where every label-0 clip gets cut, so it is
+# worth the extra compute. Bulk EDA transcription only needs to be good enough
+# to spot Latin-vs-Devanagari code-switching, and `small` on 2,000 CPU clips
+# costs about an hour we do not have.
+# `base` not `small` for alignment: we only need word BOUNDARIES, not accurate
+# transcription, and every cut lands on a word edge followed by 200-400 ms of
+# appended silence -- so a +/-100 ms timestamp error is absorbed. `small` is
+# ~3x slower on CPU for no benefit we can hear.
+ASR_MODEL = "base"              # forced alignment (segment.py)
+EDA_ASR_MODEL = "tiny"          # bulk transcription (eda.py)
+
+# ---------------------------------------------------------------- augmentation
+AUG_TELEPHONY_P = 0.30    # mu-law 8 kHz round trip  (G5)
+AUG_NOISE_P = 0.40        # ambient noise 5-20 dB SNR
+AUG_SNR_DB = (5.0, 20.0)
+AUG_SPEED = (0.95, 1.05)
+AUG_GAIN_DB = (-4.0, 4.0)
+
+# ---------------------------------------------------------------- training
+TRAIN = dict(
+    encoder=BASE_MODEL,
+    lr_encoder=5e-5,
+    lr_head=1e-3,
+    weight_decay=0.01,
+    batch_size=64,
+    epochs=3,
+    warmup_ratio=0.1,
+    early_stop_patience=2,
+    monitor="val_hinglish_acc",
+    seed=SEED,
+    amp=True,
+    grad_clip=1.0,
+    val_frac=0.1,
+)
+E2_SWEEP = [0.0, 0.05, 0.15, 0.30]     # Hinglish fraction of each batch
+BASE_SUBSET_TARGET = int(os.getenv("BASE_SUBSET_TARGET", "40000"))
+
+# ---------------------------------------------------------------- evaluation
+FALSE_INTERRUPT_BUDGET = 0.05   # pick threshold holding FI below this
+N_BOOTSTRAP = 2000
+LATENCY_RUNS = 200
+
+RUN_NAMES = ["stock", "e1", "e2_a", "e2_b", "e2_c", "e2_d", "e3"]
+QUANT_DELTA_N = 2000     # +/-1.1% at 95% CI; ample for an int8-vs-fp32 delta
+MAX_QUANT_LOSS_PP = 1.5  # int8 must stay within this of fp32, else ship fp32