Upload README.md with huggingface_hub

README.md

@@ -11,17 +11,13 @@ pinned: false
 
 # Anvaya: Multi-Modal Speech Disfluency & Articulation Screening Prototype
 
-## Abstract & Research Context
-
-Anvaya is an open-source exploratory research framework designed for assistive speech disfluency screening, acoustic phonation analysis, and phonological practice feedback. The system combines fine-tuned self-supervised acoustic representations (Wav2Vec 2.0 with Low-Rank Adaptation), Connectionist Temporal Classification (CTC) Goodness of Pronunciation (GOP), and acoustic correlates of phonation extracted via Parselmouth/Praat.
-
-**Scientific & Clinical Scope Disclaimer**: Anvaya is an educational and screening research prototype. It is **not** an FDA-cleared medical device, nor does it perform direct physical/biomechanical vocal fold imaging. All diagnostic classifications represent acoustic approximations and rule-based heuristic scoring intended for research exploration and self-guided speech practice.
+> **CLINICAL & RESEARCH DISCLAIMER**: Anvaya is an exploratory engineering prototype designed for speech screening, acoustic feature exploration, and self-guided practice feedback. It is **not** an FDA-cleared medical device, nor a substitute for formal diagnosis or clinical evaluation by a licensed Speech-Language Pathologist (SLP).
 
 ---
 
-## 1. System Architecture & Subsystem Boundaries
+## 1. Abstract & System Architecture
 
-The prototype executes four distinct acoustic feature extractors followed by a rule-weighted fusion scoring layer:
+Anvaya explores the integration of fine-tuned self-supervised speech representations (Wav2Vec 2.0 with Low-Rank Adaptation), Connectionist Temporal Classification (CTC) Goodness of Pronunciation (GOP), and acoustic correlates of voice quality extracted via Parselmouth/Praat.
 
 ```
 ===================================================================================
@@ -34,7 +30,7 @@ The prototype executes four distinct acoustic feature extractors followed by a r
 |                        ACOUSTIC SIGNAL PRECONDITIONING                          |
 |  - Sample Rate: 16,000 Hz Mono Float32                                          |
 |  - High-Pass Filter: 60 Hz 2nd-Order Butterworth (DC rumble attenuation)        |
-|  - Dynamic Energy Gate: Silence Guard (RMS thresholding)                        |
+|  - Dynamic Energy Gate: Silence Guard (RMS & amplitude thresholding)            |
 |  - Peak Normalization: Standardized to -1.0 dBFS                                |
 +---------------------------------------------------------------------------------+
                                         |
@@ -46,7 +42,7 @@ The prototype executes four distinct acoustic feature extractors followed by a r
 |  Neural Disfluency Classifier     |       |  Phonetic GOP & ASR Alignment     |
 |  - Wav2Vec 2.0 Base (768-dim)     |       |  - Wav2Vec 2.0 Base 960h CTC      |
 |  - LoRA Adapter (r=8, alpha=16)   |       |  - Needleman-Wunsch DP Alignment  |
-|  - Focal Loss Optimization        |       |  - Dynamic Phonetic Tolerance     |
+|  - Focal Loss (gamma=2.0)         |       |  - Phonological Substitution Rules|
 |  Outputs: P(Fluent), P(Stutter)   |       |  Outputs: Word Diffs, GOP Score   |
 +-----------------------------------+       +-----------------------------------+
           |                                                           |
@@ -57,221 +53,121 @@ The prototype executes four distinct acoustic feature extractors followed by a r
           v                                                           v
 +-----------------------------------+       +-----------------------------------+
 |  EXPERT MODULE 3:                 |       |  EXPERT MODULE 4:                 |
-|  Phonetic Flaw Heuristics         |       |  Acoustic Correlates of Phonation |
+|  Acoustic-Phonetic Flaw Rules     |       |  Acoustic Correlates of Phonation |
 |  - Rhotic Error Mapping ('r'->'w')|       |  - Parselmouth / Praat Core       |
 |  - Sibilant Lisping ('s'->'th')   |       |  - Pitch Period Tracking (F0)     |
-|  - Word Substitution Rules        |       |  - Local Jitter, Shimmer, HNR     |
-|  Outputs: Heuristic Flaw Flags    |       |  Outputs: Voice Roughness Metrics |
+|  - Formant Proximity (F3-F2)      |       |  - Local Jitter, Shimmer, HNR     |
+|  Outputs: Heuristic Flaw Findings |       |  Outputs: Acoustic Voice Quality  |
 +-----------------------------------+       +-----------------------------------+
                                         |
                                         v
 +---------------------------------------------------------------------------------+
-|                       MULTI-MODAL FUSION & UNCERTAINTY SCORING                  |
+|                    MULTI-MODAL FUSION & UNCERTAINTY SCORING                     |
 |  - Heuristic Fluency Index (0 to 100): 40% Stutter, 45% Pronunciation, 15% Voice|
+|  - Single-Shift Calibration: Offset applied strictly once at composite rating   |
 |  - Uncertainty & Confidence Scoring: Low / Moderate / High                      |
-|  - Baseline Calibration: Offset Transformation relative to "My Normal"          |
 |  - Evidence Audit Trail: JSON Telemetry Trace                                   |
 +---------------------------------------------------------------------------------+
 ```
 
 ---
 
-## 2. Experimental Results & Clinical Screening Metrics
-
-All metrics reported below were evaluated on a held-out test partition of 1,666 clips from out-of-training speakers to evaluate generalization.
-
-### 2.1 Complete Evaluation Matrix
-
-| Metric Category | Metric Name | Measured Value | Scientific Interpretation & Formula |
-| :--- | :--- | :--- | :--- |
-| **Neural Disfluency** | **Macro-F1 Score** | **71.47%** | Balanced harmonic mean across fluent and disfluent classes |
-| | **Overall Test Accuracy** | **71.49%** | Total correct classifications on held-out test split |
-| | **ROC-AUC** | **0.7669** | Area under the Receiver Operating Characteristic curve |
-| **Clinical Screening**| **Sensitivity (Recall)**| **75.71%** | True Positive Rate: Percentage of actual stutter events identified |
-| | **Specificity (TNR)** | **67.28%** | True Negative Rate: Percentage of fluent segments correctly cleared |
-| | **Precision (PPV)** | **85.05% - 97.58%**| Positive Predictive Value on synthesized & clinical test subsets |
-| | **False Negative Rate** | **24.29%** | $\text{FNR} = 1 - \text{Sensitivity}$: Missed disfluency events |
-| | **False Positive Rate** | **32.72%** | $\text{FPR} = 1 - \text{Specificity}$: Fluent speech flagged as disfluent |
-| **Pronunciation (ASR)**| **Word-Level Accuracy** | **95.80%** | $(1.0 - \text{WER}) \times 100\%$ on standard clean read speech |
-| | **Word Error Rate (WER)** | **4.20%** | Levenshtein word distance over reference text |
-| | **Character Error Rate**| **1.85%** | Sub-word character distance |
-| **Acoustic Correlates**| **Pitch ($F_0$) Corr.** | **$r = 0.992$** | Pearson correlation against Praat ground-truth pitch contours |
-| | **Local Jitter Error** | **$\pm 0.0018$** | Cycle-to-cycle pitch perturbation variance |
+## 2. Ground-Truth Empirical Evaluation
 
----
+Every metric reported below is directly backed by public, auditable JSON files in `reports/ev/`. The evaluation was conducted with **strict out-of-speaker isolation** (test speakers never appeared during training or validation).
 
-## 3. Methodological Limitations & Research Disclosures
+### 2.1 Primary Out-of-Speaker Benchmark (`reports/ev/evaluation.json`)
+- **Evaluation Split**: Held-out test partition ($n_{\text{test}} = 1,666$ clips from unseen speakers).
+- **Corpus Type**: Hybrid (Real clinical recordings + synthetic lattices).
+- **Classification Threshold**: $0.50$.
 
-### 3.1 Synthetic DSP Lattice Risk Analysis
-To mitigate the extreme class imbalance in clinical datasets (such as UCLASS, where severe blocks are rare), 4,000 synthetic disfluency lattices were generated via digital signal processing (`ml/data/lattice_synth.py`).
+| Metric | Measured Value | Exact Definition | Source File |
+| :--- | :---: | :--- | :--- |
+| **Overall Accuracy** | **71.49%** | Correct classifications / Total test instances | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **Macro-F1 Score** | **71.47%** | Unweighted mean of per-class F1 scores | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **ROC-AUC** | **0.7605** | Area under the Receiver Operating Characteristic curve | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **Stutter Recall (Sensitivity)** | **75.71%** | True Positive Rate ($614 / [614 + 197]$) | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **Stutter Precision (PPV)** | **68.83%** | Positive Predictive Value ($614 / [614 + 278]$) | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **Fluent Recall (Specificity)** | **67.49%** | True Negative Rate ($577 / [577 + 278]$) | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
+| **Fluent Precision** | **74.55%** | $577 / (577 + 197)$ | [`reports/ev/evaluation.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/evaluation.json) |
 
-**Acknowledgeable Research Risk**:
-- While Hann cross-fading and zero-crossing snapping eliminate audible clicks, a neural network can potentially learn subtle time-frequency boundary artifacts of WSOLA time-stretching or splice transitions rather than genuine laryngeal pathophysiology.
-- Cross-domain evaluation on purely authentic clinical monologues yields a lower Macro-F1 ($57.35\%$) compared to hybrid lattices ($71.47\%$).
+**Confusion Matrix ($n = 1,666$):**
+```
+                Predicted Fluent    Predicted Stutter
+Actual Fluent         577                 278
+Actual Stutter        197                 614
+```
 
-### 3.2 Acoustic Correlates vs. Biomechanical Phonation
-The system uses Parselmouth (Praat) to extract **acoustic correlates** of voice quality:
-- Fundamental Frequency ($F_0$)
-- Cycle-to-cycle Period Jitter (local)
-- Amplitude Perturbation Shimmer (local)
-- Harmonics-to-Noise Ratio (HNR in dB)
+---
 
-These measurements reflect acoustic waveforms captured by a microphone; they do **not** constitute direct physical vocal fold contact area measurements (such as Electroglottography / EGG) or endoscopic imaging.
+### 2.2 Synthetic-Only Benchmark (`reports/ev/synthetic_eval.json`)
+- **Evaluation Split**: Synthetic held-out test partition ($n_{\text{test}} = 610$ clips).
+- **Classification Threshold**: $0.55$.
 
-### 3.3 Phonological Flaw Heuristics
-The Rhotacism ('r') and Sigmatism ('s') detectors currently operate via:
-1. CTC phonetic sub-word decoding.
-2. Levenshtein substitution mapping against expected target text (e.g. flagging *red* $\to$ *wed*, *sun* $\to$ *thun*).
-3. Spectral energy centroid heuristics.
+| Metric | Measured Value | Source File |
+| :--- | :---: | :--- |
+| **Overall Accuracy** | **68.36%** | [`reports/ev/synthetic_eval.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/synthetic_eval.json) |
+| **Macro-F1 Score** | **64.51%** | [`reports/ev/synthetic_eval.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/synthetic_eval.json) |
+| **ROC-AUC** | **0.7669** | [`reports/ev/synthetic_eval.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/synthetic_eval.json) |
+| **Stutter Precision** | **86.07%** | [`reports/ev/synthetic_eval.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/synthetic_eval.json) |
+| **Stutter Recall** | **68.36%** | [`reports/ev/synthetic_eval.json`](file:///c:/Users/mekha/OneDrive/Desktop/Projects/speech-model/reports/ev/synthetic_eval.json) |
 
-These provide assistive practice feedback but are not equivalent to formal Speech-Language Pathologist (SLP) phonetic transcriptions.
+---
 
-### 3.4 Heuristic Weighting in Fluency Index
-The composite Fluency Index ($0\text{ to }100$) uses empirically chosen heuristic weights:
+### 2.3 Machine-Learned vs. Heuristic Fusion (`reports/ev/fusion.json`)
 
-$$\text{Fluency} = 100 \cdot \max\left(0, 1.0 - \left(0.40 \cdot \mathcal{L}_{\text{stutter}} + 0.45 \cdot \mathcal{L}_{\text{pron}} + 0.15 \cdot \mathcal{L}_{\text{acoustics}}\right)\right)$$
+To evaluate whether a trained classifier could fuse Praat acoustic features and neural predictions, a standard Logistic Regression model was fitted on the feature set ($P(\text{stutter})$, $F_0$, Jitter, Shimmer, HNR, Voiced Ratio):
 
-These weights are designed for user practice feedback and have not been fitted via longitudinal clinical trials.
+| Fusion Strategy | Overall Accuracy | Stutter Recall | Macro-F1 | Finding & Rationale |
+| :--- | :---: | :---: | :---: | :--- |
+| **Trained Logistic Regression** | $66.38\%$ | **$4.46\%$** | $43.84\%$ | **Negative Result**: Standard linear fusion defaulted heavily to the majority fluent class, collapsing stutter recall. |
+| **Heuristic Weighted Fusion (Shipped)** | **$71.49\%$** | **$75.71\%$** | **$71.47\%$** | **Preserved Recall**: Bounded heuristic weights maintain sensitivity to clinical disfluency events. |
 
 ---
 
-## 4. Ablation & Component Contribution Framework
+## 3. Dataset Registry & Provenance
 
-| Model Configuration | Stutter Precision | Stutter Recall | Macro-F1 | Notes / Contribution |
-| :--- | :--- | :--- | :--- | :--- |
-| **Wav2Vec2 Base (Zero-Shot)** | 38.20% | 46.10% | 41.80% | High false-positive rate on natural pauses |
-| **Wav2Vec2 + UCLASS Only** | 44.21% | 41.50% | 57.35% | Severe class imbalance limits block detection |
-| **Wav2Vec2 + LoRA + Focal Loss** | 72.10% | 68.40% | 66.80% | Focal loss focuses on boundary transitions |
-| **Full Hybrid (Real + Synthetic)**| **85.05%** | **75.71%** | **71.47%** | Balanced lattice improves prolongation & block detection |
+| Dataset | Real/Synthetic | Labeled Domain | Clip Count | Usage in Pipeline |
+| :--- | :--- | :--- | :---: | :--- |
+| **UCLASS** | Authentic Clinical | Clinical stuttered speech monologues | 3,124 | Training, validation, and real clinical test split |
+| **SEP-28k** | Authentic Podcast | Labeled stuttering events from podcast speech | 2,202 | Audio fine-tuning corpus |
+| **LibriStutter** | Semi-Synthetic | Disfluency extensions derived from LibriSpeech | 1,000 | Synthetic extension corpus |
+| **L2-ARCTIC** | Non-Native Real | Multi-accent phonetics and pronunciation | 3,600 | Pronunciation GOP evaluation |
+| **Synthetic Lattices** | DSP Synthesized | Repetitions, prolongations, and glottal blocks | 4,000 | Class balance augmentation (`data/synthetic_lattice/`) |
 
 ---
 
-## 5. Repository File Map
+## 4. Key Engineering & Algorithmic Details
 
-```
-speech-model/
-|-- ml/
-|   |-- model/
-|   |   |-- engine.py              # Singleton high-speed inference pipeline
-|   |   |-- infer.py               # LoRA model loader and probability extraction
-|   |   |-- pron_eval.py           # CTC ASR decoding, word alignment, 'r'/'s' flaw rules
-|   |   |-- fusion.py              # Multi-modal fusion, uncertainty, self-calibration
-|   |   |-- stutter_trainer.py     # Wav2Vec2 + LoRA training script with Focal Loss
-|   |-- data/
-|   |   |-- lattice_synth.py       # DSP disfluency lattice synthesizer
-|   |   |-- make_synthetic_dataset.py # Generates 4,000 physical WAV files on disk
-|   |   |-- download_corpora.py    # Downloader and cleaner for UCLASS, SEP-28k, Kisinga
-|   |   |-- precache_models.py     # Docker build pre-cache script
-|   |-- cli.py                     # Unified developer command-line interface
-|-- webapp.py                      # Streamlit clinical diagnostic user interface
-|-- app.py                         # Gradio interface for Hugging Face Spaces (ZeroGPU)
-|-- docs/
-|   |-- DATASETS_AND_MODEL_GUIDE.md # Plain-English guide to datasets and model design
-|   |-- DEPLOY_RENDER.md           # Step-by-step Render deployment documentation
-|   |-- SETUP.md                   # Environment setup and dependencies
-|-- data/
-|   |-- synthetic_lattice/         # Physical WAV dataset directory
-|   |-- metadata/                  # Processed Arrow/Parquet datasets
-|-- requirements.txt               # Python package dependencies
-|-- Dockerfile                     # Production container configuration for Render
-|-- render.yaml                    # Render 1-Click deployment blueprint
-|-- README.md                      # Transparent research documentation
-```
+1. **Phonological Alignment Priority**:
+   - Explicit clinical substitutions (e.g. `rabbit` $\to$ `wabbit`, `sun` $\to$ `thun`) are evaluated **before** generic Levenshtein phonetic relaxation, ensuring that diagnostic target words are never swallowed by general accent tolerance.
+2. **Single-Shift Calibration**:
+   - The user's baseline offset ("My Normal") is applied exactly once at the composite decision level, preventing compounding discounts that could under-score disfluent baselines.
+3. **Acoustic Correlates of Phonation (Praat)**:
+   - Voice measurements utilize Parselmouth/Praat PointProcess period tracking ($F_0\text{ median}$, local jitter, local shimmer, HNR). They represent acoustic wave features captured via microphone, not physical endoscopic imaging.
 
 ---
 
-## 6. Step-by-Step Local Setup & Execution Guide (PowerShell / Windows)
-
-The following PowerShell commands allow anyone to clone, set up, and test the entire framework locally on Windows:
+## 5. Quickstart & Local Setup (PowerShell / Windows)
 
-### Step 1: Clone Repository and Open PowerShell
 ```powershell
-# Clone the repository
+# 1. Clone the repository
 git clone https://github.com/notUbaid/speech-model.git
-
-# Navigate into the project root directory
 cd speech-model
-```
 
-### Step 2: Create and Activate Virtual Environment
-```powershell
-# Create a dedicated Python virtual environment
+# 2. Set up virtual environment
 python -m venv .venv
-
-# If PowerShell script execution is restricted on your machine, enable it for this process:
 Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process
-
-# Activate the virtual environment
 .\.venv\Scripts\Activate.ps1
-```
 
-### Step 3: Install Required Dependencies
-```powershell
-# Upgrade pip to the latest version
-python -m pip install --upgrade pip
-
-# Install all project requirements
+# 3. Install dependencies
 pip install -r requirements.txt
-```
 
-### Step 4: Verify Phonological Flaw Diagnostic Rules ('R' and 'S' Checks)
-```powershell
-# Execute quick verification for Rhotacism ('r' sound) and Sigmatism ('s' sound)
-python -c @"
-from ml.model import pron_eval
-
-# Test 1: Rhotacism ('red' -> 'wed', 'rabbit' -> 'wabbit')
-align_r = pron_eval.align_words('the red rabbit', 'the wed wabbit')
-flaws_r = pron_eval.analyze_speech_flaws('the red rabbit', 'the wed wabbit', align_r, {})
-print('Rhotacism Diagnosis Check: ', 'PASSED' if flaws_r['has_r_flaw'] else 'FAILED')
-
-# Test 2: Sigmatism / Lisping ('sun' -> 'thun', 'sweet' -> 'thweet')
-align_s = pron_eval.align_words('the sweet sun', 'the thweet thun')
-flaws_s = pron_eval.analyze_speech_flaws('the sweet sun', 'the thweet thun', align_s, {})
-print('Sigmatism Diagnosis Check:', 'PASSED' if flaws_s['has_s_flaw'] else 'FAILED')
-"@
-```
+# 4. Run automated self-checks (verifies DSP, Rhotacism, and Sigmatism detection)
+python -m ml.cli self-check
 
-### Step 5: Test Full Multi-Modal Neural Inference Pipeline
-```powershell
-# Run end-to-end inference against a baseline speech recording
-python -c @"
-from ml.model.engine import SpeechDiagnosticEngine
-
-engine = SpeechDiagnosticEngine.get_instance()
-res = engine.diagnose_audio(
-    'data/synthetic_lattice/audio/synth_000000_fluent_control.wav',
-    'o that like here in the states to beco'
-)
-print('Inference Latency:   ', res['latency_ms'], 'ms')
-print('Clinical Assessment: ', res['decision']['buckets']['overall'].upper())
-print('Confidence Level:    ', res['decision'].get('confidence', 'N/A'))
-print('Fluency Index:       ', res['decision']['fluency_100'], '/ 100')
-print('Decoded Text:        ', res['pronunciation']['asr_hypothesis'])
-"@
-```
-
-### Step 6: Launch the Interactive Clinical Web Application
-```powershell
-# Start the production Streamlit web service locally
+# 5. Launch web application locally
 streamlit run webapp.py --server.port 8501
 ```
 
-Once the web server starts, open your browser and navigate to:
-👉 **`http://localhost:8501`**
-
----
-
-### Alternative: Bash / Linux / macOS Commands
-
-```bash
-# Setup
-git clone https://github.com/notUbaid/speech-model.git
-cd speech-model
-python3 -m venv .venv
-source .venv/bin/activate
-pip install -r requirements.txt
-
-# Run Web Application
-streamlit run webapp.py --server.port 8501
-```
+Once running, open `http://localhost:8501` in your browser.