Initial commit

.claude/skills/phonemizer/SKILL.md

@@ -0,0 +1,517 @@
+---
+name: phonemizer
+description: Domain expert in Quranic phonetics, Tajweed rules, Arabic Unicode, and the Quranic Phonemizer system. Use for phoneme analysis, IPA mappings, Unicode character inspection, and investigating Tajweed rule patterns.
+allowed-tools:
+  - Glob
+  - Grep
+  - Read
+  - Bash
+  - Python 
+  - Python3
+---
+
+# Phonemizer Domain Expert
+
+You are a domain expert in Quranic phonetics, Tajweed rules, Arabic Unicode, and the Quranic Phonemizer system. Your role is to help understand patterns and associations between:
+- **Tajweed rules** (pronunciation rules for Quranic recitation)
+- **Unicode codepoints** (Arabic script encoding)
+- **IPA phonemes** (International Phonetic Alphabet representations)
+- **Arabic graphemes** (written characters and diacritics)
+
+---
+
+## Phonemizer API
+
+The phonemizer is located at `/mnt/c/Users/ahmed/Documents/Uni/Thesis/Code/phonemizer/`.
+
+### Getting Text and Phonemes
+
+```python
+import sys
+sys.path.insert(0, "/mnt/c/Users/ahmed/Documents/Uni/Thesis/Code/phonemizer")
+from core.phonemizer import Phonemizer
+
+pm = Phonemizer()
+
+# Get a verse by reference (surah:ayah or surah:ayah:word)
+result = pm.phonemize(ref="2:255")  # Ayat al-Kursi
+
+# Access text
+text = result.text()  # Arabic text with verse markers
+
+# Access phonemes
+phonemes_str = result.phonemes_str(phoneme_sep=" ", word_sep="")
+phonemes_list = result.phonemes_list(split='word')  # [[phonemes], ...]
+```
+
+### Reference Formats
+
+| Format | Example | Meaning |
+|--------|---------|---------|
+| Surah | `"1"` | Entire Al-Fatiha |
+| Verse | `"2:255"` | Ayat al-Kursi |
+| Word | `"1:1:2"` | Word 2 of 1:1 |
+| Range | `"1:1 - 1:5"` | Verses 1-5 of surah 1 |
+| Cross-surah | `"112 - 114"` | Last 3 surahs |
+
+### Getting Detailed Mappings
+
+```python
+mapping = result.get_mapping()
+
+# Flat phoneme sequence
+all_phonemes = mapping.phoneme_sequence  # ["b", "i", "s", "m", ...]
+
+# Word-level data
+for word in mapping.words:
+    print(word.location)   # "1:1:1"
+    print(word.text)       # "بِسْمِ"
+    print(word.phonemes)   # ["b", "i", "s", "m", "i"]
+
+    # Letter-level breakdown
+    for letter in word.letter_mappings:
+        print(f"  {letter.char} → {letter.phonemes}")
+        print(f"    diacritic: {letter.diacritic}")
+        print(f"    rules: {letter.letter_rules}")
+
+# Alignment: phoneme → source letter
+for entry in mapping.alignment:
+    print(f"Phoneme '{entry.phoneme}' from '{entry.source_char}'")
+    print(f"  Tajweed rules: {entry.rules}")
+```
+
+---
+
+## Data Structures
+
+The phonemizer produces structured mappings that the recitation app transforms into frozen dataclasses.
+
+### Phonemizer Core Structures (`phonemizer/core/mapping.py`)
+
+#### PhonemizationMapping (root)
+```python
+@dataclass
+class PhonemizationMapping:
+    ref: str                           # Reference string
+    text: str                          # Arabic text
+    words: List[WordMapping]           # Per-word breakdown
+    phoneme_sequence: List[str]        # Flat phoneme list
+    alignment: List[AlignmentEntry]    # Phoneme→letter mapping
+```
+
+#### WordMapping
+```python
+@dataclass
+class WordMapping:
+    location: str                      # "surah:ayah:word"
+    text: str                          # Arabic word
+    phonemes: List[str]                # Word's phonemes
+    letter_mappings: List[LetterMapping]
+    leading_symbols: List[OtherSymbolMapping]   # Before word
+    trailing_symbols: List[OtherSymbolMapping]  # After word (verse markers)
+    is_special_word: bool              # Allah, etc.
+    is_starting: bool                  # First word of segment
+    is_stopping: bool                  # Last word (affects phonemization)
+    madd_mappings: List[MaddMapping]   # Long vowel tracking
+```
+
+#### LetterMapping
+```python
+@dataclass
+class LetterMapping:
+    index: int                         # Position in word (0-based)
+    char: str                          # Base Arabic character
+    phonemes: List[str]                # Emitted phonemes (empty = silent)
+    diacritic: Optional[str]           # "FATHA", "SUKUN", etc.
+    has_shaddah: bool                  # Gemination marker
+    extensions: List[ExtensionSymbolMapping]  # Dagger alef, maddah, etc.
+    other_symbols: List[OtherSymbolMapping]   # Tatweel, stop signs
+    mapping_type: MappingType          # STANDARD, SILENT, ONE_TO_MANY, etc.
+    letter_rules: List[str]            # Tajweed rules for this letter
+    phoneme_rules: List[List[str]]     # Rules per emitted phoneme
+```
+
+#### ExtensionSymbolMapping
+```python
+@dataclass
+class ExtensionSymbolMapping:
+    char: str    # The extension character (e.g., "ٰ" dagger alef)
+    name: str    # Identifier: "DAGGER_ALEF", "MADDAH", "MINI_WAW", etc.
+```
+
+#### MaddMapping (Long Vowel Tracking)
+```python
+@dataclass
+class MaddMapping:
+    phoneme_index: int          # Index in word's phoneme list
+    phoneme: str                # Long vowel: "a:", "aˤ:", "u:", "i:"
+    letter_index: int           # Letter that carries this madd
+    vowel_grapheme: str         # ا, و, ي, ى, ٰ (source of the vowel)
+    has_maddah: bool            # Whether ٓ is present
+    extension_index: Optional[int]  # If vowel is an extension, its index
+    madd_type: Optional[str]    # 'wajib_muttasil', 'jaiz_munfasil', 'lazim', etc.
+    is_lafdh_jalalah: bool      # Implicit dagger alef in "Allah"
+    is_hamza_fathatan: bool     # hamza + fathatan → hamza + fatha + alef
+```
+
+---
+
+### Recitation App Structures (`recitation_analysis/result.py`)
+
+The recitation app transforms phonemizer output into **frozen (immutable)** dataclasses via `ResultBuilder`.
+
+#### RecitationResult (top-level container)
+```python
+@dataclass
+class RecitationResult:
+    verse_ref: str
+    segment_ref: Optional[str]
+    canonical_phonemes: Tuple[str, ...]
+    detected_phonemes: Tuple[str, ...]     # From ASR
+    canonical_words: Tuple[WordData, ...]
+    detected_words: Tuple[WordData, ...]   # Modified by alignment
+    alignment: Tuple[AlignmentEntry, ...]
+    madd_mappings: Tuple[MaddMappingData, ...]
+    errors: List[RecitationError]          # Mutable - added after analysis
+    ghunnah_instances: List[GhunnahInstance]
+    madd_instances: List[MaddInstance]
+```
+
+#### WordData (frozen)
+```python
+@dataclass(frozen=True)
+class WordData:
+    index: int                           # Position in verse (0-based)
+    location: str                        # "surah:verse:word"
+    letters: Tuple[LetterData, ...]      # Letters in the word
+    leading_symbols: Tuple[str, ...]     # Rare symbols before word
+    trailing_symbols: Tuple[str, ...]    # Verse markers, etc.
+    is_stopping: bool                    # True if stopping word
+
+    @property
+    def text(self) -> str:               # Derived from letters
+
+    @property
+    def phonemes(self) -> Tuple[str, ...]:  # All letter phonemes
+```
+
+#### LetterData (frozen)
+```python
+@dataclass(frozen=True)
+class LetterData:
+    index: int                           # Position in word (0-based)
+    char: str                            # Base Arabic character
+    diacritic: Optional[str]             # Diacritic name
+    diacritic_char: Optional[str]        # Diacritic character
+    shaddah: bool                        # Whether shaddah present
+    extensions: Tuple[str, ...]          # Extension CHARACTERS
+    other_symbols: Tuple[str, ...]       # Stop signs, etc.
+    phonemes: Tuple[str, ...]            # Phonemes this letter produces
+    letter_rules: Tuple[str, ...]        # Tajweed rules
+    phoneme_rules: Tuple[Tuple[str, ...], ...]  # Per-phoneme rules
+    is_silent: bool                      # True if no phonemes
+    name: str                            # "LAM", "NOON", etc.
+    extension_names: Tuple[str, ...]     # ("DAGGER_ALEF",)
+
+    # Insertion tracking (for error highlighting)
+    inserted_base: bool
+    inserted_diacritic: bool
+    inserted_shaddah: bool
+    inserted_extensions: Tuple[int, ...]
+
+    def get_full_text(self) -> str:
+        """Builds: base + shaddah + diacritic + extensions + other"""
+```
+
+---
+
+## Extensions and Vowel Phoneme Production
+
+**Key Insight:** Extensions are NOT separate letters with their own phoneme lists. They are symbols attached to a letter that **influence the parent letter's phoneme production**.
+
+### How Extensions Affect Phonemes
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Letter: ل (Lam)                                            │
+│  Diacritic: FATHA (produces "a")                           │
+│  Extensions: [DAGGER_ALEF]                                  │
+│                                                             │
+│  Phoneme production:                                        │
+│    - Lam base → "l"                                        │
+│    - Fatha + extension → "a:" (long vowel)                 │
+│                                                             │
+│  Final phonemes: ["l", "a:"]  ← stored on LETTER, not ext  │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Extension Types and Their Effects
+
+| Extension Name | Char | Effect on Phonemes |
+|---------------|------|-------------------|
+| `DAGGER_ALEF` | ٰ | Extends fatha: `a` → `a:` |
+| `MADDAH` | ٓ | Marks madd, just visual, usually accompanied with another extension |
+| `MINI_WAW` | ۥ | Extends damma: `u` → `u:` |
+| `MINI_YA_END` | ۦ | Extends kasra: `i` → `i:` |
+| `HAMZA_ABOVE` | ٔ | Adds glottal stop phoneme |
+| `HAMZA_BELOW` | ٕ | Adds glottal stop phoneme |
+
+### Vowel Grapheme Sources
+
+Long vowels can come from:
+1. **Vowel letters**: ا (alef), و (waw), ي (ya), ى (alef maksura)
+2. **Extensions**: ٰ (dagger alef), ۥ (mini waw), ۦ (mini ya)
+
+The `MaddMapping.extension_index` tracks when the vowel comes from an extension:
+
+```python
+# Example: ملٰـٓئِكَة (malaa'ika - angels)
+# The lam has dagger alef extension producing a:
+MaddMapping(
+    letter_index=0,        # Lam
+    phoneme="a:",
+    vowel_grapheme="ٰ",    # Dagger alef
+    extension_index=0,     # First extension on the letter
+)
+```
+
+### Rendering Order
+
+When rendering a letter with extensions:
+```
+base_char + shaddah? + diacritic + extensions + other_symbols
+    ل         ّ           َ           ٰ            ۟
+```
+
+---
+
+### Usage
+
+```python
+from recitation_analysis.result_builder import ResultBuilder, get_result_builder
+
+# Get singleton builder
+builder = get_result_builder()
+
+# Build from phonemizer result
+result = builder.build_from_mapping(
+    mapping=phonemizer_result.get_mapping(),
+    verse_ref="1:1",
+    is_starting_segment=True,
+)
+
+# Result contains frozen WordData/LetterData with transforms applied
+for word in result.canonical_words:
+    for letter in word.letters:
+        print(f"{letter.char} + {letter.extensions} → {letter.phonemes}")
+```
+
+---
+
+## Phoneme Inventory
+
+### Consonants (Base → Geminated)
+
+| Letter | Phoneme | Geminated | Name |
+|--------|---------|-----------|------|
+| ب | `b` | `bb` | Ba |
+| ت | `t` | `tt` | Ta |
+| ث | `θ` | `θθ` | Tha |
+| ج | `ʒ` | `ʒʒ` | Jeem |
+| ح | `ħ` | `ħħ` | Hha |
+| خ | `x` | `xx` | Kha |
+| د | `d` | `dd` | Dal |
+| ذ | `ð` | `ðð` | Thal |
+| ر | `r` | `rr` | Ra |
+| ز | `z` | `zz` | Zain |
+| س | `s` | `ss` | Seen |
+| ش | `ʃ` | `ʃʃ` | Sheen |
+| ص | `sˤ` | `sˤsˤ` | Sad (emphatic) |
+| ض | `dˤ` | `dˤdˤ` | Dad (emphatic) |
+| ط | `tˤ` | `tˤtˤ` | Tta (emphatic) |
+| ظ | `ðˤ` | `ðˤðˤ` | Dtha (emphatic) |
+| ع | `ʕ` | `ʕʕ` | Ain |
+| غ | `ɣ` | - | Ghain |
+| ف | `f` | `ff` | Fa |
+| ق | `q` | `qq` | Qaf |
+| ك | `k` | `kk` | Kaf |
+| ل | `l` | `ll` | Lam |
+| م | `m` | - | Meem |
+| ن | `n` | - | Noon |
+| ه | `h` | `hh` | Ha |
+| و | `w` | `ww` | Waw |
+| ي | `j` | `jj` | Ya |
+| ء | `ʔ` | - | Hamza |
+
+### Vowels
+
+| Diacritic | Short | Long | Name |
+|-----------|-------|------|------|
+| فَ | `a` | `a:` | Fatha |
+| فُ | `u` | `u:` | Damma |
+| فِ | `i` | `i:` | Kasra |
+| فً | `an` | - | Fathatan (tanween) |
+| فٌ | `un` | - | Dammatan (tanween) |
+| فٍ | `in` | - | Kasratan (tanween) |
+
+### Emphatic Vowels (after ص ض ط ظ)
+
+| Context | Short | Long |
+|---------|-------|------|
+| Emphatic + fatha/alef | `aˤ` | `aˤ:` |
+
+### Tajweed-Specific Phonemes
+
+| Phoneme | Rule | Description |
+|---------|------|-------------|
+| `ŋ` | Ikhfaa | Hidden noon/tanween |
+| `m̃` | Idgham | Nasalized meem (ghunnah) |
+| `ñ` | Idgham | Nasalized noon (ghunnah) |
+| `j̃` | Idgham | Nasalized ya (ghunnah) |
+| `w̃` | Idgham | Nasalized waw (ghunnah) |
+| `Q` | Qalqala | Bouncing (mid-word) |
+| `lˤlˤ` | Lam Heavy | Heavy lam in "Allah" |
+| `rˤ` | Ra Heavy | Heavy ra (tafkheem) |
+
+---
+
+
+## Searching for Specific Rules
+
+The precomputed mappings file is at `data/phonemizer_mappings.json` (118MB, covers surahs 10-114).
+
+### Search for a rule pattern
+
+```bash
+# Find verses containing idgham
+grep -o '"letter_rules":\s*\[[^\]]*idgham[^\]]*\]' data/phonemizer_mappings.json | head -5
+
+# Find specific rule type
+grep -o '"rules":\s*\[[^\]]*ikhfaa[^\]]*\]' data/phonemizer_mappings.json | head -5
+```
+
+### Python search script
+
+```python
+import json
+
+with open('data/phonemizer_mappings.json', 'r') as f:
+    raw = f.read()
+    data = json.loads(json.loads(raw)) if raw.startswith('"') else json.loads(raw)
+
+def find_rule(rule_pattern: str, limit: int = 5):
+    """Find words containing a specific Tajweed rule."""
+    results = []
+    for word in data['words']:
+        for lm in word.get('letter_mappings', []):
+            rules = lm.get('letter_rules', []) + [r for pr in lm.get('phoneme_rules', []) for r in pr]
+            if any(rule_pattern.lower() in r.lower() for r in rules):
+                results.append({
+                    'location': word['location'],
+                    'text': word['text'],
+                    'char': lm['char'],
+                    'phonemes': lm['phonemes'],
+                    'rules': rules
+                })
+                if len(results) >= limit:
+                    return results
+    return results
+
+# Example: find idgham rules
+for r in find_rule('idgham', limit=5):
+    print(f"{r['location']}: {r['text']} - {r['char']} → {r['phonemes']} ({r['rules']})")
+```
+
+---
+
+## Text to Unicode Inspection Script
+
+To inspect any Arabic text character by character:
+
+```python
+def text_to_unicode_sequence(text: str) -> list:
+    """Convert Arabic text to a sequence of Unicode codepoints with names."""
+    import unicodedata
+    result = []
+    for i, char in enumerate(text):
+        cp = ord(char)
+        try:
+            name = unicodedata.name(char)
+        except ValueError:
+            name = "UNKNOWN"
+        result.append({
+            'index': i,
+            'char': char,
+            'codepoint': f"U+{cp:04X}",
+            'decimal': cp,
+            'name': name
+        })
+    return result
+
+def print_unicode_sequence(text: str):
+    """Pretty print unicode sequence of text."""
+    seq = text_to_unicode_sequence(text)
+    print(f"Text: {text}")
+    print(f"Length: {len(seq)} codepoints")
+    print("-" * 60)
+    for item in seq:
+        print(f"[{item['index']:2d}] {item['char']!r:6} {item['codepoint']}  {item['name']}")
+```
+
+### Combined: Phonemizer + Unicode Inspection
+
+```python
+import sys
+sys.path.insert(0, "/mnt/c/Users/ahmed/Documents/Uni/Thesis/Code/phonemizer")
+from core.phonemizer import Phonemizer
+
+pm = Phonemizer()
+result = pm.phonemize("1:1:1")
+
+# Get text and inspect unicode
+text = result.text()
+print_unicode_sequence(text)
+
+# Get mapping and correlate
+mapping = result.get_mapping()
+for word in mapping.words:
+    print(f"\n=== {word.location}: {word.text} ===")
+    print_unicode_sequence(word.text)
+    print("\nLetter Mappings:")
+    for lm in word.letter_mappings:
+        print(f"  {lm.char} (U+{ord(lm.char):04X}) → {lm.phonemes}")
+```
+
+---
+
+## Investigation Workflow
+
+When investigating a specific Tajweed rule:
+
+1. **Search mappings file** for the rule pattern:
+   ```python
+   results = find_rule('qalqala', limit=10)
+   ```
+
+2. **Get the verse reference** from the results:
+   ```python
+   location = results[0]['location']  # e.g., "10:2:7"
+   ```
+
+3. **Run phonemizer** on that verse for detailed analysis:
+   ```python
+   result = pm.phonemize(location)
+   mapping = result.get_mapping()
+   ```
+
+4. **Inspect unicode** of specific words:
+   ```python
+   print_unicode_sequence(mapping.words[0].text)
+   ```
+
+5. **Examine alignments** to understand phoneme production:
+   ```python
+   for a in mapping.alignment:
+       if 'qalqala' in str(a.rules):
+           print(f"Phoneme '{a.phoneme}' from '{a.source_char}' with rules {a.rules}")
+   ```

.gitattributes

@@ -0,0 +1,8 @@
+# No LFS tracking - push files directly
+data/digital_khatt_v2_script.json filter=lfs diff=lfs merge=lfs -text
+*.mp3 filter=lfs diff=lfs merge=lfs -text
+*.otf filter=lfs diff=lfs merge=lfs -text
+*.ttf filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+data/*.mp3 filter=lfs diff=lfs merge=lfs -text
+data/qpc_hafs.json filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,54 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.env
+.venv
+env/
+venv/
+ENV/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Gradio
+flagged/
+
+# Exported models
+models/
+
+# Test API
+test_api.py
+data/api_result.json
+
+CLAUDE.md
+inference_optimization.md
+
+docs/

README.md

@@ -0,0 +1,16 @@
+---
+title: Quran Multi-Aligner
+emoji: 🎯
+colorFrom: blue
+colorTo: green
+sdk: gradio
+sdk_version: 6.5.1
+app_file: app.py
+pinned: false
+short_description: Segment recitations and extract text and word timestamps
+license: mit
+thumbnail: >-
+  https://cdn-uploads.huggingface.co/production/uploads/684abe5b6327ae8863d106d2/Rr-R8HNiyJNbaXCE5saU6.png
+---
+
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

config.py

@@ -0,0 +1,322 @@
+"""
+Configuration settings for the Segments App.
+"""
+import os
+from pathlib import Path
+
+# HF Spaces detection
+IS_HF_SPACE = os.environ.get("SPACE_ID") is not None
+
+# Get project root directory
+PROJECT_ROOT = Path(__file__).parent.absolute()
+
+# Port for local development
+PORT = 6902
+
+# =============================================================================
+# Audio settings
+# =============================================================================
+
+RESAMPLE_TYPE = "soxr_lq"
+SEGMENT_AUDIO_DIR = Path("/tmp/segments")   # WAV files written here per request
+AUDIO_PRELOAD_COUNT = 5                     # First N segments use preload="auto"
+DELETE_CACHE_FREQUENCY = 3600*5             # Gradio cache cleanup interval (seconds)
+DELETE_CACHE_AGE = 3600*5                   # Delete cached files older than this (seconds)
+
+# =============================================================================
+# Model and data paths
+# =============================================================================
+
+# VAD segmenter model
+SEGMENTER_MODEL = "obadx/recitation-segmenter-v2"
+
+# Phoneme ASR models (wav2vec2 CTC)
+PHONEME_ASR_MODELS = {
+    "Base": "hetchyy/r15_95m",
+    "Large": "hetchyy/r7",
+}
+PHONEME_ASR_MODEL_DEFAULT = "Base"
+PHONEME_ASR_MODEL = PHONEME_ASR_MODELS[PHONEME_ASR_MODEL_DEFAULT]
+
+DATA_PATH = PROJECT_ROOT / "data"
+SURAH_INFO_PATH = DATA_PATH / "surah_info.json"
+
+# Quran script paths
+QURAN_SCRIPT_PATH_COMPUTE = DATA_PATH / "qpc_hafs.json"
+QURAN_SCRIPT_PATH_DISPLAY = DATA_PATH / "qpc_hafs.json"
+
+# Pre-built phoneme cache (all 114 chapters)
+PHONEME_CACHE_PATH = DATA_PATH / "phoneme_cache.pkl"
+
+# Phoneme n-gram index for anchor detection
+NGRAM_SIZE = 5
+NGRAM_INDEX_PATH = DATA_PATH / f"phoneme_ngram_index_{NGRAM_SIZE}.pkl"
+
+# =============================================================================
+# Inference settings
+# =============================================================================
+
+def get_vad_duration(minutes):
+    """GPU seconds needed for VAD based on audio minutes."""
+    if minutes > 180:
+        return 60
+    elif minutes > 120:
+        return 40
+    elif minutes > 60:
+        return 25
+    elif minutes > 30:
+        return 15
+    elif minutes > 15:
+        return 10
+    else:
+        return 5
+
+def get_asr_duration(minutes, model_name="Base"):
+    """GPU seconds needed for ASR based on audio minutes and model size."""
+    if minutes > 180:
+        return 20
+    elif minutes > 60:
+        base = 15
+    else:
+        base = 10
+
+    if model_name == "Large":
+        LARGE_MODEL_DURATION_FACTOR = 3
+        return int(base * LARGE_MODEL_DURATION_FACTOR)
+    return base
+
+# Batching strategy
+BATCHING_STRATEGY = "dynamic"  # "naive" (fixed count) or "dynamic" (seconds + pad waste)
+
+# Naive batching
+INFERENCE_BATCH_SIZE = 32      # Fixed segments per batch (used when BATCHING_STRATEGY="naive")
+
+# Dynamic batching constraints
+MAX_BATCH_SECONDS = 300      # Max total audio seconds per batch (sum of durations)
+MAX_PAD_WASTE = 0.15         # Max fraction of padded tensor that is wasted (0=no waste, 1=all waste)
+MIN_BATCH_SIZE = 8           # Minimum segments per batch (prevents underutilization)
+
+# Model precision
+DTYPE = "float16"
+TORCH_COMPILE = True  # Apply torch.compile() to GPU models (reduce-overhead mode)
+
+# AOTInductor compilation (ZeroGPU optimization)
+AOTI_ENABLED = True            # Enable AOT compilation for VAD model on HF Space
+AOTI_MIN_AUDIO_MINUTES = 15    # Min audio duration for dynamic shapes
+AOTI_MAX_AUDIO_MINUTES = 90    # Max audio duration for dynamic shapes
+AOTI_HUB_ENABLED = True        # Enable Hub persistence (upload/download compiled models)
+AOTI_HUB_REPO = "hetchyy/quran-aligner-aoti"  # Hub repo for compiled model cache
+
+# =============================================================================
+# Phoneme-based alignment settings
+# =============================================================================
+
+ANCHOR_SEGMENTS = 5                 # N-gram voting uses first N Quran segments
+ANCHOR_RARITY_WEIGHTING = True      # Weight votes by 1/count (rarity); False = equal weight
+ANCHOR_RUN_TRIM_RATIO = 0.15        # Trim leading/trailing ayahs whose weight < ratio * max weight in run
+
+# Edit operation costs (Levenshtein hyperparameters)
+COST_SUBSTITUTION = 1.0             # Default phoneme substitution cost
+COST_INSERTION = 1.0                # Insert phoneme from reference (R)
+COST_DELETION = 0.8                 # Delete phoneme from ASR (P)
+
+# Alignment thresholds (normalized edit distance: 0 = identical, 1 = completely different)
+LOOKBACK_WORDS = 15                 # Window words to look back from pointer for starting positions
+LOOKAHEAD_WORDS = 10                # Window words to look ahead after expected end position
+MAX_EDIT_DISTANCE = 0.25            # Max normalized edit distance for valid ayah match
+MAX_SPECIAL_EDIT_DISTANCE = 0.35    # Max normalized edit distance for Basmala/Isti'adha detection
+START_PRIOR_WEIGHT = 0.005          # Penalty per word away from expected position
+
+# Failed Segments
+RETRY_LOOKBACK_WORDS = 60           # Expanded lookback for retry tier 1+2
+RETRY_LOOKAHEAD_WORDS = 40          # Expanded lookahead for retry tier 1+2
+MAX_EDIT_DISTANCE_RELAXED = 0.45    # Relaxed threshold for retry tier 2
+MAX_CONSECUTIVE_FAILURES = 2        # Re-anchor within surah after this many DP failures
+
+# Debug output
+ANCHOR_DEBUG = True                 # Show detailed n-gram voting info (votes, top candidates)
+PHONEME_ALIGNMENT_DEBUG = True      # Show detailed alignment info (R, P, edit costs)
+PHONEME_ALIGNMENT_PROFILING = True  # Track and log timing breakdown (DP, window setup, etc.)
+
+# =============================================================================
+# Segmentation slider settings
+# =============================================================================
+
+# Segmentation presets: (min_silence_ms, min_speech_ms, pad_ms)
+PRESET_MUJAWWAD = (600, 1500, 300)   # Slow / Mujawwad recitation
+PRESET_MURATTAL = (200, 1000, 100)    # Normal pace (default)
+PRESET_FAST     = (75, 750, 40)     # Fast recitation
+
+# Slider ranges (defaults come from PRESET_MURATTAL)
+MIN_SILENCE_MIN = 25
+MIN_SILENCE_MAX = 1000
+MIN_SILENCE_STEP = 25
+
+MIN_SPEECH_MIN = 500
+MIN_SPEECH_MAX = 2000
+MIN_SPEECH_STEP = 250
+
+PAD_MIN = 0
+PAD_MAX = 300
+PAD_STEP = 25
+
+# =============================================================================
+# Confidence thresholds for color coding
+# =============================================================================
+
+CONFIDENCE_HIGH = 0.8    # >= this: Green
+CONFIDENCE_MED = 0.6     # >= this: Yellow, below: Red
+REVIEW_SUMMARY_MAX_SEGMENTS = 15  # Max segment numbers to list before truncating
+
+# Undersegmentation detection thresholds
+# Flagged when (word_count >= MIN_WORDS OR ayah_span >= MIN_AYAH_SPAN) AND duration >= MIN_DURATION
+UNDERSEG_MIN_WORDS = 20         # Word count threshold
+UNDERSEG_MIN_AYAH_SPAN = 2      # Ayah span threshold (segment crosses ayah boundary)
+UNDERSEG_MIN_DURATION = 15      # Duration gate (seconds)
+
+# =============================================================================
+# MFA forced alignment (word-level timestamps via HF Space)
+# =============================================================================
+
+MFA_SPACE_URL = "https://hetchyy-quran-phoneme-mfa.hf.space"
+MFA_TIMEOUT = 120
+
+# =============================================================================
+# Usage logging (pushed to HF Hub via ParquetScheduler)
+# =============================================================================
+
+USAGE_LOG_DATASET_REPO = "hetchyy/quran-aligner-logs"
+USAGE_LOG_PUSH_INTERVAL_MINUTES = 60
+
+# =============================================================================
+# Progress bar settings
+# =============================================================================
+
+PROGRESS_PROCESS_AUDIO = {
+    "vad_asr":          (0.00, "Segmenting and transcribing..."),
+    "asr":              (0.15, "Running ASR..."),
+    "special_segments": (0.50, "Detecting special segments..."),
+    "anchor":           (0.60, "Anchor detection..."),
+    "matching":         (0.80, "Text matching..."),
+    "building":         (0.90, "Building results..."),
+    "done":             (1.00, "Done!"),
+}
+
+PROGRESS_RESEGMENT = {
+    "resegment":        (0.00, "Resegmenting..."),
+    "asr":              (0.15, "Running ASR..."),
+    "special_segments": (0.50, "Detecting special segments..."),
+    "anchor":           (0.60, "Anchor detection..."),
+    "matching":         (0.80, "Text matching..."),
+    "building":         (0.90, "Building results..."),
+    "done":             (1.00, "Done!"),
+}
+
+PROGRESS_RETRANSCRIBE = {
+    "retranscribe":     (0.00, "Retranscribing with {model} model..."),
+    "asr":              (0.15, "Running ASR..."),
+    "special_segments": (0.50, "Detecting special segments..."),
+    "anchor":           (0.60, "Anchor detection..."),
+    "matching":         (0.80, "Text matching..."),
+    "building":         (0.90, "Building results..."),
+    "done":             (1.00, "Done!"),
+}
+
+PROGRESS_COMPUTE_TIMESTAMPS = {
+    "upload":           (0.00, "Uploading audio to MFA space..."),
+    "align":            (0.20, "Running forced alignment..."),
+    "inject":           (0.90, "Applying word timestamps..."),
+    "done":             (1.00, "Done!"),
+}
+
+MFA_PROGRESS_SEGMENT_RATE = 0.05  # seconds per segment for progress bar animation
+
+# =============================================================================
+# UI settings
+# =============================================================================
+
+# Main layout column scales
+LEFT_COLUMN_SCALE = 4
+RIGHT_COLUMN_SCALE = 6
+
+# Arabic font stack
+ARABIC_FONT_STACK = "'DigitalKhatt', 'Traditional Arabic'"
+
+QURAN_TEXT_SIZE_PX = 24  # Size for Quran text in segment cards
+ARABIC_WORD_SPACING = "0.2em"  # Word spacing for Arabic text
+
+# =============================================================================
+# Animation settings
+# =============================================================================
+
+# Animation granularity
+ANIM_GRANULARITIES = ["Words", "Characters"]
+ANIM_GRANULARITY_DEFAULT = "Words"
+
+ANIM_WORD_COLOR = "#49c3b3"                # Green highlight for active word
+ANIM_STYLE_ROW_SCALES = (2, 6, 1, 1)       # Granularity, Style, Verse Only, Color
+
+ANIM_OPACITY_PREV_DEFAULT = 0.3            # Default "before" opacity
+ANIM_OPACITY_AFTER_DEFAULT = 0.3           # Default "after" opacity
+ANIM_OPACITY_STEP = 0.1                    # Opacity slider step size
+
+# Mega-card text styling sliders
+MEGA_WORD_SPACING_MIN = 0.0
+MEGA_WORD_SPACING_MAX = 1.0
+MEGA_WORD_SPACING_STEP = 0.05
+MEGA_WORD_SPACING_DEFAULT = 0.2            # matches ARABIC_WORD_SPACING
+
+MEGA_TEXT_SIZE_MIN = 12
+MEGA_TEXT_SIZE_MAX = 60
+MEGA_TEXT_SIZE_STEP = 2
+MEGA_TEXT_SIZE_DEFAULT = 30                # matches QURAN_TEXT_SIZE_PX
+MEGA_SURAH_LIGATURE_SIZE = 2               # em — surah name ligature font size in megacard
+
+MEGA_LINE_SPACING_MIN = 1.5
+MEGA_LINE_SPACING_MAX = 3.0
+MEGA_LINE_SPACING_STEP = 0.1
+MEGA_LINE_SPACING_DEFAULT = 2              # matches mega-card line-height
+
+# Window engine settings (all modes use the window engine internally)
+ANIM_WINDOW_PREV_DEFAULT = 4               # Default number of visible previous words/chars
+ANIM_WINDOW_AFTER_DEFAULT = 4              # Default number of visible after words/chars
+ANIM_WINDOW_PREV_MIN = 0
+ANIM_WINDOW_AFTER_MIN = 0
+ANIM_WINDOW_PREV_MAX = 15
+ANIM_WINDOW_AFTER_MAX = 15
+
+# Presets map mode names to window engine parameter values
+ANIM_DISPLAY_MODE_DEFAULT = "Reveal"
+ANIM_DISPLAY_MODES = ["Reveal", "Fade", "Spotlight", "Isolate", "Consume", "Custom"]
+ANIM_PRESETS = {
+    "Reveal":    {
+        "prev_opacity": 1.0,
+        "prev_words": ANIM_WINDOW_PREV_MAX,
+        "after_opacity": 0.0,
+        "after_words": 0,
+    },
+    "Fade":      {
+        "prev_opacity": 1.0,
+        "prev_words": ANIM_WINDOW_PREV_MAX,
+        "after_opacity": 0.3,
+        "after_words": ANIM_WINDOW_AFTER_MAX,
+    },
+    "Spotlight": {
+        "prev_opacity": 0.3,
+        "prev_words": ANIM_WINDOW_PREV_MAX,
+        "after_opacity": 0.3,
+        "after_words": ANIM_WINDOW_AFTER_MAX,
+    },
+    "Isolate": {
+        "prev_opacity": 0,
+        "prev_words": 0,
+        "after_opacity": 0,
+        "after_words": 0,
+    },
+    "Consume": {
+        "prev_opacity": 0,
+        "prev_words": 0,
+        "after_opacity": 0.3,
+        "after_words": ANIM_WINDOW_AFTER_MAX,
+    }
+}

data/112.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efe1176e324ffe694ef16825c02f2102cd7f1f0d9bfe66148529aed65e9893fe
+size 432301

data/7.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aef46f287881fbc2f110dbb553877c8e3a0e5ab01ba3289803e512de18bc970b
+size 46313600

data/84.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e801be0312b45546ceea22735a0d9eb4286b3b9a506ba6e98bc8965341ea9ecf
+size 2480855

data/DigitalKhattV2.otf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0935c48269a57c9808e52dfae47864c189396452901c689977156036a72dd217
+size 521832

data/Juz' 30.mp3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9493fd2161f098aa0c846998f6e4d45f8dd08517657cb10653132a2ee228e0b
+size 54459943

data/digital_khatt_v2_script.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd396239908253a07dc530580fd978c515cbac1b7751b2680296580fb62b247c
+size 14832957

data/ligatures.json

@@ -0,0 +1 @@
+{"surah-1":"surah001","surah-2":"surah002","surah-3":"surah003","surah-4":"surah004","surah-5":"surah005","surah-6":"surah006","surah-7":"surah007","surah-8":"surah008","surah-9":"surah009","surah-10":"surah010","surah-11":"surah011","surah-12":"surah012","surah-13":"surah013","surah-14":"surah014","surah-15":"surah015","surah-16":"surah016","surah-17":"surah017","surah-18":"surah018","surah-19":"surah019","surah-20":"surah020","surah-21":"surah021","surah-22":"surah022","surah-23":"surah023","surah-24":"surah024","surah-25":"surah025","surah-26":"surah026","surah-27":"surah027","surah-28":"surah028","surah-29":"surah029","surah-30":"surah030","surah-31":"surah031","surah-32":"surah032","surah-33":"surah033","surah-34":"surah034","surah-35":"surah035","surah-36":"surah036","surah-37":"surah037","surah-38":"surah038","surah-39":"surah039","surah-40":"surah040","surah-41":"surah041","surah-42":"surah042","surah-43":"surah043","surah-44":"surah044","surah-45":"surah045","surah-46":"surah046","surah-47":"surah047","surah-48":"surah048","surah-49":"surah049","surah-50":"surah050","surah-51":"surah051","surah-52":"surah052","surah-53":"surah053","surah-54":"surah054","surah-55":"surah055","surah-56":"surah056","surah-57":"surah057","surah-58":"surah058","surah-59":"surah059","surah-60":"surah060","surah-61":"surah061","surah-62":"surah062","surah-63":"surah063","surah-64":"surah064","surah-65":"surah065","surah-66":"surah066","surah-67":"surah067","surah-68":"surah068","surah-69":"surah069","surah-70":"surah070","surah-71":"surah071","surah-72":"surah072","surah-73":"surah073","surah-74":"surah074","surah-75":"surah075","surah-76":"surah076","surah-77":"surah077","surah-78":"surah078","surah-79":"surah079","surah-80":"surah080","surah-81":"surah081","surah-82":"surah082","surah-83":"surah083","surah-84":"surah084","surah-85":"surah085","surah-86":"surah086","surah-87":"surah087","surah-88":"surah088","surah-89":"surah089","surah-90":"surah090","surah-91":"surah091","surah-92":"surah092","surah-93":"surah093","surah-94":"surah094","surah-95":"surah095","surah-96":"surah096","surah-97":"surah097","surah-98":"surah098","surah-99":"surah099","surah-100":"surah100","surah-101":"surah101","surah-102":"surah102","surah-103":"surah103","surah-104":"surah104","surah-105":"surah105","surah-106":"surah106","surah-107":"surah107","surah-108":"surah108","surah-109":"surah109","surah-110":"surah110","surah-111":"surah111","surah-112":"surah112","surah-113":"surah113","surah-114":"surah114"}

data/phoneme_cache.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:027283ab3be8a239b99ba4b3ffeb869efddc3da6fce12e02473d3e335dbf3a04
+size 7964064

data/phoneme_ngram_index_5.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b522664de41f590fc18fad385f023cb1a85829623cbcf035fe18152be52bc739
+size 6205946

data/phoneme_sub_costs.json

@@ -0,0 +1,67 @@
+{
+  "_meta": {
+    "description": "Phoneme substitution costs for DP alignment. Keys are 'phA|phB' (sorted). Values are float costs (0-1). Default substitution cost is 1.0.",
+    "version": 1
+  },
+
+  "consonant_gemination": {
+    "b|bb": 0.25,
+    "t|tt": 0.25,
+    "d|dd": 0.25,
+    "f|ff": 0.25,
+    "h|hh": 0.25,
+    "j|jj": 0.25,
+    "k|kk": 0.25,
+    "l|ll": 0.25,
+    "q|qq": 0.25,
+    "r|rr": 0.25,
+    "s|ss": 0.25,
+    "w|ww": 0.25,
+    "x|xx": 0.25,
+    "z|zz": 0.25,
+    "ð|ðð": 0.25,
+    "θ|θθ": 0.25,
+    "ħ|ħħ": 0.25,
+    "ʃ|ʃʃ": 0.25,
+    "ʒ|ʒʒ": 0.25,
+    "ʕ|ʕʕ": 0.25,
+    "sˤ|sˤsˤ": 0.25,
+    "dˤ|dˤdˤ": 0.25,
+    "tˤ|tˤtˤ": 0.25,
+    "ðˤ|ðˤðˤ": 0.25
+  },
+
+  "emphatic_pairs": {
+    "a|aˤ": 0.25,
+    "a:|aˤ:": 0.25,
+    "r|rˤ": 0.25,
+    "rˤ|rˤrˤ": 0.25,
+    "r|rˤrˤ": 0.25,
+    "l|lˤlˤ": 0.25,
+    "ll|lˤlˤ": 0.25
+  },
+
+  "nasal_ghunnah": {
+    "m|m̃": 0.25,
+    "n|ñ": 0.25,
+    "n|ŋ": 0.25,
+    "ŋ|ñ": 0.25,
+    "ŋ|m̃": 0.25,
+    "j|j̃": 0.25,
+    "w|w̃": 0.25
+  },
+
+  "vowel_length": {
+    "a|a:": 0.25,
+    "aˤ|aˤ:": 0.25,
+    "a|aˤ:": 0.25,
+    "aˤ|a:": 0.25,
+    "i|i:": 0.25,
+    "u|u:": 0.25
+  },
+
+  "common_subs": {
+    "n|l": 0.25,
+    "n|m": 0.25
+  }
+}

data/qpc_hafs.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b2f91a19769275d0da57464002beacd8cec396b02b520aa14d17e3b135012a7
+size 11596756

data/surah-name-v2.ttf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d4678ef53ef76c361c32c13a9ad26317b8f8219089ab5822aafa6ed5d17502a
+size 580388

docs/api.md

@@ -0,0 +1,300 @@
+# API Documentation
+
+## Current Endpoints
+
+### `POST /process_audio_json`
+
+Stateless endpoint. Accepts audio and segmentation parameters, returns aligned JSON output.
+
+**Inputs:** audio file, min_silence_ms, min_speech_ms, pad_ms, model_name, device
+
+**Returns:** JSON with `segments` array (segment index, timestamps, Quran references, matched text, confidence, errors).
+
+**Limitation:** Every call requires re-uploading the audio. No way to resegment or retranscribe without re-sending the full file.
+
+---
+
+## Planned: Session-Based Endpoints
+
+The Gradio UI already caches intermediate results (preprocessed audio, VAD output, segment boundaries, model name) in `gr.State` so that resegment/retranscribe operations skip expensive steps. But `gr.State` is WebSocket-only — API clients using `gradio_client` can't benefit from this.
+
+### Approach: Server-Side Session Store
+
+On the first request, the server stores all intermediate data keyed by a UUID (`audio_id`) and returns it in the response. Subsequent requests reference this `audio_id` instead of re-uploading audio.
+
+**What gets stored per session:**
+- Preprocessed audio (float32, 16kHz mono) — saved to disk as `.npy`
+- Raw VAD speech intervals — in memory (small)
+- VAD completeness flags — in memory
+- Cleaned segment boundaries — in memory
+- Model name used — in memory
+
+**Lifecycle:** Sessions expire after the same TTL as the existing Gradio cache (5 hours). A background thread purges expired sessions periodically. Audio files live under `/tmp/sessions/{audio_id}/`.
+
+### `POST /process_audio_session`
+
+Full pipeline. Same as `/process_audio_json` but additionally creates a server-side session.
+
+**Inputs:** audio file, min_silence_ms, min_speech_ms, pad_ms, model_name, device
+
+**Returns:** Same JSON as `/process_audio_json` with an added `audio_id` field.
+
+### `POST /resegment_session`
+
+Re-cleans VAD boundaries with new segmentation parameters and re-runs ASR. Skips audio upload, preprocessing, and VAD inference.
+
+**Inputs:** audio_id, min_silence_ms, min_speech_ms, pad_ms, model_name, device
+
+**Returns:** JSON with `segments` array and the same `audio_id`.
+
+### `POST /retranscribe_session`
+
+Re-runs ASR with a different model on the existing segment boundaries. Skips audio upload, preprocessing, VAD, and resegmentation.
+
+**Inputs:** audio_id, model_name, device
+
+**Returns:** JSON with `segments` array and the same `audio_id`.
+
+### `POST /realign_from_timestamps`
+
+Accepts an arbitrary list of `(start, end)` timestamp pairs and runs ASR + phoneme alignment on each slice. Skips VAD entirely — the client defines the segment boundaries directly. This is the core endpoint for timeline-based editing where the user drags segment boundaries manually.
+
+**Inputs:** audio_id, timestamps (list of `{start, end}` objects in seconds), model_name, device
+
+**Returns:** JSON with `segments` array and the same `audio_id`. Session boundaries are updated to match the provided timestamps.
+
+Subsumes `/resegment_session` for most client use cases — the client can split, merge, and drag boundaries however they want, then send the final timestamp list in one call.
+
+---
+
+## Planned: Segment Editing Endpoints
+
+Fine-grained operations for modifying individual segments without reprocessing the full recitation.
+
+### `POST /split_segment`
+
+Split one segment at a given timestamp into two. Re-runs alignment on each half independently.
+
+**Inputs:** audio_id, segment_index, split_time (seconds)
+
+**Returns:** Updated `segments` array with the split segment replaced by two new segments.
+
+### `POST /merge_segments`
+
+Merge two adjacent segments into one. Re-runs alignment on the combined audio slice.
+
+**Inputs:** audio_id, segment_index_a, segment_index_b (must be adjacent)
+
+**Returns:** Updated `segments` array with the two segments replaced by one.
+
+### `POST /adjust_boundary`
+
+Shift a segment's start or end time. Re-runs alignment on the affected segment(s) and its neighbour if boundaries overlap.
+
+**Inputs:** audio_id, segment_index, new_start (seconds, optional), new_end (seconds, optional)
+
+**Returns:** Updated `segments` array.
+
+### `POST /override_segment_text`
+
+Manually assign a Quran reference range to a segment, skipping alignment entirely. For when the aligner gets it wrong and the user knows the correct ayah.
+
+**Inputs:** audio_id, segment_index, ref_from (e.g. `"2:255:1"`), ref_to (e.g. `"2:255:7"`)
+
+**Returns:** Updated segment with the overridden reference and corresponding Quran text.
+
+### `POST /bulk_update_segments`
+
+Batch update: client sends a full modified segment list (adjusted times, overridden labels). Server validates, persists to session, and optionally re-aligns changed segments.
+
+**Inputs:** audio_id, segments (list of `{start, end, ref_from?, ref_to?}`), realign (boolean, default true — re-run ASR on segments whose boundaries changed)
+
+**Returns:** Full updated `segments` array.
+
+---
+
+## Planned: Word-Level Timing
+
+### `POST /compute_word_timestamps`
+
+Compute word-level start/end times for every word in every segment. This is the backbone of karaoke-style highlighting and word-by-word caption animation.
+
+**Inputs:** audio_id, model_name, device
+
+**Returns:** JSON with per-segment word timestamps:
+```json
+{
+  "audio_id": "...",
+  "segments": [
+    {
+      "segment": 1,
+      "words": [
+        {"word": "بِسْمِ", "start": 0.81, "end": 1.12},
+        {"word": "اللَّهِ", "start": 1.12, "end": 1.45}
+      ]
+    }
+  ]
+}
+```
+
+---
+
+## Planned: Export Endpoints
+
+Generate subtitle files from session data. All accept `audio_id` and optionally use word-level timestamps if previously computed.
+
+### `POST /export_srt`
+
+Standard SRT subtitle format. One entry per segment (or per word if `word_level=true`).
+
+**Inputs:** audio_id, word_level (boolean, default false)
+
+**Returns:** SRT file content.
+
+### `POST /export_vtt`
+
+WebVTT format. Supports styling cues and is the standard for web video players.
+
+**Inputs:** audio_id, word_level (boolean, default false)
+
+**Returns:** VTT file content.
+
+### `POST /export_ass`
+
+ASS/SSA format with Arabic font and styling presets. Most useful for video editors producing styled Quran captions.
+
+**Inputs:** audio_id, word_level (boolean, default false), font_name (optional), font_size (optional)
+
+**Returns:** ASS file content.
+
+---
+
+## Planned: Quran Lookup Endpoints
+
+Utility endpoints for client-side UI (dropdowns, search, manual labelling).
+
+### `GET /quran_text`
+
+Return Quran text with diacritics for a given reference range.
+
+**Inputs:** ref_from (e.g. `"2:255:1"`), ref_to (e.g. `"2:255:7"`)
+
+**Returns:** `{"text": "...", "ref_from": "...", "ref_to": "..."}`. All 114 chapters are pre-cached in memory.
+
+### `GET /surah_info`
+
+List of all surahs with metadata.
+
+**Returns:** Array of `{number, name_arabic, name_english, ayah_count, revelation_type}`.
+
+---
+
+## Planned: Recitation Analytics
+
+### `POST /recitation_stats`
+
+Derive pace and timing analytics from an existing session's alignment results.
+
+**Inputs:** audio_id
+
+**Returns:**
+```json
+{
+  "audio_id": "...",
+  "total_duration_sec": 312.5,
+  "total_segments": 7,
+  "total_words": 86,
+  "words_per_minute": 16.5,
+  "avg_segment_duration_sec": 8.2,
+  "avg_pause_duration_sec": 1.4,
+  "per_segment": [
+    {
+      "segment": 1,
+      "ref_from": "112:1:1",
+      "ref_to": "112:1:4",
+      "duration_sec": 2.18,
+      "word_count": 4,
+      "words_per_minute": 110.1,
+      "pause_after_sec": 1.82
+    }
+  ]
+}
+```
+
+Useful for learning apps tracking student fluency, reciter comparisons, or detecting rushed/slow sections.
+
+---
+
+## Planned: Streaming
+
+### `POST /process_chunk`
+
+Streaming-friendly endpoint for incremental audio processing. The client sends audio chunks as they become available, and the server returns partial alignment results progressively. Designed for live "now playing" displays (e.g. Quran radio showing the current ayah in real time).
+
+**Inputs:** audio_id (optional — omit on first chunk to start a new session), audio_chunk (raw audio bytes), is_final (boolean)
+
+**Returns:**
+```json
+{
+  "audio_id": "...",
+  "status": "partial",
+  "latest_segments": [
+    {
+      "segment": 5,
+      "ref_from": "36:1:1",
+      "ref_to": "36:1:2",
+      "matched_text": "يسٓ",
+      "time_from": 24.3,
+      "time_to": 25.8,
+      "confidence": 0.95
+    }
+  ]
+}
+```
+
+When `is_final=true`, the server finalises the session and returns the complete aligned output (same structure as `/process_audio_session`).
+
+**Chunking notes:** The server buffers audio internally and runs VAD + ASR when enough speech has accumulated to form a segment. Earlier segments are locked in and won't change; only the trailing edge is provisional.
+
+---
+
+## Planned: Health / Status
+
+### `GET /health`
+
+Server status for monitoring dashboards and client-side availability checks.
+
+**Returns:**
+```json
+{
+  "status": "ok",
+  "gpu_available": true,
+  "gpu_quota_exhausted": false,
+  "quota_reset_time": null,
+  "active_sessions": 12,
+  "models_loaded": ["Base", "Large"],
+  "uptime_sec": 84200
+}
+```
+
+---
+
+## Error Handling
+
+If `audio_id` is missing, expired, or invalid, session endpoints return:
+
+```json
+{"error": "Session not found or expired", "segments": []}
+```
+
+The client should call `/process_audio_session` again to get a fresh session.
+
+---
+
+## Design Notes
+
+- **Thread safety:** Gradio handles concurrent requests via threading. The session store uses a lock around its internal dict.
+- **Storage:** Audio on disk (can be large), metadata in memory (always small). Audio loaded via memory-mapped reads on demand.
+- **No auth needed:** Session IDs are 128-bit random UUIDs — effectively unguessable.
+- **HF Spaces compatibility:** `/tmp` is ephemeral and cleared on restart, which is fine since sessions are transient. The existing `allowed_paths=["/tmp"]` covers the new directory.
+- **Backward compatible:** `/process_audio_json` remains unchanged.

docs/usage-logging.md

@@ -0,0 +1,370 @@
+# Usage Logging
+
+## Part 1 — Reference: `recitation_app` Logging
+
+Documents the recitation logging system used in `recitation_app` to collect anonymised analysis data on HuggingFace Hub. Included here as a reference for the quran_aligner schema below.
+
+### Dataset
+
+| Property | Value |
+|----------|-------|
+| Repo | `hetchyy/recitation-logs` (private) |
+| Type | HuggingFace Dataset |
+| Format | Parquet files in `data/` |
+| Push interval | 1 minute |
+
+Configured in `config.py`:
+
+```python
+USAGE_LOG_DATASET_REPO = "hetchyy/recitation-logs"
+USAGE_LOG_PUSH_INTERVAL_MINUTES = 1
+USAGE_LOG_AUDIO = False  # toggleable at runtime
+```
+
+### Schema
+
+Defined in `utils/usage_logger.py` as `_RECITATION_SCHEMA`:
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `audio` | `Audio` | Optional FLAC-encoded audio bytes embedded in parquet |
+| `timestamp` | `Value(string)` | ISO 8601 datetime of the analysis |
+| `user_id` | `Value(string)` | SHA-256 hash (12-char) of username or IP+UA |
+| `verse_ref` | `Value(string)` | Quranic reference, e.g. `"1:1"` |
+| `canonical_text` | `Value(string)` | Arabic text of the verse |
+| `segments` | `Value(string)` | JSON array of segment results (see below) |
+| `multi_model` | `Value(bool)` | Whether multiple ASR models were used |
+| `settings` | `Value(string)` | JSON dict of Tajweed settings |
+| `vad_timestamps` | `Value(string)` | JSON list of VAD segment boundaries |
+
+#### Segment object (inside `segments` JSON)
+
+```json
+{
+  "segment_ref": "1:1",
+  "canonical_phonemes": "b i s m i ...",
+  "detected_phonemes": "b i s m i ..."
+}
+```
+
+#### Settings object (inside `settings` JSON)
+
+```json
+{
+  "tolerance": 0.15,
+  "iqlab_sound": "m",
+  "ghunnah_length": 2,
+  "jaiz_length": 4,
+  "wajib_length": 4,
+  "arid_length": 2,
+  "leen_length": 2
+}
+```
+
+### ParquetScheduler
+
+Custom subclass of `huggingface_hub.CommitScheduler` (`utils/usage_logger.py`).
+
+#### How it works
+
+1. **Buffer** — Rows accumulate in an in-memory list via `.append(row)`. Access is protected by a threading lock.
+2. **Flush** — On each scheduler tick (every `USAGE_LOG_PUSH_INTERVAL_MINUTES`):
+   - Lock the buffer, swap it out, release the lock.
+   - For any `audio` field containing a file path, read the file and convert to `{"path": filename, "bytes": binary_data}`.
+   - Build a PyArrow table from the rows.
+   - Embed the HF feature schema in parquet metadata:
+     ```python
+     table.replace_schema_metadata(
+         {"huggingface": json.dumps({"info": {"features": schema}})}
+     )
+     ```
+   - Write to a temp parquet file, then upload via `api.upload_file()` to `data/{uuid4()}.parquet`.
+   - Clean up temp audio files.
+
+#### Audio encoding
+
+When `USAGE_LOG_AUDIO` is enabled:
+
+```python
+sf.write(filepath, audio_array, sample_rate, format="FLAC")
+row["audio"] = str(filepath)  # ParquetScheduler reads and embeds the bytes
+```
+
+The audio is 16kHz mono, encoded as FLAC, and stored as embedded bytes inside the parquet file.
+
+### Lazy Initialisation
+
+Schedulers are **not** created at import time. They are initialised on first call to `_ensure_schedulers()` using double-checked locking:
+
+```python
+_recitation_scheduler = None
+_schedulers_initialized = False
+_init_lock = threading.Lock()
+
+def _ensure_schedulers():
+    global _recitation_scheduler, _schedulers_initialized
+    if _schedulers_initialized:
+        return
+    with _init_lock:
+        if _schedulers_initialized:
+            return
+        _schedulers_initialized = True
+        _recitation_scheduler = ParquetScheduler(
+            repo_id=USAGE_LOG_DATASET_REPO,
+            schema=_RECITATION_SCHEMA,
+            every=USAGE_LOG_PUSH_INTERVAL_MINUTES,
+            path_in_repo="data",
+            repo_type="dataset",
+            private=True,
+        )
+```
+
+This avoids interfering with ZeroGPU, which is sensitive to early network calls.
+
+### Error Logging
+
+Errors use a separate `CommitScheduler` (not `ParquetScheduler`) that watches a local directory:
+
+- Local path: `/usage_logs/errors/error_log-{uuid4()}.jsonl`
+- Remote path: `data/errors/`
+- Format: JSONL with fields `timestamp`, `user_id`, `verse_ref`, `error_message`
+
+Errors are appended to the JSONL file under a file lock. The `CommitScheduler` syncs the directory to Hub periodically.
+
+### User Anonymisation
+
+```python
+def get_user_id(request) -> str:
+    username = getattr(request, "username", None)
+    if username:
+        return hashlib.sha256(username.encode()).hexdigest()[:12]
+    ip = headers.get("x-forwarded-for", "").split(",")[0].strip()
+    ua = headers.get("user-agent", "")
+    return hashlib.sha256(f"{ip}|{ua}".encode()).hexdigest()[:12]
+```
+
+- Logged-in HF users: hash of username
+- Anonymous users: hash of IP + User-Agent
+- Always truncated to 12 hex characters
+
+### Fallback
+
+If the scheduler fails to initialise (no HF token, network issues), rows are written to a local JSONL file at `usage_logs/recitations_fallback.jsonl` (without audio).
+
+### Integration Point
+
+Logging is called from the audio processing handler (`ui/handlers/audio_processing.py`) after each analysis completes:
+
+```python
+log_analysis(
+    user_id, ref, text, segments,
+    multi_model=bool(use_multi),
+    settings=_settings,
+    audio=audio_for_log,       # tuple of (sample_rate, np.ndarray) or None
+    vad_timestamps=vad_ts,     # list of [start, end] pairs
+)
+```
+
+Errors are logged separately:
+
+```python
+log_error(user_id, ref, "Audio loading failed")
+```
+
+### Dependencies
+
+- `huggingface_hub` — `CommitScheduler` base class and Hub API
+- `pyarrow` — Parquet table creation and schema metadata
+- `soundfile` — FLAC audio encoding
+
+---
+
+## Part 2 — `quran_aligner` Logging Schema
+
+Schema for logging alignment runs from this project. One row per audio upload. The row is mutated in-place while it sits in the `ParquetScheduler` buffer (before the next push-to-Hub tick). Run-level fields (profiling, reciter stats, quality stats, settings) are **overwritten** to reflect the latest run. Segment results are **appended** so every setting combination is preserved.
+
+### Run-level fields
+
+#### Identity
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `audio` | `Audio` | FLAC-encoded audio (16kHz mono) |
+| `audio_id` | `Value(string)` | `{sha256(audio_bytes)[:16]}:{timestamp}`, e.g. `a3f7b2c91e04d8f2:20260203T141532` |
+| `timestamp` | `Value(string)` | ISO 8601 datetime truncated to seconds, e.g. `2026-02-03T01:50:45` |
+| `user_id` | `Value(string)` | SHA-256 hash (12-char) of IP+UA |
+
+The `audio_id` hash prefix enables grouping/deduplication of the same recording across runs; the timestamp suffix makes each run unique. Cost is ~90ms for a 5-minute recording.
+
+#### Input metadata
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `audio_duration_s` | `Value(float64)` | Total audio duration in seconds |
+| `num_segments` | `Value(int32)` | Number of VAD segments |
+| `surah` | `Value(int32)` | Detected surah (1-114) |
+
+#### Segmentation settings
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `min_silence_ms` | `Value(int32)` | Minimum silence duration to split |
+| `min_speech_ms` | `Value(int32)` | Minimum speech duration for a valid segment |
+| `pad_ms` | `Value(int32)` | Padding around speech segments |
+| `asr_model` | `Value(string)` | `"Base"` (`hetchyy/r15_95m`) or `"Large"` (`hetchyy/r7`) |
+| `device` | `Value(string)` | `"GPU"` or `"CPU"` |
+
+#### Profiling (seconds)
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `total_time` | `Value(float64)` | End-to-end pipeline wall time |
+| `vad_queue_time` | `Value(float64)` | VAD queue wait time |
+| `vad_gpu_time` | `Value(float64)` | VAD actual GPU execution |
+| `asr_gpu_time` | `Value(float64)` | ASR actual GPU execution |
+| `dp_total_time` | `Value(float64)` | Total DP alignment across all segments |
+
+#### Quality & retry stats
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `segments_passed` | `Value(int32)` | Segments with confidence > 0 |
+| `segments_failed` | `Value(int32)` | Segments with confidence <= 0 |
+| `mean_confidence` | `Value(float64)` | Average confidence across all segments |
+| `tier1_retries` | `Value(int32)` | Expanded-window retry attempts |
+| `tier1_passed` | `Value(int32)` | Successful tier 1 retries |
+| `tier2_retries` | `Value(int32)` | Relaxed-threshold retry attempts |
+| `tier2_passed` | `Value(int32)` | Successful tier 2 retries |
+| `reanchors` | `Value(int32)` | Re-anchor events (after consecutive failures) |
+| `special_merges` | `Value(int32)` | Basmala-fused segments |
+
+#### Reciter stats
+
+Computed from matched segments (those with `word_count > 0`). Already calculated in `app.py:877-922` for console output.
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `words_per_minute` | `Value(float64)` | `total_words / (total_speech_s / 60)` |
+| `phonemes_per_second` | `Value(float64)` | `total_phonemes / total_speech_s` |
+| `avg_segment_duration` | `Value(float64)` | Mean duration of matched segments |
+| `std_segment_duration` | `Value(float64)` | Std dev of matched segment durations |
+| `avg_pause_duration` | `Value(float64)` | Mean inter-segment silence gap |
+| `std_pause_duration` | `Value(float64)` | Std dev of pause durations |
+
+#### Session flags
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `resegmented` | `Value(bool)` | User resegmented with different VAD settings |
+| `retranscribed` | `Value(bool)` | User retranscribed with a different ASR model |
+
+#### Segments, timestamps & error
+
+| Field | HF Type | Description |
+|-------|---------|-------------|
+| `segments` | `Value(string)` | JSON array of run objects (see below) — **appended** on resegment/retranscribe |
+| `word_timestamps` | `Value(string)` | JSON array of per-segment MFA word timings (see below), null until computed |
+| `error` | `Value(string)` | Top-level error message if the pipeline failed |
+
+### Segment runs (inside `segments` JSON)
+
+Each run with different settings appends a new run object. The array preserves the full history so every setting combination is available.
+
+```json
+[
+  {
+    "min_silence_ms": 200,
+    "min_speech_ms": 1000,
+    "pad_ms": 100,
+    "asr_model": "Base",
+    "segments": [
+      {
+        "idx": 1,
+        "start": 0.512,
+        "end": 3.841,
+        "duration": 3.329,
+        "ref": "2:255:1-2:255:5",
+        "confidence": 0.87,
+        "word_count": 5,
+        "ayah_span": 1,
+        "phoneme_count": 42,
+        "undersegmented": false,
+        "missing_words": false,
+        "special_type": null,
+        "error": null
+      }
+    ]
+  },
+  {
+    "min_silence_ms": 600,
+    "min_speech_ms": 1500,
+    "pad_ms": 300,
+    "asr_model": "Base",
+    "segments": [...]
+  }
+]
+```
+
+#### Run object
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `min_silence_ms` | int | Silence setting used for this run |
+| `min_speech_ms` | int | Speech setting used for this run |
+| `pad_ms` | int | Pad setting used for this run |
+| `asr_model` | string | `"Base"` or `"Large"` |
+| `segments` | array | Per-segment objects for this run |
+
+#### Per-segment object
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `idx` | int | 1-indexed segment number |
+| `start` | float | Segment start time in seconds |
+| `end` | float | Segment end time in seconds |
+| `duration` | float | `end - start` |
+| `ref` | string | Matched reference `"S:A:W1-S:A:W2"`, empty if failed |
+| `confidence` | float | Alignment confidence [0.0, 1.0] |
+| `word_count` | int | Number of words matched |
+| `ayah_span` | int | Number of ayahs spanned |
+| `phoneme_count` | int | Length of ASR phoneme sequence |
+| `undersegmented` | bool | Flagged if word_count >= 20 or ayah_span >= 2 and duration >= 15s |
+| `missing_words` | bool | Gaps detected in word alignment |
+| `special_type` | string\|null | `"Basmala"`, `"Isti'adha"`, `"Isti'adha+Basmala"`, or null |
+| `error` | string\|null | Per-segment error message |
+
+### Word timestamps (inside `word_timestamps` JSON)
+
+Populated when the user computes MFA timestamps. Array of per-segment word timing arrays:
+
+```json
+[
+  {
+    "segment_idx": 1,
+    "ref": "2:255:1-2:255:5",
+    "words": [
+      {"word": "ٱللَّهُ", "start": 0.512, "end": 0.841},
+      {"word": "لَآ", "start": 0.870, "end": 1.023}
+    ]
+  }
+]
+```
+
+### In-place mutation
+
+The row dict is appended to `ParquetScheduler` on the initial run, and a reference is stored in `gr.State`. Subsequent actions (resegment, retranscribe, compute timestamps) mutate the dict in-place before the next push-to-Hub tick (every 1 minute).
+
+- **Overwritten on each run:** profiling, quality/retry stats, reciter stats, run-level settings (`min_silence_ms`, `asr_model`, etc.), `num_segments`, `surah`.
+- **Appended on each run:** `segments` JSON array gains a new run object with its settings and per-segment results.
+- **Set once:** `word_timestamps` is populated when the user computes MFA timestamps (null until then).
+- **If the push already fired** before a subsequent action, the mutation is a no-op on the already-uploaded row. The new results are lost for that row — acceptable since the initial run is always captured.
+
+### Design rationale
+
+- **Settings are denormalised** into each row so config changes can be correlated with quality without joins.
+- **Profiling fields are flat columns**, not nested JSON, so they are directly queryable in the HF dataset viewer and pandas.
+- **Segments are an array of run objects** — each run includes its settings alongside the per-segment results, so different setting combinations are preserved even though run-level fields reflect the latest state.
+- **`mean_confidence` is pre-computed** at the run level for easy filtering and sorting without parsing the segments array.
+- **Audio is always uploaded** as the first column so every run is reproducible and the dataset is playable in the HF viewer.
+- **`audio_id`** combines a content hash with a timestamp — the hash prefix groups re-runs of the same recording, the suffix makes each row unique.
+- **All sources are from existing objects** — `ProfilingData` (segment_processor.py), `SegmentInfo` (segment_processor.py), and `config.py` values. No new computation is required beyond assembling the row.

requirements.txt

@@ -0,0 +1,13 @@
+gradio>=6.5.1
+spaces>=0.44.0
+torch==2.8.0
+transformers==5.0.0
+accelerate==1.11.0
+librosa==0.10.2
+numpy>=1.24.0,<2.0.0
+requests>=2.28.0
+pyarrow>=14.0.0
+soundfile>=0.12.0
+cython>=3.0.0
+recitations_segmenter==1.0.0
+git+https://github.com/Hetchy/Quranic-Phonemizer.git@1b6a8cc

scripts/add_open_tanween.py

@@ -0,0 +1,57 @@
+"""Replace regular tanween in qpc_hafs.json with open tanween where digital_khatt uses them."""
+
+import json
+from pathlib import Path
+
+DATA_DIR = Path(__file__).resolve().parent.parent / "data"
+
+OPEN_TO_REGULAR = {
+    "\u08F0": "\u064B",  # open fathatan → regular fathatan
+    "\u08F1": "\u064C",  # open dammatan → regular dammatan
+    "\u08F2": "\u064D",  # open kasratan → regular kasratan
+}
+REGULAR_TO_OPEN = {v: k for k, v in OPEN_TO_REGULAR.items()}
+
+def main():
+    khatt = json.loads((DATA_DIR / "digital_khatt_v2_script.json").read_text("utf-8"))
+    qpc = json.loads((DATA_DIR / "qpc_hafs.json").read_text("utf-8"))
+
+    counts = {"\u08F0": 0, "\u08F1": 0, "\u08F2": 0}
+    mismatches = []
+
+    for key, khatt_entry in khatt.items():
+        if key not in qpc:
+            continue
+        khatt_text = khatt_entry["text"]
+        qpc_text = qpc[key]["text"]
+
+        for open_char, regular_char in OPEN_TO_REGULAR.items():
+            if open_char in khatt_text:
+                if regular_char in qpc_text:
+                    qpc_text = qpc_text.replace(regular_char, open_char)
+                    counts[open_char] += 1
+                else:
+                    mismatches.append((key, open_char, khatt_text, qpc[key]["text"]))
+
+        qpc[key]["text"] = qpc_text
+
+    print("Replacements:")
+    for char, count in counts.items():
+        name = {"\u08F0": "fathatan", "\u08F1": "dammatan", "\u08F2": "kasratan"}[char]
+        print(f"  open {name} (U+{ord(char):04X}): {count} words")
+    print(f"  total: {sum(counts.values())} words")
+
+    if mismatches:
+        print(f"\nMismatches ({len(mismatches)}):")
+        for key, char, kt, qt in mismatches[:10]:
+            print(f"  {key}: khatt has U+{ord(char):04X} but qpc missing regular equivalent")
+            print(f"    khatt: {kt}")
+            print(f"    qpc:   {qt}")
+
+    out_path = DATA_DIR / "qpc_hafs.json"
+    out_path.write_text(json.dumps(qpc, ensure_ascii=False, indent=2) + "\n", "utf-8")
+    print(f"\nSaved to {out_path}")
+
+
+if __name__ == "__main__":
+    main()

scripts/build_phoneme_cache.py

@@ -0,0 +1,95 @@
+"""
+Build phoneme cache for all 114 chapters.
+
+Phonemizes the entire Quran in a single call and saves per-chapter
+ChapterReference objects to a pickle file for fast loading at runtime.
+
+Usage:
+    python scripts/build_phoneme_cache.py
+"""
+
+import pickle
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+_project_root = Path(__file__).parent.parent.resolve()
+sys.path.insert(0, str(_project_root))
+
+from config import PHONEME_CACHE_PATH
+from src.alignment.phoneme_matcher import ChapterReference, RefWord
+from src.phonemizer_utils import get_phonemizer
+
+
+def build_all_chapters() -> dict[int, ChapterReference]:
+    """Phonemize entire Quran and build all ChapterReference objects."""
+    pm = get_phonemizer()
+
+    print("Phonemizing entire Quran (1-114)...")
+    result = pm.phonemize(ref="1-114", stops=["verse"])
+
+    words = result._words
+    nested = result._nested
+    print(f"Total words: {len(words)}")
+
+    # Group by surah
+    surah_words: dict[int, list[RefWord]] = defaultdict(list)
+    for word, phonemes in zip(words, nested):
+        loc = word.location
+        surah_words[loc.surah_num].append(RefWord(
+            text=word.text,
+            phonemes=phonemes,
+            surah=loc.surah_num,
+            ayah=loc.ayah_num,
+            word_num=loc.word_num,
+        ))
+
+    # Build ChapterReference for each surah
+    chapters: dict[int, ChapterReference] = {}
+    for surah_num in sorted(surah_words):
+        ref_words = surah_words[surah_num]
+
+        total_phones = sum(len(w.phonemes) for w in ref_words)
+        avg_phones_per_word = total_phones / len(ref_words) if ref_words else 4.0
+
+        flat_phonemes = []
+        flat_phone_to_word = []
+        word_phone_offsets = []
+
+        for word_idx, word in enumerate(ref_words):
+            word_phone_offsets.append(len(flat_phonemes))
+            for ph in word.phonemes:
+                flat_phonemes.append(ph)
+                flat_phone_to_word.append(word_idx)
+
+        # Sentinel offset
+        word_phone_offsets.append(len(flat_phonemes))
+
+        chapters[surah_num] = ChapterReference(
+            surah=surah_num,
+            words=ref_words,
+            avg_phones_per_word=avg_phones_per_word,
+            flat_phonemes=flat_phonemes,
+            flat_phone_to_word=flat_phone_to_word,
+            word_phone_offsets=word_phone_offsets,
+        )
+
+    print(f"Built {len(chapters)} chapter references")
+    return chapters
+
+
+def main():
+    chapters = build_all_chapters()
+
+    output_path = Path(PHONEME_CACHE_PATH)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    with open(output_path, "wb") as f:
+        pickle.dump(chapters, f, protocol=pickle.HIGHEST_PROTOCOL)
+
+    print(f"Saved to {output_path}")
+    print(f"File size: {output_path.stat().st_size / 1024 / 1024:.2f} MB")
+
+
+if __name__ == "__main__":
+    main()

scripts/build_phoneme_ngram_index.py

@@ -0,0 +1,110 @@
+"""
+Build phoneme n-gram index for the entire Quran.
+
+Usage:
+    python scripts/build_phoneme_ngram_index.py
+    python scripts/build_phoneme_ngram_index.py --ngram-size 4
+"""
+
+import argparse
+import pickle
+import sys
+from collections import defaultdict
+from pathlib import Path
+from typing import Dict, List, Tuple
+
+from tqdm import tqdm
+
+# Add project root to path
+_project_root = Path(__file__).parent.parent.resolve()
+sys.path.insert(0, str(_project_root))
+
+from config import NGRAM_SIZE, NGRAM_INDEX_PATH
+from src.alignment.ngram_index import PhonemeNgramIndex
+from src.phonemizer_utils import get_phonemizer
+
+
+def build_index(ngram_size: int) -> PhonemeNgramIndex:
+    """Build the phoneme n-gram index from the entire Quran."""
+    pm = get_phonemizer()
+
+    print("Phonemizing entire Quran (surahs 1-114)...")
+    result = pm.phonemize(ref="1-114", stops=["verse"])
+
+    # Use _words (locations) + _nested (phonemes) directly — avoids slow get_mapping()
+    words = result._words
+    nested = result._nested
+    print(f"Total words from phonemizer: {len(words)}")
+
+    # Group phonemes by (surah, ayah) from word locations
+    verse_phonemes: Dict[Tuple[int, int], List[str]] = defaultdict(list)
+    for word, phonemes in tqdm(zip(words, nested), total=len(words), desc="Grouping words by verse"):
+        if not phonemes:
+            continue
+        loc = word.location
+        verse_phonemes[(loc.surah_num, loc.ayah_num)].extend(phonemes)
+
+    print(f"Total verses: {len(verse_phonemes)}")
+
+    # Extract n-grams per verse
+    ngram_positions: Dict[Tuple[str, ...], List[Tuple[int, int]]] = defaultdict(list)
+    total_ngrams = 0
+
+    for (surah, ayah), phonemes in tqdm(verse_phonemes.items(), desc="Building n-grams"):
+        if len(phonemes) < ngram_size:
+            continue
+        for i in range(len(phonemes) - ngram_size + 1):
+            ng = tuple(phonemes[i : i + ngram_size])
+            ngram_positions[ng].append((surah, ayah))
+            total_ngrams += 1
+
+    # Build counts
+    ngram_counts: Dict[Tuple[str, ...], int] = {
+        ng: len(positions) for ng, positions in ngram_positions.items()
+    }
+
+    print(f"Total n-gram occurrences: {total_ngrams}")
+    print(f"Unique n-grams: {len(ngram_positions)}")
+    if ngram_counts:
+        min_count = min(ngram_counts.values())
+        max_count = max(ngram_counts.values())
+        print(f"Count range: {min_count} - {max_count}")
+
+    return PhonemeNgramIndex(
+        ngram_positions=dict(ngram_positions),
+        ngram_counts=ngram_counts,
+        ngram_size=ngram_size,
+        total_ngrams=total_ngrams,
+    )
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Build phoneme n-gram index for Quran")
+    parser.add_argument(
+        "--ngram-size",
+        type=int,
+        default=NGRAM_SIZE,
+        help=f"N-gram size (default: {NGRAM_SIZE})",
+    )
+    parser.add_argument(
+        "--output",
+        type=str,
+        default=str(NGRAM_INDEX_PATH),
+        help=f"Output path (default: {NGRAM_INDEX_PATH})",
+    )
+    args = parser.parse_args()
+
+    index = build_index(args.ngram_size)
+
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    with open(output_path, "wb") as f:
+        pickle.dump(index, f, protocol=pickle.HIGHEST_PROTOCOL)
+
+    print(f"Saved index to {output_path}")
+    print(f"File size: {output_path.stat().st_size / 1024 / 1024:.2f} MB")
+
+
+if __name__ == "__main__":
+    main()

scripts/export_onnx.py

@@ -0,0 +1,160 @@
+"""Export phoneme ASR models to optimized ONNX format and upload to HF Hub.
+
+Usage:
+    pip install optimum[onnxruntime] onnxruntime
+    HF_TOKEN=... python scripts/export_onnx.py
+    HF_TOKEN=... python scripts/export_onnx.py --quantize        # + dynamic INT8
+    HF_TOKEN=... python scripts/export_onnx.py --models Base     # single model
+
+Exports fp32 ONNX with ORT graph optimizations baked in.
+Optionally applies dynamic INT8 quantization for CPU inference.
+
+Note: ORTOptimizer's transformer-specific fusions (attention, LayerNorm, GELU)
+do NOT support wav2vec2. We use ORT's general graph optimizations instead
+(constant folding, redundant node elimination, common subexpression elimination).
+Runtime ORT_ENABLE_ALL adds further optimizations at session load time.
+"""
+
+import argparse
+import os
+import shutil
+import sys
+from pathlib import Path
+
+# Add project root to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from config import PHONEME_ASR_MODELS
+
+
+def get_hf_token():
+    """Get HF token from env or cached login."""
+    hf_token = os.environ.get("HF_TOKEN")
+    if not hf_token:
+        try:
+            from huggingface_hub import HfFolder
+            hf_token = HfFolder.get_token()
+        except Exception:
+            pass
+    if not hf_token:
+        print("WARNING: No HF token found. Set HF_TOKEN env var or run `huggingface-cli login`.")
+    return hf_token
+
+
+def _optimize_graph(model_path: Path):
+    """Apply general ORT graph optimizations (no transformer-specific fusions).
+
+    Bakes constant folding, redundant node elimination, and common subexpression
+    elimination into the model file so they don't need to run at session load time.
+    """
+    import onnxruntime as ort
+
+    model_file = str(model_path / "model.onnx")
+    optimized_file = str(model_path / "model_optimized.onnx")
+
+    sess_options = ort.SessionOptions()
+    sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+    sess_options.optimized_model_filepath = optimized_file
+
+    # Create session just to trigger optimization and save
+    ort.InferenceSession(model_file, sess_options, providers=["CPUExecutionProvider"])
+
+    # Replace original with optimized
+    os.replace(optimized_file, model_file)
+
+
+def export_model(model_name: str, model_path: str, output_dir: Path, hf_token: str,
+                 quantize: bool = False):
+    """Export a single model to optimized fp32 ONNX and push to HF Hub."""
+    from optimum.onnxruntime import ORTModelForCTC
+
+    print(f"\n{'='*60}")
+    print(f"Exporting '{model_name}' ({model_path})")
+    print(f"{'='*60}")
+
+    # Clean output dir for fresh export
+    if output_dir.exists():
+        shutil.rmtree(output_dir)
+    output_dir.mkdir(parents=True)
+
+    # Step 1: Export to ONNX (fp32)
+    print(f"  [1/5] Exporting fp32 ONNX...")
+    model = ORTModelForCTC.from_pretrained(model_path, export=True, token=hf_token)
+    model.save_pretrained(output_dir)
+    print(f"        Saved to {output_dir}")
+
+    # Step 2: Apply general ORT graph optimizations
+    # (wav2vec2 is not supported by ORTOptimizer's transformer-specific fusions,
+    #  so we use ORT's built-in graph optimizations directly)
+    print(f"  [2/5] Applying ORT graph optimizations...")
+    _optimize_graph(output_dir)
+    print(f"        Graph optimization complete")
+
+    # Step 3: Optional dynamic INT8 quantization
+    model_file = output_dir / "model.onnx"
+    if quantize:
+        print(f"  [3/5] Applying dynamic INT8 quantization (avx2)...")
+        from onnxruntime.quantization import QuantType, quantize_dynamic
+
+        quantized_file = output_dir / "model_quantized.onnx"
+        quantize_dynamic(
+            model_input=str(model_file),
+            model_output=str(quantized_file),
+            weight_type=QuantType.QInt8,
+        )
+        # Replace original with quantized
+        os.replace(str(quantized_file), str(model_file))
+        print(f"        INT8 quantization complete")
+    else:
+        print(f"  [3/5] Skipping quantization (use --quantize to enable)")
+
+    # Step 4: Verify with dummy forward pass
+    print(f"  [4/5] Verifying model...")
+    import numpy as np
+    import onnxruntime as ort
+
+    sess = ort.InferenceSession(str(model_file), providers=["CPUExecutionProvider"])
+    input_info = sess.get_inputs()[0]
+    print(f"        Input: name={input_info.name}, type={input_info.type}, shape={input_info.shape}")
+    dummy = np.random.randn(1, 16000).astype(np.float32)
+    out = sess.run(None, {"input_values": dummy})
+    print(f"        Output shape: {out[0].shape} (dtype={out[0].dtype})")
+    del sess, dummy, out
+
+    # Step 5: Push to HF Hub
+    print(f"  [5/5] Uploading to HF Hub...")
+    from huggingface_hub import HfApi
+
+    repo_name = model_path.split("/")[-1]
+    hub_repo = f"hetchyy/{repo_name}-onnx"
+    api = HfApi(token=hf_token)
+    api.create_repo(repo_id=hub_repo, repo_type="model", private=True, exist_ok=True)
+    api.upload_folder(folder_path=str(output_dir), repo_id=hub_repo, repo_type="model")
+    print(f"        Pushed to {hub_repo}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Export phoneme ASR models to optimized ONNX")
+    parser.add_argument("--quantize", action="store_true",
+                        help="Apply dynamic INT8 quantization after graph optimization")
+    parser.add_argument("--models", nargs="+", choices=list(PHONEME_ASR_MODELS.keys()),
+                        default=list(PHONEME_ASR_MODELS.keys()),
+                        help="Which models to export (default: all)")
+    args = parser.parse_args()
+
+    hf_token = get_hf_token()
+
+    models_dir = Path(__file__).parent.parent / "models"
+    models_dir.mkdir(exist_ok=True)
+
+    for name in args.models:
+        path = PHONEME_ASR_MODELS[name]
+        output_dir = models_dir / f"onnx_{name}"
+        export_model(name, path, output_dir, hf_token, quantize=args.quantize)
+
+    suffix = " + INT8 quantized" if args.quantize else ""
+    print(f"\nDone. ONNX fp32 optimized{suffix} models exported and uploaded.")
+
+
+if __name__ == "__main__":
+    main()

scripts/fix_stop_sign_spacing.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+"""
+Fix stop sign spacing in qpc_hafs.json.
+
+The DigitalKhatt font expects stop signs (U+06D6 to U+06DB) as combining marks
+directly attached to words, but the current data has spaces before them.
+
+This script removes spaces before stop signs:
+  'رَيْبَ ۛ' → 'رَيْبَۛ'
+"""
+
+import json
+import re
+from pathlib import Path
+
+# Stop sign characters (U+06D6 to U+06DB)
+STOP_SIGNS = '\u06D6\u06D7\u06D8\u06D9\u06DA\u06DB'
+
+# Pattern to match space followed by stop sign
+SPACE_BEFORE_STOP_PATTERN = re.compile(f' ([{STOP_SIGNS}])')
+
+
+def fix_stop_sign_spacing(text: str) -> str:
+    """Remove spaces before stop signs."""
+    return SPACE_BEFORE_STOP_PATTERN.sub(r'\1', text)
+
+
+def main():
+    data_path = Path(__file__).parent.parent / 'data' / 'qpc_hafs.json'
+
+    print(f"Loading {data_path}...")
+    with open(data_path, 'r', encoding='utf-8') as f:
+        data = json.load(f)
+
+    modified_count = 0
+
+    for key, entry in data.items():
+        if 'text' in entry:
+            original = entry['text']
+            fixed = fix_stop_sign_spacing(original)
+            if fixed != original:
+                entry['text'] = fixed
+                modified_count += 1
+                if modified_count <= 5:  # Show first 5 examples
+                    print(f"  {key}: {repr(original)} → {repr(fixed)}")
+
+    if modified_count > 5:
+        print(f"  ... and {modified_count - 5} more entries")
+
+    print(f"\nModified {modified_count} entries")
+
+    print(f"Saving to {data_path}...")
+    with open(data_path, 'w', encoding='utf-8') as f:
+        json.dump(data, f, ensure_ascii=False, indent=2)
+
+    print("Done!")
+
+
+if __name__ == '__main__':
+    main()

setup.py

@@ -0,0 +1,15 @@
+"""Build Cython extensions (DP alignment core)."""
+
+from setuptools import setup, Extension
+from Cython.Build import cythonize
+
+extensions = [
+    Extension(
+        "src._dp_core",
+        ["src/_dp_core.pyx"],
+    ),
+]
+
+setup(
+    ext_modules=cythonize(extensions, language_level="3"),
+)

src/__init__.py

@@ -0,0 +1 @@
+# Core processing module

src/_dp_core.pyx

@@ -0,0 +1,357 @@
+# cython: boundscheck=False, wraparound=False, cdivision=True
+"""
+Cython-accelerated word-boundary-constrained substring Levenshtein DP.
+
+Drop-in replacement for the pure-Python align_with_word_boundaries() in
+phoneme_matcher.py.  Callers still pass plain Python lists of strings;
+encoding to integer arrays happens inside this module.
+"""
+
+from libc.stdlib cimport malloc, free
+from libc.math cimport INFINITY, fabs
+
+# ---------------------------------------------------------------------------
+# Phoneme → integer encoding (built lazily on first call)
+# ---------------------------------------------------------------------------
+
+cdef dict _phoneme_to_id = {}
+cdef int _num_phonemes = 0
+cdef double *_sub_matrix = NULL   # flat _num_phonemes × _num_phonemes
+cdef double _default_sub = 1.0
+
+
+cdef int _encode_phoneme(str p):
+    """Return integer id for *p*, assigning a new one if unseen."""
+    global _num_phonemes
+    cdef int pid
+    try:
+        pid = _phoneme_to_id[p]
+    except KeyError:
+        pid = _num_phonemes
+        _phoneme_to_id[p] = pid
+        _num_phonemes += 1
+    return pid
+
+
+def init_substitution_matrix(dict sub_costs, double default_sub):
+    """Build the dense substitution-cost matrix from the Python dict.
+
+    Must be called once before the first DP call (phoneme_matcher.py does
+    this at import time).
+
+    Parameters
+    ----------
+    sub_costs : dict[(str, str), float]
+        Phoneme-pair substitution costs (both orderings already present).
+    default_sub : float
+        Cost used for pairs not in *sub_costs*.
+    """
+    global _sub_matrix, _default_sub, _num_phonemes
+
+    _default_sub = default_sub
+
+    # First pass: make sure every phoneme in sub_costs has an id
+    for (a, b) in sub_costs:
+        _encode_phoneme(a)
+        _encode_phoneme(b)
+
+    # Allocate matrix (will be re-allocated if new phonemes appear later)
+    _rebuild_matrix(sub_costs)
+
+
+cdef void _rebuild_matrix(dict sub_costs):
+    """(Re)allocate and fill the dense cost matrix."""
+    global _sub_matrix, _num_phonemes, _default_sub
+
+    cdef int size = _num_phonemes
+    cdef int i, j
+
+    if _sub_matrix != NULL:
+        free(_sub_matrix)
+
+    _sub_matrix = <double *>malloc(size * size * sizeof(double))
+    if _sub_matrix == NULL:
+        raise MemoryError("Failed to allocate substitution matrix")
+
+    # Fill with default
+    for i in range(size * size):
+        _sub_matrix[i] = _default_sub
+
+    # Diagonal = 0 (match)
+    for i in range(size):
+        _sub_matrix[i * size + i] = 0.0
+
+    # Overrides from dict
+    cdef int aid, bid
+    cdef double cost
+    for (a, b), cost in sub_costs.items():
+        aid = _phoneme_to_id.get(a, -1)
+        bid = _phoneme_to_id.get(b, -1)
+        if aid >= 0 and bid >= 0:
+            _sub_matrix[aid * size + bid] = cost
+
+
+cdef inline double _get_sub_cost(int pid, int rid, int size) nogil:
+    """Look up substitution cost from the dense matrix."""
+    if pid == rid:
+        return 0.0
+    if pid < size and rid < size:
+        return _sub_matrix[pid * size + rid]
+    return _default_sub
+
+
+# ---------------------------------------------------------------------------
+# Main DP function
+# ---------------------------------------------------------------------------
+
+def cy_align_with_word_boundaries(
+    list P_list,
+    list R_list,
+    list R_phone_to_word_list,
+    int expected_word,
+    double prior_weight,
+    double cost_sub,
+    double cost_del,
+    double cost_ins,
+):
+    """Word-boundary-constrained substring alignment (Cython).
+
+    Identical semantics to the pure-Python version.  Returns the same
+    (best_j, best_j_start, best_cost, best_norm_dist) tuple, with
+    ``(None, None, INF, INF)`` on failure.
+    """
+    cdef int m = len(P_list)
+    cdef int n = len(R_list)
+    cdef double INF_VAL = INFINITY
+
+    if m == 0 or n == 0:
+        return (None, None, float('inf'), float('inf'))
+
+    # ------------------------------------------------------------------
+    # Encode string lists → C arrays
+    # ------------------------------------------------------------------
+    cdef int *P_ids = <int *>malloc(m * sizeof(int))
+    cdef int *R_ids = <int *>malloc(n * sizeof(int))
+    cdef int *R_w   = <int *>malloc(n * sizeof(int))
+    if P_ids == NULL or R_ids == NULL or R_w == NULL:
+        if P_ids != NULL: free(P_ids)
+        if R_ids != NULL: free(R_ids)
+        if R_w   != NULL: free(R_w)
+        raise MemoryError()
+
+    cdef int i, j
+    cdef bint need_rebuild = False
+
+    for i in range(m):
+        p = P_list[i]
+        if p not in _phoneme_to_id:
+            _encode_phoneme(p)
+            need_rebuild = True
+        P_ids[i] = _phoneme_to_id[p]
+
+    for j in range(n):
+        r = R_list[j]
+        if r not in _phoneme_to_id:
+            _encode_phoneme(r)
+            need_rebuild = True
+        R_ids[j] = _phoneme_to_id[r]
+        R_w[j] = <int>R_phone_to_word_list[j]
+
+    # If new phonemes appeared, rebuild the matrix so ids are covered
+    if need_rebuild and _sub_matrix != NULL:
+        # We need the original sub_costs dict, but we don't have it here.
+        # The safest approach: expand matrix with defaults for new phonemes.
+        _grow_matrix()
+
+    cdef int mat_size = _num_phonemes
+
+    # ------------------------------------------------------------------
+    # Precompute boundary flags
+    # ------------------------------------------------------------------
+    cdef char *start_boundary = <char *>malloc((n + 1) * sizeof(char))
+    cdef char *end_boundary   = <char *>malloc((n + 1) * sizeof(char))
+    if start_boundary == NULL or end_boundary == NULL:
+        free(P_ids); free(R_ids); free(R_w)
+        if start_boundary != NULL: free(start_boundary)
+        if end_boundary   != NULL: free(end_boundary)
+        raise MemoryError()
+
+    # start_boundary[j]: can alignment start at column j?
+    start_boundary[0] = 1  # column 0 always valid
+    for j in range(1, n):
+        start_boundary[j] = 1 if R_w[j] != R_w[j - 1] else 0
+    start_boundary[n] = 0  # can't start at or past end
+
+    # end_boundary[j]: can alignment end at column j?
+    end_boundary[0] = 0    # can't end before consuming anything
+    for j in range(1, n):
+        end_boundary[j] = 1 if R_w[j] != R_w[j - 1] else 0
+    end_boundary[n] = 1    # end of reference always valid
+
+    # ------------------------------------------------------------------
+    # DP arrays (two-row rolling)
+    # ------------------------------------------------------------------
+    cdef double *prev_cost = <double *>malloc((n + 1) * sizeof(double))
+    cdef double *curr_cost = <double *>malloc((n + 1) * sizeof(double))
+    cdef int    *prev_start = <int *>malloc((n + 1) * sizeof(int))
+    cdef int    *curr_start = <int *>malloc((n + 1) * sizeof(int))
+    if (prev_cost == NULL or curr_cost == NULL or
+            prev_start == NULL or curr_start == NULL):
+        free(P_ids); free(R_ids); free(R_w)
+        free(start_boundary); free(end_boundary)
+        if prev_cost  != NULL: free(prev_cost)
+        if curr_cost  != NULL: free(curr_cost)
+        if prev_start != NULL: free(prev_start)
+        if curr_start != NULL: free(curr_start)
+        raise MemoryError()
+
+    # Initialise row 0
+    for j in range(n + 1):
+        if start_boundary[j]:
+            prev_cost[j] = 0.0
+            prev_start[j] = j
+        else:
+            prev_cost[j] = INF_VAL
+            prev_start[j] = -1
+
+    # ------------------------------------------------------------------
+    # Core DP loop (no Python objects touched → runs at C speed)
+    # ------------------------------------------------------------------
+    cdef double del_option, ins_option, sub_option, sc
+    cdef double *tmp_d
+    cdef int    *tmp_i
+    cdef bint col0_start = start_boundary[0]
+
+    for i in range(1, m + 1):
+        if col0_start:
+            curr_cost[0] = i * cost_del
+            curr_start[0] = 0
+        else:
+            curr_cost[0] = INF_VAL
+            curr_start[0] = -1
+
+        for j in range(1, n + 1):
+            del_option = prev_cost[j] + cost_del
+            ins_option = curr_cost[j - 1] + cost_ins
+            sc = _get_sub_cost(P_ids[i - 1], R_ids[j - 1], mat_size)
+            sub_option = prev_cost[j - 1] + sc
+
+            if sub_option <= del_option and sub_option <= ins_option:
+                curr_cost[j] = sub_option
+                curr_start[j] = prev_start[j - 1]
+            elif del_option <= ins_option:
+                curr_cost[j] = del_option
+                curr_start[j] = prev_start[j]
+            else:
+                curr_cost[j] = ins_option
+                curr_start[j] = curr_start[j - 1]
+
+        # Swap rows
+        tmp_d = prev_cost;  prev_cost = curr_cost;  curr_cost = tmp_d
+        tmp_i = prev_start; prev_start = curr_start; curr_start = tmp_i
+
+    # ------------------------------------------------------------------
+    # Best-match selection (end boundaries only)
+    # ------------------------------------------------------------------
+    cdef double best_score = INF_VAL
+    cdef int best_j = -1
+    cdef int best_j_start = -1
+    cdef double best_cost_val = INF_VAL
+    cdef double best_norm = INF_VAL
+
+    cdef double dist, norm_dist, prior, score
+    cdef int j_start_val, ref_len, denom, start_word
+
+    for j in range(1, n + 1):
+        if not end_boundary[j]:
+            continue
+        if prev_cost[j] >= INF_VAL:
+            continue
+
+        dist = prev_cost[j]
+        j_start_val = prev_start[j]
+
+        ref_len = j - j_start_val
+        denom = m if m > ref_len else ref_len
+        if denom < 1:
+            denom = 1
+        norm_dist = dist / denom
+
+        if j_start_val < n:
+            start_word = R_w[j_start_val]
+        else:
+            start_word = R_w[j - 1]
+
+        prior = prior_weight * fabs(<double>(start_word - expected_word))
+        score = norm_dist + prior
+
+        if score < best_score:
+            best_score = score
+            best_j = j
+            best_j_start = j_start_val
+            best_cost_val = dist
+            best_norm = norm_dist
+
+    # ------------------------------------------------------------------
+    # Cleanup
+    # ------------------------------------------------------------------
+    free(P_ids); free(R_ids); free(R_w)
+    free(start_boundary); free(end_boundary)
+    free(prev_cost); free(curr_cost)
+    free(prev_start); free(curr_start)
+
+    if best_j < 0:
+        return (None, None, float('inf'), float('inf'))
+
+    return (best_j, best_j_start, best_cost_val, best_norm)
+
+
+# ---------------------------------------------------------------------------
+# Helper: grow matrix when new phonemes are encountered at runtime
+# ---------------------------------------------------------------------------
+
+cdef void _grow_matrix():
+    """Expand the substitution matrix to cover newly added phonemes.
+
+    New rows/columns are filled with the default substitution cost,
+    diagonal with 0.0.  Existing entries are preserved.
+    """
+    global _sub_matrix, _num_phonemes
+
+    cdef int old_size
+    cdef int new_size = _num_phonemes
+    cdef double *new_mat
+
+    if _sub_matrix == NULL:
+        # No matrix yet — allocate fresh with defaults
+        _sub_matrix = <double *>malloc(new_size * new_size * sizeof(double))
+        if _sub_matrix == NULL:
+            return
+        for i in range(new_size * new_size):
+            _sub_matrix[i] = _default_sub
+        for i in range(new_size):
+            _sub_matrix[i * new_size + i] = 0.0
+        return
+
+    # Figure out old size from current allocation
+    # We track it implicitly: old_size = new_size - (number of phonemes added since last build)
+    # Simpler: just rebuild from scratch with defaults + diagonal
+    new_mat = <double *>malloc(new_size * new_size * sizeof(double))
+    if new_mat == NULL:
+        return
+
+    cdef int i, j_idx
+    for i in range(new_size * new_size):
+        new_mat[i] = _default_sub
+    for i in range(new_size):
+        new_mat[i * new_size + i] = 0.0
+
+    # Copy old entries (old matrix was some smaller size).
+    # We don't know old_size exactly, so we just keep the new defaults.
+    # The original sub_costs were already written; since we don't have
+    # the dict here, the known-pair costs are lost for the new matrix.
+    # This only happens if a completely new phoneme appears at runtime,
+    # which is extremely rare.  The init call covers all 69 known phonemes.
+
+    free(_sub_matrix)
+    _sub_matrix = new_mat

src/alignment/alignment_pipeline.py

@@ -0,0 +1,377 @@
+"""Orchestration for phoneme-based alignment and retries."""
+
+from typing import List, Tuple
+
+from config import (
+    ANCHOR_SEGMENTS,
+    MAX_CONSECUTIVE_FAILURES,
+    RETRY_LOOKBACK_WORDS,
+    RETRY_LOOKAHEAD_WORDS,
+    MAX_EDIT_DISTANCE_RELAXED,
+    PHONEME_ALIGNMENT_PROFILING,
+)
+
+
+def run_phoneme_matching(
+    phoneme_texts: List[List[str]],
+    detected_surah: int,
+    first_quran_idx: int = 0,
+    special_results: List[tuple] = None,
+    start_pointer: int = 0,
+) -> Tuple[List[tuple], dict, set]:
+    """
+    Phoneme-based segment matching using substring DP.
+
+    Args:
+        phoneme_texts: List of phoneme lists (each is a list of phoneme strings)
+        detected_surah: Surah number from anchor search
+        first_quran_idx: Index where Quran segments start (after specials)
+        special_results: Results for special segments (Isti'adha/Basmala)
+        start_pointer: Initial word pointer from anchor voting
+
+    Returns:
+        (results, profiling_dict, gap_segments)
+        results: List[(matched_text, score, matched_ref), ...]
+    """
+    from .phoneme_matcher import align_segment, get_matched_text
+    from .phoneme_matcher_cache import get_chapter_reference
+    from .phoneme_anchor import reanchor_within_surah, verse_to_word_index, find_anchor_by_voting
+    from .ngram_index import get_ngram_index
+
+    # Only import time if profiling enabled
+    if PHONEME_ALIGNMENT_PROFILING:
+        import time
+        total_start = time.perf_counter()
+        ref_build_start = time.perf_counter()
+
+    # Build/get cached chapter reference (includes phonemizer call if not cached)
+    chapter_ref = get_chapter_reference(detected_surah)
+
+    if PHONEME_ALIGNMENT_PROFILING:
+        ref_build_time = time.perf_counter() - ref_build_start
+
+    # Initialize results with special segments
+    results = list(special_results) if special_results else []
+    # Parallel list: None for specials/failures, (start_word_idx, end_word_idx) for matches
+    word_indices = [None] * len(results)
+
+    # Timing accumulators (only used if profiling enabled)
+    if PHONEME_ALIGNMENT_PROFILING:
+        dp_times = []
+        window_setup_total = 0.0
+        result_build_total = 0.0
+
+    # Track whether the next segment might have Basmala fused with verse content
+    from .special_segments import SPECIAL_PHONEMES, SPECIAL_TEXT
+    basmala_already_detected = any(
+        r[2] in ("Basmala", "Isti'adha+Basmala") for r in (special_results or [])
+    )
+    is_first_after_transition = not basmala_already_detected
+
+    special_merges = 0
+
+    # Process Quran segments with phoneme alignment
+    pointer = start_pointer
+    num_segments = 0
+    consecutive_failures = 0
+    skip_count = 0
+    pending_specials = []
+    tier1_attempts = 0
+    tier1_passed = 0
+    tier1_segments = []
+    tier2_attempts = 0
+    tier2_passed = 0
+    tier2_segments = []
+    consec_reanchors = 0
+    segments_attempted = 0
+    segments_passed = 0
+
+    for i, asr_phonemes in enumerate(phoneme_texts[first_quran_idx:]):
+        # Handle segments consumed by inter-chapter special detection
+        if skip_count > 0:
+            results.append(pending_specials.pop(0))
+            word_indices.append(None)
+            skip_count -= 1
+            continue
+
+        segment_idx = first_quran_idx + i + 1  # 1-indexed for display
+        segments_attempted += 1
+
+        alignment, timing = align_segment(asr_phonemes, chapter_ref, pointer, segment_idx)
+        num_segments += 1
+
+        # Accumulate timing if profiling enabled
+        if PHONEME_ALIGNMENT_PROFILING:
+            dp_times.append(timing['dp_time'])
+            window_setup_total += timing['window_setup_time']
+            result_build_total += timing['result_build_time']
+
+        # Chapter transition: pointer past end of chapter
+        if alignment is None and pointer >= chapter_ref.num_words:
+            from .special_segments import detect_inter_chapter_specials
+            remaining_phonemes = phoneme_texts[first_quran_idx + i:]
+            inter_specials, num_consumed = detect_inter_chapter_specials(remaining_phonemes)
+
+            if chapter_ref.surah == 1:
+                # After Al-Fatiha, the next chapter could be anything — global reanchor
+                print(f"  [CHAPTER-END] Surah 1 complete at segment {segment_idx}, "
+                      f"running global reanchor...")
+
+                # Use segments after specials for anchor voting
+                anchor_offset = first_quran_idx + i + num_consumed
+                anchor_remaining = phoneme_texts[anchor_offset:]
+
+                reanchor_surah, reanchor_ayah = find_anchor_by_voting(
+                    anchor_remaining, get_ngram_index(), ANCHOR_SEGMENTS,
+                )
+
+                if reanchor_surah > 0:
+                    next_surah = reanchor_surah
+                    chapter_ref = get_chapter_reference(next_surah)
+                    pointer = verse_to_word_index(chapter_ref, reanchor_ayah)
+                    print(f"  [GLOBAL-REANCHOR] Anchored to Surah {next_surah}, "
+                          f"Ayah {reanchor_ayah}, word {pointer}")
+                else:
+                    # Fallback: assume chapter 2
+                    next_surah = 2
+                    chapter_ref = get_chapter_reference(next_surah)
+                    pointer = 0
+                    print(f"  [GLOBAL-REANCHOR] No anchor found, falling back to Surah 2")
+            else:
+                next_surah = chapter_ref.surah + 1
+                if next_surah > 114:
+                    pass  # No more chapters — fall through to failure handling
+                else:
+                    print(f"  [CHAPTER-END] Surah {chapter_ref.surah} complete at segment {segment_idx}, "
+                          f"transitioning to Surah {next_surah}")
+                    chapter_ref = get_chapter_reference(next_surah)
+                    pointer = 0
+
+            if next_surah <= 114:
+                detected_surah = next_surah
+                consecutive_failures = 0
+
+                if num_consumed > 0:
+                    has_basmala = any(s[2] in ("Basmala", "Isti'adha+Basmala") for s in inter_specials)
+                    is_first_after_transition = not has_basmala
+                    # Current segment is a special — append its result
+                    results.append(inter_specials[0])
+                    word_indices.append(None)
+                    # Queue remaining specials for subsequent segments
+                    if num_consumed > 1:
+                        pending_specials = list(inter_specials[1:])
+                        skip_count = num_consumed - 1
+
+                    continue
+                else:
+                    is_first_after_transition = True
+                    # No specials — re-try alignment on this segment against the new chapter
+                    alignment, timing = align_segment(asr_phonemes, chapter_ref, pointer, segment_idx)
+                    num_segments += 1
+                    if PHONEME_ALIGNMENT_PROFILING:
+                        dp_times.append(timing['dp_time'])
+                        window_setup_total += timing['window_setup_time']
+                        result_build_total += timing['result_build_time']
+                    # Fall through to existing if/else below
+
+        # Basmala-fused retry: if this is the first segment after a transition
+        # and Basmala wasn't detected, the reciter may have merged Basmala with
+        # the first verse. Always try prepending Basmala phonemes to R and pick
+        # the better result (even if the plain alignment already succeeded).
+        if is_first_after_transition:
+            is_first_after_transition = False
+
+            basmala_alignment, basmala_timing = align_segment(
+                asr_phonemes, chapter_ref, pointer, segment_idx,
+                basmala_prefix=True)
+            num_segments += 1
+            if PHONEME_ALIGNMENT_PROFILING:
+                dp_times.append(basmala_timing['dp_time'])
+                window_setup_total += basmala_timing['window_setup_time']
+                result_build_total += basmala_timing['result_build_time']
+
+            if basmala_alignment and basmala_alignment.basmala_consumed:
+                existing_conf = alignment.confidence if alignment else 0.0
+                if basmala_alignment.confidence > existing_conf:
+                    matched_text = SPECIAL_TEXT["Basmala"] + " " + get_matched_text(chapter_ref, basmala_alignment)
+                    result = (matched_text, basmala_alignment.confidence, basmala_alignment.matched_ref)
+                    pointer = basmala_alignment.end_word_idx + 1
+                    consecutive_failures = 0
+                    word_indices.append((basmala_alignment.start_word_idx, basmala_alignment.end_word_idx))
+                    results.append(result)
+                    special_merges += 1
+                    segments_passed += 1
+                    print(f"  [BASMALA-FUSED] Segment {segment_idx}: Basmala merged with verse "
+                          f"(fused conf={basmala_alignment.confidence:.2f} > plain conf={existing_conf:.2f})")
+                    continue
+            # Basmala-fused didn't win — fall through with original alignment
+
+        if alignment:
+            is_first_after_transition = False
+            matched_text = get_matched_text(chapter_ref, alignment)
+            result = (matched_text, alignment.confidence, alignment.matched_ref)
+            pointer = alignment.end_word_idx + 1  # Advance pointer
+            consecutive_failures = 0
+            word_indices.append((alignment.start_word_idx, alignment.end_word_idx))
+            segments_passed += 1
+        else:
+            # === Graduated retry ===
+            # Tier 1: expanded window, same threshold
+            tier1_attempts += 1
+            tier1_segments.append(segment_idx)
+            alignment, timing = align_segment(
+                asr_phonemes, chapter_ref, pointer, segment_idx,
+                lookback_override=RETRY_LOOKBACK_WORDS,
+                lookahead_override=RETRY_LOOKAHEAD_WORDS,
+            )
+            num_segments += 1
+            if PHONEME_ALIGNMENT_PROFILING:
+                dp_times.append(timing['dp_time'])
+                window_setup_total += timing['window_setup_time']
+                result_build_total += timing['result_build_time']
+
+            # Tier 2: expanded window + relaxed threshold
+            tier2_entered = False
+            if alignment is None:
+                tier2_entered = True
+                tier2_attempts += 1
+                tier2_segments.append(segment_idx)
+                alignment, timing = align_segment(
+                    asr_phonemes, chapter_ref, pointer, segment_idx,
+                    lookback_override=RETRY_LOOKBACK_WORDS,
+                    lookahead_override=RETRY_LOOKAHEAD_WORDS,
+                    max_edit_distance_override=MAX_EDIT_DISTANCE_RELAXED,
+                )
+                num_segments += 1
+                if PHONEME_ALIGNMENT_PROFILING:
+                    dp_times.append(timing['dp_time'])
+                    window_setup_total += timing['window_setup_time']
+                    result_build_total += timing['result_build_time']
+
+            if alignment:
+                # Retry succeeded
+                is_first_after_transition = False
+                matched_text = get_matched_text(chapter_ref, alignment)
+                result = (matched_text, alignment.confidence, alignment.matched_ref)
+                pointer = alignment.end_word_idx + 1
+                consecutive_failures = 0
+                word_indices.append((alignment.start_word_idx, alignment.end_word_idx))
+                segments_passed += 1
+                if tier2_entered:
+                    tier2_passed += 1
+                else:
+                    tier1_passed += 1
+                print(f"  [RETRY-OK] Segment {segment_idx}: recovered via expanded window/relaxed threshold")
+            else:
+                # Real failure after all retries
+                result = ("", 0.0, "")
+                consecutive_failures += 1
+                word_indices.append(None)
+
+                if consecutive_failures >= MAX_CONSECUTIVE_FAILURES:
+                    consec_reanchors += 1
+                    # Global re-anchor (not constrained to current surah)
+                    remaining_idx = first_quran_idx + i + 1
+                    remaining_texts = phoneme_texts[remaining_idx:]
+                    if remaining_texts:
+                        reanchor_surah, reanchor_ayah = find_anchor_by_voting(
+                            remaining_texts, get_ngram_index(), ANCHOR_SEGMENTS,
+                        )
+                        if reanchor_surah > 0:
+                            if reanchor_surah != detected_surah:
+                                detected_surah = reanchor_surah
+                                chapter_ref = get_chapter_reference(detected_surah)
+                            pointer = verse_to_word_index(chapter_ref, reanchor_ayah)
+                            print(f"  [GLOBAL-REANCHOR] Jumped to Surah {detected_surah}, "
+                                  f"Ayah {reanchor_ayah}, word {pointer}")
+                    consecutive_failures = 0
+
+        results.append(result)
+
+    # Post-processing: detect consecutive segments with reference gaps
+    gap_segments = set()
+
+    prev_matched_idx = None
+    for idx in range(len(results)):
+        if word_indices[idx] is None:
+            continue
+
+        if prev_matched_idx is not None:
+            prev_end = word_indices[prev_matched_idx][1]
+            curr_start = word_indices[idx][0]
+            gap = curr_start - prev_end - 1
+
+            if gap > 0:
+                gap_segments.add(prev_matched_idx)
+                gap_segments.add(idx)
+
+                print(f"  [GAP] {gap} word(s) missing between segments "
+                      f"{prev_matched_idx + 1} and {idx + 1}")
+
+        prev_matched_idx = idx
+
+    # Edge case: missing words at start of expected range
+    first_matched = next((i for i, w in enumerate(word_indices) if w is not None), None)
+    if first_matched is not None:
+        first_start = word_indices[first_matched][0]
+        if first_start > start_pointer:
+            gap_segments.add(first_matched)
+            print(f"  [GAP] {first_start - start_pointer} word(s) missing before first segment {first_matched + 1}")
+
+    # Edge case: missing words at end of current verse
+    # Only flag if the last matched segment is also the final segment overall.
+    # If there are trailing no-match segments after it, those account for the
+    # remaining audio — the words aren't missing, they just failed to align.
+    # Compare against the verse boundary (not chapter end), since a recitation
+    # doesn't necessarily cover the entire chapter.
+    last_matched = next((i for i in range(len(word_indices) - 1, -1, -1) if word_indices[i] is not None), None)
+    if last_matched is not None and last_matched == len(word_indices) - 1:
+        last_end = word_indices[last_matched][1]
+        last_ayah = chapter_ref.words[last_end].ayah
+        # Find the last word index that belongs to the same verse
+        verse_end = last_end
+        while verse_end + 1 < chapter_ref.num_words and chapter_ref.words[verse_end + 1].ayah == last_ayah:
+            verse_end += 1
+        if last_end < verse_end:
+            gap_segments.add(last_matched)
+            print(f"  [GAP] {verse_end - last_end} word(s) missing after last segment {last_matched + 1}")
+
+    # Build profiling dict
+    if PHONEME_ALIGNMENT_PROFILING:
+        total_time = time.perf_counter() - total_start
+        profiling = {
+            "total_time": total_time,
+            "ref_build_time": ref_build_time,
+            "dp_total_time": sum(dp_times),
+            "dp_min_time": min(dp_times) if dp_times else 0.0,
+            "dp_max_time": max(dp_times) if dp_times else 0.0,
+            "window_setup_time": window_setup_total,
+            "result_build_time": result_build_total,
+            "num_segments": num_segments,
+            "tier1_attempts": tier1_attempts,
+            "tier1_passed": tier1_passed,
+            "tier1_segments": tier1_segments,
+            "tier2_attempts": tier2_attempts,
+            "tier2_passed": tier2_passed,
+            "tier2_segments": tier2_segments,
+            "consec_reanchors": consec_reanchors,
+            "segments_attempted": segments_attempted,
+            "segments_passed": segments_passed,
+            "special_merges": special_merges,
+        }
+    else:
+        profiling = {
+            "num_segments": num_segments,
+            "tier1_attempts": tier1_attempts,
+            "tier1_passed": tier1_passed,
+            "tier1_segments": tier1_segments,
+            "tier2_attempts": tier2_attempts,
+            "tier2_passed": tier2_passed,
+            "tier2_segments": tier2_segments,
+            "consec_reanchors": consec_reanchors,
+            "segments_attempted": segments_attempted,
+            "segments_passed": segments_passed,
+            "special_merges": special_merges,
+        }
+
+    return results, profiling, gap_segments

src/alignment/ngram_index.py

@@ -0,0 +1,39 @@
+"""
+Phoneme n-gram index: dataclass and cached loader.
+"""
+
+import pickle
+from dataclasses import dataclass
+from typing import Dict, List, Optional, Tuple
+
+from config import NGRAM_INDEX_PATH
+
+
+@dataclass
+class PhonemeNgramIndex:
+    """Pre-computed n-gram index for the entire Quran."""
+
+    # n-gram -> list of (surah, ayah) positions where it occurs
+    ngram_positions: Dict[Tuple[str, ...], List[Tuple[int, int]]]
+
+    # n-gram -> total occurrence count (for rarity weighting)
+    ngram_counts: Dict[Tuple[str, ...], int]
+
+    # Metadata
+    ngram_size: int
+    total_ngrams: int
+
+
+_INDEX: Optional[PhonemeNgramIndex] = None
+
+
+def get_ngram_index() -> PhonemeNgramIndex:
+    """Get or load the phoneme n-gram index."""
+    global _INDEX
+    if _INDEX is None:
+        print(f"[NGRAM] Loading index from {NGRAM_INDEX_PATH}...")
+        with open(NGRAM_INDEX_PATH, "rb") as f:
+            _INDEX = pickle.load(f)
+        print(f"[NGRAM] Loaded: {len(_INDEX.ngram_positions)} unique {_INDEX.ngram_size}-grams, "
+              f"{_INDEX.total_ngrams} total occurrences")
+    return _INDEX

src/alignment/phoneme_anchor.py

@@ -0,0 +1,293 @@
+"""
+Phoneme n-gram voting for global anchor detection.
+
+Replaces Whisper-based text matching for chapter/verse identification.
+Each ASR n-gram that matches the Quran index votes for (surah, ayah)
+weighted by rarity (1/count). The highest vote total wins the surah.
+Then we find the best contiguous run of voted ayahs in that surah and
+anchor to the first ayah of that run.
+"""
+
+from collections import defaultdict
+from typing import Dict, List, Tuple
+
+from config import ANCHOR_DEBUG, ANCHOR_RARITY_WEIGHTING, ANCHOR_RUN_TRIM_RATIO
+from .ngram_index import PhonemeNgramIndex
+from .phoneme_matcher import ChapterReference
+
+
+def _find_best_contiguous_run(
+    ayah_weights: Dict[int, float],
+) -> Tuple[int, int, float]:
+    """
+    Find the contiguous run of consecutive ayahs with highest total weight.
+
+    Args:
+        ayah_weights: {ayah_number: vote_weight} for a single surah
+
+    Returns:
+        (start_ayah, end_ayah, total_weight) of the best run
+    """
+    if not ayah_weights:
+        return (0, 0, 0.0)
+
+    sorted_ayahs = sorted(ayah_weights.keys())
+
+    # Build runs of consecutive ayahs
+    runs: List[Tuple[int, int, float]] = []  # (start, end, total_weight)
+    run_start = sorted_ayahs[0]
+    run_end = sorted_ayahs[0]
+    run_weight = ayah_weights[sorted_ayahs[0]]
+
+    for i in range(1, len(sorted_ayahs)):
+        ayah = sorted_ayahs[i]
+        if ayah == run_end + 1:
+            # Extends current run
+            run_end = ayah
+            run_weight += ayah_weights[ayah]
+        else:
+            # Gap — save current run, start new one
+            runs.append((run_start, run_end, run_weight))
+            run_start = ayah
+            run_end = ayah
+            run_weight = ayah_weights[ayah]
+
+    # Don't forget the last run
+    runs.append((run_start, run_end, run_weight))
+
+    # Pick run with highest total weight
+    best_start, best_end, best_weight = max(runs, key=lambda r: r[2])
+
+    # Trim leading/trailing ayahs whose weight < ANCHOR_RUN_TRIM_RATIO * max
+    max_w = max(ayah_weights[a] for a in range(best_start, best_end + 1))
+    threshold = ANCHOR_RUN_TRIM_RATIO * max_w
+
+    while best_start < best_end and ayah_weights[best_start] < threshold:
+        best_weight -= ayah_weights[best_start]
+        best_start += 1
+
+    while best_end > best_start and ayah_weights[best_end] < threshold:
+        best_weight -= ayah_weights[best_end]
+        best_end -= 1
+
+    return (best_start, best_end, best_weight)
+
+
+def find_anchor_by_voting(
+    phoneme_texts: List[List[str]],
+    ngram_index: PhonemeNgramIndex,
+    n_segments: int,
+) -> Tuple[int, int]:
+    """
+    Vote on (surah, ayah) using n-gram rarity weighting.
+
+    Two-phase selection:
+    1. Raw voting determines the winning surah (highest total weight across all ayahs)
+    2. Within that surah, find the best contiguous run of ayahs and return
+       the first ayah of that run as the anchor point.
+
+    Args:
+        phoneme_texts: Phoneme lists for segments (starting from first Quran segment)
+        ngram_index: Pre-built n-gram index
+        n_segments: Number of segments to use for voting
+
+    Returns:
+        (surah, ayah) of best match, or (0, 0) if nothing found
+    """
+    # Concatenate first N non-empty segments
+    combined: List[str] = []
+    segments_used = 0
+    for phonemes in phoneme_texts[:n_segments]:
+        if phonemes:
+            combined.extend(phonemes)
+            segments_used += 1
+
+    n = ngram_index.ngram_size
+
+    if ANCHOR_DEBUG:
+        print(f"\n{'=' * 60}")
+        print(f"ANCHOR VOTING DEBUG")
+        print(f"{'=' * 60}")
+        print(f"  Segments used: {segments_used}/{n_segments}")
+        print(f"  Combined phonemes: {len(combined)}")
+        print(f"  N-gram size: {n}")
+        if combined:
+            print(f"  ASR phonemes: {' '.join(combined[:30])}{'...' if len(combined) > 30 else ''}")
+
+    # Extract n-grams from ASR
+    asr_ngrams = [
+        tuple(combined[i : i + n])
+        for i in range(len(combined) - n + 1)
+    ]
+
+    if ANCHOR_DEBUG:
+        print(f"  ASR n-grams extracted: {len(asr_ngrams)}")
+
+    # =========================================================================
+    # Phase 1: Raw voting — accumulate (surah, ayah) votes
+    # =========================================================================
+    votes: Dict[Tuple[int, int], float] = defaultdict(float)
+    matched_ngrams = 0
+    missed_ngrams = 0
+
+    for ng in asr_ngrams:
+        if ng not in ngram_index.ngram_positions:
+            missed_ngrams += 1
+            continue
+
+        matched_ngrams += 1
+        weight = (1.0 / ngram_index.ngram_counts[ng]) if ANCHOR_RARITY_WEIGHTING else 1.0
+
+        for surah, ayah in ngram_index.ngram_positions[ng]:
+            votes[(surah, ayah)] += weight
+
+    if ANCHOR_DEBUG:
+        print(f"  N-grams matched: {matched_ngrams}/{len(asr_ngrams)} "
+              f"({missed_ngrams} missed)")
+        print(f"  Distinct (surah, ayah) voted for: {len(votes)}")
+
+    if not votes:
+        if ANCHOR_DEBUG:
+            print(f"  RESULT: No votes cast — returning (0, 0)")
+            print(f"{'=' * 60}\n")
+        return (0, 0)
+
+    # =========================================================================
+    # Phase 1b: Determine winning surah (by total weight across all ayahs)
+    # =========================================================================
+    surah_totals: Dict[int, float] = defaultdict(float)
+    for (s, a), w in votes.items():
+        surah_totals[s] += w
+
+    winning_surah = max(surah_totals, key=surah_totals.get)
+
+    if ANCHOR_DEBUG:
+        ranked_surahs = sorted(surah_totals.items(), key=lambda kv: kv[1], reverse=True)
+        print(f"\n  Surah vote totals (top 5):")
+        print(f"  {'Surah':>5}  {'Total Weight':>12}")
+        print(f"  {'-' * 20}")
+        for s, w in ranked_surahs[:5]:
+            marker = " <-- winner" if s == winning_surah else ""
+            print(f"  {s:>5}  {w:>12.3f}{marker}")
+
+    # =========================================================================
+    # Phase 2: Within winning surah, find best contiguous ayah run
+    # =========================================================================
+    ayah_weights: Dict[int, float] = {}
+    for (s, a), w in votes.items():
+        if s == winning_surah:
+            ayah_weights[a] = w
+
+    run_start, run_end, run_weight = _find_best_contiguous_run(ayah_weights)
+
+    if ANCHOR_DEBUG:
+        # Show per-ayah votes in this surah
+        print(f"\n  Surah {winning_surah} ayah votes:")
+        print(f"  {'Ayah':>5}  {'Weight':>8}  {'In Best Run':>11}")
+        print(f"  {'-' * 28}")
+        for a in sorted(ayah_weights.keys()):
+            in_run = "***" if run_start <= a <= run_end else ""
+            print(f"  {a:>5}  {ayah_weights[a]:>8.3f}  {in_run:>11}")
+
+        print(f"\n  Best contiguous run (after trim): ayahs {run_start}-{run_end} "
+              f"(weight={run_weight:.3f}, trim_ratio={ANCHOR_RUN_TRIM_RATIO})")
+        print(f"  RESULT: Surah {winning_surah}, Ayah {run_start} (start of run)")
+        print(f"{'=' * 60}\n")
+
+    return (winning_surah, run_start)
+
+
+def reanchor_within_surah(
+    phoneme_texts: List[List[str]],
+    ngram_index: PhonemeNgramIndex,
+    surah: int,
+    n_segments: int,
+) -> int:
+    """
+    Re-anchor within a known surah after consecutive DP failures.
+
+    Same n-gram voting as find_anchor_by_voting but:
+    - Only counts votes for the given surah (skip all others)
+    - Returns ayah (start of best contiguous run), or 0 if no votes
+
+    Args:
+        phoneme_texts: Remaining unprocessed phoneme lists
+        ngram_index: Pre-built n-gram index
+        surah: Current surah (fixed)
+        n_segments: How many segments to use for voting
+
+    Returns:
+        ayah number to re-anchor to (0 = failed)
+    """
+    # Concatenate first N non-empty segments
+    combined: List[str] = []
+    segments_used = 0
+    for phonemes in phoneme_texts[:n_segments]:
+        if phonemes:
+            combined.extend(phonemes)
+            segments_used += 1
+
+    n = ngram_index.ngram_size
+
+    if ANCHOR_DEBUG:
+        print(f"\n{'=' * 60}")
+        print(f"RE-ANCHOR WITHIN SURAH {surah}")
+        print(f"{'=' * 60}")
+        print(f"  Segments used: {segments_used}/{n_segments}")
+        print(f"  Combined phonemes: {len(combined)}")
+
+    # Extract n-grams from ASR
+    asr_ngrams = [
+        tuple(combined[i : i + n])
+        for i in range(len(combined) - n + 1)
+    ]
+
+    # Vote — only accumulate weight for positions in the given surah
+    ayah_weights: Dict[int, float] = defaultdict(float)
+    matched_ngrams = 0
+
+    for ng in asr_ngrams:
+        if ng not in ngram_index.ngram_positions:
+            continue
+        matched_ngrams += 1
+        weight = (1.0 / ngram_index.ngram_counts[ng]) if ANCHOR_RARITY_WEIGHTING else 1.0
+        for s, a in ngram_index.ngram_positions[ng]:
+            if s == surah:
+                ayah_weights[a] += weight
+
+    if ANCHOR_DEBUG:
+        print(f"  N-grams matched: {matched_ngrams}/{len(asr_ngrams)}")
+        print(f"  Ayahs with votes: {len(ayah_weights)}")
+
+    if not ayah_weights:
+        if ANCHOR_DEBUG:
+            print(f"  RESULT: No votes — returning 0")
+            print(f"{'=' * 60}\n")
+        return 0
+
+    run_start, run_end, run_weight = _find_best_contiguous_run(dict(ayah_weights))
+
+    if ANCHOR_DEBUG:
+        print(f"  Best contiguous run (after trim): ayahs {run_start}-{run_end} "
+              f"(weight={run_weight:.3f}, trim_ratio={ANCHOR_RUN_TRIM_RATIO})")
+        print(f"  RESULT: Ayah {run_start}")
+        print(f"{'=' * 60}\n")
+
+    return run_start
+
+
+def verse_to_word_index(chapter_ref: ChapterReference, ayah: int) -> int:
+    """
+    Find word index of the first word in a given ayah.
+
+    Args:
+        chapter_ref: Pre-built chapter reference
+        ayah: Verse number to find
+
+    Returns:
+        Word index into chapter_ref.words, or 0 if not found
+    """
+    for idx, word in enumerate(chapter_ref.words):
+        if word.ayah == ayah:
+            return idx
+    return 0

src/alignment/phoneme_asr.py

@@ -0,0 +1,355 @@
+"""Phoneme ASR using wav2vec2 CTC model."""
+
+import os
+import time
+import torch
+import numpy as np
+from typing import List, Dict, Any
+
+from config import (
+    PHONEME_ASR_MODELS, PHONEME_ASR_MODEL_DEFAULT, DTYPE, IS_HF_SPACE, TORCH_COMPILE,
+    BATCHING_STRATEGY, INFERENCE_BATCH_SIZE,
+    MAX_BATCH_SECONDS, MAX_PAD_WASTE, MIN_BATCH_SIZE,
+)
+from ..zero_gpu import ZERO_GPU_AVAILABLE, is_quota_exhausted
+
+
+_cache = {}  # model_name -> {"model": Model, "processor": Processor, "device": str}
+
+_TORCH_DTYPE = torch.float16 if DTYPE == "float16" else torch.float32
+
+
+def _get_hf_token():
+    """Get HF token from env var or stored login."""
+    token = os.environ.get("HF_TOKEN")
+    if not token:
+        try:
+            from huggingface_hub import HfFolder
+            token = HfFolder.get_token()
+        except Exception:
+            pass
+    return token
+
+
+def _get_device_and_dtype():
+    """Get the best available device and dtype.
+
+    On HF Spaces with ZeroGPU, returns CPU to defer CUDA init
+    until inside a @gpu_decorator function.
+    """
+    if IS_HF_SPACE or ZERO_GPU_AVAILABLE:
+        return torch.device("cpu"), _TORCH_DTYPE
+    if torch.cuda.is_available():
+        return torch.device("cuda"), _TORCH_DTYPE
+    return torch.device("cpu"), _TORCH_DTYPE
+
+
+def load_phoneme_asr(model_name=PHONEME_ASR_MODEL_DEFAULT):
+    """Load phoneme ASR model on CPU. Returns (model, processor).
+
+    Models are loaded once and cached per model_name. Both base and large
+    can be cached simultaneously. Use move_phoneme_asr_to_gpu() inside
+    GPU-decorated functions to move to CUDA.
+    """
+    if model_name in _cache:
+        entry = _cache[model_name]
+        return entry["model"], entry["processor"]
+
+    import logging
+    from transformers import AutoModelForCTC, AutoProcessor
+
+    # Suppress verbose transformers logging during load
+    logging.getLogger("transformers").setLevel(logging.WARNING)
+
+    model_path = PHONEME_ASR_MODELS[model_name]
+    print(f"Loading phoneme ASR: {model_path} ({model_name})")
+
+    # Use HF_TOKEN for private model access
+    hf_token = _get_hf_token()
+
+    device, dtype = _get_device_and_dtype()
+
+    model = AutoModelForCTC.from_pretrained(
+        model_path, token=hf_token, attn_implementation="sdpa"
+    )
+    model.to(device, dtype=dtype)
+    model.eval()
+    if TORCH_COMPILE and not (IS_HF_SPACE or ZERO_GPU_AVAILABLE):
+        model = torch.compile(model, mode="reduce-overhead")
+
+    processor = AutoProcessor.from_pretrained(model_path, token=hf_token)
+
+    _cache[model_name] = {
+        "model": model,
+        "processor": processor,
+        "device": device.type,
+    }
+
+    print(f"Phoneme ASR ({model_name}) loaded on {device}")
+    return model, processor
+
+
+def move_phoneme_asr_to_gpu(model_name=None):
+    """Move cached phoneme ASR model(s) to GPU.
+
+    Args:
+        model_name: Move only this model. If None, move all cached models.
+
+    Call this inside @gpu_decorator functions on HF Spaces.
+    Idempotent: checks current device before moving.
+    Skips if quota exhausted or CUDA unavailable.
+    """
+    if is_quota_exhausted() or not torch.cuda.is_available():
+        return
+
+    names = [model_name] if model_name else list(_cache.keys())
+    device = torch.device("cuda")
+
+    for name in names:
+        if name not in _cache:
+            continue
+        entry = _cache[name]
+        model = entry["model"]
+        if next(model.parameters()).device.type != "cuda":
+            entry["model"] = model.to(device, dtype=_TORCH_DTYPE)
+            entry["device"] = "cuda"
+            print(f"[PHONEME ASR] Moved '{name}' to CUDA")
+
+
+def move_phoneme_asr_to_cpu(model_name=None):
+    """Move cached phoneme ASR model(s) back to CPU.
+
+    Args:
+        model_name: Move only this model. If None, move all cached models.
+
+    Called when GPU lease fails or quota is exhausted so that
+    CPU fallback inference can proceed.
+    Idempotent: checks current device before moving.
+    """
+    names = [model_name] if model_name else list(_cache.keys())
+    device = torch.device("cpu")
+
+    for name in names:
+        if name not in _cache:
+            continue
+        entry = _cache[name]
+        model = entry["model"]
+        if next(model.parameters()).device.type != "cpu":
+            entry["model"] = model.to(device, dtype=_TORCH_DTYPE)
+            entry["device"] = "cpu"
+            print(f"[PHONEME ASR] Moved '{name}' to CPU")
+
+
+def ids_to_phoneme_list(ids: List[int], tokenizer, pad_id: int) -> List[str]:
+    """
+    Convert token IDs to phoneme list with CTC collapse.
+
+    CTC decoding:
+    1. Remove pad/blank tokens
+    2. Collapse consecutive duplicates
+    3. Filter out word delimiter "|"
+    """
+    # Convert all IDs to tokens first (do not skip any yet)
+    toks = tokenizer.convert_ids_to_tokens(ids)
+
+    if not toks:
+        return []
+
+    # Get the actual token string for pad
+    pad_tok = tokenizer.convert_ids_to_tokens([pad_id])[0] if pad_id is not None else "[PAD]"
+
+    # CTC collapse: remove consecutive duplicates and special tokens
+    collapsed: List[str] = []
+    prev = None
+    for t in toks:
+        # Skip pad/blank token
+        if t == pad_tok:
+            prev = t
+            continue
+        # Skip word delimiter
+        if t == "|":
+            prev = t
+            continue
+        # Skip consecutive duplicates (CTC collapse)
+        if t == prev:
+            continue
+        collapsed.append(t)
+        prev = t
+
+    return collapsed
+
+
+def build_batches_naive(sorted_indices: List[int], batch_size: int) -> List[List[int]]:
+    """Fixed-count batching (original behavior)."""
+    return [sorted_indices[i:i + batch_size]
+            for i in range(0, len(sorted_indices), batch_size)]
+
+
+def build_batches(sorted_indices: List[int], durations: List[float]) -> List[List[int]]:
+    """Build dynamic batches from duration-sorted indices.
+
+    Constraints:
+        - sum(durations) per batch <= MAX_BATCH_SECONDS
+        - pad waste fraction <= MAX_PAD_WASTE  (1 - sum/[n*max], measures wasted tensor compute)
+        - batch won't be cut below MIN_BATCH_SIZE (avoids underutilization)
+    """
+    batches: List[List[int]] = []
+    current: List[int] = []
+    current_seconds = 0.0
+
+    for i in sorted_indices:
+        dur = durations[i]
+
+        if not current:
+            current.append(i)
+            current_seconds = dur
+            continue
+
+        max_dur = dur                      # candidate is the new longest (sorted ascending)
+        new_seconds = current_seconds + dur
+        new_size = len(current) + 1
+        pad_waste = 1.0 - new_seconds / (new_size * max_dur) if max_dur > 0 else 0.0
+
+        seconds_exceeded = new_seconds > MAX_BATCH_SECONDS
+        waste_exceeded = pad_waste > MAX_PAD_WASTE
+
+        if (seconds_exceeded or waste_exceeded) and len(current) >= MIN_BATCH_SIZE:
+            batches.append(current)
+            current = [i]
+            current_seconds = dur
+        else:
+            current.append(i)
+            current_seconds = new_seconds
+
+    if current:
+        batches.append(current)
+
+    return batches
+
+
+def _transcribe_batch_pytorch(
+    segment_audios: List[np.ndarray],
+    durations: List[float],
+    batches: List[List[int]],
+    model,
+    processor,
+    tokenizer,
+    pad_id: int,
+    device: torch.device,
+    dtype: torch.dtype,
+) -> tuple:
+    """PyTorch inference path (GPU or CPU fallback)."""
+    results: List[List[str]] = [[] for _ in segment_audios]
+    batch_profiling = []
+
+    for batch_num_idx, batch_idx in enumerate(batches):
+        batch_audios = [segment_audios[i] for i in batch_idx]
+        batch_durations = [durations[i] for i in batch_idx]
+
+        batch_num = batch_num_idx + 1
+        t0 = time.time()
+
+        # Feature extraction + GPU transfer
+        t_feat_start = time.time()
+        inputs = processor(
+            batch_audios,
+            sampling_rate=16000,
+            return_tensors="pt",
+            padding=True,
+        )
+        input_values = inputs.input_values.to(device=device, dtype=dtype)
+        attention_mask = inputs.get("attention_mask")
+        if attention_mask is not None:
+            attention_mask = attention_mask.to(device=device)
+        feat_time = time.time() - t_feat_start
+
+        # Model inference
+        t_infer_start = time.time()
+        with torch.no_grad():
+            outputs = model(input_values, attention_mask=attention_mask)
+            logits = outputs.logits
+        if device.type == "cuda":
+            torch.cuda.synchronize()
+        infer_time = time.time() - t_infer_start
+
+        # CTC greedy decode
+        t_decode_start = time.time()
+        predicted_ids = torch.argmax(logits, dim=-1)
+
+        for j in range(predicted_ids.shape[0]):
+            ids_list = predicted_ids[j].cpu().tolist()
+            phoneme_list = ids_to_phoneme_list(ids_list, tokenizer, pad_id)
+            results[batch_idx[j]] = phoneme_list
+        decode_time = time.time() - t_decode_start
+
+        del input_values, attention_mask, outputs, logits, predicted_ids
+
+        batch_time = time.time() - t0
+
+        batch_profiling.append({
+            "batch_num": batch_num,
+            "size": len(batch_audios),
+            "time": batch_time,
+            "feat_time": feat_time,
+            "infer_time": infer_time,
+            "decode_time": decode_time,
+            "min_dur": min(batch_durations),
+            "max_dur": max(batch_durations),
+            "avg_dur": sum(batch_durations) / len(batch_durations),
+            "total_seconds": sum(batch_durations),
+            "pad_waste": 1.0 - sum(batch_durations) / (len(batch_durations) * max(batch_durations)) if max(batch_durations) > 0 else 0.0,
+        })
+
+    return results, batch_profiling
+
+
+def transcribe_batch(segment_audios: List[np.ndarray], sample_rate: int, model_name: str = PHONEME_ASR_MODEL_DEFAULT) -> tuple:
+    """Transcribe audio segments to phoneme lists, sorted by duration for efficiency.
+
+    Args:
+        segment_audios: List of audio arrays
+        sample_rate: Audio sample rate
+        model_name: Which ASR model to use ("base" or "large")
+
+    Returns:
+        (results, batch_profiling) where results is List[List[str]] and
+        batch_profiling is a list of dicts with per-batch timing and duration stats.
+    """
+    if not segment_audios:
+        return [], [], 0.0, 0.0
+
+    model, processor = load_phoneme_asr(model_name)
+    if model is None:
+        return [[] for _ in segment_audios], [], 0.0, 0.0
+
+    device = next(model.parameters()).device
+    dtype = next(model.parameters()).dtype
+    tokenizer = processor.tokenizer
+    pad_id = tokenizer.pad_token_id if tokenizer.pad_token_id is not None else 0
+
+    # Compute durations (audio assumed to be 16kHz — resampled at source)
+    durations = [len(audio) / 16000.0 for audio in segment_audios]
+
+    # Sort indices by duration, then build dynamic batches
+    t_sort = time.time()
+    sorted_indices = sorted(range(len(segment_audios)), key=lambda i: durations[i])
+    sorting_time = time.time() - t_sort
+
+    t_batch_build = time.time()
+    if BATCHING_STRATEGY == "dynamic":
+        batches = build_batches(sorted_indices, durations)
+    else:
+        batches = build_batches_naive(sorted_indices, INFERENCE_BATCH_SIZE)
+    batch_build_time = time.time() - t_batch_build
+
+    backend = "PyTorch" + (f" ({device.type})" if device.type != "cpu" else " (CPU)")
+    print(f"[PHONEME ASR] Using {backend}")
+    results, batch_profiling = _transcribe_batch_pytorch(
+        segment_audios, durations, batches,
+        model, processor, tokenizer, pad_id, device, dtype,
+    )
+
+    sizes = [p["size"] for p in batch_profiling]
+    print(f"[PHONEME ASR] {len(segment_audios)} segments in {len(batch_profiling)} batches "
+          f"(sizes: {min(sizes)}-{max(sizes)}, sort: {sorting_time:.3f}s, batch build: {batch_build_time:.3f}s)")
+    return results, batch_profiling, sorting_time, batch_build_time

src/alignment/phoneme_matcher.py

@@ -0,0 +1,590 @@
+"""
+Phoneme-based alignment using substring Levenshtein DP.
+
+This module implements the core alignment algorithm for matching ASR phoneme
+sequences to reference Quranic text phonemes with word-boundary constraints.
+"""
+
+import json
+from dataclasses import dataclass
+from typing import Dict, List, Optional, Tuple
+
+from config import (
+    DATA_PATH,
+    LOOKBACK_WORDS,
+    LOOKAHEAD_WORDS,
+    MAX_EDIT_DISTANCE,
+    START_PRIOR_WEIGHT,
+    COST_SUBSTITUTION,
+    COST_DELETION,
+    COST_INSERTION,
+    PHONEME_ALIGNMENT_DEBUG,
+    PHONEME_ALIGNMENT_PROFILING,
+)
+
+from ..phonemizer_utils import get_phonemizer
+
+
+# =============================================================================
+# Phoneme Substitution Cost Lookup
+# =============================================================================
+
+
+def _load_substitution_costs() -> Dict[Tuple[str, str], float]:
+    """Load phoneme pair substitution costs from JSON data file.
+
+    Stores both orderings (a,b) and (b,a) so lookups need only a plain tuple.
+    """
+    path = DATA_PATH / "phoneme_sub_costs.json"
+    if not path.exists():
+        return {}
+    with open(path) as f:
+        raw = json.load(f)
+    costs = {}
+    for key, section in raw.items():
+        if key == "_meta":
+            continue
+        for pair_str, cost in section.items():
+            a, b = pair_str.split("|")
+            c = float(cost)
+            costs[(a, b)] = c
+            costs[(b, a)] = c
+    return costs
+
+
+_SUBSTITUTION_COSTS: Dict[Tuple[str, str], float] = _load_substitution_costs()
+
+# Try to load Cython-accelerated DP; fall back to pure Python silently.
+try:
+    from .._dp_core import cy_align_with_word_boundaries, init_substitution_matrix
+    init_substitution_matrix(_SUBSTITUTION_COSTS, COST_SUBSTITUTION)
+    _USE_CYTHON_DP = True
+except ImportError:
+    _USE_CYTHON_DP = False
+
+
+def get_sub_cost(p: str, r: str, default: float) -> float:
+    """Look up substitution cost for a phoneme pair.
+
+    Returns 0.0 for exact match, pair-specific cost if defined, otherwise default.
+    """
+    if p == r:
+        return 0.0
+    return _SUBSTITUTION_COSTS.get((p, r), default)
+
+
+# =============================================================================
+# Data Structures
+# =============================================================================
+
+
+@dataclass
+class RefWord:
+    """Reference word with phoneme metadata."""
+    text: str              # Arabic text
+    phonemes: List[str]    # Phoneme list for this word
+    surah: int             # Surah number
+    ayah: int              # Verse number within surah
+    word_num: int          # Word number within verse (1-indexed)
+
+    @property
+    def location(self) -> str:
+        """Format as 'surah:ayah:word'."""
+        return f"{self.surah}:{self.ayah}:{self.word_num}"
+
+
+@dataclass
+class ChapterReference:
+    """Pre-built reference data for a chapter."""
+    surah: int
+    words: List[RefWord]
+    avg_phones_per_word: float
+
+    # Pre-flattened phoneme data (avoids rebuilding per segment)
+    flat_phonemes: List[str]        # All phonemes concatenated
+    flat_phone_to_word: List[int]   # Word index for each phoneme (GLOBAL indices)
+    word_phone_offsets: List[int]   # Prefix sum: word i starts at offset[i]
+
+    @property
+    def num_words(self) -> int:
+        return len(self.words)
+
+
+@dataclass
+class AlignmentResult:
+    """Result of aligning a segment."""
+    start_word_idx: int        # Index into ChapterReference.words
+    end_word_idx: int          # Index into ChapterReference.words (inclusive)
+    edit_cost: float           # Raw edit distance (may be non-integer with substitution costs)
+    confidence: float          # 1.0 - (edit_cost / max(asr_len, ref_len))
+
+    # For debugging
+    j_start: int               # Start phoneme index in R window
+    best_j: int                # End phoneme index in R window (exclusive)
+
+    # Resolved word references
+    start_word: RefWord
+    end_word: RefWord
+
+    # Whether Basmala prefix was consumed by the alignment
+    basmala_consumed: bool = False
+
+    @property
+    def ref_from(self) -> str:
+        """Start reference as 'surah:ayah:word'."""
+        return self.start_word.location
+
+    @property
+    def ref_to(self) -> str:
+        """End reference as 'surah:ayah:word'."""
+        return self.end_word.location
+
+    @property
+    def matched_ref(self) -> str:
+        """Combined reference as 'start-end'."""
+        return f"{self.ref_from}-{self.ref_to}"
+
+
+# =============================================================================
+# Helper Functions
+# =============================================================================
+
+
+def parse_location(location: str) -> Tuple[int, int, int]:
+    """Parse 'surah:ayah:word' into (surah, ayah, word_num)."""
+    parts = location.split(":")
+    return int(parts[0]), int(parts[1]), int(parts[2])
+
+
+def get_matched_text(chapter_ref: ChapterReference, result: AlignmentResult) -> str:
+    """Get Arabic text for aligned words."""
+    words = chapter_ref.words[result.start_word_idx : result.end_word_idx + 1]
+    return ' '.join(w.text for w in words)
+
+
+def _word_loc(words: List, idx: int) -> str:
+    """Format word index as 'ayah:word_num'."""
+    if idx < 0 or idx >= len(words):
+        return f"?({idx})"
+    w = words[idx]
+    return f"{w.ayah}:{w.word_num}"
+
+
+def print_debug_info(
+    P: List[str],
+    R: List[str],
+    result: Optional[AlignmentResult],
+    segment_idx: int,
+    pointer: int,
+    win_start: int,
+    win_end: int,
+    words: List = None,
+) -> None:
+    """Print detailed alignment debug info."""
+    if not PHONEME_ALIGNMENT_DEBUG:
+        return
+
+    print("\n" + "━" * 60)
+    print(f"[PHONEME ALIGN] Segment {segment_idx}")
+    print("─" * 60)
+    loc_range = ""
+    if words:
+        loc_range = f" = {_word_loc(words, win_start)}-{_word_loc(words, win_end)}"
+    print(f"  Window: words [{win_start}-{win_end}]{loc_range} "
+          f"({win_end - win_start} words, {len(R)} phonemes)")
+    ptr_loc = ""
+    if words:
+        ptr_loc = f" = {_word_loc(words, pointer)}"
+    print(f"  Expected start: word {pointer}{ptr_loc}")
+    print()
+    if len(R) <= 40:
+        print(f"  R:        {' '.join(R)}")
+    else:
+        print(f"  R:        {' '.join(R[:20])} ... {' '.join(R[-20:])}")
+    print(f"  P:        {' '.join(P)}  ({len(P)} phonemes)")
+    print()
+
+    if result:
+        recovered = R[result.j_start:result.best_j]
+        print(f"  ✓ MATCH: words [{result.start_word_idx}-{result.end_word_idx}] "
+              f"({result.end_word_idx - result.start_word_idx + 1} words)")
+        print(f"  Recovered: {' '.join(recovered)}  ({len(recovered)} phonemes)")
+        print(f"  Edit cost: {result.edit_cost}")
+        print(f"  Confidence: {result.confidence:.2f}")
+    else:
+        print(f"  ✗ NO MATCH (no candidates passed threshold)")
+
+    print("━" * 60)
+
+
+# =============================================================================
+# Chapter Reference Building
+# =============================================================================
+
+
+def build_chapter_reference(surah_num: int) -> ChapterReference:
+    """Build phoneme reference for entire chapter."""
+    pm = get_phonemizer()
+
+    # Phonemize entire chapter with stopping rules at verse boundaries
+    result = pm.phonemize(
+        ref=str(surah_num),
+        stops=["verse"]
+    )
+
+    # Get mapping - provides word metadata and phonemes directly
+    mapping = result.get_mapping()
+
+    # Build RefWord list - WordMapping already has phonemes as List[str]
+    words = []
+    for word in mapping.words:
+        surah, ayah, word_num = parse_location(word.location)
+        words.append(RefWord(
+            text=word.text,
+            phonemes=word.phonemes,  # Direct access, no string parsing needed
+            surah=surah,
+            ayah=ayah,
+            word_num=word_num,
+        ))
+
+    # Compute average phonemes per word
+    total_phones = sum(len(w.phonemes) for w in words)
+    avg_phones_per_word = total_phones / len(words) if words else 4.0
+
+    # Pre-flatten phonemes for efficient windowing (avoids per-segment rebuilds)
+    flat_phonemes = []
+    flat_phone_to_word = []
+    word_phone_offsets = []
+
+    for word_idx, word in enumerate(words):
+        word_phone_offsets.append(len(flat_phonemes))  # Start offset for this word
+        for ph in word.phonemes:
+            flat_phonemes.append(ph)
+            flat_phone_to_word.append(word_idx)
+
+    # Sentinel: offset past last phoneme (for slicing convenience)
+    word_phone_offsets.append(len(flat_phonemes))
+
+    return ChapterReference(
+        surah=surah_num,
+        words=words,
+        avg_phones_per_word=avg_phones_per_word,
+        flat_phonemes=flat_phonemes,
+        flat_phone_to_word=flat_phone_to_word,
+        word_phone_offsets=word_phone_offsets,
+    )
+
+
+# =============================================================================
+# Word-Boundary-Constrained Alignment (DP)
+# =============================================================================
+
+
+def align_with_word_boundaries(
+    P: List[str],
+    R: List[str],
+    R_phone_to_word: List[int],
+    expected_word: int = 0,
+    prior_weight: float = START_PRIOR_WEIGHT,
+    cost_sub: float = COST_SUBSTITUTION,
+    cost_del: float = COST_DELETION,
+    cost_ins: float = COST_INSERTION,
+) -> Tuple[Optional[int], Optional[int], float, float]:
+    """
+    Word-boundary-constrained substring alignment.
+
+    Combines DP computation with best-match selection:
+    - Start: only word-start positions allowed (INF cost otherwise)
+    - End: only word-end positions evaluated as candidates
+
+    Args:
+        P: ASR phoneme sequence
+        R: Reference phoneme window
+        R_phone_to_word: Maps phoneme index -> word index (GLOBAL indices)
+        expected_word: Expected starting word index (for position prior)
+        prior_weight: Penalty per word distance from expected
+        cost_sub: Substitution cost
+        cost_del: Deletion cost (delete from P)
+        cost_ins: Insertion cost (insert from R)
+
+    Returns:
+        (best_j, best_j_start, best_cost, best_norm_dist) or (None, None, INF, INF)
+    """
+    if _USE_CYTHON_DP:
+        return cy_align_with_word_boundaries(
+            P, R, R_phone_to_word,
+            expected_word, prior_weight,
+            cost_sub, cost_del, cost_ins,
+        )
+
+    # --- Pure Python fallback ---
+    m, n = len(P), len(R)
+    INF = float('inf')
+
+    if m == 0 or n == 0:
+        return None, None, INF, float('inf')
+
+    # DP column semantics:
+    #   Column j represents "consumed j phonemes" / boundary after phoneme j-1
+    #   Column 0 = before any phonemes, Column n = after all phonemes
+    #   Phoneme indices are 0..n-1, DP columns are 0..n
+
+    def is_start_boundary(j: int) -> bool:
+        """Can alignment START at DP column j? (before phoneme j)"""
+        if j >= n:
+            return False  # Can't start at or past end
+        if j == 0:
+            return True   # Column 0 is always valid start (first word)
+        # Valid if phoneme j begins a new word
+        return R_phone_to_word[j] != R_phone_to_word[j - 1]
+
+    def is_end_boundary(j: int) -> bool:
+        """Can alignment END at DP column j? (after phoneme j-1)"""
+        if j == 0:
+            return False  # Can't end before consuming anything
+        if j == n:
+            return True   # Column n (end of reference) always valid
+        # Valid if phoneme j starts a new word (meaning j-1 ended a word)
+        return R_phone_to_word[j] != R_phone_to_word[j - 1]
+
+    # Initialize: free start ONLY at word boundaries
+    prev_cost = [0.0 if is_start_boundary(j) else INF for j in range(n + 1)]
+    prev_start = [j if is_start_boundary(j) else -1 for j in range(n + 1)]
+
+    curr_cost = [0.0] * (n + 1)
+    curr_start = [0] * (n + 1)
+
+    # DP computation
+    for i in range(1, m + 1):
+        curr_cost[0] = i * cost_del if is_start_boundary(0) else INF
+        curr_start[0] = 0 if is_start_boundary(0) else -1
+
+        for j in range(1, n + 1):
+            del_option = prev_cost[j] + cost_del
+            ins_option = curr_cost[j-1] + cost_ins
+            sub_option = prev_cost[j-1] + get_sub_cost(P[i-1], R[j-1], cost_sub)
+
+            if sub_option <= del_option and sub_option <= ins_option:
+                curr_cost[j] = sub_option
+                curr_start[j] = prev_start[j-1]
+            elif del_option <= ins_option:
+                curr_cost[j] = del_option
+                curr_start[j] = prev_start[j]
+            else:
+                curr_cost[j] = ins_option
+                curr_start[j] = curr_start[j-1]
+
+        prev_cost, curr_cost = curr_cost, prev_cost
+        prev_start, curr_start = curr_start, prev_start
+
+    # After DP: evaluate only valid end boundary positions
+    # prev_cost/prev_start now contain the final row (after m iterations)
+    best_score = float('inf')  # Score includes float norm_dist, so keep as float
+    best_j = None
+    best_j_start = None
+    best_cost = INF
+    best_norm_dist = float('inf')
+
+    for j in range(1, n + 1):
+        # Skip non-end-boundary positions
+        if not is_end_boundary(j):
+            continue
+
+        # Skip infinite cost (no valid alignment ends here)
+        if prev_cost[j] >= INF:
+            continue
+
+        dist = prev_cost[j]
+        j_start = prev_start[j]
+
+        # Compute normalized edit distance
+        ref_len = j - j_start
+        denom = max(m, ref_len, 1)
+        norm_dist = dist / denom
+
+        # Position prior on start word
+        start_word = R_phone_to_word[j_start] if j_start < n else R_phone_to_word[j - 1]
+        prior = prior_weight * abs(start_word - expected_word)
+        score = norm_dist + prior
+
+        if score < best_score:
+            best_score = score
+            best_j = j
+            best_j_start = j_start
+            best_cost = dist
+            best_norm_dist = norm_dist
+
+    return best_j, best_j_start, best_cost, best_norm_dist
+
+
+# =============================================================================
+# Per-Segment Alignment
+# =============================================================================
+
+
+def align_segment(
+    asr_phonemes: List[str],
+    chapter_ref: ChapterReference,
+    pointer: int,
+    segment_idx: int = 0,
+    basmala_prefix: bool = False,
+    lookback_override: Optional[int] = None,
+    lookahead_override: Optional[int] = None,
+    max_edit_distance_override: Optional[float] = None,
+) -> Tuple[Optional[AlignmentResult], dict]:
+    """
+    Align ASR phonemes to reference using substring Levenshtein DP.
+
+    Args:
+        asr_phonemes: Phoneme sequence from ASR for this segment
+        chapter_ref: Pre-built chapter reference data
+        pointer: First unprocessed word index (0 at start of chapter)
+        segment_idx: Segment number for debug output
+        basmala_prefix: If True, prepend Basmala phonemes to the R window
+            so the DP can consume a fused Basmala+verse segment
+        lookback_override: Override LOOKBACK_WORDS for this call
+        lookahead_override: Override LOOKAHEAD_WORDS for this call
+        max_edit_distance_override: Override MAX_EDIT_DISTANCE for this call
+
+    Returns: (AlignmentResult or None, timing_dict)
+    """
+    timing = {'window_setup_time': 0.0, 'dp_time': 0.0, 'result_build_time': 0.0}
+
+    # Only import time if profiling is enabled
+    if PHONEME_ALIGNMENT_PROFILING:
+        import time
+
+    P = asr_phonemes
+    m = len(P)
+
+    if m == 0:
+        return None, timing
+
+    words = chapter_ref.words
+    avg_phones = chapter_ref.avg_phones_per_word
+    num_words = chapter_ref.num_words
+
+    # === WINDOW SETUP ===
+    if PHONEME_ALIGNMENT_PROFILING:
+        t0 = time.perf_counter()
+
+    # 1. Estimate word count from phoneme count
+    est_words = max(1, round(m / avg_phones))
+
+    # 2. Define search window (word indices)
+    lb = lookback_override if lookback_override is not None else LOOKBACK_WORDS
+    la = lookahead_override if lookahead_override is not None else LOOKAHEAD_WORDS
+    win_start = max(0, pointer - lb)
+    win_end = min(num_words, pointer + est_words + la)
+
+    # End of chapter check
+    if win_start >= num_words:
+        if PHONEME_ALIGNMENT_PROFILING:
+            timing['window_setup_time'] = time.perf_counter() - t0
+        if PHONEME_ALIGNMENT_DEBUG:
+            print(f"[PHONEME ALIGN] Segment {segment_idx}: Past end of chapter")
+        return None, timing
+
+    # 3. Slice pre-flattened phoneme window
+    phone_start = chapter_ref.word_phone_offsets[win_start]
+    phone_end = chapter_ref.word_phone_offsets[win_end]
+
+    R = chapter_ref.flat_phonemes[phone_start:phone_end]
+    R_phone_to_word = chapter_ref.flat_phone_to_word[phone_start:phone_end]
+
+    # Optionally prepend Basmala phonemes so the DP can consume fused Basmala+verse
+    BASMALA_SENTINEL = -1
+    prefix_phonemes = None
+    if basmala_prefix:
+        from .special_segments import SPECIAL_PHONEMES
+        prefix_phonemes = SPECIAL_PHONEMES["Basmala"]
+
+    if prefix_phonemes is not None:
+        prefix_len = len(prefix_phonemes)
+        R = list(prefix_phonemes) + list(R)
+        R_phone_to_word = [BASMALA_SENTINEL] * prefix_len + list(R_phone_to_word)
+
+    n = len(R)
+
+    if n == 0:
+        if PHONEME_ALIGNMENT_PROFILING:
+            timing['window_setup_time'] = time.perf_counter() - t0
+        if PHONEME_ALIGNMENT_DEBUG:
+            print(f"[PHONEME ALIGN] Segment {segment_idx}: Empty reference window")
+        return None, timing
+
+    if PHONEME_ALIGNMENT_PROFILING:
+        timing['window_setup_time'] = time.perf_counter() - t0
+
+    # === DP ===
+    if PHONEME_ALIGNMENT_PROFILING:
+        t0 = time.perf_counter()
+
+    # 4. Run word-boundary-constrained alignment (DP + selection in one pass)
+    best_j, j_start, best_cost, norm_dist = align_with_word_boundaries(
+        P, R, R_phone_to_word,
+        expected_word=pointer,
+        prior_weight=START_PRIOR_WEIGHT
+    )
+
+    if PHONEME_ALIGNMENT_PROFILING:
+        timing['dp_time'] = time.perf_counter() - t0
+
+    # === RESULT BUILD ===
+    if PHONEME_ALIGNMENT_PROFILING:
+        t0 = time.perf_counter()
+
+    if best_j is None:
+        if PHONEME_ALIGNMENT_PROFILING:
+            timing['result_build_time'] = time.perf_counter() - t0
+        print_debug_info(P, R, None, segment_idx, pointer, win_start, win_end, words)
+        return None, timing
+
+    # 5. Check acceptance threshold
+    threshold = max_edit_distance_override if max_edit_distance_override is not None else MAX_EDIT_DISTANCE
+    if norm_dist > threshold:
+        if PHONEME_ALIGNMENT_PROFILING:
+            timing['result_build_time'] = time.perf_counter() - t0
+        print_debug_info(P, R, None, segment_idx, pointer, win_start, win_end, words)
+        return None, timing
+
+    # 6. Confidence is 1 - normalized distance
+    confidence = 1.0 - norm_dist
+
+    # 7. Map phoneme indices to word indices
+    start_word_idx = R_phone_to_word[j_start]
+    end_word_idx = R_phone_to_word[best_j - 1]
+
+    # Handle prefix: if alignment starts in the prefix region, find the first real word
+    basmala_consumed = False
+    if prefix_phonemes is not None and start_word_idx == BASMALA_SENTINEL:
+        basmala_consumed = True
+        for k in range(j_start, best_j):
+            if R_phone_to_word[k] != BASMALA_SENTINEL:
+                start_word_idx = R_phone_to_word[k]
+                break
+        else:
+            # Entire match is just Basmala with no verse content — reject
+            if PHONEME_ALIGNMENT_PROFILING:
+                timing['result_build_time'] = time.perf_counter() - t0
+            return None, timing
+
+    result = AlignmentResult(
+        start_word_idx=start_word_idx,
+        end_word_idx=end_word_idx,
+        edit_cost=best_cost,
+        confidence=confidence,
+        j_start=j_start,
+        best_j=best_j,
+        start_word=words[start_word_idx],
+        end_word=words[end_word_idx],
+        basmala_consumed=basmala_consumed,
+    )
+
+    if PHONEME_ALIGNMENT_PROFILING:
+        timing['result_build_time'] = time.perf_counter() - t0
+
+    # Debug output
+    print_debug_info(P, R, result, segment_idx, pointer, win_start, win_end, words)
+
+    return result, timing

src/alignment/phoneme_matcher_cache.py

@@ -0,0 +1,59 @@
+"""
+Cache for ChapterReference objects.
+
+Loads pre-built chapter references from a pickle file (built by
+scripts/build_phoneme_cache.py) to avoid runtime phonemization.
+"""
+
+import pickle
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from .phoneme_matcher import ChapterReference
+
+# Global cache: surah number -> ChapterReference
+_chapter_cache: dict[int, "ChapterReference"] = {}
+
+
+def get_chapter_reference(surah: int) -> "ChapterReference":
+    """
+    Get chapter reference from cache.
+
+    Args:
+        surah: Surah number (1-114)
+
+    Returns:
+        ChapterReference with pre-built phoneme data
+    """
+    if surah not in _chapter_cache:
+        # Fallback: build at runtime if cache wasn't preloaded
+        from .phoneme_matcher import build_chapter_reference
+        print(f"[CACHE] WARNING: Building reference for Surah {surah} at runtime "
+              "(phoneme cache not loaded — run scripts/build_phoneme_cache.py)")
+        _chapter_cache[surah] = build_chapter_reference(surah)
+    return _chapter_cache[surah]
+
+
+def preload_all_chapters() -> None:
+    """Load all 114 chapter references from the pre-built cache file."""
+    from config import PHONEME_CACHE_PATH
+
+    if PHONEME_CACHE_PATH.exists():
+        print(f"[CACHE] Loading phoneme cache from {PHONEME_CACHE_PATH}...")
+        with open(PHONEME_CACHE_PATH, "rb") as f:
+            loaded: dict[int, "ChapterReference"] = pickle.load(f)
+        _chapter_cache.update(loaded)
+        print(f"[CACHE] Loaded {len(loaded)} chapters from cache")
+    else:
+        print(f"[CACHE] WARNING: {PHONEME_CACHE_PATH} not found, "
+              "falling back to runtime phonemization")
+        print("[CACHE] Run: python scripts/build_phoneme_cache.py")
+        for surah in range(1, 115):
+            get_chapter_reference(surah)
+        print(f"[CACHE] All 114 chapters built at runtime")
+
+
+def clear_chapter_cache() -> None:
+    """Clear cache (for memory management)."""
+    _chapter_cache.clear()
+    print("[CACHE] Cleared chapter cache")

src/alignment/special_segments.py

@@ -0,0 +1,295 @@
+"""
+Phoneme-based special segment detection for Basmala and Isti'adha.
+
+These are common recitation openers that need special handling:
+- Isti'adha: "أَعُوذُ بِٱللَّهِ مِنَ الشَّيْطَانِ الرَّجِيم" (I seek refuge in Allah)
+- Basmala: "بِسْمِ ٱللَّهِ ٱلرَّحْمَٰنِ ٱلرَّحِيم" (In the name of Allah)
+
+Detection uses phoneme edit distance for robustness against ASR errors.
+"""
+
+from __future__ import annotations
+
+from typing import List, Tuple, Optional
+
+# =============================================================================
+# Constants
+# =============================================================================
+
+from config import MAX_SPECIAL_EDIT_DISTANCE
+
+# Special phoneme sequences
+SPECIAL_PHONEMES = {
+    "Isti'adha": [
+        "ʔ", "a", "ʕ", "u:", "ð", "u", "b", "i", "ll", "a:", "h", "i",
+        "m", "i", "n", "a", "ʃʃ", "a", "j", "tˤ", "aˤ:", "n", "i",
+        "rˤrˤ", "aˤ", "ʒ", "i:", "m"
+    ],
+    "Basmala": [
+        "b", "i", "s", "m", "i", "ll", "a:", "h", "i", "rˤrˤ", "aˤ",
+        "ħ", "m", "a:", "n", "i", "rˤrˤ", "aˤ", "ħ", "i:", "m"
+    ],
+}
+
+# Combined = Isti'adha + Basmala (for detecting both in one segment)
+COMBINED_PHONEMES = SPECIAL_PHONEMES["Isti'adha"] + SPECIAL_PHONEMES["Basmala"]
+
+# Arabic text for display
+SPECIAL_TEXT = {
+    "Isti'adha": "أَعُوذُ بِٱللَّهِ مِنَ الشَّيْطَانِ الرَّجِيم",
+    "Basmala": "بِسْمِ ٱللَّهِ ٱلرَّحْمَٰنِ ٱلرَّحِيم",
+}
+
+
+# =============================================================================
+# Levenshtein Distance
+# =============================================================================
+
+def levenshtein_distance(seq1: List[str], seq2: List[str]) -> int:
+    """
+    Compute standard Levenshtein edit distance between two sequences.
+
+    Args:
+        seq1: First sequence (list of phonemes)
+        seq2: Second sequence (list of phonemes)
+
+    Returns:
+        Edit distance (number of insertions, deletions, substitutions)
+    """
+    m, n = len(seq1), len(seq2)
+
+    # Handle edge cases
+    if m == 0:
+        return n
+    if n == 0:
+        return m
+
+    # Use two-row optimization for memory efficiency
+    prev = list(range(n + 1))
+    curr = [0] * (n + 1)
+
+    for i in range(1, m + 1):
+        curr[0] = i
+        for j in range(1, n + 1):
+            if seq1[i - 1] == seq2[j - 1]:
+                curr[j] = prev[j - 1]  # No operation needed
+            else:
+                curr[j] = 1 + min(
+                    prev[j],      # Deletion
+                    curr[j - 1],  # Insertion
+                    prev[j - 1],  # Substitution
+                )
+        prev, curr = curr, prev
+
+    return prev[n]
+
+
+def phoneme_edit_distance(asr_phonemes: List[str], ref_phonemes: List[str]) -> float:
+    """
+    Compute normalized edit distance between two phoneme sequences.
+
+    Args:
+        asr_phonemes: ASR output phoneme sequence
+        ref_phonemes: Reference phoneme sequence
+
+    Returns:
+        Normalized edit distance (0.0 = identical, 1.0 = completely different)
+    """
+    if not asr_phonemes or not ref_phonemes:
+        return 1.0
+
+    edit_dist = levenshtein_distance(asr_phonemes, ref_phonemes)
+    max_len = max(len(asr_phonemes), len(ref_phonemes))
+
+    return edit_dist / max_len
+
+
+# =============================================================================
+# Special Segment Detection
+# =============================================================================
+
+def detect_special_segments(
+    phoneme_texts: List[List[str]],
+    vad_segments: List,
+    segment_audios: List,
+) -> Tuple[List, List, List[Tuple[str, float, str]], int]:
+    """
+    Detect special segments (Isti'adha/Basmala) using phoneme edit distance.
+
+    Detection order:
+    1. Try COMBINED (Isti'adha + Basmala) on segment 0 → split if match
+    2. Else try Isti'adha on segment 0 → if match, try Basmala on segment 1
+    3. Else try Basmala on segment 0
+    4. Else no specials
+
+    Args:
+        phoneme_texts: List of phoneme lists from ASR
+        vad_segments: List of VadSegment objects
+        segment_audios: List of audio arrays
+
+    Returns:
+        (updated_vad_segments, updated_audios, special_results, first_quran_idx)
+
+        special_results: List of tuples (matched_text, score, ref) for compatibility
+        first_quran_idx: Index where Quran segments start (after specials)
+    """
+    # Import here to avoid circular imports
+    from ..segment_types import VadSegment
+
+    if not phoneme_texts or not vad_segments or not segment_audios:
+        return vad_segments, segment_audios, [], 0
+
+    special_results: List[Tuple[str, float, str]] = []
+
+    # Segment 0 phonemes (already a list)
+    seg0_phonemes = phoneme_texts[0] if phoneme_texts[0] else []
+
+    # ==========================================================================
+    # 1. Try COMBINED (Isti'adha + Basmala in one segment)
+    # ==========================================================================
+    combined_dist = phoneme_edit_distance(seg0_phonemes, COMBINED_PHONEMES)
+
+    if combined_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[SPECIAL] Combined Isti'adha+Basmala detected (dist={combined_dist:.2f})")
+
+        # Split segment 0 by midpoint
+        seg = vad_segments[0]
+        audio = segment_audios[0]
+        mid_time = (seg.start_time + seg.end_time) / 2.0
+        mid_sample = max(1, len(audio) // 2)
+
+        # Create two new segments
+        new_vads = [
+            VadSegment(start_time=seg.start_time, end_time=mid_time, segment_idx=0),
+            VadSegment(start_time=mid_time, end_time=seg.end_time, segment_idx=1),
+        ]
+        new_audios = [
+            audio[:mid_sample],
+            audio[mid_sample:],
+        ]
+
+        # Add remaining segments with reindexed segment_idx
+        for i, vs in enumerate(vad_segments[1:], start=2):
+            new_vads.append(VadSegment(
+                start_time=vs.start_time,
+                end_time=vs.end_time,
+                segment_idx=i
+            ))
+        new_audios.extend(segment_audios[1:])
+
+        # Special results for both (confidence = 1 - distance)
+        confidence = 1.0 - combined_dist
+        special_results = [
+            (SPECIAL_TEXT["Isti'adha"], confidence, "Isti'adha"),
+            (SPECIAL_TEXT["Basmala"], confidence, "Basmala"),
+        ]
+
+        return new_vads, new_audios, special_results, 2
+
+    # ==========================================================================
+    # 2. Try Isti'adha on segment 0
+    # ==========================================================================
+    istiadha_dist = phoneme_edit_distance(seg0_phonemes, SPECIAL_PHONEMES["Isti'adha"])
+
+    if istiadha_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[SPECIAL] Isti'adha detected on segment 0 (dist={istiadha_dist:.2f})")
+        special_results.append(
+            (SPECIAL_TEXT["Isti'adha"], 1.0 - istiadha_dist, "Isti'adha")
+        )
+
+        # Try Basmala on segment 1
+        if len(phoneme_texts) >= 2 and phoneme_texts[1]:
+            seg1_phonemes = phoneme_texts[1]
+            basmala_dist = phoneme_edit_distance(seg1_phonemes, SPECIAL_PHONEMES["Basmala"])
+
+            if basmala_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+                print(f"[SPECIAL] Basmala detected on segment 1 (dist={basmala_dist:.2f})")
+                special_results.append(
+                    (SPECIAL_TEXT["Basmala"], 1.0 - basmala_dist, "Basmala")
+                )
+                return vad_segments, segment_audios, special_results, 2
+            else:
+                print(f"[SPECIAL] No Basmala on segment 1 (dist={basmala_dist:.2f})")
+
+        return vad_segments, segment_audios, special_results, 1
+
+    # ==========================================================================
+    # 3. Try Basmala on segment 0
+    # ==========================================================================
+    basmala_dist = phoneme_edit_distance(seg0_phonemes, SPECIAL_PHONEMES["Basmala"])
+
+    if basmala_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[SPECIAL] Basmala detected on segment 0 (dist={basmala_dist:.2f})")
+        special_results.append(
+            (SPECIAL_TEXT["Basmala"], 1.0 - basmala_dist, "Basmala")
+        )
+        return vad_segments, segment_audios, special_results, 1
+
+    # ==========================================================================
+    # 4. No specials detected
+    # ==========================================================================
+    print(f"[SPECIAL] No special segments detected "
+          f"(istiadha={istiadha_dist:.2f}, basmala={basmala_dist:.2f})")
+
+    return vad_segments, segment_audios, [], 0
+
+
+def detect_inter_chapter_specials(
+    phoneme_texts: List[List[str]],
+) -> Tuple[List[Tuple[str, float, str]], int]:
+    """
+    Detect special segments between chapters (phoneme-only, no audio splitting).
+
+    Same detection order as detect_special_segments:
+    1. Try COMBINED on segment 0
+    2. Else try Isti'adha on seg 0 -> if match, try Basmala on seg 1
+    3. Else try Basmala on seg 0
+    4. Else no specials
+
+    Returns:
+        (special_results, num_consumed)
+        special_results: List of (matched_text, score, ref) tuples
+        num_consumed: Number of segments consumed as specials
+    """
+    if not phoneme_texts or not phoneme_texts[0]:
+        return [], 0
+
+    seg0_phonemes = phoneme_texts[0]
+
+    # 1. Try COMBINED (Isti'adha + Basmala in one segment)
+    combined_dist = phoneme_edit_distance(seg0_phonemes, COMBINED_PHONEMES)
+    if combined_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[INTER-CHAPTER] Combined Isti'adha+Basmala detected (dist={combined_dist:.2f})")
+        combined_text = SPECIAL_TEXT["Isti'adha"] + " ۝ " + SPECIAL_TEXT["Basmala"]
+        return [(combined_text, 1.0 - combined_dist, "Isti'adha+Basmala")], 1
+
+    # 2. Try Isti'adha on segment 0
+    istiadha_dist = phoneme_edit_distance(seg0_phonemes, SPECIAL_PHONEMES["Isti'adha"])
+    if istiadha_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[INTER-CHAPTER] Isti'adha detected (dist={istiadha_dist:.2f})")
+        results = [(SPECIAL_TEXT["Isti'adha"], 1.0 - istiadha_dist, "Isti'adha")]
+        consumed = 1
+
+        # Try Basmala on segment 1
+        if len(phoneme_texts) >= 2 and phoneme_texts[1]:
+            seg1_phonemes = phoneme_texts[1]
+            basmala_dist = phoneme_edit_distance(seg1_phonemes, SPECIAL_PHONEMES["Basmala"])
+            if basmala_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+                print(f"[INTER-CHAPTER] Basmala detected on next segment (dist={basmala_dist:.2f})")
+                results.append((SPECIAL_TEXT["Basmala"], 1.0 - basmala_dist, "Basmala"))
+                consumed = 2
+            else:
+                print(f"[INTER-CHAPTER] No Basmala on next segment (dist={basmala_dist:.2f})")
+
+        return results, consumed
+
+    # 3. Try Basmala on segment 0
+    basmala_dist = phoneme_edit_distance(seg0_phonemes, SPECIAL_PHONEMES["Basmala"])
+    if basmala_dist <= MAX_SPECIAL_EDIT_DISTANCE:
+        print(f"[INTER-CHAPTER] Basmala detected (dist={basmala_dist:.2f})")
+        return [(SPECIAL_TEXT["Basmala"], 1.0 - basmala_dist, "Basmala")], 1
+
+    # 4. No specials
+    print(f"[INTER-CHAPTER] No special segments detected "
+          f"(istiadha={istiadha_dist:.2f}, basmala={basmala_dist:.2f})")
+    return [], 0

src/phonemizer_utils.py

@@ -0,0 +1,12 @@
+"""Phonemizer integration for reference phonemes."""
+
+_pm = None
+
+
+def get_phonemizer():
+    """Get or create Phonemizer instance."""
+    global _pm
+    if _pm is None:
+        from core.phonemizer import Phonemizer
+        _pm = Phonemizer()
+    return _pm

src/quran_index.py

@@ -0,0 +1,150 @@
+"""
+QuranIndex: Pre-indexed Quran words for reference lookup and display.
+
+Uses dual-script loading:
+- QPC Hafs (qpc_hafs.json) for computation (indices, word counts, lookups)
+- Digital Khatt (digital_khatt_v2_script.json) for display (renders correctly with DK font)
+
+Stop signs in Digital Khatt are combining marks attached to words, while QPC Hafs
+has spaces before stop signs. The DigitalKhatt font renders DK text correctly.
+"""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Optional
+
+from config import QURAN_SCRIPT_PATH_COMPUTE, QURAN_SCRIPT_PATH_DISPLAY
+
+
+# Verse marker prefix to filter out (end-of-ayah markers)
+VERSE_MARKER_PREFIX = '۝'
+
+
+@dataclass
+class WordInfo:
+    """Information about a single Quran word."""
+    global_idx: int       # Position in flat word array
+    surah: int
+    ayah: int
+    word: int
+    text: str             # QPC Hafs text (computation)
+    display_text: str     # Digital Khatt text (display)
+
+
+@dataclass
+class QuranIndex:
+    """
+    Pre-indexed Quran words for reference lookup and display.
+
+    Used to convert matched references (e.g. "2:255:1-2:255:5") back to
+    original Arabic text with verse markers for UI rendering.
+    """
+    words: list[WordInfo]                           # All words in order
+    word_lookup: dict[tuple[int, int, int], int]    # (surah, ayah, word) -> global_idx
+
+    @classmethod
+    def load(cls, compute_path: Optional[Path] = None, display_path: Optional[Path] = None) -> "QuranIndex":
+        """
+        Load and index the Quran from dual script sources.
+
+        Uses QPC Hafs as primary (determines word structure) and Digital Khatt
+        for display text. Falls back to QPC text if DK entry is missing.
+
+        Filters out verse markers (۝) - they're not real words.
+        """
+        if compute_path is None:
+            compute_path = QURAN_SCRIPT_PATH_COMPUTE
+        if display_path is None:
+            display_path = QURAN_SCRIPT_PATH_DISPLAY
+
+        with open(compute_path, "r", encoding="utf-8") as f:
+            compute_data = json.load(f)
+        with open(display_path, "r", encoding="utf-8") as f:
+            display_data = json.load(f)
+
+        words: list[WordInfo] = []
+        word_lookup: dict[tuple[int, int, int], int] = {}
+
+        # Sort by location key to ensure order (1:1:1, 1:1:2, ..., 114:6:3)
+        sorted_keys = sorted(compute_data.keys(), key=_parse_location_key)
+
+        for key in sorted_keys:
+            entry = compute_data[key]
+            text = entry["text"]
+
+            # Skip verse markers (QPC shouldn't have any, but safety check)
+            if text.startswith(VERSE_MARKER_PREFIX):
+                continue
+
+            surah = int(entry["surah"])
+            ayah = int(entry["ayah"])
+            word = int(entry["word"])
+
+            # Get display text from Digital Khatt, fallback to QPC text
+            dk_entry = display_data.get(key)
+            display_text = dk_entry["text"] if dk_entry else text
+
+            word_info = WordInfo(
+                global_idx=len(words),
+                surah=surah,
+                ayah=ayah,
+                word=word,
+                text=text,
+                display_text=display_text,
+            )
+            words.append(word_info)
+            word_lookup[(surah, ayah, word)] = word_info.global_idx
+
+        print(f"[QuranIndex] Loaded {len(words)} words")
+
+        return cls(
+            words=words,
+            word_lookup=word_lookup,
+        )
+
+    def ref_to_indices(self, ref: str) -> Optional[tuple[int, int]]:
+        """
+        Convert a ref like '2:255:1-2:255:5' or '2:255:5' to global start/end indices.
+        """
+        if not ref or ":" not in ref:
+            return None
+        try:
+            if "-" in ref:
+                start_ref, end_ref = ref.split("-")
+            else:
+                start_ref = end_ref = ref
+
+            def _lookup(r: str) -> Optional[int]:
+                parts = r.split(":")
+                if len(parts) < 3:
+                    return None
+                return self.word_lookup.get((int(parts[0]), int(parts[1]), int(parts[2])))
+
+            start_idx = _lookup(start_ref)
+            end_idx = _lookup(end_ref)
+            if start_idx is None or end_idx is None:
+                return None
+            return start_idx, end_idx
+        except Exception:
+            return None
+
+
+def _parse_location_key(key: str) -> tuple[int, int, int]:
+    """Parse location key like '2:255:3' into (surah, ayah, word) for sorting."""
+    parts = key.split(":")
+    return (int(parts[0]), int(parts[1]), int(parts[2]))
+
+
+# Global singleton - loaded on first access
+_quran_index_cache: Optional[QuranIndex] = None
+
+
+def get_quran_index() -> QuranIndex:
+    """Get or create the global QuranIndex singleton."""
+    global _quran_index_cache
+    if _quran_index_cache is None:
+        _quran_index_cache = QuranIndex.load()
+    return _quran_index_cache

src/segment_processor.py

@@ -0,0 +1,20 @@
+"""Compatibility façade for segmentation pipeline modules."""
+
+from .segment_types import VadSegment, SegmentInfo, ProfilingData
+from .segmenter.segmenter_model import load_segmenter, ensure_models_on_gpu, ensure_models_on_cpu
+from .segmenter.segmenter_aoti import test_vad_aoti_export, apply_aoti_compiled
+from .segmenter.vad import detect_speech_segments
+from .alignment.alignment_pipeline import run_phoneme_matching
+
+__all__ = [
+    "VadSegment",
+    "SegmentInfo",
+    "ProfilingData",
+    "load_segmenter",
+    "ensure_models_on_gpu",
+    "ensure_models_on_cpu",
+    "detect_speech_segments",
+    "run_phoneme_matching",
+    "test_vad_aoti_export",
+    "apply_aoti_compiled",
+]

src/segment_types.py

@@ -0,0 +1,153 @@
+"""Data types for the segmentation pipeline."""
+
+from dataclasses import dataclass
+from typing import Optional
+
+
+@dataclass
+class VadSegment:
+    """Raw VAD segment with timing info."""
+    start_time: float
+    end_time: float
+    segment_idx: int
+
+
+@dataclass
+class SegmentInfo:
+    """Processed segment with transcription and matching results."""
+    start_time: float
+    end_time: float
+    transcribed_text: str
+    matched_text: str
+    matched_ref: str  # e.g. "2:255:1-2:255:5"
+    match_score: float
+    error: Optional[str] = None
+    has_missing_words: bool = False
+    potentially_undersegmented: bool = False
+
+
+@dataclass
+class ProfilingData:
+    """Profiling metrics for the processing pipeline."""
+    # Preprocessing
+    resample_time: float = 0.0               # Audio resampling time
+    # VAD profiling
+    vad_model_load_time: float = 0.0
+    vad_model_move_time: float = 0.0
+    vad_inference_time: float = 0.0
+    vad_gpu_time: float = 0.0               # Actual GPU lease execution time
+    vad_wall_time: float = 0.0              # Wall-clock time (includes queue wait)
+    # Phoneme ASR profiling
+    asr_time: float = 0.0                    # Wav2vec wall-clock time (includes queue wait)
+    asr_gpu_time: float = 0.0               # Actual GPU lease execution time
+    asr_model_move_time: float = 0.0         # ASR model GPU move time
+    asr_sorting_time: float = 0.0            # Duration-sorting time
+    asr_batch_build_time: float = 0.0        # Dynamic batch construction time
+    asr_batch_profiling: list = None         # Per-batch timing details
+    # Global anchor profiling
+    anchor_time: float = 0.0                 # N-gram voting anchor detection
+    # Phoneme alignment profiling
+    phoneme_total_time: float = 0.0          # Overall phoneme matching time
+    phoneme_ref_build_time: float = 0.0      # Time to build chapter reference
+    phoneme_dp_total_time: float = 0.0       # Total DP time across all segments
+    phoneme_dp_min_time: float = 0.0         # Min DP time per segment
+    phoneme_dp_max_time: float = 0.0         # Max DP time per segment
+    phoneme_window_setup_time: float = 0.0   # Total window slicing time
+    phoneme_result_build_time: float = 0.0   # Total result construction time
+    phoneme_num_segments: int = 0            # Number of segments aligned
+    match_wall_time: float = 0.0             # Total matching wall-clock time
+    # Retry / reanchor counters
+    tier1_attempts: int = 0
+    tier1_passed: int = 0
+    tier1_segments: list = None
+    tier2_attempts: int = 0
+    tier2_passed: int = 0
+    tier2_segments: list = None
+    consec_reanchors: int = 0
+    segments_attempted: int = 0
+    segments_passed: int = 0
+    special_merges: int = 0
+    # Result building profiling
+    result_build_time: float = 0.0           # Total result building time
+    result_audio_encode_time: float = 0.0    # Audio-to-data-URL encoding
+    # Total pipeline time
+    total_time: float = 0.0                  # End-to-end pipeline time
+
+    @property
+    def phoneme_dp_avg_time(self) -> float:
+        """Average DP time per segment."""
+        if self.phoneme_num_segments == 0:
+            return 0.0
+        return self.phoneme_dp_total_time / self.phoneme_num_segments
+
+    @staticmethod
+    def _fmt(seconds):
+        """Format seconds as m:ss.fff when >= 60s, else as s.fffs."""
+        if seconds >= 60:
+            m, s = divmod(seconds, 60)
+            return f"{int(m)}:{s:06.3f}"
+        return f"{seconds:.3f}s"
+
+    def summary(self) -> str:
+        """Return a formatted profiling summary."""
+        _fmt = self._fmt
+        lines = [
+            "\n" + "=" * 60,
+            "PROFILING SUMMARY",
+            "=" * 60,
+            f"  Preprocessing:",
+            f"    Resample:        {self.resample_time:.3f}s",
+            f"  VAD:                                 wall {_fmt(self.vad_wall_time)}",
+            f"    GPU Time:        {self.vad_gpu_time:.3f}s   (queue {self.vad_wall_time - self.vad_gpu_time:.3f}s)",
+            f"    Model Load:      {self.vad_model_load_time:.3f}s",
+            f"    Model Move:      {self.vad_model_move_time:.3f}s",
+            f"    Inference:       {self.vad_inference_time:.3f}s",
+            f"  Phoneme ASR:                         wall {_fmt(self.asr_time)}",
+            f"    GPU Time:        {self.asr_gpu_time:.3f}s   (queue {self.asr_time - self.asr_gpu_time:.3f}s)",
+            f"    Model Move:      {self.asr_model_move_time:.3f}s",
+            f"    Sorting:         {self.asr_sorting_time:.3f}s",
+            f"    Batch Build:     {self.asr_batch_build_time:.3f}s",
+            f"    Batches:         {len(self.asr_batch_profiling) if self.asr_batch_profiling else 0}",
+        ]
+        if self.asr_batch_profiling:
+            for b in self.asr_batch_profiling:
+                lines.append(
+                    f"    Batch {b['batch_num']:>2}: {b['size']:>3} segs | "
+                    f"{b['time']:.3f}s | "
+                    f"{b['min_dur']:.2f}-{b['max_dur']:.2f}s "
+                    f"(A {b['avg_dur']:.2f}s, T {b['total_seconds']:.1f}s, W {b['pad_waste']:.0%})"
+                )
+        lines += [
+            f"  Global Anchor:",
+            f"    N-gram Voting:   {self.anchor_time:.3f}s",
+            f"  Phoneme Alignment:                   wall {_fmt(self.match_wall_time)}",
+            f"    Ref Build:       {self.phoneme_ref_build_time:.3f}s",
+            f"    Window Setup:    {self.phoneme_window_setup_time:.3f}s",
+            f"    DP Total:        {self.phoneme_dp_total_time:.3f}s",
+            f"    Segments:        {self.phoneme_num_segments}",
+            f"    DP Avg/segment:  {1000*self.phoneme_dp_avg_time:.3f}ms",
+            f"    DP Min:          {1000*self.phoneme_dp_min_time:.3f}ms",
+            f"    DP Max:          {1000*self.phoneme_dp_max_time:.3f}ms",
+        ]
+        pct = 100 * self.segments_passed / self.segments_attempted if self.segments_attempted else 0
+        t1_segs = self.tier1_segments or []
+        t2_segs = self.tier2_segments or []
+        lines += [
+            f"  Alignment Stats:",
+            f"    Attempted:       {self.segments_attempted}",
+            f"    Passed:          {self.segments_passed}  ({pct:.1f}%)",
+            f"    Tier 1 Retries:  {self.tier1_passed}/{self.tier1_attempts} passed   segments: {t1_segs}",
+            f"    Tier 2 Retries:  {self.tier2_passed}/{self.tier2_attempts} passed   segments: {t2_segs}",
+            f"    Reanchors (consec failures): {self.consec_reanchors}",
+            f"    Special Merges:  {self.special_merges}",
+            "-" * 60,
+        ]
+        profiled_sum = (self.resample_time + self.vad_wall_time + self.asr_time
+                        + self.anchor_time + self.match_wall_time + self.result_build_time)
+        unaccounted = self.total_time - profiled_sum
+        lines += [
+            f"  PROFILED SUM:      {_fmt(profiled_sum)}",
+            f"  TOTAL (wall):      {_fmt(self.total_time)}   (unaccounted: {_fmt(unaccounted)})",
+            "=" * 60,
+        ]
+        return "\n".join(lines)

src/segmenter/segmenter_aoti.py

@@ -0,0 +1,379 @@
+"""AoTInductor compilation utilities for the VAD segmenter."""
+
+import torch
+
+from config import (
+    AOTI_ENABLED, AOTI_MIN_AUDIO_MINUTES, AOTI_MAX_AUDIO_MINUTES,
+    AOTI_HUB_REPO, AOTI_HUB_ENABLED,
+)
+from .segmenter_model import _segmenter_cache
+
+
+# =============================================================================
+# AoT Compilation Test
+# =============================================================================
+
+_aoti_cache = {
+    "exported": None,
+    "compiled": None,
+    "tested": False,
+}
+
+
+def is_aoti_applied() -> bool:
+    """Return True if a compiled AoTI model has been applied."""
+    return bool(_aoti_cache.get("applied"))
+
+
+def _get_aoti_hub_filename():
+    """Generate Hub filename encoding min/max audio duration."""
+    return f"vad_aoti_{AOTI_MIN_AUDIO_MINUTES}min_{AOTI_MAX_AUDIO_MINUTES}min.pt2"
+
+
+def _try_load_aoti_from_hub(model):
+    """
+    Try to load a pre-compiled AoTI model from Hub.
+    Returns True if successful, False otherwise.
+    """
+    import os
+    import time
+
+    if not AOTI_HUB_ENABLED:
+        print("[AoTI] Hub persistence disabled")
+        return False
+
+    token = os.environ.get("HF_TOKEN")
+    if not token:
+        print("[AoTI] HF_TOKEN not set, cannot access Hub")
+        return False
+
+    filename = _get_aoti_hub_filename()
+    print(f"[AoTI] Checking Hub for pre-compiled model: {AOTI_HUB_REPO}/{filename}")
+
+    try:
+        from huggingface_hub import hf_hub_download, HfApi
+
+        # Check if file exists in repo
+        api = HfApi(token=token)
+        try:
+            files = api.list_repo_files(AOTI_HUB_REPO, token=token)
+            if filename not in files:
+                print(f"[AoTI] Compiled model not found on Hub (available: {files})")
+                return False
+        except Exception as e:
+            print(f"[AoTI] Could not list Hub repo: {e}")
+            return False
+
+        # Download the compiled graph
+        t0 = time.time()
+        compiled_graph_file = hf_hub_download(
+            AOTI_HUB_REPO, filename, token=token
+        )
+        download_time = time.time() - t0
+        print(f"[AoTI] Downloaded from Hub in {download_time:.1f}s: {compiled_graph_file}")
+
+        # Load using ZeroGPU AOTI utilities
+        from spaces.zero.torch.aoti import ZeroGPUCompiledModel, ZeroGPUWeights, drain_module_parameters
+
+        state_dict = model.state_dict()
+        zerogpu_weights = ZeroGPUWeights({name: weight for name, weight in state_dict.items()})
+        compiled = ZeroGPUCompiledModel(compiled_graph_file, zerogpu_weights)
+
+        # Replace forward method
+        setattr(model, "forward", compiled)
+        drain_module_parameters(model)
+
+        _aoti_cache["compiled"] = compiled
+        _aoti_cache["applied"] = True
+        print(f"[AoTI] Loaded and applied compiled model from Hub")
+        return True
+
+    except Exception as e:
+        print(f"[AoTI] Failed to load from Hub: {type(e).__name__}: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+
+def _push_aoti_to_hub(compiled):
+    """
+    Push compiled AoTI model to Hub for future reuse.
+    """
+    import os
+    import time
+    import tempfile
+
+    if not AOTI_HUB_ENABLED:
+        print("[AoTI] Hub persistence disabled, skipping upload")
+        return False
+
+    token = os.environ.get("HF_TOKEN")
+    if not token:
+        print("[AoTI] HF_TOKEN not set, cannot upload to Hub")
+        return False
+
+    filename = _get_aoti_hub_filename()
+    print(f"[AoTI] Uploading compiled model to Hub: {AOTI_HUB_REPO}/{filename}")
+
+    try:
+        from huggingface_hub import HfApi, create_repo
+
+        api = HfApi(token=token)
+
+        # Create repo if it doesn't exist
+        try:
+            create_repo(AOTI_HUB_REPO, exist_ok=True, token=token)
+        except Exception as e:
+            print(f"[AoTI] Repo creation note: {e}")
+
+        # Get the archive file from the compiled object
+        archive = compiled.archive_file
+        if archive is None:
+            print("[AoTI] Compiled object has no archive_file, cannot upload")
+            return False
+
+        t0 = time.time()
+
+        # Write archive to temp file and upload
+        with tempfile.TemporaryDirectory() as tmpdir:
+            output_path = os.path.join(tmpdir, filename)
+
+            # archive is a BytesIO object
+            with open(output_path, "wb") as f:
+                f.write(archive.getvalue())
+
+            info = api.upload_file(
+                repo_id=AOTI_HUB_REPO,
+                path_or_fileobj=output_path,
+                path_in_repo=filename,
+                commit_message=f"Add compiled VAD model ({AOTI_MIN_AUDIO_MINUTES}-{AOTI_MAX_AUDIO_MINUTES} min)",
+                token=token,
+            )
+
+        upload_time = time.time() - t0
+        print(f"[AoTI] Uploaded to Hub in {upload_time:.1f}s: {info}")
+        return True
+
+    except Exception as e:
+        print(f"[AoTI] Failed to upload to Hub: {type(e).__name__}: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+
+def test_vad_aoti_export():
+    """
+    Test torch.export AoT compilation for VAD model using spaces.aoti_capture.
+    Must be called AFTER model is on GPU (inside GPU-decorated function).
+
+    Checks Hub for pre-compiled model first. If found, loads it directly.
+    Otherwise, compiles fresh and uploads to Hub for future reuse.
+
+    Uses aoti_capture to capture the EXACT call signature from a real inference
+    call to segment_recitations, ensuring the export matches what the model
+    actually receives during inference.
+
+    Returns dict with test results and timing.
+    """
+    import time
+
+    results = {
+        "export_success": False,
+        "export_time": 0.0,
+        "compile_success": False,
+        "compile_time": 0.0,
+        "hub_loaded": False,
+        "hub_uploaded": False,
+        "error": None,
+    }
+
+    if not AOTI_ENABLED:
+        results["error"] = "AoTI disabled in config"
+        print("[AoTI] Disabled via AOTI_ENABLED=False")
+        return results
+
+    if _aoti_cache["tested"]:
+        print("[AoTI] Already tested this session, skipping")
+        return {"skipped": True, **results}
+
+    _aoti_cache["tested"] = True
+
+    # Check model is loaded and on GPU
+    if not _segmenter_cache["loaded"] or _segmenter_cache["model"] is None:
+        results["error"] = "Model not loaded"
+        print(f"[AoTI] {results['error']}")
+        return results
+
+    model = _segmenter_cache["model"]
+    processor = _segmenter_cache["processor"]
+    device = next(model.parameters()).device
+    dtype = next(model.parameters()).dtype
+
+    if device.type != "cuda":
+        results["error"] = f"Model not on GPU (device={device})"
+        print(f"[AoTI] {results['error']}")
+        return results
+
+    print(f"[AoTI] Testing torch.export on VAD model (device={device}, dtype={dtype})")
+
+    # Import spaces for aoti_capture
+    try:
+        import spaces
+    except ImportError:
+        results["error"] = "spaces module not available"
+        print(f"[AoTI] {results['error']}")
+        return results
+
+    # Try to load pre-compiled model from Hub first
+    if _try_load_aoti_from_hub(model):
+        results["hub_loaded"] = True
+        results["compile_success"] = True
+        print("[AoTI] Using pre-compiled model from Hub")
+        return results
+
+    # No cached model found - compile fresh
+    print("[AoTI] No cached model on Hub, compiling fresh...")
+
+    # Convert config minutes to samples (16kHz audio)
+    SAMPLES_PER_MINUTE = 16000 * 60
+    min_samples = int(AOTI_MIN_AUDIO_MINUTES * SAMPLES_PER_MINUTE)
+    max_samples = int(AOTI_MAX_AUDIO_MINUTES * SAMPLES_PER_MINUTE)
+
+    # Create test audio for capture - use min duration to save memory
+    # MUST be on CPU - segment_recitations moves to GPU internally
+    test_audio = torch.randn(min_samples, device="cpu")
+    print(f"[AoTI] Test audio: {min_samples} samples ({AOTI_MIN_AUDIO_MINUTES} min)")
+
+    # Capture the exact args/kwargs used by segment_recitations
+    try:
+        from recitations_segmenter import segment_recitations
+
+        print("[AoTI] Capturing call signature via aoti_capture...")
+        with spaces.aoti_capture(model) as call:
+            segment_recitations(
+                [test_audio], model, processor,
+                device=device, dtype=dtype, batch_size=1,
+            )
+
+        print(f"[AoTI] Captured args: {len(call.args)} positional, {list(call.kwargs.keys())} kwargs")
+
+    except Exception as e:
+        results["error"] = f"aoti_capture failed: {type(e).__name__}: {e}"
+        print(f"[AoTI] {results['error']}")
+        import traceback
+        traceback.print_exc()
+        return results
+
+    # Build dynamic shapes from captured tensors
+    # The sequence dimension (T) varies with audio length
+    try:
+        from torch.export import export, Dim
+
+        # Derive frame rate from captured tensor (model's actual output rate)
+        # Find the first 2D+ tensor to get the captured frame count
+        captured_frames = None
+        for val in list(call.kwargs.values()) + list(call.args):
+            if isinstance(val, torch.Tensor) and val.dim() >= 2:
+                captured_frames = val.shape[1]
+                break
+
+        if captured_frames is None:
+            raise ValueError("No 2D+ tensor found in captured args/kwargs")
+
+        # Calculate frames per minute from captured data
+        frames_per_minute = captured_frames / AOTI_MIN_AUDIO_MINUTES
+        min_frames = captured_frames  # Already at min duration
+        max_frames = int(AOTI_MAX_AUDIO_MINUTES * frames_per_minute)
+        dynamic_T = Dim("T", min=min_frames, max=max_frames)
+        print(f"[AoTI] Captured {captured_frames} frames for {AOTI_MIN_AUDIO_MINUTES} min = {frames_per_minute:.1f} frames/min")
+        print(f"[AoTI] Dynamic shape range: {min_frames}-{max_frames} frames")
+
+        # Build dynamic_shapes dict matching the captured signature
+        dynamic_shapes_args = []
+        for arg in call.args:
+            if isinstance(arg, torch.Tensor) and arg.dim() >= 2:
+                # Assume sequence dim is dim 1 for 2D+ tensors
+                dynamic_shapes_args.append({1: dynamic_T})
+            else:
+                dynamic_shapes_args.append(None)
+
+        dynamic_shapes_kwargs = {}
+        for key, val in call.kwargs.items():
+            if isinstance(val, torch.Tensor) and val.dim() >= 2:
+                dynamic_shapes_kwargs[key] = {1: dynamic_T}
+            else:
+                dynamic_shapes_kwargs[key] = None
+
+        print(f"[AoTI] Dynamic shapes - args: {dynamic_shapes_args}, kwargs: {list(dynamic_shapes_kwargs.keys())}")
+
+        t0 = time.time()
+        # Export using captured signature - guarantees match with inference
+        exported = export(
+            model,
+            args=call.args,
+            kwargs=call.kwargs,
+            dynamic_shapes=(dynamic_shapes_args, dynamic_shapes_kwargs) if dynamic_shapes_args else dynamic_shapes_kwargs,
+            strict=False,
+        )
+        results["export_time"] = time.time() - t0
+        results["export_success"] = True
+        _aoti_cache["exported"] = exported
+        print(f"[AoTI] torch.export SUCCESS in {results['export_time']:.1f}s")
+
+    except Exception as e:
+        results["error"] = f"torch.export failed: {type(e).__name__}: {e}"
+        print(f"[AoTI] {results['error']}")
+        import traceback
+        traceback.print_exc()
+        return results
+
+    # Attempt spaces.aoti_compile
+    try:
+        t0 = time.time()
+        compiled = spaces.aoti_compile(exported)
+        results["compile_time"] = time.time() - t0
+        results["compile_success"] = True
+        _aoti_cache["compiled"] = compiled
+        print(f"[AoTI] spaces.aoti_compile SUCCESS in {results['compile_time']:.1f}s")
+
+        # Return compiled object - apply happens OUTSIDE GPU lease (in main process)
+        results["compiled"] = compiled
+        print(f"[AoTI] Compiled object ready for apply")
+
+        # Upload to Hub for future reuse
+        if _push_aoti_to_hub(compiled):
+            results["hub_uploaded"] = True
+
+    except Exception as e:
+        results["error"] = f"aoti_compile failed: {type(e).__name__}: {e}"
+        print(f"[AoTI] {results['error']}")
+        import traceback
+        traceback.print_exc()
+
+    return results
+
+
+def apply_aoti_compiled(compiled):
+    """
+    Apply AoTI compiled model to VAD segmenter.
+    Must be called OUTSIDE GPU lease, in main process.
+    """
+    if compiled is None:
+        print("[AoTI] No compiled object to apply")
+        return False
+
+    model = _segmenter_cache.get("model")
+    if model is None:
+        print("[AoTI] Model not loaded, cannot apply")
+        return False
+
+    try:
+        import spaces
+        spaces.aoti_apply(compiled, model)
+        _aoti_cache["compiled"] = compiled
+        _aoti_cache["applied"] = True
+        print(f"[AoTI] Compiled model applied to VAD (model_id={id(model)})")
+        return True
+    except Exception as e:
+        print(f"[AoTI] Apply failed: {e}")
+        return False

src/segmenter/segmenter_model.py

@@ -0,0 +1,158 @@
+"""Model lifecycle and device management for the VAD segmenter."""
+
+import torch
+
+from config import SEGMENTER_MODEL, DTYPE, IS_HF_SPACE, TORCH_COMPILE
+from ..zero_gpu import ZERO_GPU_AVAILABLE, is_quota_exhausted, is_user_forced_cpu
+
+
+# =============================================================================
+# Model caches
+# =============================================================================
+
+_segmenter_cache = {"model": None, "processor": None, "loaded": False, "load_time": 0.0, "device": None}
+_env_logged = False
+
+
+def _log_env_once():
+    """Log library and GPU versions once for debugging HF Space mismatches."""
+    global _env_logged
+    if _env_logged:
+        return
+    _env_logged = True
+    try:
+        import importlib.metadata as _im
+
+        def _ver(pkg: str) -> str:
+            try:
+                return _im.version(pkg)
+            except Exception:
+                return "unknown"
+
+        print(f"[ENV] torch={torch.__version__} cuda={torch.version.cuda} cudnn={torch.backends.cudnn.version() if torch.backends.cudnn.is_available() else 'none'}")
+        print(f"[ENV] transformers={_ver('transformers')} recitations_segmenter={_ver('recitations_segmenter')}")
+        if torch.cuda.is_available():
+            print(f"[ENV] GPU={torch.cuda.get_device_name(0)}")
+    except Exception as e:
+        print(f"[ENV] Failed to log env: {e}")
+
+
+_TORCH_DTYPE = torch.float16 if DTYPE == "float16" else torch.float32
+
+
+def _get_device_and_dtype():
+    """Get the best available device and dtype."""
+    if IS_HF_SPACE or ZERO_GPU_AVAILABLE:
+        return torch.device("cpu"), _TORCH_DTYPE
+    if torch.cuda.is_available():
+        return torch.device("cuda"), _TORCH_DTYPE
+    return torch.device("cpu"), _TORCH_DTYPE
+
+
+def ensure_models_on_gpu(asr_model_name=None):
+    """
+    Move models to GPU. Call this INSIDE a GPU-decorated function
+    after ZeroGPU lease is acquired.
+
+    Args:
+        asr_model_name: If provided, move only this ASR model to GPU.
+            If None, skip ASR model movement (e.g. during VAD-only lease).
+
+    Skips if quota exhausted or CUDA unavailable.
+    Idempotent: checks current device before moving.
+
+    Returns:
+        float: Time in seconds spent moving models to GPU.
+    """
+    import time
+    from ..alignment.phoneme_asr import move_phoneme_asr_to_gpu
+
+    if is_user_forced_cpu() or is_quota_exhausted() or not torch.cuda.is_available():
+        return 0.0
+
+    device = torch.device("cuda")
+    dtype = _TORCH_DTYPE
+    move_start = time.time()
+
+    # Move segmenter to GPU
+    if _segmenter_cache["loaded"] and _segmenter_cache["model"] is not None:
+        model = _segmenter_cache["model"]
+        if next(model.parameters()).device.type != "cuda":
+            print("[GPU] Moving segmenter to CUDA...")
+            model.to(device, dtype=dtype)
+            _segmenter_cache["model"] = model
+            _segmenter_cache["device"] = "cuda"
+            print("[GPU] Segmenter on CUDA")
+
+    # Move phoneme ASR to GPU (only the requested model)
+    if asr_model_name is not None:
+        move_phoneme_asr_to_gpu(asr_model_name)
+
+    return time.time() - move_start
+
+
+def ensure_models_on_cpu():
+    """
+    Move all models back to CPU. Called when GPU lease fails or quota
+    is exhausted so that CPU fallback inference can proceed.
+
+    Idempotent: checks current device before moving.
+    """
+    from ..alignment.phoneme_asr import move_phoneme_asr_to_cpu
+
+    device = torch.device("cpu")
+    dtype = _TORCH_DTYPE
+
+    # Move segmenter to CPU
+    if _segmenter_cache["loaded"] and _segmenter_cache["model"] is not None:
+        model = _segmenter_cache["model"]
+        if next(model.parameters()).device.type != "cpu":
+            print("[CPU] Moving segmenter to CPU...")
+            model.to(device, dtype=dtype)
+            _segmenter_cache["model"] = model
+            _segmenter_cache["device"] = "cpu"
+            print("[CPU] Segmenter on CPU")
+
+    # Move phoneme ASR to CPU
+    move_phoneme_asr_to_cpu()
+
+
+def load_segmenter():
+    """Load the VAD segmenter model on CPU. Returns (model, processor, load_time).
+
+    Models are loaded once and cached. Use ensure_models_on_gpu()
+    inside GPU-decorated functions to move to CUDA.
+    """
+    if _segmenter_cache["loaded"]:
+        return _segmenter_cache["model"], _segmenter_cache["processor"], 0.0
+
+    import time
+    start_time = time.time()
+
+    try:
+        from transformers import AutoModelForAudioFrameClassification, AutoFeatureExtractor
+
+        print(f"Loading segmenter: {SEGMENTER_MODEL}")
+        device, dtype = _get_device_and_dtype()
+
+        model = AutoModelForAudioFrameClassification.from_pretrained(SEGMENTER_MODEL)
+        model.to(device, dtype=dtype)
+        model.eval()
+        if TORCH_COMPILE and not (IS_HF_SPACE or ZERO_GPU_AVAILABLE):
+            model = torch.compile(model, mode="reduce-overhead")
+
+        processor = AutoFeatureExtractor.from_pretrained(SEGMENTER_MODEL)
+
+        load_time = time.time() - start_time
+        _segmenter_cache["model"] = model
+        _segmenter_cache["processor"] = processor
+        _segmenter_cache["loaded"] = True
+        _segmenter_cache["load_time"] = load_time
+        _segmenter_cache["device"] = device.type
+
+        print(f"Segmenter loaded on {device} in {load_time:.2f}s")
+        return model, processor, load_time
+
+    except Exception as e:
+        print(f"Failed to load segmenter: {e}")
+        return None, None, 0.0

src/segmenter/vad.py

@@ -0,0 +1,97 @@
+"""VAD inference utilities."""
+
+from typing import List, Tuple
+
+import numpy as np
+import torch
+
+from .segmenter_aoti import is_aoti_applied
+from .segmenter_model import load_segmenter, _log_env_once
+
+
+def detect_speech_segments(
+    audio: np.ndarray,
+    sample_rate: int,
+    min_silence_ms: int,
+    min_speech_ms: int,
+    pad_ms: int
+) -> tuple[List[Tuple[float, float]], dict]:
+    """
+    Detect speech segments in audio using VAD.
+
+    Args:
+        audio: Audio waveform (mono, float32)
+        sample_rate: Sample rate of audio
+        min_silence_ms: Minimum silence duration to split segments
+        min_speech_ms: Minimum speech duration for a valid segment
+        pad_ms: Padding around speech segments
+
+    Returns:
+        Tuple of (intervals, profiling_dict, raw_speech_intervals, raw_is_complete) where:
+        - intervals: List of (start_time, end_time) tuples in seconds
+        - profiling_dict: {"model_load_time": float, "inference_time": float}
+        - raw_speech_intervals: Raw VAD intervals before cleaning (for resegmentation)
+        - raw_is_complete: Raw VAD completeness flags (for resegmentation)
+    """
+    import time
+
+    model, processor, model_load_time = load_segmenter()
+    if model is None:
+        # Fallback: treat whole audio as one segment
+        return [(0, len(audio) / sample_rate)], {"model_load_time": 0.0, "inference_time": 0.0}, None, None
+
+    inference_start = time.time()
+    _log_env_once()
+
+    try:
+        from recitations_segmenter import segment_recitations, clean_speech_intervals
+
+        audio_tensor = torch.from_numpy(audio).float()
+
+        device = next(model.parameters()).device
+        dtype = next(model.parameters()).dtype
+
+        # Log AoTI status
+        if is_aoti_applied():
+            print("[VAD] Using AOTInductor-compiled model")
+
+        # Run segmentation
+        outputs = segment_recitations(
+            [audio_tensor], model, processor,
+            device=device, dtype=dtype, batch_size=1,
+        )
+
+        if not outputs:
+            inference_time = time.time() - inference_start
+            return [(0, len(audio) / sample_rate)], {"model_load_time": model_load_time, "inference_time": inference_time}, None, None
+
+        # Clean speech intervals with user parameters
+        clean_out = clean_speech_intervals(
+            outputs[0].speech_intervals,
+            outputs[0].is_complete,
+            min_silence_duration_ms=min_silence_ms,
+            min_speech_duration_ms=min_speech_ms,
+            pad_duration_ms=pad_ms,
+            return_seconds=True,
+        )
+
+        inference_time = time.time() - inference_start
+        intervals = clean_out.clean_speech_intervals.tolist()
+
+        raw_count = len(outputs[0].speech_intervals)
+        final_count = len(intervals)
+        removed = raw_count - final_count
+        print(f"[VAD] Raw model intervals: {raw_count}, after cleaning: {final_count} "
+              f"({removed} removed by silence merge + min_speech={min_speech_ms}ms filter)")
+
+        raw_speech_intervals = outputs[0].speech_intervals
+        raw_is_complete = outputs[0].is_complete
+
+        return [(start, end) for start, end in intervals], {"model_load_time": model_load_time, "inference_time": inference_time}, raw_speech_intervals, raw_is_complete
+
+    except Exception as e:
+        print(f"VAD error: {e}")
+        import traceback
+        traceback.print_exc()
+        # Let gpu_with_fallback handle retries on CPU
+        raise

src/zero_gpu.py

@@ -0,0 +1,146 @@
+"""
+Utilities for integrating Hugging Face Spaces ZeroGPU without breaking
+local or non-ZeroGPU environments.
+"""
+
+import re
+from typing import Callable, TypeVar
+from functools import wraps
+
+T = TypeVar("T", bound=Callable)
+
+# Default values in case the spaces package is unavailable (e.g., local runs).
+ZERO_GPU_AVAILABLE = False
+
+# Track whether we've fallen back to CPU due to quota exhaustion
+_gpu_quota_exhausted = False
+_quota_reset_time = None  # e.g. "13:53:59"
+_user_forced_cpu = False
+
+try:
+    import spaces  # type: ignore
+
+    gpu_decorator = spaces.GPU  # pragma: no cover
+    ZERO_GPU_AVAILABLE = True
+except Exception:
+    def gpu_decorator(*decorator_args, **decorator_kwargs):
+        """
+        No-op replacement for spaces.GPU so code can run without the package
+        or outside of a ZeroGPU Space.
+        """
+
+        def wrapper(func: T) -> T:
+            return func
+
+        # Support both bare @gpu_decorator and @gpu_decorator(...)
+        if decorator_args and callable(decorator_args[0]) and not decorator_kwargs:
+            return decorator_args[0]
+        return wrapper
+
+
+def is_quota_exhausted() -> bool:
+    """Check if GPU quota has been exhausted this session."""
+    return _gpu_quota_exhausted
+
+
+def is_user_forced_cpu() -> bool:
+    """Check if the user manually selected CPU mode."""
+    return _user_forced_cpu
+
+
+def get_quota_reset_time() -> str | None:
+    """Return the quota reset time string (e.g. '13:53:59'), or None."""
+    return _quota_reset_time
+
+
+def reset_quota_flag():
+    """Reset the quota exhausted flag (e.g., after quota resets)."""
+    global _gpu_quota_exhausted, _quota_reset_time, _user_forced_cpu
+    _gpu_quota_exhausted = False
+    _quota_reset_time = None
+    _user_forced_cpu = False
+
+
+def force_cpu_mode():
+    """Force all GPU-decorated functions to skip GPU and run on CPU."""
+    global _user_forced_cpu
+    _user_forced_cpu = True
+    _move_models_to_cpu()
+
+
+def _move_models_to_cpu():
+    """Move all models back to CPU for fallback inference."""
+    try:
+        from .segmenter.segmenter_model import ensure_models_on_cpu
+        ensure_models_on_cpu()
+    except Exception as e:
+        print(f"[CPU] Failed to move models to CPU: {e}")
+
+
+def gpu_with_fallback(duration=60):
+    """
+    Decorator that wraps a GPU function with automatic CPU fallback.
+
+    If ZeroGPU quota is exceeded, the function runs on CPU instead.
+    The decorated function should call ensure_models_on_gpu() internally,
+    which checks is_quota_exhausted() to decide whether to move to CUDA.
+
+    Usage:
+        @gpu_with_fallback(duration=60)
+        def my_gpu_func(data):
+            ensure_models_on_gpu()  # Moves to CUDA if quota not exhausted
+            # ... inference using model's current device ...
+    """
+    def decorator(func: T) -> T:
+        # Create the GPU-wrapped version
+        if ZERO_GPU_AVAILABLE:
+            gpu_func = gpu_decorator(duration=duration)(func)
+        else:
+            gpu_func = func
+
+        @wraps(func)
+        def wrapper(*args, **kwargs):
+            global _gpu_quota_exhausted, _quota_reset_time
+
+            # If user explicitly chose CPU mode, skip GPU entirely
+            if _user_forced_cpu:
+                print("[CPU] User selected CPU mode")
+                return func(*args, **kwargs)
+
+            # If quota already exhausted, go straight to CPU
+            if _gpu_quota_exhausted:
+                print("[GPU] Quota exhausted, using CPU fallback")
+                _move_models_to_cpu()
+                return func(*args, **kwargs)
+
+            # Try GPU first
+            try:
+                return gpu_func(*args, **kwargs)
+            except Exception as e:
+                # ZeroGPU raises gradio.Error with title="ZeroGPU quota exceeded"
+                is_quota_error = getattr(e, 'title', '') == "ZeroGPU quota exceeded"
+                if not is_quota_error:
+                    is_quota_error = 'quota' in str(e).lower()
+
+                if is_quota_error:
+                    print(f"[GPU] Quota exceeded, falling back to CPU: {e}")
+                    _gpu_quota_exhausted = True
+                    # Parse reset time from message like "Try again in 13:53:59"
+                    match = re.search(r'Try again in (\d+:\d{2}:\d{2})', str(e))
+                    if match:
+                        _quota_reset_time = match.group(1)
+                    _move_models_to_cpu()
+                    return func(*args, **kwargs)
+                else:
+                    err_lower = str(e).lower()
+                    is_timeout = (
+                        'timeout' in err_lower
+                        or 'duration' in err_lower
+                        or 'time limit' in err_lower
+                    )
+                    if is_timeout:
+                        print(f"[GPU] Timeout error in {func.__name__}: {e}")
+                    raise
+
+        return wrapper
+    return decorator

utils/usage_logger.py

@@ -0,0 +1,593 @@
+"""
+Usage logger that pushes alignment runs to a HF Dataset repo.
+
+Uses a ParquetScheduler (subclass of CommitScheduler) to buffer rows in memory
+and periodically write+upload parquet files with embedded audio to the Hub.
+Error logs use a separate CommitScheduler with JSONL files.
+Falls back to local-only logging if schedulers can't initialize.
+
+Scheduler creation is deferred to first use so that background threads don't
+interfere with ZeroGPU's startup function scan.
+"""
+
+import hashlib
+import io
+import json
+import threading
+from datetime import datetime
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple, Union
+from uuid import uuid4
+
+import numpy as np
+
+# =========================================================================
+# Directory setup
+# =========================================================================
+
+LOG_DIR = Path("usage_logs")
+LOG_DIR.mkdir(parents=True, exist_ok=True)
+
+ERROR_DIR = LOG_DIR / "errors"
+ERROR_DIR.mkdir(parents=True, exist_ok=True)
+
+ERROR_LOG_PATH = ERROR_DIR / f"error_log-{uuid4()}.jsonl"
+
+# =========================================================================
+# ParquetScheduler class definition (no instances created at import time)
+# =========================================================================
+
+_HAS_DEPS = False
+try:
+    import pyarrow as pa
+    import pyarrow.parquet as pq
+    from huggingface_hub import CommitScheduler
+    from config import USAGE_LOG_DATASET_REPO, USAGE_LOG_PUSH_INTERVAL_MINUTES
+
+    _HAS_DEPS = True
+except Exception:
+    pass
+
+# HF features schema (column order matters — audio first for HF viewer widget)
+_ALIGNER_SCHEMA: Dict[str, Dict[str, str]] = {
+    # Identity
+    "audio": {"_type": "Audio"},
+    "audio_id": {"_type": "Value", "dtype": "string"},
+    "timestamp": {"_type": "Value", "dtype": "string"},
+    "user_id": {"_type": "Value", "dtype": "string"},
+    # Input metadata
+    "audio_duration_s": {"_type": "Value", "dtype": "float64"},
+    "num_segments": {"_type": "Value", "dtype": "int32"},
+    "surah": {"_type": "Value", "dtype": "int32"},
+    # Segmentation settings
+    "min_silence_ms": {"_type": "Value", "dtype": "int32"},
+    "min_speech_ms": {"_type": "Value", "dtype": "int32"},
+    "pad_ms": {"_type": "Value", "dtype": "int32"},
+    "asr_model": {"_type": "Value", "dtype": "string"},
+    "device": {"_type": "Value", "dtype": "string"},
+    # Profiling
+    "total_time": {"_type": "Value", "dtype": "float64"},
+    "vad_queue_time": {"_type": "Value", "dtype": "float64"},
+    "vad_gpu_time": {"_type": "Value", "dtype": "float64"},
+    "asr_gpu_time": {"_type": "Value", "dtype": "float64"},
+    "dp_total_time": {"_type": "Value", "dtype": "float64"},
+    # Quality & retry
+    "segments_passed": {"_type": "Value", "dtype": "int32"},
+    "segments_failed": {"_type": "Value", "dtype": "int32"},
+    "mean_confidence": {"_type": "Value", "dtype": "float64"},
+    "tier1_retries": {"_type": "Value", "dtype": "int32"},
+    "tier1_passed": {"_type": "Value", "dtype": "int32"},
+    "tier2_retries": {"_type": "Value", "dtype": "int32"},
+    "tier2_passed": {"_type": "Value", "dtype": "int32"},
+    "reanchors": {"_type": "Value", "dtype": "int32"},
+    "special_merges": {"_type": "Value", "dtype": "int32"},
+    # Reciter stats
+    "words_per_minute": {"_type": "Value", "dtype": "float64"},
+    "phonemes_per_second": {"_type": "Value", "dtype": "float64"},
+    "avg_segment_duration": {"_type": "Value", "dtype": "float64"},
+    "std_segment_duration": {"_type": "Value", "dtype": "float64"},
+    "avg_pause_duration": {"_type": "Value", "dtype": "float64"},
+    "std_pause_duration": {"_type": "Value", "dtype": "float64"},
+    # Session flags
+    "resegmented": {"_type": "Value", "dtype": "bool"},
+    "retranscribed": {"_type": "Value", "dtype": "bool"},
+    # Segments, timestamps & error
+    "segments": {"_type": "Value", "dtype": "string"},
+    "word_timestamps": {"_type": "Value", "dtype": "string"},
+    "char_timestamps": {"_type": "Value", "dtype": "string"},
+    "error": {"_type": "Value", "dtype": "string"},
+}
+
+if _HAS_DEPS:
+    class ParquetScheduler(CommitScheduler):
+        """Buffers rows in memory and uploads a parquet file each interval.
+
+        Audio values are stored as file paths in the row dict; on push they are
+        read as bytes and embedded in the parquet using the HF Audio struct.
+        """
+
+        def __init__(
+            self,
+            *,
+            repo_id: str,
+            schema: Optional[Dict[str, Dict[str, str]]] = None,
+            every: Union[int, float] = 5,
+            path_in_repo: Optional[str] = "data",
+            repo_type: Optional[str] = "dataset",
+            private: bool = False,
+        ) -> None:
+            super().__init__(
+                repo_id=repo_id,
+                folder_path="dummy",  # not used — we upload directly
+                every=every,
+                path_in_repo=path_in_repo,
+                repo_type=repo_type,
+                private=private,
+            )
+            self._rows: List[Dict[str, Any]] = []
+            self._schema = schema
+
+        def append(self, row: Dict[str, Any]) -> None:
+            with self.lock:
+                self._rows.append(row)
+
+        def push_to_hub(self) -> None:
+            with self.lock:
+                rows = self._rows
+                self._rows = []
+            if not rows:
+                return
+
+            print(f"[USAGE_LOG] Pushing {len(rows)} alignment row(s) to Hub.")
+
+            schema: Dict[str, Dict] = dict(self._schema) if self._schema else {}
+            paths_to_cleanup: List[Path] = []
+
+            for row in rows:
+                for key, value in row.items():
+                    if key not in schema:
+                        schema[key] = _infer_schema(key, value)
+
+                    if value is not None and schema[key].get("_type") in ("Image", "Audio"):
+                        file_path = Path(value)
+                        if file_path.is_file():
+                            row[key] = {
+                                "path": file_path.name,
+                                "bytes": file_path.read_bytes(),
+                            }
+                            paths_to_cleanup.append(file_path)
+                        else:
+                            row[key] = None
+
+            for row in rows:
+                for feature in schema:
+                    if feature not in row:
+                        row[feature] = None
+
+            table = pa.Table.from_pylist(rows)
+            table = table.replace_schema_metadata(
+                {"huggingface": json.dumps({"info": {"features": schema}})}
+            )
+
+            archive = None
+            try:
+                import tempfile
+                archive = tempfile.NamedTemporaryFile(suffix=".parquet", delete=False)
+                pq.write_table(table, archive.name)
+                self.api.upload_file(
+                    repo_id=self.repo_id,
+                    repo_type=self.repo_type,
+                    revision=self.revision,
+                    path_in_repo=f"{self.path_in_repo}/{uuid4()}.parquet",
+                    path_or_fileobj=archive.name,
+                )
+                print("[USAGE_LOG] Parquet commit completed.")
+            except Exception as e:
+                print(f"[USAGE_LOG] Failed to upload parquet: {e}")
+            finally:
+                if archive:
+                    archive.close()
+                    Path(archive.name).unlink(missing_ok=True)
+
+            for path in paths_to_cleanup:
+                path.unlink(missing_ok=True)
+
+    def _infer_schema(key: str, value: Any) -> Dict[str, str]:
+        if "image" in key:
+            return {"_type": "Image"}
+        if "audio" in key:
+            return {"_type": "Audio"}
+        if isinstance(value, bool):
+            return {"_type": "Value", "dtype": "bool"}
+        if isinstance(value, int):
+            return {"_type": "Value", "dtype": "int64"}
+        if isinstance(value, float):
+            return {"_type": "Value", "dtype": "float64"}
+        if isinstance(value, bytes):
+            return {"_type": "Value", "dtype": "binary"}
+        return {"_type": "Value", "dtype": "string"}
+
+
+# =========================================================================
+# Lazy scheduler initialization (deferred to first use)
+# =========================================================================
+
+_aligner_scheduler = None
+_error_scheduler = None
+_schedulers_initialized = False
+_init_lock = threading.Lock()
+_fallback_lock = threading.Lock()
+
+
+def _ensure_schedulers() -> None:
+    global _aligner_scheduler, _error_scheduler, _schedulers_initialized
+    if _schedulers_initialized:
+        return
+    with _init_lock:
+        if _schedulers_initialized:
+            return
+        _schedulers_initialized = True
+        if not _HAS_DEPS:
+            print("[USAGE_LOG] Dependencies missing (local-only mode).")
+            return
+        try:
+            _aligner_scheduler = ParquetScheduler(
+                repo_id=USAGE_LOG_DATASET_REPO,
+                schema=_ALIGNER_SCHEMA,
+                every=USAGE_LOG_PUSH_INTERVAL_MINUTES,
+                path_in_repo="data",
+                repo_type="dataset",
+                private=True,
+            )
+            _error_scheduler = CommitScheduler(
+                repo_id=USAGE_LOG_DATASET_REPO,
+                repo_type="dataset",
+                folder_path=ERROR_DIR,
+                path_in_repo="data/errors",
+                private=True,
+                every=USAGE_LOG_PUSH_INTERVAL_MINUTES,
+            )
+        except Exception as e:
+            print(f"[USAGE_LOG] Scheduler init failed (local-only mode): {e}")
+
+
+# =========================================================================
+# Helpers
+# =========================================================================
+
+
+def _get_error_lock():
+    _ensure_schedulers()
+    if _error_scheduler is not None:
+        return _error_scheduler.lock
+    return _fallback_lock
+
+
+def get_user_id(request) -> str:
+    """SHA-256 hash (12-char) of IP+UA from a gr.Request, or 'unknown'."""
+    try:
+        headers = request.headers
+        ip = (
+            headers.get("x-forwarded-for", "").split(",")[0].strip()
+            or headers.get("x-real-ip", "")
+            or ""
+        )
+        ua = headers.get("user-agent", "")
+        return hashlib.sha256(f"{ip}|{ua}".encode()).hexdigest()[:12]
+    except Exception:
+        return "unknown"
+
+
+def _compute_audio_id(audio: np.ndarray, ts: datetime) -> str:
+    """Content hash (16-char) + compact timestamp."""
+    audio_hash = hashlib.sha256(audio.tobytes()).hexdigest()[:16]
+    return f"{audio_hash}:{ts.strftime('%Y%m%dT%H%M%S')}"
+
+
+def _encode_audio_flac(audio: np.ndarray, sample_rate: int, audio_id: str) -> str:
+    """Encode audio to a temp FLAC file; returns the file path."""
+    import soundfile as sf
+
+    tmp_dir = LOG_DIR / "tmp_audio"
+    tmp_dir.mkdir(parents=True, exist_ok=True)
+    safe_id = audio_id.replace(":", "-")
+    filepath = tmp_dir / f"{safe_id}.flac"
+    sf.write(str(filepath), audio, sample_rate, format="FLAC")
+    return str(filepath)
+
+
+def _sync_row_to_scheduler(row: Dict[str, Any]) -> None:
+    """Ensure *row* is represented in the scheduler buffer.
+
+    gr.State may deserialize the dict (creating a copy), and push_to_hub
+    detaches rows from the buffer.  This helper finds the original row by
+    audio_id and updates it, or re-appends if it was already pushed.
+    """
+    if _aligner_scheduler is None:
+        return
+    audio_id = row.get("audio_id")
+    if not audio_id:
+        return
+    with _aligner_scheduler.lock:
+        for buffered in _aligner_scheduler._rows:
+            if buffered.get("audio_id") == audio_id:
+                # Update the buffered row in-place (handles gr.State copies)
+                buffered.update(row)
+                return
+        # Row was already pushed — re-append (audio file may be gone, that's ok)
+        _aligner_scheduler._rows.append(row)
+
+
+# =========================================================================
+# Public logging API
+# =========================================================================
+
+
+def log_alignment(
+    *,
+    audio: np.ndarray,
+    sample_rate: int,
+    request=None,
+    # Input metadata
+    audio_duration_s: float,
+    num_segments: int,
+    surah: int,
+    # Settings
+    min_silence_ms: int,
+    min_speech_ms: int,
+    pad_ms: int,
+    asr_model: str,
+    device: str,
+    # Profiling
+    total_time: float,
+    vad_queue_time: float,
+    vad_gpu_time: float,
+    asr_gpu_time: float,
+    dp_total_time: float,
+    # Quality & retry
+    segments_passed: int,
+    segments_failed: int,
+    mean_confidence: float,
+    tier1_retries: int,
+    tier1_passed: int,
+    tier2_retries: int,
+    tier2_passed: int,
+    reanchors: int,
+    special_merges: int,
+    # Reciter stats
+    words_per_minute: float,
+    phonemes_per_second: float,
+    avg_segment_duration: float,
+    std_segment_duration: float,
+    avg_pause_duration: float,
+    std_pause_duration: float,
+    # Segments
+    log_segments: List[dict],
+) -> Optional[Dict[str, Any]]:
+    """Log an alignment run. Returns the row dict reference for in-place mutation.
+
+    The returned dict can be stored in gr.State and mutated on
+    resegment/retranscribe/timestamps before the scheduler pushes.
+    """
+    _ensure_schedulers()
+    try:
+        ts = datetime.now()
+        audio_id = _compute_audio_id(audio, ts)
+        user_id = get_user_id(request) if request else "unknown"
+
+        # Build the segments JSON: array of run objects
+        segments_runs = [{
+            "min_silence_ms": int(min_silence_ms),
+            "min_speech_ms": int(min_speech_ms),
+            "pad_ms": int(pad_ms),
+            "asr_model": asr_model,
+            "segments": log_segments,
+        }]
+
+        # Encode audio to FLAC temp file (scheduler embeds bytes on push)
+        audio_path = _encode_audio_flac(audio, sample_rate, audio_id)
+
+        row: Dict[str, Any] = {
+            "audio": audio_path,
+            "audio_id": audio_id,
+            "timestamp": ts.isoformat(timespec="seconds"),
+            "user_id": user_id,
+            # Input metadata
+            "audio_duration_s": audio_duration_s,
+            "num_segments": num_segments,
+            "surah": surah,
+            # Settings (latest)
+            "min_silence_ms": int(min_silence_ms),
+            "min_speech_ms": int(min_speech_ms),
+            "pad_ms": int(pad_ms),
+            "asr_model": asr_model,
+            "device": device,
+            # Profiling
+            "total_time": total_time,
+            "vad_queue_time": vad_queue_time,
+            "vad_gpu_time": vad_gpu_time,
+            "asr_gpu_time": asr_gpu_time,
+            "dp_total_time": dp_total_time,
+            # Quality & retry
+            "segments_passed": segments_passed,
+            "segments_failed": segments_failed,
+            "mean_confidence": mean_confidence,
+            "tier1_retries": tier1_retries,
+            "tier1_passed": tier1_passed,
+            "tier2_retries": tier2_retries,
+            "tier2_passed": tier2_passed,
+            "reanchors": reanchors,
+            "special_merges": special_merges,
+            # Reciter stats
+            "words_per_minute": words_per_minute,
+            "phonemes_per_second": phonemes_per_second,
+            "avg_segment_duration": avg_segment_duration,
+            "std_segment_duration": std_segment_duration,
+            "avg_pause_duration": avg_pause_duration,
+            "std_pause_duration": std_pause_duration,
+            # Session flags
+            "resegmented": False,
+            "retranscribed": False,
+            # Segments & error
+            "segments": json.dumps(segments_runs),
+            "word_timestamps": None,
+            "char_timestamps": None,
+            "error": None,
+        }
+
+        if _aligner_scheduler is not None:
+            _aligner_scheduler.append(row)
+        else:
+            _write_fallback(row)
+
+        return row
+
+    except Exception as e:
+        print(f"[USAGE_LOG] Failed to log alignment: {e}")
+        return None
+
+
+def update_alignment_row(
+    row: Dict[str, Any],
+    *,
+    action: str,
+    # Input metadata (overwritten)
+    audio_duration_s: float,
+    num_segments: int,
+    surah: int,
+    # Settings for this run
+    min_silence_ms: int,
+    min_speech_ms: int,
+    pad_ms: int,
+    asr_model: str,
+    device: str,
+    # Profiling
+    total_time: float,
+    vad_queue_time: float,
+    vad_gpu_time: float,
+    asr_gpu_time: float,
+    dp_total_time: float,
+    # Quality & retry
+    segments_passed: int,
+    segments_failed: int,
+    mean_confidence: float,
+    tier1_retries: int,
+    tier1_passed: int,
+    tier2_retries: int,
+    tier2_passed: int,
+    reanchors: int,
+    special_merges: int,
+    # Reciter stats
+    words_per_minute: float,
+    phonemes_per_second: float,
+    avg_segment_duration: float,
+    std_segment_duration: float,
+    avg_pause_duration: float,
+    std_pause_duration: float,
+    # Segments
+    log_segments: List[dict],
+) -> None:
+    """Mutate an existing row dict in-place and ensure it's in the scheduler buffer.
+
+    After mutation, syncs the row into the scheduler's buffer so the update
+    is captured even if gr.State returned a deserialized copy or if the
+    original row was already pushed to Hub.
+
+    Args:
+        row: The dict returned by log_alignment(), stored in gr.State.
+        action: "resegment" or "retranscribe".
+    """
+    try:
+        # Overwrite run-level fields
+        row["audio_duration_s"] = audio_duration_s
+        row["num_segments"] = num_segments
+        row["surah"] = surah
+        row["min_silence_ms"] = int(min_silence_ms)
+        row["min_speech_ms"] = int(min_speech_ms)
+        row["pad_ms"] = int(pad_ms)
+        row["asr_model"] = asr_model
+        row["device"] = device
+        row["total_time"] = total_time
+        row["vad_queue_time"] = vad_queue_time
+        row["vad_gpu_time"] = vad_gpu_time
+        row["asr_gpu_time"] = asr_gpu_time
+        row["dp_total_time"] = dp_total_time
+        row["segments_passed"] = segments_passed
+        row["segments_failed"] = segments_failed
+        row["mean_confidence"] = mean_confidence
+        row["tier1_retries"] = tier1_retries
+        row["tier1_passed"] = tier1_passed
+        row["tier2_retries"] = tier2_retries
+        row["tier2_passed"] = tier2_passed
+        row["reanchors"] = reanchors
+        row["special_merges"] = special_merges
+        row["words_per_minute"] = words_per_minute
+        row["phonemes_per_second"] = phonemes_per_second
+        row["avg_segment_duration"] = avg_segment_duration
+        row["std_segment_duration"] = std_segment_duration
+        row["avg_pause_duration"] = avg_pause_duration
+        row["std_pause_duration"] = std_pause_duration
+
+        # Set session flag
+        if action == "resegment":
+            row["resegmented"] = True
+        elif action == "retranscribe":
+            row["retranscribed"] = True
+
+        # Append new run to segments array
+        segments_runs = json.loads(row.get("segments") or "[]")
+        segments_runs.append({
+            "min_silence_ms": int(min_silence_ms),
+            "min_speech_ms": int(min_speech_ms),
+            "pad_ms": int(pad_ms),
+            "asr_model": asr_model,
+            "segments": log_segments,
+        })
+        row["segments"] = json.dumps(segments_runs)
+
+        # Sync with scheduler buffer — the row from gr.State may be a
+        # deserialized copy, or the original may have already been pushed.
+        _sync_row_to_scheduler(row)
+
+    except Exception as e:
+        print(f"[USAGE_LOG] Failed to update alignment row: {e}")
+
+
+def update_word_timestamps(
+    row: Dict[str, Any],
+    word_timestamps_json: str,
+    char_timestamps_json: Optional[str] = None,
+) -> None:
+    """Set word and char timestamps fields on an existing row and sync to scheduler."""
+    try:
+        row["word_timestamps"] = word_timestamps_json
+        if char_timestamps_json is not None:
+            row["char_timestamps"] = char_timestamps_json
+        _sync_row_to_scheduler(row)
+    except Exception as e:
+        print(f"[USAGE_LOG] Failed to update word timestamps: {e}")
+
+
+def log_error(user_id: str, error_message: str) -> None:
+    """Log a pipeline error to JSONL."""
+    try:
+        with _get_error_lock():
+            with ERROR_LOG_PATH.open("a") as f:
+                json.dump({
+                    "timestamp": datetime.now().isoformat(timespec="seconds"),
+                    "user_id": user_id,
+                    "error_message": error_message or "",
+                }, f)
+                f.write("\n")
+    except Exception:
+        pass
+
+
+def _write_fallback(row: Dict[str, Any]) -> None:
+    """Local-only fallback: write JSONL (without audio)."""
+    fallback_path = LOG_DIR / "alignments_fallback.jsonl"
+    with _fallback_lock:
+        with fallback_path.open("a") as f:
+            fallback_row = {k: v for k, v in row.items() if k != "audio"}
+            json.dump(fallback_row, f)
+            f.write("\n")