Update to Pro-TeVA branding and add architecture diagram
Browse files- README.md +31 -13
- app.py +40 -23
- proteva_archi.png +3 -0
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: 🎵
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: green
|
|
@@ -8,12 +8,25 @@ sdk_version: 5.49.1
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
| 11 |
-
short_description: '
|
| 12 |
---
|
| 13 |
|
| 14 |
-
#
|
| 15 |
|
| 16 |
-
This Space demonstrates **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
## Features
|
| 19 |
|
|
@@ -33,17 +46,22 @@ Yoruba is a tonal language with three contrastive tones:
|
|
| 33 |
|
| 34 |
## Model Architecture
|
| 35 |
|
| 36 |
-
-
|
| 37 |
-
|
| 38 |
-
-
|
| 39 |
-
|
| 40 |
-
- **
|
| 41 |
-
- **
|
| 42 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
## Space Detection
|
| 45 |
|
| 46 |
-
|
| 47 |
|
| 48 |
### Detection Methods
|
| 49 |
|
|
@@ -125,7 +143,7 @@ If you use this model in your research, please cite:
|
|
| 125 |
|
| 126 |
```bibtex
|
| 127 |
@article{proteva2025,
|
| 128 |
-
title={
|
| 129 |
author={Your Name},
|
| 130 |
year={2025},
|
| 131 |
note={Hugging Face Space}
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Pro-TeVA Yoruba Tone Recognition
|
| 3 |
emoji: 🎵
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: green
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
| 11 |
+
short_description: 'Pro-TeVA: AI-powered tone recognition for Yoruba language.'
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# Pro-TeVA: Yoruba Tone Recognition
|
| 15 |
|
| 16 |
+
This Space demonstrates **Pro-TeVA** (Prototype-based Temporal Variational Autoencoder), an explainable neural model for recognizing tone patterns in Yoruba language.
|
| 17 |
+
|
| 18 |
+
## About Pro-TeVA
|
| 19 |
+
|
| 20 |
+
Many sub-Saharan African languages, including Yoruba, are tonal languages where pitch variations determine word meanings. These languages are typically low-resourced, making automatic tone recognition challenging yet crucial for speech processing applications.
|
| 21 |
+
|
| 22 |
+
**Pro-TeVA** addresses this challenge by providing both high-performance tone recognition and explainable results that linguists can understand and trust. Unlike black-box models, Pro-TeVA offers transparency through:
|
| 23 |
+
|
| 24 |
+
- **Interpretable Features**: Uses fundamental frequency (F0) as an acoustic feature that linguists can easily understand
|
| 25 |
+
- **Visualizable Prototypes**: 10 learnable tone prototypes that cluster sound patterns in an interpretable way
|
| 26 |
+
- **F0 Reconstruction**: Decoder outputs that explain model decisions through pitch contour predictions
|
| 27 |
+
- **High Performance**: Achieves 17.74% Tone Error Rate, comparable to black-box models
|
| 28 |
+
|
| 29 |
+
This transparency facilitates collaboration between computational and linguistic disciplines and accelerates dataset creation for low-resource tonal languages by enabling efficient pre-labeling of tones on transcriptions.
|
| 30 |
|
| 31 |
## Features
|
| 32 |
|
|
|
|
| 46 |
|
| 47 |
## Model Architecture
|
| 48 |
|
| 49 |
+

|
| 50 |
+
|
| 51 |
+
Pro-TeVA integrates multiple components for explainable tone recognition:
|
| 52 |
+
|
| 53 |
+
- **Feature Extractor**: HuBERT (Orange/SSA-HuBERT-base-60k) - Self-supervised speech representation
|
| 54 |
+
- **Encoder**: 2-layer Bidirectional GRU (512 hidden units) - Captures temporal dependencies
|
| 55 |
+
- **Variational Autoencoder**: Learns compact latent representations (z) of tone patterns
|
| 56 |
+
- **Prototype Layer**: 10 learnable tone prototypes (P₁, P₂, ..., P₁₀) for clustering sound patterns
|
| 57 |
+
- **Decoder**: VanillaNN (2 blocks, 512 neurons) - Reconstructs F0 contours for interpretability
|
| 58 |
+
- **Classification Layer**: Linear + Softmax → CTC for sequence prediction
|
| 59 |
+
- **F0 Extraction**: TorchYIN pitch estimation for interpretable acoustic features
|
| 60 |
+
- **Space Detection**: Multi-method acoustic boundary detection (post-processing)
|
| 61 |
|
| 62 |
## Space Detection
|
| 63 |
|
| 64 |
+
Pro-TeVA uses intelligent post-processing to detect word boundaries:
|
| 65 |
|
| 66 |
### Detection Methods
|
| 67 |
|
|
|
|
| 143 |
|
| 144 |
```bibtex
|
| 145 |
@article{proteva2025,
|
| 146 |
+
title={Pro-TeVA: Prototype-based Temporal Variational Autoencoder for Yoruba Tone Recognition},
|
| 147 |
author={Your Name},
|
| 148 |
year={2025},
|
| 149 |
note={Hugging Face Space}
|
app.py
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
"""
|
| 2 |
-
Gradio App for
|
| 3 |
Hugging Face Spaces deployment
|
| 4 |
"""
|
| 5 |
|
|
@@ -17,7 +17,7 @@ TONE_INFO = config.TONE_INFO
|
|
| 17 |
|
| 18 |
# ============ MODEL LOADING ============
|
| 19 |
|
| 20 |
-
print("Loading
|
| 21 |
print(f"Checkpoint folder: {config.CHECKPOINT_FOLDER}")
|
| 22 |
|
| 23 |
try:
|
|
@@ -204,11 +204,11 @@ custom_css = """
|
|
| 204 |
}
|
| 205 |
"""
|
| 206 |
|
| 207 |
-
with gr.Blocks(css=custom_css, title="
|
| 208 |
|
| 209 |
gr.Markdown(
|
| 210 |
f"""
|
| 211 |
-
# 🎵
|
| 212 |
|
| 213 |
Upload an audio file or record your voice to detect Yoruba tone patterns.
|
| 214 |
|
|
@@ -272,31 +272,48 @@ with gr.Blocks(css=custom_css, title="ProTeVa Tone Recognition") as demo:
|
|
| 272 |
gr.Markdown("### 📚 Example Audios")
|
| 273 |
gr.Markdown("*Upload Yoruba speech samples to test the model*")
|
| 274 |
|
| 275 |
-
gr.
|
| 276 |
-
|
| 277 |
-
|
|
|
|
|
|
|
| 278 |
|
| 279 |
-
|
| 280 |
|
| 281 |
-
|
| 282 |
|
| 283 |
-
|
| 284 |
-
|
| 285 |
-
|
| 286 |
-
|
| 287 |
-
|
| 288 |
-
- Output: CTC-based tone sequence prediction + acoustic space detection
|
| 289 |
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
|
| 293 |
-
|
|
|
|
|
|
|
|
|
|
| 294 |
|
| 295 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 296 |
|
| 297 |
-
**Model Checkpoint:** {config.CHECKPOINT_FOLDER}
|
| 298 |
-
"""
|
| 299 |
-
)
|
| 300 |
|
| 301 |
if __name__ == "__main__":
|
| 302 |
demo.launch(
|
|
|
|
| 1 |
"""
|
| 2 |
+
Gradio App for Pro-TeVA Yoruba Tone Recognition
|
| 3 |
Hugging Face Spaces deployment
|
| 4 |
"""
|
| 5 |
|
|
|
|
| 17 |
|
| 18 |
# ============ MODEL LOADING ============
|
| 19 |
|
| 20 |
+
print("Loading Pro-TeVA tone recognition model...")
|
| 21 |
print(f"Checkpoint folder: {config.CHECKPOINT_FOLDER}")
|
| 22 |
|
| 23 |
try:
|
|
|
|
| 204 |
}
|
| 205 |
"""
|
| 206 |
|
| 207 |
+
with gr.Blocks(css=custom_css, title="Pro-TeVA Tone Recognition") as demo:
|
| 208 |
|
| 209 |
gr.Markdown(
|
| 210 |
f"""
|
| 211 |
+
# 🎵 Pro-TeVA: Yoruba Tone Recognition
|
| 212 |
|
| 213 |
Upload an audio file or record your voice to detect Yoruba tone patterns.
|
| 214 |
|
|
|
|
| 272 |
gr.Markdown("### 📚 Example Audios")
|
| 273 |
gr.Markdown("*Upload Yoruba speech samples to test the model*")
|
| 274 |
|
| 275 |
+
with gr.Row():
|
| 276 |
+
with gr.Column(scale=1):
|
| 277 |
+
gr.Markdown(
|
| 278 |
+
f"""
|
| 279 |
+
---
|
| 280 |
|
| 281 |
+
**About Pro-TeVA:**
|
| 282 |
|
| 283 |
+
**Pro-TeVA** (Prototype-based Temporal Variational Autoencoder) is an explainable neural model for tone recognition.
|
| 284 |
|
| 285 |
+
Unlike black-box models, Pro-TeVA provides transparency through:
|
| 286 |
+
- Interpretable F0 (pitch) features
|
| 287 |
+
- Visualizable tone prototypes
|
| 288 |
+
- F0 reconstruction for explainability
|
| 289 |
+
- High performance: 17.74% Tone Error Rate
|
|
|
|
| 290 |
|
| 291 |
+
**Model Architecture:**
|
| 292 |
+
- Feature Extractor: HuBERT (Orange/SSA-HuBERT-base-60k)
|
| 293 |
+
- Encoder: {config.RNN_LAYERS}-layer Bidirectional GRU ({config.RNN_NEURONS} neurons)
|
| 294 |
+
- Variational Autoencoder: Compact latent representations
|
| 295 |
+
- Prototype Layer: {config.N_PROTOTYPES} learnable tone prototypes
|
| 296 |
+
- Decoder: F0 reconstruction (VanillaNN)
|
| 297 |
+
- Output: CTC-based sequence prediction
|
| 298 |
|
| 299 |
+
**Space Detection:**
|
| 300 |
+
- Method: {config.SPACE_DETECTION_METHOD if config.ENABLE_SPACE_DETECTION else 'Disabled'}
|
| 301 |
+
- Uses F0 contours, silence patterns, and tone duration
|
| 302 |
+
- Automatically detects word boundaries in continuous speech
|
| 303 |
+
|
| 304 |
+
Built with ❤️ using SpeechBrain and Gradio
|
| 305 |
+
|
| 306 |
+
**Model Checkpoint:** {config.CHECKPOINT_FOLDER}
|
| 307 |
+
"""
|
| 308 |
+
)
|
| 309 |
+
|
| 310 |
+
with gr.Column(scale=1):
|
| 311 |
+
gr.Image(
|
| 312 |
+
value="proteva_archi.png",
|
| 313 |
+
label="Pro-TeVA Architecture",
|
| 314 |
+
show_label=True
|
| 315 |
+
)
|
| 316 |
|
|
|
|
|
|
|
|
|
|
| 317 |
|
| 318 |
if __name__ == "__main__":
|
| 319 |
demo.launch(
|
proteva_archi.png
ADDED
|
Git LFS Details
|