Obiang commited on
Commit
ee09b74
·
1 Parent(s): e1d7139

Update to Pro-TeVA branding and add architecture diagram

Browse files
Files changed (3) hide show
  1. README.md +31 -13
  2. app.py +40 -23
  3. proteva_archi.png +3 -0
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: ProTeVa Yoruba Tone Recognition
3
  emoji: 🎵
4
  colorFrom: blue
5
  colorTo: green
@@ -8,12 +8,25 @@ sdk_version: 5.49.1
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
- short_description: 'ProTeVa: AI-powered tone recognition for Yoruba language.'
12
  ---
13
 
14
- # ProTeVa: Yoruba Tone Recognition
15
 
16
- This Space demonstrates **ProTeVa** (Prototype-based Tone Variant Autoencoder), a neural model for recognizing tone patterns in Yoruba language with intelligent word boundary detection.
 
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
  ## Features
19
 
@@ -33,17 +46,22 @@ Yoruba is a tonal language with three contrastive tones:
33
 
34
  ## Model Architecture
35
 
36
- - **Feature Extractor**: HuBERT (Orange/SSA-HuBERT-base-60k)
37
- - **Encoder**: 2-layer Bidirectional GRU (512 hidden units)
38
- - **Decoder**: VanillaNN (2 blocks, 512 neurons)
39
- - **Prototype Layer**: 10 learnable tone prototypes
40
- - **F0 Reconstruction**: TorchYIN pitch estimation
41
- - **Output**: CTC-based sequence prediction
42
- - **Space Detection**: Multi-method acoustic boundary detection
 
 
 
 
 
43
 
44
  ## Space Detection
45
 
46
- ProTeVa uses intelligent post-processing to detect word boundaries:
47
 
48
  ### Detection Methods
49
 
@@ -125,7 +143,7 @@ If you use this model in your research, please cite:
125
 
126
  ```bibtex
127
  @article{proteva2025,
128
- title={ProTeVa: Prototype-based Tone Variant Autoencoder for Yoruba Tone Recognition},
129
  author={Your Name},
130
  year={2025},
131
  note={Hugging Face Space}
 
1
  ---
2
+ title: Pro-TeVA Yoruba Tone Recognition
3
  emoji: 🎵
4
  colorFrom: blue
5
  colorTo: green
 
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
+ short_description: 'Pro-TeVA: AI-powered tone recognition for Yoruba language.'
12
  ---
13
 
14
+ # Pro-TeVA: Yoruba Tone Recognition
15
 
16
+ This Space demonstrates **Pro-TeVA** (Prototype-based Temporal Variational Autoencoder), an explainable neural model for recognizing tone patterns in Yoruba language.
17
+
18
+ ## About Pro-TeVA
19
+
20
+ Many sub-Saharan African languages, including Yoruba, are tonal languages where pitch variations determine word meanings. These languages are typically low-resourced, making automatic tone recognition challenging yet crucial for speech processing applications.
21
+
22
+ **Pro-TeVA** addresses this challenge by providing both high-performance tone recognition and explainable results that linguists can understand and trust. Unlike black-box models, Pro-TeVA offers transparency through:
23
+
24
+ - **Interpretable Features**: Uses fundamental frequency (F0) as an acoustic feature that linguists can easily understand
25
+ - **Visualizable Prototypes**: 10 learnable tone prototypes that cluster sound patterns in an interpretable way
26
+ - **F0 Reconstruction**: Decoder outputs that explain model decisions through pitch contour predictions
27
+ - **High Performance**: Achieves 17.74% Tone Error Rate, comparable to black-box models
28
+
29
+ This transparency facilitates collaboration between computational and linguistic disciplines and accelerates dataset creation for low-resource tonal languages by enabling efficient pre-labeling of tones on transcriptions.
30
 
31
  ## Features
32
 
 
46
 
47
  ## Model Architecture
48
 
49
+ ![Pro-TeVA Architecture](proteva_archi.png)
50
+
51
+ Pro-TeVA integrates multiple components for explainable tone recognition:
52
+
53
+ - **Feature Extractor**: HuBERT (Orange/SSA-HuBERT-base-60k) - Self-supervised speech representation
54
+ - **Encoder**: 2-layer Bidirectional GRU (512 hidden units) - Captures temporal dependencies
55
+ - **Variational Autoencoder**: Learns compact latent representations (z) of tone patterns
56
+ - **Prototype Layer**: 10 learnable tone prototypes (P₁, P₂, ..., P₁₀) for clustering sound patterns
57
+ - **Decoder**: VanillaNN (2 blocks, 512 neurons) - Reconstructs F0 contours for interpretability
58
+ - **Classification Layer**: Linear + Softmax → CTC for sequence prediction
59
+ - **F0 Extraction**: TorchYIN pitch estimation for interpretable acoustic features
60
+ - **Space Detection**: Multi-method acoustic boundary detection (post-processing)
61
 
62
  ## Space Detection
63
 
64
+ Pro-TeVA uses intelligent post-processing to detect word boundaries:
65
 
66
  ### Detection Methods
67
 
 
143
 
144
  ```bibtex
145
  @article{proteva2025,
146
+ title={Pro-TeVA: Prototype-based Temporal Variational Autoencoder for Yoruba Tone Recognition},
147
  author={Your Name},
148
  year={2025},
149
  note={Hugging Face Space}
app.py CHANGED
@@ -1,5 +1,5 @@
1
  """
2
- Gradio App for ProTeVa Yoruba Tone Recognition
3
  Hugging Face Spaces deployment
4
  """
5
 
@@ -17,7 +17,7 @@ TONE_INFO = config.TONE_INFO
17
 
18
  # ============ MODEL LOADING ============
19
 
20
- print("Loading ProTeVa tone recognition model...")
21
  print(f"Checkpoint folder: {config.CHECKPOINT_FOLDER}")
22
 
23
  try:
@@ -204,11 +204,11 @@ custom_css = """
204
  }
205
  """
206
 
207
- with gr.Blocks(css=custom_css, title="ProTeVa Tone Recognition") as demo:
208
 
209
  gr.Markdown(
210
  f"""
211
- # 🎵 ProTeVa: Yoruba Tone Recognition
212
 
213
  Upload an audio file or record your voice to detect Yoruba tone patterns.
214
 
@@ -272,31 +272,48 @@ with gr.Blocks(css=custom_css, title="ProTeVa Tone Recognition") as demo:
272
  gr.Markdown("### 📚 Example Audios")
273
  gr.Markdown("*Upload Yoruba speech samples to test the model*")
274
 
275
- gr.Markdown(
276
- f"""
277
- ---
 
 
278
 
279
- **About ProTeVa:**
280
 
281
- ProTeVa (Prototype-based Tone Variant Autoencoder) is a neural model for tone recognition.
282
 
283
- **Model Architecture:**
284
- - Feature Extractor: HuBERT (Orange/SSA-HuBERT-base-60k)
285
- - Encoder: {config.RNN_LAYERS}-layer Bidirectional GRU ({config.RNN_NEURONS} neurons)
286
- - Prototype Layer: {config.N_PROTOTYPES} learnable tone prototypes
287
- - Decoder: F0 reconstruction
288
- - Output: CTC-based tone sequence prediction + acoustic space detection
289
 
290
- **Space Detection:**
291
- - Method: {config.SPACE_DETECTION_METHOD if config.ENABLE_SPACE_DETECTION else 'Disabled'}
292
- - Uses F0 contours, silence patterns, and tone duration
293
- - Automatically detects word boundaries in continuous speech
 
 
 
294
 
295
- Built with ❤️ using SpeechBrain and Gradio
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
296
 
297
- **Model Checkpoint:** {config.CHECKPOINT_FOLDER}
298
- """
299
- )
300
 
301
  if __name__ == "__main__":
302
  demo.launch(
 
1
  """
2
+ Gradio App for Pro-TeVA Yoruba Tone Recognition
3
  Hugging Face Spaces deployment
4
  """
5
 
 
17
 
18
  # ============ MODEL LOADING ============
19
 
20
+ print("Loading Pro-TeVA tone recognition model...")
21
  print(f"Checkpoint folder: {config.CHECKPOINT_FOLDER}")
22
 
23
  try:
 
204
  }
205
  """
206
 
207
+ with gr.Blocks(css=custom_css, title="Pro-TeVA Tone Recognition") as demo:
208
 
209
  gr.Markdown(
210
  f"""
211
+ # 🎵 Pro-TeVA: Yoruba Tone Recognition
212
 
213
  Upload an audio file or record your voice to detect Yoruba tone patterns.
214
 
 
272
  gr.Markdown("### 📚 Example Audios")
273
  gr.Markdown("*Upload Yoruba speech samples to test the model*")
274
 
275
+ with gr.Row():
276
+ with gr.Column(scale=1):
277
+ gr.Markdown(
278
+ f"""
279
+ ---
280
 
281
+ **About Pro-TeVA:**
282
 
283
+ **Pro-TeVA** (Prototype-based Temporal Variational Autoencoder) is an explainable neural model for tone recognition.
284
 
285
+ Unlike black-box models, Pro-TeVA provides transparency through:
286
+ - Interpretable F0 (pitch) features
287
+ - Visualizable tone prototypes
288
+ - F0 reconstruction for explainability
289
+ - High performance: 17.74% Tone Error Rate
 
290
 
291
+ **Model Architecture:**
292
+ - Feature Extractor: HuBERT (Orange/SSA-HuBERT-base-60k)
293
+ - Encoder: {config.RNN_LAYERS}-layer Bidirectional GRU ({config.RNN_NEURONS} neurons)
294
+ - Variational Autoencoder: Compact latent representations
295
+ - Prototype Layer: {config.N_PROTOTYPES} learnable tone prototypes
296
+ - Decoder: F0 reconstruction (VanillaNN)
297
+ - Output: CTC-based sequence prediction
298
 
299
+ **Space Detection:**
300
+ - Method: {config.SPACE_DETECTION_METHOD if config.ENABLE_SPACE_DETECTION else 'Disabled'}
301
+ - Uses F0 contours, silence patterns, and tone duration
302
+ - Automatically detects word boundaries in continuous speech
303
+
304
+ Built with ❤️ using SpeechBrain and Gradio
305
+
306
+ **Model Checkpoint:** {config.CHECKPOINT_FOLDER}
307
+ """
308
+ )
309
+
310
+ with gr.Column(scale=1):
311
+ gr.Image(
312
+ value="proteva_archi.png",
313
+ label="Pro-TeVA Architecture",
314
+ show_label=True
315
+ )
316
 
 
 
 
317
 
318
  if __name__ == "__main__":
319
  demo.launch(
proteva_archi.png ADDED

Git LFS Details

  • SHA256: dbc1c93c74814f9396b5922e7c28767de201f3f0922a21655c3163cc917a465d
  • Pointer size: 131 Bytes
  • Size of remote file: 278 kB