Initial model upload

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,211 @@
+---
+language:
+  - am  # Amharic
+  - ar  # Arabic
+  - tw  # Asante Twi
+  - bm  # Bambara
+  - fr  # French
+  - lg  # Ganda
+  - ha  # Hausa
+  - ig  # Igbo
+  - rw  # Kinyarwanda
+  - kg  # Kongo
+  - ln  # Lingala
+  - lu  # Luba-Katanga
+  - mg  # Malagasy
+  - nso # Northern Sotho
+  - ny  # Nyanja
+  - om  # Oromo
+  - pt  # Portuguese
+  - sn  # Shona
+  - so  # Somali
+  - st  # Southern Sotho
+  - sw  # Swahili
+  - ss  # Swati
+  - ti  # Tigrinya
+  - ts  # Tsonga
+  - tn  # Tswana
+  - ak  # Twi
+  - ve  # Venda
+  - wo  # Wolof
+  - xh  # Xhosa
+  - yo  # Yoruba
+  - zu  # Zulu
+  - tzm # Tamazight
+  - sg  # Sango
+  - din # Dinka
+  - ee  # Ewe
+  - fo  # Fon
+  - luo # Luo
+  - mos # Mossi
+  - umb # Umbundu
+license: cc-by-4.0
+tags:
+  - automatic-speech-recognition
+  - audio
+  - speech
+  - african-languages
+  - multilingual
+  - simba
+  - low-resource
+  - speech-recognition
+  - asr
+datasets:
+  - UBC-NLP/SimbaBench
+metrics:
+  - wer
+  - cer
+library_name: transformers
+pipeline_tag: automatic-speech-recognition
+---
+<div align="center">
+
+<img src="https://africa.dlnlp.ai/simba/images/VoC_simba" alt="VoC Simba Models Logo">
+
+
+[![EMNLP 2025 Paper](https://img.shields.io/badge/EMNLP_2025-Paper-B31B1B?style=for-the-badge&logo=arxiv&logoColor=B31B1B&labelColor=FFCDD2)](https://aclanthology.org/2025.emnlp-main.559/)
+[![Official Website](https://img.shields.io/badge/Official-Website-2EA44F?style=for-the-badge&logo=googlechrome&logoColor=2EA44F&labelColor=C8E6C9)](https://africa.dlnlp.ai/simba/)
+[![SimbaBench](https://img.shields.io/badge/SimbaBench-Benchmark-8A2BE2?style=for-the-badge&logo=googlecharts&logoColor=8A2BE2&labelColor=E1BEE7)](#simbabench)
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-FFD21E?style=for-the-badge&logoColor=black&labelColor=FFF9C4)](https://huggingface.co/collections/UBC-NLP/simba-speech-series)
+[![YouTube Video](https://img.shields.io/badge/YouTube-Video-FF0000?style=for-the-badge&logo=youtube&logoColor=FF0000&labelColor=FFCCBC)](#demo)
+
+</div>
+
+## *Bridging the Digital Divide for African AI*
+
+**Voice of a Continent** is a comprehensive open-source ecosystem designed to bring African languages to the forefront of artificial intelligence. By providing a unified suite of benchmarking tools and state-of-the-art models, we ensure that the future of speech technology is inclusive, representative, and accessible to over a billion people.
+
+## Best-in-Class Multilingual Models
+
+Introduced in our EMNLP 2025 paper *[Voice of a Continent](https://aclanthology.org/2025.emnlp-main.559/)*, the **Simba Series** represents the current state-of-the-art for African speech AI.
+
+- **Unified Suite:** Models optimized for African languages.
+- **Superior Accuracy:** Outperforms generic multilingual models by leveraging SimbaBench's high-quality, domain-diverse datasets.
+- **Multitask Capability:** Designed for high performance in ASR (Automatic Speech Recognition) and TTS (Text-to-Speech).
+- **Inclusion-First:** Specifically built to mitigate the "digital divide" by empowering speakers of underrepresented languages.
+
+The **Simba** family consists of state-of-the-art models fine-tuned using SimbaBench. These models achieve superior performance by leveraging dataset quality, domain diversity, and language family relationships.
+
+### 🗣️✍️ Simba-ASR
+> **The New Standard for African Speech-to-Text**
+
+**🎯 Task** `Automatic Speech Recognition` — Powering high-accuracy transcription across the continent.
+
+**🌍 Language Coverage (43 African languages)**
+>  **Amharic** (`amh`), **Arabic** (`ara`), **Asante Twi** (`asanti`), **Bambara** (`bam`), **Baoulé** (`bau`), **Bemba** (`bem`), **Ewe** (`ewe`), **Fanti** (`fat`), **Fon** (`fon`), **French** (`fra`), **Ganda** (`lug`), **Hausa** (`hau`), **Igbo** (`ibo`), **Kabiye** (`kab`), **Kinyarwanda** (`kin`), **Kongo** (`kon`), **Lingala** (`lin`), **Luba-Katanga** (`lub`), **Luo** (`luo`), **Malagasy** (`mlg`), **Mossi** (`mos`), **Northern Sotho** (`nso`), **Nyanja** (`nya`), **Oromo** (`orm`), **Portuguese** (`por`), **Shona** (`sna`), **Somali** (`som`), **Southern Sotho** (`sot`), **Swahili** (`swa`), **Swati** (`ssw`), **Tigrinya** (`tir`), **Tsonga** (`tso`), **Tswana** (`tsn`), **Twi** (`twi`), **Umbundu** (`umb`), **Venda** (`ven`), **Wolof** (`wol`), **Xhosa** (`xho`), **Yoruba** (`yor`), **Zulu** (`zul`), **Tamazight** (`tzm`), **Sango** (`sag`), **Dinka** (`din`).
+
+**🏗️ Base Architectures**
+
+  -  **Simba-S** (SeamlessM4T-v2-MT) — *Top Performer*
+  - **Simba-W** (Whisper-v3-large)
+  - **Simba-X** (Wav2Vec2-XLS-R-2b)
+  - **Simba-M** (MMS-1b-all)
+  - **Simba-H** (AfriHuBERT)
+      
+🌐 Explore the Frontier
+
+| **ASR Models**   | **Architecture**  | **#Parameters** | **🤗 Hugging Face Model Card** | **Status** |
+|---------|:------------------:| :------------------:| :------------------:|:------------------:|    
+| 🔥**Simba-S**🔥|    SeamlessM4T-v2  |  2.3B | 🤗 [https://huggingface.co/UBC-NLP/Simba-S](https://huggingface.co/UBC-NLP/Simba-S) | ✅ Released |
+| 🔥**Simba-W**🔥|    Whisper         |  1.5B | 🤗 [https://huggingface.co/UBC-NLP/Simba-W](https://huggingface.co/UBC-NLP/Simba-W) | ✅ Released | 
+| 🔥**Simba-X**🔥|    Wav2Vec2        |  1B | 🤗 [https://huggingface.co/UBC-NLP/Simba-X](https://huggingface.co/UBC-NLP/Simba-X) | ✅ Released |   
+| 🔥**Simba-M**🔥|    MMS             |  1B | 🤗 [https://huggingface.co/UBC-NLP/Simba-M](https://huggingface.co/UBC-NLP/Simba-M) | ✅ Released |   
+| 🔥**Simba-H**🔥|    HuBERT          |  94M | 🤗 [https://huggingface.co/UBC-NLP/Simba-H](https://huggingface.co/UBC-NLP/Simba-H) | ✅ Released |   
+
+* **Simba-S** emerged as the best-performing ASR model overall.
+
+
+**🧩 Usage Example**
+
+You can easily run inference using the Hugging Face `transformers` library.
+
+```python
+from transformers import pipeline
+
+# Load Simba-S for ASR
+asr_pipeline = pipeline(
+    "automatic-speech-recognition",
+    model="UBC-NLP/Simba-S" #Simba mdoels `UBC-NLP/Simba-S`, `UBC-NLP/Simba-W`, `UBC-NLP/Simba-X`, `UBC-NLP/Simba-H`, `UBC-NLP/Simba-M`
+)
+
+##### Load the multilingual African adapter (Only for  `UBC-NLP/Simba-M`)
+asr_pipeline.model.load_adapter("multilingual_african")  # Only for  `UBC-NLP/Simba-M`
+###########################
+
+# Transcribe audio from file
+result = asr_pipeline("https://africa.dlnlp.ai/simba/audio/afr_Lwazi_afr_test_idx3889.wav")
+print(result["text"])
+
+
+# Transcribe audio from audio array
+result = asr_pipeline({
+    "array": audio_array,
+    "sampling_rate": 16_000
+})
+print(result["text"])
+
+```
+
+#### Example Outputs
+
+Using the same audio file with different Simba models:
+
+```python
+# Simba-S
+{'text': 'watter verontwaardiging sou daar, in ons binneste gewees het.'}
+```
+
+```python
+# Simba-W
+{'text': 'watter veronwaardigingsel daar, in ons binneste gewees het.'}
+```
+
+```python
+# Simba-X
+{'text': 'fator fr on ar taamsodr is'}
+```
+
+```python
+# Simba-M
+{'text': 'watter veronwaardiging sodaar in ons binniste gewees het'}
+```
+
+```python
+# Simba-H
+{'text': 'watter vironwaardiging so daar in ons binneste geweeshet'}
+```
+
+Get started with Simba models in minutes using our interactive Colab notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/UBC-NLP/simba/edit/main/simba_models.ipynb)
+
+
+## Citation
+
+If you use the Simba models or SimbaBench  benchmark for your scientific publication, or if you find the resources in this website useful, please cite our paper.
+
+```bibtex
+
+@inproceedings{elmadany-etal-2025-voice,
+    title = "Voice of a Continent: Mapping {A}frica{'}s Speech Technology Frontier",
+    author = "Elmadany, AbdelRahim A.  and
+      Kwon, Sang Yun  and
+      Toyin, Hawau Olamide  and
+      Alcoba Inciarte, Alcides  and
+      Aldarmaki, Hanan  and
+      Abdul-Mageed, Muhammad",
+    editor = "Christodoulopoulos, Christos  and
+      Chakraborty, Tanmoy  and
+      Rose, Carolyn  and
+      Peng, Violet",
+    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
+    month = nov,
+    year = "2025",
+    address = "Suzhou, China",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2025.emnlp-main.559/",
+    doi = "10.18653/v1/2025.emnlp-main.559",
+    pages = "11039--11061",
+    ISBN = "979-8-89176-332-6",
+}
+
+```
+

README.md

@@ -0,0 +1,211 @@
+---
+language:
+  - am  # Amharic
+  - ar  # Arabic
+  - tw  # Asante Twi
+  - bm  # Bambara
+  - fr  # French
+  - lg  # Ganda
+  - ha  # Hausa
+  - ig  # Igbo
+  - rw  # Kinyarwanda
+  - kg  # Kongo
+  - ln  # Lingala
+  - lu  # Luba-Katanga
+  - mg  # Malagasy
+  - nso # Northern Sotho
+  - ny  # Nyanja
+  - om  # Oromo
+  - pt  # Portuguese
+  - sn  # Shona
+  - so  # Somali
+  - st  # Southern Sotho
+  - sw  # Swahili
+  - ss  # Swati
+  - ti  # Tigrinya
+  - ts  # Tsonga
+  - tn  # Tswana
+  - ak  # Twi
+  - ve  # Venda
+  - wo  # Wolof
+  - xh  # Xhosa
+  - yo  # Yoruba
+  - zu  # Zulu
+  - tzm # Tamazight
+  - sg  # Sango
+  - din # Dinka
+  - ee  # Ewe
+  - fo  # Fon
+  - luo # Luo
+  - mos # Mossi
+  - umb # Umbundu
+license: cc-by-4.0
+tags:
+  - automatic-speech-recognition
+  - audio
+  - speech
+  - african-languages
+  - multilingual
+  - simba
+  - low-resource
+  - speech-recognition
+  - asr
+datasets:
+  - UBC-NLP/SimbaBench
+metrics:
+  - wer
+  - cer
+library_name: transformers
+pipeline_tag: automatic-speech-recognition
+---
+<div align="center">
+
+<img src="https://africa.dlnlp.ai/simba/images/VoC_simba" alt="VoC Simba Models Logo">
+
+
+[![EMNLP 2025 Paper](https://img.shields.io/badge/EMNLP_2025-Paper-B31B1B?style=for-the-badge&logo=arxiv&logoColor=B31B1B&labelColor=FFCDD2)](https://aclanthology.org/2025.emnlp-main.559/)
+[![Official Website](https://img.shields.io/badge/Official-Website-2EA44F?style=for-the-badge&logo=googlechrome&logoColor=2EA44F&labelColor=C8E6C9)](https://africa.dlnlp.ai/simba/)
+[![SimbaBench](https://img.shields.io/badge/SimbaBench-Benchmark-8A2BE2?style=for-the-badge&logo=googlecharts&logoColor=8A2BE2&labelColor=E1BEE7)](#simbabench)
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-FFD21E?style=for-the-badge&logoColor=black&labelColor=FFF9C4)](https://huggingface.co/collections/UBC-NLP/simba-speech-series)
+[![YouTube Video](https://img.shields.io/badge/YouTube-Video-FF0000?style=for-the-badge&logo=youtube&logoColor=FF0000&labelColor=FFCCBC)](#demo)
+
+</div>
+
+## *Bridging the Digital Divide for African AI*
+
+**Voice of a Continent** is a comprehensive open-source ecosystem designed to bring African languages to the forefront of artificial intelligence. By providing a unified suite of benchmarking tools and state-of-the-art models, we ensure that the future of speech technology is inclusive, representative, and accessible to over a billion people.
+
+## Best-in-Class Multilingual Models
+
+Introduced in our EMNLP 2025 paper *[Voice of a Continent](https://aclanthology.org/2025.emnlp-main.559/)*, the **Simba Series** represents the current state-of-the-art for African speech AI.
+
+- **Unified Suite:** Models optimized for African languages.
+- **Superior Accuracy:** Outperforms generic multilingual models by leveraging SimbaBench's high-quality, domain-diverse datasets.
+- **Multitask Capability:** Designed for high performance in ASR (Automatic Speech Recognition) and TTS (Text-to-Speech).
+- **Inclusion-First:** Specifically built to mitigate the "digital divide" by empowering speakers of underrepresented languages.
+
+The **Simba** family consists of state-of-the-art models fine-tuned using SimbaBench. These models achieve superior performance by leveraging dataset quality, domain diversity, and language family relationships.
+
+### 🗣️✍️ Simba-ASR
+> **The New Standard for African Speech-to-Text**
+
+**🎯 Task** `Automatic Speech Recognition` — Powering high-accuracy transcription across the continent.
+
+**🌍 Language Coverage (43 African languages)**
+>  **Amharic** (`amh`), **Arabic** (`ara`), **Asante Twi** (`asanti`), **Bambara** (`bam`), **Baoulé** (`bau`), **Bemba** (`bem`), **Ewe** (`ewe`), **Fanti** (`fat`), **Fon** (`fon`), **French** (`fra`), **Ganda** (`lug`), **Hausa** (`hau`), **Igbo** (`ibo`), **Kabiye** (`kab`), **Kinyarwanda** (`kin`), **Kongo** (`kon`), **Lingala** (`lin`), **Luba-Katanga** (`lub`), **Luo** (`luo`), **Malagasy** (`mlg`), **Mossi** (`mos`), **Northern Sotho** (`nso`), **Nyanja** (`nya`), **Oromo** (`orm`), **Portuguese** (`por`), **Shona** (`sna`), **Somali** (`som`), **Southern Sotho** (`sot`), **Swahili** (`swa`), **Swati** (`ssw`), **Tigrinya** (`tir`), **Tsonga** (`tso`), **Tswana** (`tsn`), **Twi** (`twi`), **Umbundu** (`umb`), **Venda** (`ven`), **Wolof** (`wol`), **Xhosa** (`xho`), **Yoruba** (`yor`), **Zulu** (`zul`), **Tamazight** (`tzm`), **Sango** (`sag`), **Dinka** (`din`).
+
+**🏗️ Base Architectures**
+
+  -  **Simba-S** (SeamlessM4T-v2-MT) — *Top Performer*
+  - **Simba-W** (Whisper-v3-large)
+  - **Simba-X** (Wav2Vec2-XLS-R-2b)
+  - **Simba-M** (MMS-1b-all)
+  - **Simba-H** (AfriHuBERT)
+      
+🌐 Explore the Frontier
+
+| **ASR Models**   | **Architecture**  | **#Parameters** | **🤗 Hugging Face Model Card** | **Status** |
+|---------|:------------------:| :------------------:| :------------------:|:------------------:|    
+| 🔥**Simba-S**🔥|    SeamlessM4T-v2  |  2.3B | 🤗 [https://huggingface.co/UBC-NLP/Simba-S](https://huggingface.co/UBC-NLP/Simba-S) | ✅ Released |
+| 🔥**Simba-W**🔥|    Whisper         |  1.5B | 🤗 [https://huggingface.co/UBC-NLP/Simba-W](https://huggingface.co/UBC-NLP/Simba-W) | ✅ Released | 
+| 🔥**Simba-X**🔥|    Wav2Vec2        |  1B | 🤗 [https://huggingface.co/UBC-NLP/Simba-X](https://huggingface.co/UBC-NLP/Simba-X) | ✅ Released |   
+| 🔥**Simba-M**🔥|    MMS             |  1B | 🤗 [https://huggingface.co/UBC-NLP/Simba-M](https://huggingface.co/UBC-NLP/Simba-M) | ✅ Released |   
+| 🔥**Simba-H**🔥|    HuBERT          |  94M | 🤗 [https://huggingface.co/UBC-NLP/Simba-H](https://huggingface.co/UBC-NLP/Simba-H) | ✅ Released |   
+
+* **Simba-S** emerged as the best-performing ASR model overall.
+
+
+**🧩 Usage Example**
+
+You can easily run inference using the Hugging Face `transformers` library.
+
+```python
+from transformers import pipeline
+
+# Load Simba-S for ASR
+asr_pipeline = pipeline(
+    "automatic-speech-recognition",
+    model="UBC-NLP/Simba-S" #Simba mdoels `UBC-NLP/Simba-S`, `UBC-NLP/Simba-W`, `UBC-NLP/Simba-X`, `UBC-NLP/Simba-H`, `UBC-NLP/Simba-M`
+)
+
+##### Load the multilingual African adapter (Only for  `UBC-NLP/Simba-M`)
+asr_pipeline.model.load_adapter("multilingual_african")  # Only for  `UBC-NLP/Simba-M`
+###########################
+
+# Transcribe audio from file
+result = asr_pipeline("https://africa.dlnlp.ai/simba/audio/afr_Lwazi_afr_test_idx3889.wav")
+print(result["text"])
+
+
+# Transcribe audio from audio array
+result = asr_pipeline({
+    "array": audio_array,
+    "sampling_rate": 16_000
+})
+print(result["text"])
+
+```
+
+#### Example Outputs
+
+Using the same audio file with different Simba models:
+
+```python
+# Simba-S
+{'text': 'watter verontwaardiging sou daar, in ons binneste gewees het.'}
+```
+
+```python
+# Simba-W
+{'text': 'watter veronwaardigingsel daar, in ons binneste gewees het.'}
+```
+
+```python
+# Simba-X
+{'text': 'fator fr on ar taamsodr is'}
+```
+
+```python
+# Simba-M
+{'text': 'watter veronwaardiging sodaar in ons binniste gewees het'}
+```
+
+```python
+# Simba-H
+{'text': 'watter vironwaardiging so daar in ons binneste geweeshet'}
+```
+
+Get started with Simba models in minutes using our interactive Colab notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/UBC-NLP/simba/edit/main/simba_models.ipynb)
+
+
+## Citation
+
+If you use the Simba models or SimbaBench  benchmark for your scientific publication, or if you find the resources in this website useful, please cite our paper.
+
+```bibtex
+
+@inproceedings{elmadany-etal-2025-voice,
+    title = "Voice of a Continent: Mapping {A}frica{'}s Speech Technology Frontier",
+    author = "Elmadany, AbdelRahim A.  and
+      Kwon, Sang Yun  and
+      Toyin, Hawau Olamide  and
+      Alcoba Inciarte, Alcides  and
+      Aldarmaki, Hanan  and
+      Abdul-Mageed, Muhammad",
+    editor = "Christodoulopoulos, Christos  and
+      Chakraborty, Tanmoy  and
+      Rose, Carolyn  and
+      Peng, Violet",
+    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
+    month = nov,
+    year = "2025",
+    address = "Suzhou, China",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2025.emnlp-main.559/",
+    doi = "10.18653/v1/2025.emnlp-main.559",
+    pages = "11039--11061",
+    ISBN = "979-8-89176-332-6",
+}
+
+```
+

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "</s>": 427,
+  "<s>": 426
+}

config.json

@@ -0,0 +1,76 @@
+{
+  "_name_or_path": "ajesujoba/AfriHuBERT",
+  "activation_dropout": 0.0,
+  "add_adapter": false,
+  "apply_spec_augment": true,
+  "architectures": [
+    "HubertForCTC"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "classifier_proj_size": 256,
+  "conv_bias": false,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "mean",
+  "ctc_zero_infinity": false,
+  "do_stable_layer_norm": false,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_dropout": 0.0,
+  "feat_extract_norm": "group",
+  "feat_proj_dropout": 0.0,
+  "feat_proj_layer_norm": true,
+  "final_dropout": 0.0,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.0,
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.0,
+  "mask_feature_length": 10,
+  "mask_feature_min_masks": 0,
+  "mask_feature_prob": 0.0,
+  "mask_time_length": 10,
+  "mask_time_min_masks": 2,
+  "mask_time_prob": 0.05,
+  "model_type": "hubert",
+  "num_attention_heads": 12,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 12,
+  "pad_token_id": 425,
+  "tokenizer_class": "Wav2Vec2CTCTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.33.2",
+  "use_weighted_layer_sum": false,
+  "vocab_size": 428
+}

preprocessor_config.json

@@ -0,0 +1,9 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0,
+  "return_attention_mask": false,
+  "sampling_rate": 16000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bd01b633fc5af37d2d3c9d95ea9275fe8b0fe10d0b04bba04f42520b02d2c35
+size 378876385

special_tokens_map.json

@@ -0,0 +1,22 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "[PAD]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "replace_word_delimiter_char": " ",
+  "target_lang": null,
+  "tokenizer_class": "Wav2Vec2CTCTokenizer",
+  "unk_token": "[UNK]",
+  "word_delimiter_token": "|"
+}

vocab.json

@@ -0,0 +1,428 @@
+{
+  "<": 1,
+  "=": 2,
+  ">": 3,
+  "[": 4,
+  "[PAD]": 425,
+  "[UNK]": 424,
+  "\\": 5,
+  "]": 6,
+  "_": 7,
+  "`": 8,
+  "a": 9,
+  "b": 10,
+  "c": 11,
+  "d": 12,
+  "e": 13,
+  "f": 14,
+  "g": 15,
+  "h": 16,
+  "i": 17,
+  "j": 18,
+  "k": 19,
+  "l": 20,
+  "m": 21,
+  "n": 22,
+  "o": 23,
+  "p": 24,
+  "q": 25,
+  "r": 26,
+  "s": 27,
+  "t": 28,
+  "u": 29,
+  "v": 30,
+  "w": 31,
+  "x": 32,
+  "y": 33,
+  "z": 34,
+  "|": 0,
+  "~": 35,
+  "«": 36,
+  "²": 37,
+  "µ": 38,
+  "·": 39,
+  "»": 40,
+  "à": 41,
+  "á": 42,
+  "â": 43,
+  "ã": 44,
+  "ç": 45,
+  "è": 46,
+  "é": 47,
+  "ê": 48,
+  "ë": 49,
+  "ì": 50,
+  "í": 51,
+  "ï": 52,
+  "ñ": 53,
+  "ò": 54,
+  "ó": 55,
+  "ô": 56,
+  "õ": 57,
+  "ö": 58,
+  "ù": 59,
+  "ú": 60,
+  "û": 61,
+  "ć": 62,
+  "č": 63,
+  "ĕ": 64,
+  "ğ": 65,
+  "ĩ": 66,
+  "ī": 67,
+  "ĭ": 68,
+  "ĺ": 69,
+  "ń": 70,
+  "ŋ": 71,
+  "ŏ": 72,
+  "ś": 73,
+  "š": 74,
+  "ŧ": 75,
+  "ũ": 76,
+  "ŭ": 77,
+  "ŵ": 78,
+  "ƙ": 79,
+  "ƥ": 80,
+  "ƭ": 81,
+  "ƴ": 82,
+  "ǧ": 83,
+  "ǹ": 84,
+  "ɓ": 85,
+  "ɔ": 86,
+  "ɖ": 87,
+  "ɗ": 88,
+  "ɛ": 89,
+  "ɣ": 90,
+  "ɲ": 91,
+  "ʹ": 92,
+  "ʻ": 93,
+  "̀": 94,
+  "́": 95,
+  "̆": 96,
+  "̈": 97,
+  "̣": 98,
+  "έ": 99,
+  "γ": 100,
+  "ε": 101,
+  "ԑ": 102,
+  "ሀ": 103,
+  "ሁ": 104,
+  "ሂ": 105,
+  "ሃ": 106,
+  "ሄ": 107,
+  "ህ": 108,
+  "ሆ": 109,
+  "ለ": 110,
+  "ሉ": 111,
+  "ሊ": 112,
+  "ላ": 113,
+  "ሌ": 114,
+  "ል": 115,
+  "ሎ": 116,
+  "ሏ": 117,
+  "ሐ": 118,
+  "ሑ": 119,
+  "ሒ": 120,
+  "ሓ": 121,
+  "ሔ": 122,
+  "ሕ": 123,
+  "ሖ": 124,
+  "መ": 125,
+  "ሙ": 126,
+  "ሚ": 127,
+  "ማ": 128,
+  "ሜ": 129,
+  "ም": 130,
+  "ሞ": 131,
+  "ሟ": 132,
+  "ሠ": 133,
+  "ሡ": 134,
+  "ሣ": 135,
+  "ሥ": 136,
+  "ሦ": 137,
+  "ረ": 138,
+  "ሩ": 139,
+  "ሪ": 140,
+  "ራ": 141,
+  "ሬ": 142,
+  "ር": 143,
+  "ሮ": 144,
+  "ሯ": 145,
+  "ሰ": 146,
+  "ሱ": 147,
+  "ሲ": 148,
+  "ሳ": 149,
+  "ሴ": 150,
+  "ስ": 151,
+  "ሶ": 152,
+  "ሷ": 153,
+  "ሸ": 154,
+  "ሹ": 155,
+  "ሺ": 156,
+  "ሻ": 157,
+  "ሼ": 158,
+  "ሽ": 159,
+  "ሾ": 160,
+  "ሿ": 161,
+  "ቀ": 162,
+  "ቁ": 163,
+  "ቂ": 164,
+  "ቃ": 165,
+  "ቄ": 166,
+  "ቅ": 167,
+  "ቆ": 168,
+  "ቋ": 169,
+  "ቐ": 170,
+  "ቒ": 171,
+  "ቓ": 172,
+  "ቕ": 173,
+  "ቚ": 174,
+  "በ": 175,
+  "ቡ": 176,
+  "ቢ": 177,
+  "ባ": 178,
+  "ቤ": 179,
+  "ብ": 180,
+  "ቦ": 181,
+  "ቧ": 182,
+  "ቨ": 183,
+  "ቩ": 184,
+  "ቪ": 185,
+  "ቫ": 186,
+  "ቬ": 187,
+  "ቭ": 188,
+  "ቮ": 189,
+  "ቯ": 190,
+  "ተ": 191,
+  "ቱ": 192,
+  "ቲ": 193,
+  "ታ": 194,
+  "ቴ": 195,
+  "ት": 196,
+  "ቶ": 197,
+  "ቷ": 198,
+  "ቸ": 199,
+  "ቹ": 200,
+  "ቺ": 201,
+  "ቻ": 202,
+  "ቼ": 203,
+  "ች": 204,
+  "ቾ": 205,
+  "ቿ": 206,
+  "ኃ": 207,
+  "ኅ": 208,
+  "ኋ": 209,
+  "ነ": 210,
+  "ኑ": 211,
+  "ኒ": 212,
+  "ና": 213,
+  "ኔ": 214,
+  "ን": 215,
+  "ኖ": 216,
+  "ኗ": 217,
+  "ኘ": 218,
+  "ኙ": 219,
+  "ኚ": 220,
+  "ኛ": 221,
+  "ኜ": 222,
+  "ኝ": 223,
+  "ኞ": 224,
+  "ኟ": 225,
+  "አ": 226,
+  "ኡ": 227,
+  "ኢ": 228,
+  "ኣ": 229,
+  "ኤ": 230,
+  "እ": 231,
+  "ኦ": 232,
+  "ከ": 233,
+  "ኩ": 234,
+  "ኪ": 235,
+  "ካ": 236,
+  "ኬ": 237,
+  "ክ": 238,
+  "ኮ": 239,
+  "ኰ": 240,
+  "ኲ": 241,
+  "ኳ": 242,
+  "ኸ": 243,
+  "ኻ": 244,
+  "ኽ": 245,
+  "ኾ": 246,
+  "ወ": 247,
+  "ዉ": 248,
+  "ዊ": 249,
+  "ዋ": 250,
+  "ዌ": 251,
+  "ው": 252,
+  "ዎ": 253,
+  "ዐ": 254,
+  "ዑ": 255,
+  "ዒ": 256,
+  "ዓ": 257,
+  "ዔ": 258,
+  "ዕ": 259,
+  "ዖ": 260,
+  "ዘ": 261,
+  "ዙ": 262,
+  "ዚ": 263,
+  "ዛ": 264,
+  "ዜ": 265,
+  "ዝ": 266,
+  "ዞ": 267,
+  "ዟ": 268,
+  "ዠ": 269,
+  "ዡ": 270,
+  "ዢ": 271,
+  "ዣ": 272,
+  "ዤ": 273,
+  "ዥ": 274,
+  "ዦ": 275,
+  "ዧ": 276,
+  "የ": 277,
+  "ዩ": 278,
+  "ዪ": 279,
+  "ያ": 280,
+  "ዬ": 281,
+  "ይ": 282,
+  "ዮ": 283,
+  "ደ": 284,
+  "ዱ": 285,
+  "ዲ": 286,
+  "ዳ": 287,
+  "ዴ": 288,
+  "ድ": 289,
+  "ዶ": 290,
+  "ዷ": 291,
+  "ጀ": 292,
+  "ጁ": 293,
+  "ጂ": 294,
+  "ጃ": 295,
+  "ጄ": 296,
+  "ጅ": 297,
+  "ጆ": 298,
+  "ጇ": 299,
+  "ገ": 300,
+  "ጉ": 301,
+  "ጊ": 302,
+  "ጋ": 303,
+  "ጌ": 304,
+  "ግ": 305,
+  "ጎ": 306,
+  "ጐ": 307,
+  "ጓ": 308,
+  "ጔ": 309,
+  "ጠ": 310,
+  "ጡ": 311,
+  "ጢ": 312,
+  "ጣ": 313,
+  "ጤ": 314,
+  "ጥ": 315,
+  "ጦ": 316,
+  "ጧ": 317,
+  "ጨ": 318,
+  "ጩ": 319,
+  "ጪ": 320,
+  "ጫ": 321,
+  "ጬ": 322,
+  "ጭ": 323,
+  "ጮ": 324,
+  "ጯ": 325,
+  "ጰ": 326,
+  "ጱ": 327,
+  "ጲ": 328,
+  "ጳ": 329,
+  "ጴ": 330,
+  "ጵ": 331,
+  "ጶ": 332,
+  "ጸ": 333,
+  "ጹ": 334,
+  "ጺ": 335,
+  "ጻ": 336,
+  "ጼ": 337,
+  "ጽ": 338,
+  "ጾ": 339,
+  "ጿ": 340,
+  "ፀ": 341,
+  "ፁ": 342,
+  "ፃ": 343,
+  "ፅ": 344,
+  "ፈ": 345,
+  "ፉ": 346,
+  "ፊ": 347,
+  "ፋ": 348,
+  "ፌ": 349,
+  "ፍ": 350,
+  "ፎ": 351,
+  "ፏ": 352,
+  "ፐ": 353,
+  "ፑ": 354,
+  "ፒ": 355,
+  "ፓ": 356,
+  "ፔ": 357,
+  "ፕ": 358,
+  "ፖ": 359,
+  "፡": 360,
+  "።": 361,
+  "፣": 362,
+  "፤": 363,
+  "ḅ": 364,
+  "ḍ": 365,
+  "ḓ": 366,
+  "ḥ": 367,
+  "ḷ": 368,
+  "ḽ": 369,
+  "ṅ": 370,
+  "ṋ": 371,
+  "ṕ": 372,
+  "ṛ": 373,
+  "ṣ": 374,
+  "ṭ": 375,
+  "ṱ": 376,
+  "ẃ": 377,
+  "ẓ": 378,
+  "ạ": 379,
+  "ẹ": 380,
+  "ị": 381,
+  "ọ": 382,
+  "ụ": 383,
+  "ὲ": 384,
+  "–": 385,
+  "—": 386,
+  "’": 387,
+  "‟": 388,
+  "•": 389,
+  "…": 390,
+  "‽": 391,
+  "ⴰ": 392,
+  "ⴱ": 393,
+  "ⴳ": 394,
+  "ⴷ": 395,
+  "ⴹ": 396,
+  "ⴻ": 397,
+  "ⴼ": 398,
+  "ⴽ": 399,
+  "ⵀ": 400,
+  "ⵃ": 401,
+  "ⵄ": 402,
+  "ⵅ": 403,
+  "ⵇ": 404,
+  "ⵉ": 405,
+  "ⵊ": 406,
+  "ⵍ": 407,
+  "ⵎ": 408,
+  "ⵏ": 409,
+  "ⵓ": 410,
+  "ⵔ": 411,
+  "ⵕ": 412,
+  "ⵖ": 413,
+  "ⵙ": 414,
+  "ⵚ": 415,
+  "ⵛ": 416,
+  "ⵜ": 417,
+  "ⵟ": 418,
+  "ⵡ": 419,
+  "ⵢ": 420,
+  "ⵣ": 421,
+  "ⵥ": 422,
+  "ⵯ": 423
+}