| --- |
| license: mit |
| tags: |
| - audio |
| - automatic-speech-recognition |
| widget: |
| - example_title: sample 1 |
| src: https://huggingface.co/bangla-speech-processing/BanglaASR/resolve/main/mp3/common_voice_bn_31515636.mp3 |
| - example_title: sample 2 |
| src: https://huggingface.co/bangla-speech-processing/BanglaASR/resolve/main/mp3/common_voice_bn_31549899.mp3 |
| - example_title: sample 3 |
| src: https://huggingface.co/bangla-speech-processing/BanglaASR/resolve/main/mp3/common_voice_bn_31617644.mp3 |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| Bangla ASR model which was trained Bangla Mozilla Common Voice Dataset. This is Fine-tuning Whisper model using Bangla mozilla common voice dataset. |
| For training this model used 40k training and 7k Validation of around 400 hours of data. We trained 12000 steps and get word |
| error rate 4.58%. This model was whisper small[244 M] variant model. |
|
|
|
|
| ```py |
| |
| import os |
| import librosa |
| import torch |
| import torchaudio |
| import numpy as np |
| |
| from transformers import WhisperTokenizer |
| from transformers import WhisperProcessor |
| from transformers import WhisperFeatureExtractor |
| from transformers import WhisperForConditionalGeneration |
| |
| device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') |
| |
| mp3_path = "https://huggingface.co/bangla-speech-processing/BanglaASR/resolve/main/mp3/common_voice_bn_31515636.mp3" |
| |
| model_path = "bangla-speech-processing/BanglaASR" |
| |
| |
| feature_extractor = WhisperFeatureExtractor.from_pretrained(model_path) |
| tokenizer = WhisperTokenizer.from_pretrained(model_path) |
| processor = WhisperProcessor.from_pretrained(model_path) |
| model = WhisperForConditionalGeneration.from_pretrained(model_path).to(device) |
| |
| |
| speech_array, sampling_rate = torchaudio.load(mp3_path, format="mp3") |
| speech_array = speech_array[0].numpy() |
| speech_array = librosa.resample(np.asarray(speech_array), orig_sr=sampling_rate, target_sr=16000) |
| input_features = feature_extractor(speech_array, sampling_rate=16000, return_tensors="pt").input_features |
| |
| # batch = processor.feature_extractor.pad(input_features, return_tensors="pt") |
| predicted_ids = model.generate(inputs=input_features.to(device))[0] |
| |
| |
| transcription = processor.decode(predicted_ids, skip_special_tokens=True) |
| |
| print(transcription) |
| |
| ``` |
|
|
|
|
| # Dataset |
| Used Mozilla common voice dataset around 400 hours data both training[40k] and validation[7k] mp3 samples. |
| For more information about dataser please [click here](https://commonvoice.mozilla.org/bn/datasets) |
|
|
| # Training Model Information |
|
|
|
|
| | Size | Layers | Width | Heads | Parameters | Bangla-only | Training Status | |
| | ------------- | ------------- | -------- |-------- | ------------- | ------------- | -------- | |
| tiny | 4 |384 | 6 | 39 M | X | X |
| base | 6 |512 | 8 |74 M | X | X |
| small | 12 |768 | 12 |244 M | ✓ | ✓ |
| medium | 24 |1024 | 16 |769 M | X | X |
| large | 32 |1280 | 20 |1550 M | X | X |
|
|
| # Evaluation |
|
|
| Word Error Rate 4.58 % |
|
|
| For More please check the [github](https://github.com/saiful9379/BanglaASR/tree/main) |
|
|
| ``` |
| @misc{BanglaASR , |
| title={Transformer Based Whisper Bangla ASR Model}, |
| author={Md Saiful Islam}, |
| howpublished={}, |
| year={2023} |
| } |
| ``` |
|
|
