| --- |
| language: |
| - en |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - automatic-speech-recognition |
| - speech |
| - audio |
| - whisper |
| - fine-tuning |
| - pytorch |
| - transformers |
| - huggingface |
| - deep-learning |
| - asr |
| --- |
| |
| # Whisper Fine-Tuned Automatic Speech Recognition Model |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| This model is a fine-tuned version of OpenAI's Whisper model for Automatic Speech Recognition (ASR). It has been trained on a custom speech dataset to improve transcription accuracy for domain-specific speech and conversational audio. |
|
|
| The model leverages the strong multilingual capabilities of Whisper while adapting to the characteristics of the target dataset through supervised fine-tuning. |
|
|
| - **Developed by:** Gad Amr , Osama Mohamed , Rojeh Wael |
| - **Funded by:** Self-funded Academic Graduation Project |
| - **Shared by:** Gad Amr |
| - **Model type:** Automatic Speech Recognition (ASR) |
| - **Base model:** OpenAI Whisper Medium |
| - **Language(s):** Arabic / English |
| - **Framework:** Hugging Face Transformers |
| - **License:** Apache-2.0 |
| - **Fine-tuned from model:** openai/whisper-Medium |
|
|
| --- |
|
|
| ## Direct Use |
|
|
| This model is designed for automatic speech recognition tasks, including: |
|
|
| - Speech-to-text transcription |
|
|
| --- |
|
|
| ## Downstream Use |
|
|
| The model can be integrated into: |
|
|
| - Meezan Graduation Project |
|
|
| --- |
|
|
| ## Out-of-Scope Use |
|
|
|
|
|
|
| # Bias, Risks, and Limitations |
|
|
| Like all speech recognition models, performance depends on: |
|
|
| - Audio quality |
| - Background noise |
| - Microphone quality |
| - Accent diversity |
| - Speaking speed |
| - Domain mismatch |
|
|
| Potential limitations include: |
|
|
| - Reduced accuracy on unseen accents but covered Egyptian ones . |
| - Difficulty with overlapping speakers. |
|
|
| --- |
|
|
| ## Recommendations |
|
|
|
|
| # How to Get Started with the Model |
|
|
| ```python |
| from transformers import pipeline |
| |
| pipe = pipeline( |
| "automatic-speech-recognition", |
| model="DBGad/Meezan-ASR" |
| ) |
| |
| result = pipe("sample.wav") |
| |
| print(result["text"]) |
| ```` |
|
|
| --- |
|
|
|
|
|
|
| ### Preprocessing |
|
|
| The following preprocessing steps were applied: |
|
|
| * Resampling audio to 16 kHz |
| * Removing corrupted samples |
| * Feature extraction using WhisperFeatureExtractor |
| * Text normalization |
| * Tokenization using WhisperTokenizer |
| * Dynamic padding with DataCollatorSpeechSeq2SeqWithPadding |
|
|
| --- |
| ### LoRA Configuration |
|
|
| | Parameter | Value | |
| |------------|-------| |
| | Rank (r) | 8 | |
| | Alpha | 16 | |
| | Dropout | 0.1 | |
|
|
| Target Modules: |
|
|
| - encoder_attn.q_proj |
| - encoder_attn.v_proj |
| - encoder_attn.out_proj |
|
|
| --- |
|
|
| ## Training Hyperparameters |
|
|
| | Hyperparameter | Value | |
| |----------------|-------| |
| | Base Model | openai/whisper-medium | |
| | Epochs | 12 | |
| | Batch Size (per device) | 16 | |
| | Gradient Accumulation | 8 | |
| | Effective Batch Size | 128 | |
| | Learning Rate | 5e-4 | |
| | Warmup Steps | 150 | |
| | Evaluation Strategy | Every 200 Steps | |
| | Checkpoint Saving | Every 200 Steps | |
| | Logging Steps | Every 50 Steps | |
| | Best Model Selection Metric | Word Error Rate (WER) | |
| | Load Best Model | Enabled | |
|
|
| --- |
|
|
| ## Optimization |
|
|
| The model was trained using: |
|
|
| - Mixed Precision (FP16/BF16) |
| - Gradient Checkpointing |
| - LoRA Parameter-Efficient Fine-Tuning |
|
|
| These techniques significantly reduced GPU memory consumption while maintaining strong model performance. |
|
|
| --- |
|
|
| ## Evaluation |
|
|
| Validation was performed every **200 training steps**. |
|
|
| The best checkpoint was automatically selected using the **lowest Word Error Rate (WER)**. |
|
|
| Evaluation Metric: |
|
|
| - Word Error Rate (WER) |
|
|
| --- |
|
|
| ## Hardware & Software |
|
|
| ### Hardware |
|
|
| Training was performed on an NVIDIA GPU with CUDA support. |
|
|
| ### Software |
|
|
| - Python |
| - PyTorch |
| - Hugging Face Transformers |
| - PEFT (LoRA) |
| - Datasets |
| - Evaluate |
| - Accelerate |
|
|
| --- |
|
|
| ## Model Objective |
|
|
| The objective of the model is to convert spoken English audio into accurate text transcriptions. |
|
|
| The model minimizes sequence-to-sequence cross-entropy loss while optimizing transcription quality measured by Word Error Rate (WER). |
|
|
|
|
| # Model Card Authors |
|
|
| * Gad Amr |
|
|
| --- |
|
|
| # Model Card Contact |
|
|
| For questions, suggestions, or collaboration: |
|
|
| * GitHub: https://github.com/DBGad |
| * GitHub: https://github.com/Osamamoo |
| * GitHub: https://github.com/Rojeh-wael |
| * Gad Email : [gadelhaq.work@gmail.com](mailto:gadelhaq.work@gmail.com) |
| * Osama Email: [osama-m-ali@outlook.com](mailto:osama-m-ali@outlook.com) |
| * Rojeh Email: [rojehwael@yahoo.com](mailto:rojehwael@yahoo.com) |
|
|
| ``` |
| ``` |
|
|
