Update README.md
Browse files
README.md
CHANGED
|
@@ -30,59 +30,19 @@ model-index:
|
|
| 30 |
# Whisper Base Bn - by BanglaBridge
|
| 31 |
|
| 32 |
This model is a fine-tuned version of [openai/whisper-base](https://huggingface.co/openai/whisper-base) on the Common Voice 17.0 dataset.
|
| 33 |
-
|
|
|
|
|
|
|
| 34 |
It achieves the following results on the test set:
|
| 35 |
- Wer: 44.93734
|
| 36 |
- Normalized Wer: 22.56397
|
| 37 |
|
| 38 |
-
|
| 39 |
-
### Training hyperparameters
|
| 40 |
-
|
| 41 |
-
The following hyperparameters were used during training:
|
| 42 |
-
- learning_rate: 1e-03
|
| 43 |
-
- train_batch_size: 32
|
| 44 |
-
- eval_batch_size: 32
|
| 45 |
-
- warmup_steps: 500
|
| 46 |
-
- training_steps: 20000
|
| 47 |
-
|
| 48 |
-
LoraConfig:
|
| 49 |
-
- r: 32
|
| 50 |
-
- lora_alpha: 64
|
| 51 |
-
- target_modules: `["q_proj", "v_proj"]`
|
| 52 |
-
- lora_dropout: 0.005
|
| 53 |
-
- bias: none
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
### Training results
|
| 57 |
-
|
| 58 |
-
| Step | Training Loss | Validation Loss |
|
| 59 |
-
|:------:|:-------------:|:---------------:|
|
| 60 |
-
| 1000 | 0.240200 | 0.251211 |
|
| 61 |
-
| 2000 | 0.178700 | 0.210411 |
|
| 62 |
-
| 3000 | 0.150000 | 0.193197 |
|
| 63 |
-
| 4000 | 0.122500 | 0.184060 |
|
| 64 |
-
| 5000 | 0.122300 | 0.177079 |
|
| 65 |
-
| 6000 | 0.097100 | 0.181073 |
|
| 66 |
-
| 7000 | 0.095800 | 0.175566 |
|
| 67 |
-
| 8000 | 0.071400 | 0.173997 |
|
| 68 |
-
| 9000 | 0.082600 | 0.175677 |
|
| 69 |
-
| 10000 | 0.064400 | 0.178262 |
|
| 70 |
-
| 11000 | 0.064700 | 0.177943 |
|
| 71 |
-
| 12000 | 0.046900 | 0.185763 |
|
| 72 |
-
| 13000 | 0.047200 | 0.186843 |
|
| 73 |
-
| 14000 | 0.037500 | 0.193575 |
|
| 74 |
-
| 15000 | 0.036000 | 0.199084 |
|
| 75 |
-
| 16000 | 0.027500 | 0.208745 |
|
| 76 |
-
| 17000 | 0.025200 | 0.215685 |
|
| 77 |
-
| 18000 | 0.017400 | 0.227938 |
|
| 78 |
-
| 19000 | 0.016500 | 0.236160 |
|
| 79 |
-
| 20000 | 0.013000 | 0.240447 |
|
| 80 |
|
| 81 |
|
| 82 |
### Framework versions
|
| 83 |
|
| 84 |
- Transformers 4.40.2
|
| 85 |
- Pytorch 2.6.0+cu124
|
| 86 |
-
- Datasets 3.5.1
|
| 87 |
- Tokenizers 0.19.1
|
| 88 |
- Peft 0.10.0
|
|
|
|
| 30 |
# Whisper Base Bn - by BanglaBridge
|
| 31 |
|
| 32 |
This model is a fine-tuned version of [openai/whisper-base](https://huggingface.co/openai/whisper-base) on the Common Voice 17.0 dataset.
|
| 33 |
+
|
| 34 |
+
It is the merged model from this fine-tuned PEFT LoRA adapter: [banglabridge/base-bn-lora-adapter](https://huggingface.co/banglabridge/base-bn-lora-adapter)
|
| 35 |
+
|
| 36 |
It achieves the following results on the test set:
|
| 37 |
- Wer: 44.93734
|
| 38 |
- Normalized Wer: 22.56397
|
| 39 |
|
| 40 |
+
Refer to the adapter repository for more details on the finetuning: [banglabridge/base-bn-lora-adapter](https://huggingface.co/banglabridge/base-bn-lora-adapter)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
|
| 43 |
### Framework versions
|
| 44 |
|
| 45 |
- Transformers 4.40.2
|
| 46 |
- Pytorch 2.6.0+cu124
|
|
|
|
| 47 |
- Tokenizers 0.19.1
|
| 48 |
- Peft 0.10.0
|