asierhv commited on
Commit
08a9877
·
verified ·
1 Parent(s): f4bd572

Added description and "how to use" example

Browse files
Files changed (1) hide show
  1. README.md +143 -37
README.md CHANGED
@@ -28,45 +28,109 @@ model-index:
28
  value: 14.119648426424725
29
  ---
30
 
31
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
32
- should probably proofread and complete it, then remove this comment. -->
33
-
34
  # Whisper Medium Basque
35
 
36
- This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on the mozilla-foundation/common_voice_13_0 eu dataset.
37
- It achieves the following results on the evaluation set:
38
- - Loss: 0.4119
39
- - Wer: 14.1196
 
 
 
40
 
41
  ## Model description
42
 
43
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
- ## Intended uses & limitations
46
 
47
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
 
49
  ## Training and evaluation data
50
 
51
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
  ## Training procedure
54
 
55
  ### Training hyperparameters
56
 
57
- The following hyperparameters were used during training:
58
- - learning_rate: 1e-05
59
- - train_batch_size: 64
60
- - eval_batch_size: 32
61
- - seed: 42
62
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
63
- - lr_scheduler_type: linear
64
- - lr_scheduler_warmup_steps: 500
65
- - training_steps: 10000
66
 
67
- ### Training results
68
 
69
- | Training Loss | Epoch | Step | Validation Loss | Wer |
70
  |:-------------:|:-----:|:-----:|:---------------:|:-------:|
71
  | 0.0206 | 4.02 | 1000 | 0.2998 | 16.9995 |
72
  | 0.0036 | 9.01 | 2000 | 0.3235 | 15.5211 |
@@ -79,27 +143,57 @@ The following hyperparameters were used during training:
79
  | 0.0001 | 43.01 | 9000 | 0.4119 | 14.1196 |
80
  | 0.0001 | 48.01 | 10000 | 0.4150 | 14.1358 |
81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
 
83
- ### Framework versions
 
 
 
 
 
 
84
 
85
- - Transformers 4.33.0.dev0
86
- - Pytorch 2.0.1+cu117
87
- - Datasets 2.14.4
88
- - Tokenizers 0.13.3
 
89
 
90
  ## Citation
91
 
92
- If you use these models in your research, please cite:
93
 
94
  ```bibtex
95
  @misc{dezuazo2025whisperlmimprovingasrmodels,
96
- title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
97
- author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
98
- year={2025},
99
- eprint={2503.23542},
100
- archivePrefix={arXiv},
101
- primaryClass={cs.CL},
102
- url={https://arxiv.org/abs/2503.23542},
103
  }
104
  ```
105
 
@@ -107,9 +201,21 @@ Please, check the related paper preprint in
107
  [arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
108
  for more details.
109
 
110
- ## Licensing
 
 
111
 
112
  This model is available under the
113
  [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
114
  You are free to use, modify, and distribute this model as long as you credit
115
- the original creators.
 
 
 
 
 
 
 
 
 
 
 
28
  value: 14.119648426424725
29
  ---
30
 
 
 
 
31
  # Whisper Medium Basque
32
 
33
+ ## Model summary
34
+
35
+ **Whisper Medium Basque** is an automatic speech recognition (ASR) model for **Basque (eu)** speech. It is fine-tuned from [openai/whisper-medium] on the **Basque portion of Mozilla Common Voice 13.0**, achieving a **Word Error Rate (WER) of 14.12%** on the Common Voice evaluation split.
36
+
37
+ This model offers a balance between transcription accuracy and computational requirements, providing significantly improved ASR performance over smaller Whisper variants while remaining practical for offline or batch processing.
38
+
39
+ ---
40
 
41
  ## Model description
42
 
43
+ * **Architecture:** Transformer-based encoder–decoder (Whisper)
44
+ * **Base model:** openai/whisper-medium
45
+ * **Language:** Basque (eu)
46
+ * **Task:** Automatic Speech Recognition (ASR)
47
+ * **Output:** Text transcription in Basque
48
+ * **Decoding:** Autoregressive sequence-to-sequence decoding
49
+
50
+ This medium-sized model leverages Whisper’s multilingual pretraining and is fine-tuned on Basque speech data, delivering higher transcription quality for a low-resource language while remaining manageable for typical GPU or CPU environments.
51
+
52
+ ---
53
+
54
+ ## Intended use
55
+
56
+ ### Primary use cases
57
+
58
+ * High-quality transcription of Basque audio recordings
59
+ * Offline or batch ASR pipelines
60
+ * Research and development in Basque ASR
61
+ * Media, educational, and archival transcription tasks
62
+
63
+ ### Intended users
64
+
65
+ * Researchers working on Basque or low-resource ASR
66
+ * Developers building Basque speech applications
67
+ * Academic and institutional users
68
 
69
+ ### Out-of-scope use
70
 
71
+ * Real-time or low-latency ASR without additional optimization
72
+ * Speech translation tasks
73
+ * Safety-critical applications without validation
74
+
75
+ ---
76
+
77
+ ## Limitations and known issues
78
+
79
+ * Performance may degrade on:
80
+ * Noisy or low-quality recordings
81
+ * Conversational or spontaneous speech
82
+ * Accents underrepresented in Common Voice
83
+ * While highly accurate for a medium-sized model, errors can still occur under challenging acoustic conditions
84
+ * Dataset biases from Common Voice may be reflected in outputs
85
+
86
+ Users are encouraged to evaluate the model on their own data before deployment.
87
+
88
+ ---
89
 
90
  ## Training and evaluation data
91
 
92
+ ### Training data
93
+
94
+ * **Dataset:** Mozilla Common Voice 13.0 (Basque subset)
95
+ * **Data type:** Crowd-sourced, read speech
96
+ * **Preprocessing:**
97
+ * Audio resampled to 16 kHz
98
+ * Text normalized using Whisper tokenizer
99
+ * Filtering of invalid or problematic samples
100
+
101
+ ### Evaluation data
102
+
103
+ * **Dataset:** Mozilla Common Voice 13.0 (Basque evaluation split)
104
+ * **Metric:** Word Error Rate (WER)
105
+
106
+ ---
107
+
108
+ ## Evaluation results
109
+
110
+ | Metric | Value |
111
+ | ---------- | ---------- |
112
+ | WER (eval) | **14.12%** |
113
+
114
+ These results indicate strong transcription performance for a medium-sized Whisper model fine-tuned for Basque.
115
+
116
+ ---
117
 
118
  ## Training procedure
119
 
120
  ### Training hyperparameters
121
 
122
+ * Learning rate: 1e-5
123
+ * Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
124
+ * LR scheduler: Linear
125
+ * Warmup steps: 500
126
+ * Training steps: 10,000
127
+ * Train batch size: 64
128
+ * Evaluation batch size: 32
129
+ * Seed: 42
 
130
 
131
+ ### Training results (summary)
132
 
133
+ | Training Loss | Epoch | Step | Validation Loss | WER |
134
  |:-------------:|:-----:|:-----:|:---------------:|:-------:|
135
  | 0.0206 | 4.02 | 1000 | 0.2998 | 16.9995 |
136
  | 0.0036 | 9.01 | 2000 | 0.3235 | 15.5211 |
 
143
  | 0.0001 | 43.01 | 9000 | 0.4119 | 14.1196 |
144
  | 0.0001 | 48.01 | 10000 | 0.4150 | 14.1358 |
145
 
146
+ ---
147
+
148
+ ## Framework versions
149
+
150
+ - Transformers 4.33.0.dev0
151
+ - PyTorch 2.0.1+cu117
152
+ - Datasets 2.14.4
153
+ - Tokenizers 0.13.3
154
+
155
+ ---
156
+
157
+ ## How to use
158
+
159
+ ```python
160
+ from transformers import pipeline
161
+
162
+ hf_model = "HiTZ/whisper-medium-eu" # replace with actual repo ID
163
+ device = 0 # set to -1 for CPU
164
+
165
+ pipe = pipeline(
166
+ task="automatic-speech-recognition",
167
+ model=hf_model,
168
+ device=device
169
+ )
170
 
171
+ result = pipe("audio.wav")
172
+ print(result["text"])
173
+ ```
174
+
175
+ ---
176
+
177
+ ## Ethical considerations and risks
178
 
179
+ * This model transcribes speech and may process personal data.
180
+ * Users should ensure compliance with applicable data protection laws (e.g., GDPR).
181
+ * The model should not be used for surveillance or non-consensual audio processing.
182
+
183
+ ---
184
 
185
  ## Citation
186
 
187
+ If you use this model in your research, please cite:
188
 
189
  ```bibtex
190
  @misc{dezuazo2025whisperlmimprovingasrmodels,
191
+ title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
192
+ author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
193
+ year={2025},
194
+ eprint={2503.23542},
195
+ archivePrefix={arXiv},
196
+ primaryClass={cs.CL}
 
197
  }
198
  ```
199
 
 
201
  [arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
202
  for more details.
203
 
204
+ ---
205
+
206
+ ## License
207
 
208
  This model is available under the
209
  [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
210
  You are free to use, modify, and distribute this model as long as you credit
211
+ the original creators.
212
+
213
+ ---
214
+
215
+ ## Contact and attribution
216
+
217
+ * Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
218
+ * Base model: OpenAI Whisper
219
+ * Dataset: Mozilla Common Voice
220
+
221
+ For questions or issues, please open an issue in the model repository.