asierhv commited on
Commit
bc1ecdc
·
verified ·
1 Parent(s): 35a796b

Added description and "how to use" example

Browse files
Files changed (1) hide show
  1. README.md +147 -37
README.md CHANGED
@@ -28,45 +28,113 @@ model-index:
28
  value: 32.26943173073028
29
  ---
30
 
31
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
32
- should probably proofread and complete it, then remove this comment. -->
33
-
34
  # Whisper Tiny Basque
35
 
36
- This model is a fine-tuned version of [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) on the mozilla-foundation/common_voice_13_0 eu dataset.
37
- It achieves the following results on the evaluation set:
38
- - Loss: 0.6522
39
- - Wer: 32.2694
 
 
 
40
 
41
  ## Model description
42
 
43
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
- ## Intended uses & limitations
46
 
47
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
 
49
  ## Training and evaluation data
50
 
51
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
  ## Training procedure
54
 
55
  ### Training hyperparameters
56
 
57
- The following hyperparameters were used during training:
58
- - learning_rate: 3.75e-05
59
- - train_batch_size: 256
60
- - eval_batch_size: 128
61
- - seed: 42
62
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
63
- - lr_scheduler_type: linear
64
- - lr_scheduler_warmup_steps: 500
65
- - training_steps: 5000
66
 
67
- ### Training results
68
 
69
- | Training Loss | Epoch | Step | Validation Loss | Wer |
70
  |:-------------:|:-----:|:----:|:---------------:|:-------:|
71
  | 0.0096 | 19.0 | 1000 | 0.5796 | 32.8952 |
72
  | 0.0011 | 38.0 | 2000 | 0.6522 | 32.2694 |
@@ -74,27 +142,57 @@ The following hyperparameters were used during training:
74
  | 0.0003 | 76.01 | 4000 | 0.7217 | 33.0734 |
75
  | 0.0003 | 96.0 | 5000 | 0.7321 | 33.1585 |
76
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
 
78
- ### Framework versions
 
 
 
 
 
 
79
 
80
- - Transformers 4.33.0.dev0
81
- - Pytorch 2.0.1+cu117
82
- - Datasets 2.14.4
83
- - Tokenizers 0.13.3
 
84
 
85
  ## Citation
86
 
87
- If you use these models in your research, please cite:
88
 
89
  ```bibtex
90
  @misc{dezuazo2025whisperlmimprovingasrmodels,
91
- title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
92
- author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
93
- year={2025},
94
- eprint={2503.23542},
95
- archivePrefix={arXiv},
96
- primaryClass={cs.CL},
97
- url={https://arxiv.org/abs/2503.23542},
98
  }
99
  ```
100
 
@@ -102,9 +200,21 @@ Please, check the related paper preprint in
102
  [arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
103
  for more details.
104
 
105
- ## Licensing
 
 
106
 
107
  This model is available under the
108
  [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
109
  You are free to use, modify, and distribute this model as long as you credit
110
- the original creators.
 
 
 
 
 
 
 
 
 
 
 
28
  value: 32.26943173073028
29
  ---
30
 
 
 
 
31
  # Whisper Tiny Basque
32
 
33
+ ## Model summary
34
+
35
+ **Whisper Tiny Basque** is an automatic speech recognition (ASR) model for **Basque (eu)** speech. It is fine-tuned from [openai/whisper-tiny] on the **Basque portion of Mozilla Common Voice 13.0**, achieving a **Word Error Rate (WER) of 32.27%** on the Common Voice evaluation split.
36
+
37
+ The model is designed for lightweight transcription of Basque speech, prioritizing low computational cost over transcription accuracy.
38
+
39
+ ---
40
 
41
  ## Model description
42
 
43
+ * **Architecture:** Transformer-based encoder–decoder (Whisper)
44
+ * **Base model:** openai/whisper-tiny
45
+ * **Language:** Basque (eu)
46
+ * **Task:** Automatic Speech Recognition (ASR)
47
+ * **Output:** Text transcription in Basque
48
+ * **Decoding:** Autoregressive sequence-to-sequence decoding
49
+
50
+ This model leverages Whisper’s multilingual pretraining and is further fine-tuned on Basque speech data to enable ASR for a low-resource language, using a compact model size suitable for constrained environments.
51
+
52
+ ---
53
+
54
+ ## Intended use
55
+
56
+ ### Primary use cases
57
+
58
+ * Basque speech transcription in low-resource or experimental settings
59
+ * Lightweight ASR pipelines with limited computational resources
60
+ * Research on Basque ASR and low-resource speech recognition
61
+ * Dataset exploration and preprocessing
62
+
63
+ ### Intended users
64
+
65
+ * Researchers working on Basque or low-resource ASR
66
+ * Developers experimenting with compact ASR models
67
+ * Academic and educational use
68
 
69
+ ### Out-of-scope use
70
 
71
+ * High-accuracy transcription requirements
72
+ * Real-time or production-grade ASR without further optimization
73
+ * Speech translation tasks
74
+ * Safety-critical applications
75
+
76
+ ---
77
+
78
+ ## Limitations and known issues
79
+
80
+ * Relatively **high WER** compared to larger Whisper variants
81
+ * Performance may degrade significantly on:
82
+ * Noisy audio
83
+ * Conversational or spontaneous speech
84
+ * Accents underrepresented in Common Voice
85
+ * As a tiny model, it may:
86
+ * Miss words
87
+ * Produce incomplete or inaccurate transcriptions
88
+ * Dataset biases from Common Voice may be reflected in outputs
89
+
90
+ Users are encouraged to evaluate the model on their own data before deployment.
91
+
92
+ ---
93
 
94
  ## Training and evaluation data
95
 
96
+ ### Training data
97
+
98
+ * **Dataset:** Mozilla Common Voice 13.0 (Basque subset)
99
+ * **Data type:** Crowd-sourced, read speech
100
+ * **Preprocessing:**
101
+ * Audio resampled to 16 kHz
102
+ * Text normalized using Whisper tokenizer
103
+ * Filtering of invalid samples
104
+
105
+ ### Evaluation data
106
+
107
+ * **Dataset:** Mozilla Common Voice 13.0 (Basque evaluation split)
108
+ * **Metric:** Word Error Rate (WER)
109
+
110
+ ---
111
+
112
+ ## Evaluation results
113
+
114
+ | Metric | Value |
115
+ | ---------- | ---------- |
116
+ | WER (eval) | **32.27%** |
117
+
118
+ These results reflect the expected performance of a tiny Whisper model fine-tuned for a low-resource language.
119
+
120
+ ---
121
 
122
  ## Training procedure
123
 
124
  ### Training hyperparameters
125
 
126
+ * Learning rate: 3.75e-5
127
+ * Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
128
+ * LR scheduler: Linear
129
+ * Warmup steps: 500
130
+ * Training steps: 5,000
131
+ * Train batch size: 256
132
+ * Evaluation batch size: 128
133
+ * Seed: 42
 
134
 
135
+ ### Training results (summary)
136
 
137
+ | Training Loss | Epoch | Step | Validation Loss | WER |
138
  |:-------------:|:-----:|:----:|:---------------:|:-------:|
139
  | 0.0096 | 19.0 | 1000 | 0.5796 | 32.8952 |
140
  | 0.0011 | 38.0 | 2000 | 0.6522 | 32.2694 |
 
142
  | 0.0003 | 76.01 | 4000 | 0.7217 | 33.0734 |
143
  | 0.0003 | 96.0 | 5000 | 0.7321 | 33.1585 |
144
 
145
+ ---
146
+
147
+ ## Framework versions
148
+
149
+ - Transformers 4.33.0.dev0
150
+ - PyTorch 2.0.1+cu117
151
+ - Datasets 2.14.4
152
+ - Tokenizers 0.13.3
153
+
154
+ ---
155
+
156
+ ## How to use
157
+
158
+ ```python
159
+ from transformers import pipeline
160
+
161
+ hf_model = "ORG_OR_USER/whisper-tiny-eu" # replace with actual repo ID
162
+ device = 0 # set to -1 for CPU
163
+
164
+ pipe = pipeline(
165
+ task="automatic-speech-recognition",
166
+ model=hf_model,
167
+ device=device
168
+ )
169
 
170
+ result = pipe("audio.wav")
171
+ print(result["text"])
172
+ ```
173
+
174
+ ---
175
+
176
+ ## Ethical considerations and risks
177
 
178
+ * This model transcribes speech and may process personal data.
179
+ * Users should ensure compliance with applicable data protection laws (e.g., GDPR).
180
+ * The model should not be used for surveillance or non-consensual audio processing.
181
+
182
+ ---
183
 
184
  ## Citation
185
 
186
+ If you use this model in your research, please cite:
187
 
188
  ```bibtex
189
  @misc{dezuazo2025whisperlmimprovingasrmodels,
190
+ title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
191
+ author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
192
+ year={2025},
193
+ eprint={2503.23542},
194
+ archivePrefix={arXiv},
195
+ primaryClass={cs.CL}
 
196
  }
197
  ```
198
 
 
200
  [arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
201
  for more details.
202
 
203
+ ---
204
+
205
+ ## License
206
 
207
  This model is available under the
208
  [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
209
  You are free to use, modify, and distribute this model as long as you credit
210
+ the original creators.
211
+
212
+ ---
213
+
214
+ ## Contact and attribution
215
+
216
+ * Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
217
+ * Base model: OpenAI Whisper
218
+ * Dataset: Mozilla Common Voice
219
+
220
+ For questions or issues, please open an issue in the model repository.