Whisper Tiny Spanish

Model summary

Whisper Tiny Spanish is an automatic speech recognition (ASR) model for Spanish (es) speech. It is fine-tuned from [openai/whisper-tiny] on the Spanish subset of Mozilla Common Voice 13.0, achieving a Word Error Rate (WER) of 19.5904% on the evaluation split.

This variant is optimized for low-latency and lightweight ASR applications on Spanish audio.

Model description

Architecture: Transformer-based encoder–decoder (Whisper Tiny)
Base model: openai/whisper-tiny
Language: Spanish (es)
Task: Automatic Speech Recognition (ASR)
Output: Text transcription in Spanish
Decoding: Autoregressive sequence-to-sequence decoding

Fine-tuned to improve transcription quality while maintaining a small model footprint.

Intended use

Primary use cases

Lightweight Spanish speech transcription
Research and experimentation with Spanish ASR
Applications on devices with limited compute resources

Out-of-scope use

High-accuracy or professional transcription (WER ~20%)
Real-time transcription without latency optimization
Safety-critical applications

Limitations and known issues

Performance may be limited on:
- Noisy recordings or overlapping speech
- Rapid or conversational Spanish
- Regional dialects not well-represented in Common Voice
Occasional errors due to small model capacity and low parameter count.

Training and evaluation data

Dataset: Mozilla Common Voice 13.0 (Spanish subset)
Data type: Crowd-sourced read speech
Preprocessing:
- Audio resampled to 16 kHz
- Text normalized using Whisper tokenizer
- Invalid samples removed
Evaluation metric: Word Error Rate (WER) on held-out evaluation set

Evaluation results

Metric	Value
WER (eval)	19.5904%

Training procedure

Training hyperparameters

Learning rate: 3.75e-5
Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
LR scheduler: Linear
Warmup steps: 500
Training steps: 5000
Train batch size: 256
Eval batch size: 128
Seed: 42

Training results (summary)

Training Loss	Epoch	Step	Validation Loss	WER
0.1801	8.0	1000	0.4318	22.1861
0.1627	16.01	2000	0.4218	19.5904
0.0691	24.01	3000	0.4374	20.7170
0.0124	32.01	4000	0.4635	20.0459
0.0129	40.02	5000	0.4568	20.4135

Framework versions

Transformers 4.33.0.dev0
PyTorch 2.0.1+cu117
Datasets 2.14.4
Tokenizers 0.13.3

How to use

from transformers import pipeline

hf_model = "HiTZ/whisper-tiny-es"  # replace with actual repo ID
device = 0  # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

This model transcribes speech and may process personal data.
Users should ensure compliance with applicable data protection laws (e.g., GDPR).
The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.