Whisper Base — Romanian (whisper-base_ro-80mel)

Fine-tuned openai/whisper-base for Romanian automatic speech recognition. Trained as part of the Dorele project — a fully Romanian-language, low-latency smart-home voice assistant — where Whisper models serve as on-premise STT engines.

Results

Evaluated on a held-out validation split of 25,348 samples, beam size 5:

Metric	Value
WER	15.58 %
WER (normalized: lowercase, punctuation stripped)	15.09 %
CER	5.82 %

The validation set is a deliberately hard multi-source mix (read speech, parliamentary/spontaneous speech, command-domain utterances), so numbers are not directly comparable to single-corpus benchmarks like Common Voice test.

Training data

185 K training samples drawn from a combined Romanian ASR corpus (290 K samples / ~619 h total across splits), assembled from:

• Common Voice 25 (Romanian)
• Google FLEURS (ro_ro)
• VoxPopuli (Romanian)
• Echo read-speech corpus
• RODigits (spoken digits)
• USPDATRO
• CRoWL
• RO-Smart synthetic TTS data (smart-home command domain)

Audio pre-processed to 80-bin log-mel features (30 s windows, Whisper standard). Splits: train 185,175 / validation 25,348 / test 78,167.

Training procedure

Full fine-tune (no LoRA) on a single RTX 5070 Ti 16 GB, BF16 autocast, SDPA attention.

Hyperparameter	Value
Parameters	74 M
Effective batch size	144 (16 × 9 grad accum)
Learning rate	3e-5, cosine schedule
Warmup steps	1000
Max epochs	12 (early stopping, patience 5 on eval WER)
Optimizer	AdamW (fused)
Label smoothing	0.0
SpecAugment	2 freq masks (≤27 bins), 2 time masks (≤100 frames)
Selection metric	raw WER, best checkpoint reloaded at end

Labels use the canonical Whisper target sequence (<|startoftranscript|><|ro|><|transcribe|><|notimestamps|> … <|endoftext|>) with teacher-forcing shift, and decoder_start_token_id set to <|startoftranscript|> so training and generation are exactly aligned.

Usage

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="IonGrozea/whisper-base_ro-80mel",
    device="cuda",
)
print(asr("audio.wav", generate_kwargs={"language": "romanian", "task": "transcribe"}))

For low-latency deployment, the model converts cleanly to CTranslate2 / faster-whisper and whisper.cpp (GGUF).

Intended use & limitations

• Romanian speech only; language token is forced to <|ro|>.
• Strongest on read speech and short command-style utterances matching the training distribution; expect degraded accuracy on noisy far-field audio, heavy dialects, and code-switching.

Citation

If you use this model, please also credit the underlying datasets (Common Voice, FLEURS, VoxPopuli, etc.) and OpenAI Whisper:

Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision", 2022.