| --- |
| language: |
| - dv |
| - en |
| - ar |
| license: apache-2.0 |
| tags: |
| - whisper |
| - dhivehi |
| - code-switching |
| - automatic-speech-recognition |
| base_model: openai/whisper-small |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| # Whisper Dhivehi Code-Switching ASR |
|
|
| Whisper-small fine-tuned for code-switched Dhivehi (with English and Arabic). |
| Adds a custom `<|dv|>` language token to the tokenizer. |
|
|
| ## Usage |
|
|
| ~~~python |
| from transformers import pipeline |
| |
| asr = pipeline( |
| task="automatic-speech-recognition", |
| model="Serialtechlab/whisper-dhivehi-code-switch-v2", |
| device=0, |
| chunk_length_s=10, |
| stride_length_s=(1, 1), |
| generate_kwargs={"num_beams": 3, "repetition_penalty": 1.05}, |
| ) |
| |
| result = asr("audio.wav") |
| print(result["text"]) |
| ~~~ |
|
|
| ## Training data |
|
|
| Fine-tuned on a synthetic code-switched dataset combining: |
| - Dhivehi: Serialtechlab/dhivehi-mms-v5-combined, dhivehi-tts-preprocessed, dv-syn-female2-for-tts |
| - English/Arabic loan words: google/fleurs (en_us, ar_eg) |
|
|
| Trained for 20,000 steps from `whisper-small` base, with a custom `<|dv|>` language token added. |
