| --- |
| language: it |
| license: mit |
| tags: |
| - whisper |
| - automatic-speech-recognition |
| - italian |
| - ctranslate2 |
| - faster-whisper |
| - whisperx |
| - localai |
| datasets: |
| - mozilla-foundation/common_voice_25_0 |
| base_model: openai/whisper-tiny |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| # whisper-tiny-it |
|
|
| Fine-tuned [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) (39M params) for Italian automatic speech recognition (ASR). |
|
|
| **Author:** Ettore Di Giacinto |
|
|
| Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team. This model can be used directly with [LocalAI](https://localai.io). |
|
|
| ## Usage with LocalAI |
|
|
| This model is ready to use with [LocalAI](https://localai.io) via the `whisperx` backend. |
|
|
| Save the following as `whisperx-tiny-it.yaml` in your LocalAI models directory: |
|
|
| ```yaml |
| name: whisperx-tiny-it |
| backend: whisperx |
| known_usecases: |
| - transcript |
| parameters: |
| model: LocalAI-io/whisper-tiny-it-ct2-int8 |
| language: it |
| ``` |
|
|
| Then transcribe audio via the OpenAI-compatible endpoint: |
|
|
| ```bash |
| curl http://localhost:8080/v1/audio/transcriptions \ |
| -H "Content-Type: multipart/form-data" \ |
| -F file="@audio.mp3" \ |
| -F model="whisperx-tiny-it" |
| ``` |
|
|
| ## Results |
|
|
| Evaluated on Common Voice 25.0 Italian test set (15,184 samples): |
|
|
| | Step | Train Loss | Eval Loss | WER | |
| |------|-----------|-----------|-----| |
| | 1000 | — | 0.59 | 37.1% | |
| | 3000 | 0.42 | 0.47 | 30.8% | |
| | 5000 | — | 0.43 | 28.7% | |
| | 10000 | 0.29 | 0.40 | **27.1%** | |
|
|
| ## Training Details |
|
|
| - **Base model:** openai/whisper-tiny (39M parameters) |
| - **Dataset:** Common Voice 25.0 Italian (173k train, 15k dev, 15k test) |
| - **Steps:** 10,000 (batch size 32, ~1.8 epochs) |
| - **Learning rate:** 1e-5 with 500 warmup steps |
| - **Precision:** bf16 on NVIDIA GB10 |
| - **Training time:** ~2 hours |
|
|
| ## Usage |
|
|
| ### Transformers |
|
|
| ```python |
| from transformers import pipeline |
| |
| pipe = pipeline("automatic-speech-recognition", model="LocalAI-io/whisper-tiny-it") |
| result = pipe("audio.mp3", generate_kwargs={"language": "it", "task": "transcribe"}) |
| print(result["text"]) |
| ``` |
|
|
| ### CTranslate2 / faster-whisper |
|
|
| For optimized CPU inference, use the INT8 quantized version: [LocalAI-io/whisper-tiny-it-ct2-int8](https://huggingface.co/LocalAI-io/whisper-tiny-it-ct2-int8) (39MB). |
|
|
| ### LocalAI |
|
|
| This model is compatible with [LocalAI](https://github.com/mudler/LocalAI) for local, self-hosted AI inference. |
|
|
| ## Links |
|
|
| - **Code:** [github.com/localai-org/italian-whisper](https://github.com/localai-org/italian-whisper) |
| - **CTranslate2 INT8:** [LocalAI-io/whisper-tiny-it-ct2-int8](https://huggingface.co/LocalAI-io/whisper-tiny-it-ct2-int8) |
| - **LocalAI:** [github.com/mudler/LocalAI](https://github.com/mudler/LocalAI) |
|
|
