| --- |
| language: it |
| license: mit |
| tags: |
| - whisper |
| - automatic-speech-recognition |
| - italian |
| - localai |
| datasets: |
| - mozilla-foundation/common_voice_25_0 |
| base_model: openai/whisper-base |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| # whisper-base-it |
|
|
| Fine-tuned [openai/whisper-base](https://huggingface.co/openai/whisper-base) (74M params) for Italian automatic speech recognition (ASR). |
|
|
| **Author:** Ettore Di Giacinto |
|
|
| Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team. This model can be used directly with [LocalAI](https://localai.io). |
|
|
| ## Usage with LocalAI |
|
|
| This model is ready to use with [LocalAI](https://localai.io) via the `whisperx` backend. |
|
|
| Save the following as `whisperx-base-it.yaml` in your LocalAI models directory: |
|
|
| ```yaml |
| name: whisperx-base-it |
| backend: whisperx |
| known_usecases: |
| - transcript |
| parameters: |
| model: LocalAI-io/whisper-base-it-ct2-int8 |
| language: it |
| ``` |
|
|
| Then transcribe audio via the OpenAI-compatible endpoint: |
|
|
| ```bash |
| curl http://localhost:8080/v1/audio/transcriptions \ |
| -H "Content-Type: multipart/form-data" \ |
| -F file="@audio.mp3" \ |
| -F model="whisperx-base-it" |
| ``` |
|
|
| ## Results |
|
|
| Evaluated on Common Voice 25.0 Italian test set (15,184 samples): |
|
|
| | Step | WER | |
| |------|-----| |
| | 1000 | 26.5% | |
| | 2000 | 24.0% | |
| | 3000 | 22.4% | |
| | 5000 | 20.6% | |
| | 7000 | 19.9% | |
| | 10000 | **19.2%** | |
|
|
| ## Training Details |
|
|
| - **Base model:** openai/whisper-base (74M parameters) |
| - **Dataset:** Common Voice 25.0 Italian (173k train, 15k dev, 15k test) |
| - **Steps:** 10,000 (batch size 32, ~1.8 epochs) |
| - **Learning rate:** 1e-5 with 500 warmup steps |
| - **Precision:** bf16 on NVIDIA GB10 |
|
|
| ## Usage |
|
|
| ### Transformers |
|
|
| ```python |
| from transformers import pipeline |
| |
| pipe = pipeline("automatic-speech-recognition", model="LocalAI-io/whisper-base-it") |
| result = pipe("audio.mp3", generate_kwargs={"language": "it", "task": "transcribe"}) |
| print(result["text"]) |
| ``` |
|
|
| ### CTranslate2 / faster-whisper |
|
|
| For optimized CPU inference, use the INT8 quantized version: [LocalAI-io/whisper-base-it-ct2-int8](https://huggingface.co/LocalAI-io/whisper-base-it-ct2-int8) (79MB). |
|
|
| ### LocalAI |
|
|
| This model is compatible with [LocalAI](https://github.com/mudler/LocalAI) for local, self-hosted AI inference. |
|
|
| ## Links |
|
|
| - **CTranslate2 INT8:** [LocalAI-io/whisper-base-it-ct2-int8](https://huggingface.co/LocalAI-io/whisper-base-it-ct2-int8) |
| - **Code:** [github.com/localai-org/italian-whisper](https://github.com/localai-org/italian-whisper) |
| - **LocalAI:** [github.com/mudler/LocalAI](https://github.com/mudler/LocalAI) |
|
|
