| --- |
| language: |
| - az |
| license: cc-by-nc-4.0 |
| library_name: nemo |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - asr |
| - ctc |
| - fastconformer |
| - kenlm |
| - onnx |
| - azerbaijani |
| --- |
| |
| # speakwrite-azerbaijani |
|
|
|
|
| Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with a 3-pass loop refiner, product-key acoustic memory |
| (700 x 700 sub-keys) and a 6-gram token KenLM. Input is 16 kHz mono; output has case and `. , ? !`, with numbers written as words. |
|
|
| ## Results |
|
|
| [LocalDoc/fleurs-azerbaijani-asr](https://huggingface.co/datasets/LocalDoc/fleurs-azerbaijani-asr), not used for training. Reference: `text_norm` |
| (numbers as words). WER and CER ignore case and punctuation. Clips with `norm_status = failed` are excluded (11 of 3977). |
|
|
| WER / CER: |
|
|
| | split | clips | hours | PyTorch, no LM | PyTorch + KenLM | ONNX fast + KenLM | |
| |---|---|---|---|---|---| |
| | test | 914 | 3.20 | 16.34 / 4.75 | **10.98 / 3.57** | **11.03 / 3.61** | |
| | dev | 397 | 1.34 | 16.52 / 4.75 | 11.05 / 3.50 | 11.25 / 3.59 | |
| | train | 2655 | 9.27 | 15.82 / 4.80 | 11.30 / 3.78 | 11.36 / 3.83 | |
| | all | 3966 | 13.81 | 16.01 / 4.79 | 11.20 / 3.70 | 11.27 / 3.75 | |
|
|
| - Decoder weights (`alpha` 1.0, `beta` 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote. |
| - The ONNX plain graph also gives 11.03 / 3.61 on FLEURS test; it was not run on the other splits. |
| - Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not. |
| - Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01. |
|
|
| Speed on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM: |
|
|
| | runtime | threads | speed | weights | measured on | |
| |---|---|---|---|---| |
| | ONNX fast | 8 | 84-87x | 268 MB | this checkpoint, FLEURS test, dev and train | |
| | ONNX plain | 8 | ~68x | 548 MB | this checkpoint, FLEURS test | |
| | PyTorch float32 | 8 | ~45x | 865 MB | this checkpoint, all FLEURS | |
| | PyTorch float32 | 1 | 12.7x | 865 MB | an earlier checkpoint of this model, 100 clips | |
|
|
| The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time in the PyTorch runs and adds 1 GB to the download for every runtime. ONNX was not measured on 1 thread. |
|
|
| ## Use |
|
|
| ```bash |
| pip install balbal |
| ``` |
|
|
| ```python |
| import balbal |
| |
| model = balbal.Model("LocalDoc/speakwrite-azerbaijani", threads=8) |
| print(model.transcribe("speech.wav")["text"]) |
| ``` |
|
|
| Tested with balbal 0.3.0 on Python 3.12 and Linux. The first run downloads about 1.3 GB: the fast graph and the KenLM. Pass `runtime="onnx"` for the plain graph, which uses standard ONNX operators only. |
|
|
| `transcribe` takes a file path or a waveform array with its sample rate: `model.transcribe(wave, 16000)`. WAV, FLAC and OGG are read directly; other formats need FFmpeg. The result also has `words`, with `word`, `start`, `end` and `confidence` for each word. |
|
|
| Audio longer than 30 s is cut into 30 s windows with 10 s overlap. `overlap_seconds=0` cuts fixed 30 s chunks instead, which is faster but gave a higher WER on our long recordings. |
|
|
| ### PyTorch |
|
|
| `model.pt` is the original checkpoint, for further training and experiments. |
|
|
| Needs [balbal](https://github.com/erayyap/balbal) at commit `1be9f88b77c63fbafa7713c6e46e79847f6c390e` (Python 3.11-3.12, Linux; install |
| `requirements/full.txt`, then run `inference/build_native.py`). Copy this repository's `refiner.json` over balbal's |
| `configs/balbal-v2-manas/refiner.json`: this model's memory size differs from balbal's default, and the weights will not load otherwise. |
|
|
| ```python |
| import json, sys |
| sys.path[:0] = ["balbal/inference", "balbal"] |
| from balbal.audio import read_audio |
| from runtime import Engine |
| |
| engine = Engine("speakwrite-azerbaijani", runtime="torch", settings=json.load(open("speakwrite-azerbaijani/decoder.json"))) |
| print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"]) |
| ``` |
|
|
| Keep `itn=False`: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks. |
|
|
| To use the PyTorch model without the KenLM, take the argmax of the CTC output instead of `Engine`: without an LM, WER on FLEURS test is 16.34. The `lm/` folder is then not needed. |
|
|
| ## Training |
|
|
| About 2800 hours of Azerbaijani speech. Encoder initialized from |
| [nvidia/stt_en_fastconformer_ctc_large](https://huggingface.co/nvidia/stt_en_fastconformer_ctc_large), then modified and trained for 4.5 epochs |
| (127552 updates) with the balbal-v2-manas recipe. balbal's own weights were not used. |
|
|
|
|
| ## Acknowledgements |
|
|
| The ONNX graphs and the `pip install balbal` setup were contributed by [erayyapagci](https://huggingface.co/erayyapagci), the author of [balbal](https://github.com/erayyap/balbal). |
|
|
|
|
| ## License |
|
|
| [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; credit is required in any project using this model: |
| "speakwrite-azerbaijani by LocalDoc" with a link to this page. For commercial use, contact [LocalDoc](https://huggingface.co/LocalDoc). |
|
|
| Based on `nvidia/stt_en_fastconformer_ctc_large` (CC BY 4.0, NVIDIA) and on code from balbal (Apache 2.0). |
|
|
