File size: 5,080 Bytes
8147f61 1258f2f 8147f61 dc85dfa 8147f61 1258f2f 8147f61 1258f2f 8147f61 1258f2f 8147f61 1258f2f 8147f61 1258f2f 8147f61 1258f2f 8147f61 1258f2f 8147f61 8071a6e 8147f61 4ad010e 8147f61 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
language:
- az
license: cc-by-nc-4.0
library_name: nemo
pipeline_tag: automatic-speech-recognition
tags:
- asr
- ctc
- fastconformer
- kenlm
- onnx
- azerbaijani
---
# speakwrite-azerbaijani
Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with a 3-pass loop refiner, product-key acoustic memory
(700 x 700 sub-keys) and a 6-gram token KenLM. Input is 16 kHz mono; output has case and `. , ? !`, with numbers written as words.
## Results
[LocalDoc/fleurs-azerbaijani-asr](https://huggingface.co/datasets/LocalDoc/fleurs-azerbaijani-asr), not used for training. Reference: `text_norm`
(numbers as words). WER and CER ignore case and punctuation. Clips with `norm_status = failed` are excluded (11 of 3977).
WER / CER:
| split | clips | hours | PyTorch, no LM | PyTorch + KenLM | ONNX fast + KenLM |
|---|---|---|---|---|---|
| test | 914 | 3.20 | 16.34 / 4.75 | **10.98 / 3.57** | **11.03 / 3.61** |
| dev | 397 | 1.34 | 16.52 / 4.75 | 11.05 / 3.50 | 11.25 / 3.59 |
| train | 2655 | 9.27 | 15.82 / 4.80 | 11.30 / 3.78 | 11.36 / 3.83 |
| all | 3966 | 13.81 | 16.01 / 4.79 | 11.20 / 3.70 | 11.27 / 3.75 |
- Decoder weights (`alpha` 1.0, `beta` 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote.
- The ONNX plain graph also gives 11.03 / 3.61 on FLEURS test; it was not run on the other splits.
- Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not.
- Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01.
Speed on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM:
| runtime | threads | speed | weights | measured on |
|---|---|---|---|---|
| ONNX fast | 8 | 84-87x | 268 MB | this checkpoint, FLEURS test, dev and train |
| ONNX plain | 8 | ~68x | 548 MB | this checkpoint, FLEURS test |
| PyTorch float32 | 8 | ~45x | 865 MB | this checkpoint, all FLEURS |
| PyTorch float32 | 1 | 12.7x | 865 MB | an earlier checkpoint of this model, 100 clips |
The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time in the PyTorch runs and adds 1 GB to the download for every runtime. ONNX was not measured on 1 thread.
## Use
```bash
pip install balbal
```
```python
import balbal
model = balbal.Model("LocalDoc/speakwrite-azerbaijani", threads=8)
print(model.transcribe("speech.wav")["text"])
```
Tested with balbal 0.3.0 on Python 3.12 and Linux. The first run downloads about 1.3 GB: the fast graph and the KenLM. Pass `runtime="onnx"` for the plain graph, which uses standard ONNX operators only.
`transcribe` takes a file path or a waveform array with its sample rate: `model.transcribe(wave, 16000)`. WAV, FLAC and OGG are read directly; other formats need FFmpeg. The result also has `words`, with `word`, `start`, `end` and `confidence` for each word.
Audio longer than 30 s is cut into 30 s windows with 10 s overlap. `overlap_seconds=0` cuts fixed 30 s chunks instead, which is faster but gave a higher WER on our long recordings.
### PyTorch
`model.pt` is the original checkpoint, for further training and experiments.
Needs [balbal](https://github.com/erayyap/balbal) at commit `1be9f88b77c63fbafa7713c6e46e79847f6c390e` (Python 3.11-3.12, Linux; install
`requirements/full.txt`, then run `inference/build_native.py`). Copy this repository's `refiner.json` over balbal's
`configs/balbal-v2-manas/refiner.json`: this model's memory size differs from balbal's default, and the weights will not load otherwise.
```python
import json, sys
sys.path[:0] = ["balbal/inference", "balbal"]
from balbal.audio import read_audio
from runtime import Engine
engine = Engine("speakwrite-azerbaijani", runtime="torch", settings=json.load(open("speakwrite-azerbaijani/decoder.json")))
print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"])
```
Keep `itn=False`: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks.
To use the PyTorch model without the KenLM, take the argmax of the CTC output instead of `Engine`: without an LM, WER on FLEURS test is 16.34. The `lm/` folder is then not needed.
## Training
About 2800 hours of Azerbaijani speech. Encoder initialized from
[nvidia/stt_en_fastconformer_ctc_large](https://huggingface.co/nvidia/stt_en_fastconformer_ctc_large), then modified and trained for 4.5 epochs
(127552 updates) with the balbal-v2-manas recipe. balbal's own weights were not used.
## Acknowledgements
The ONNX graphs and the `pip install balbal` setup were contributed by [erayyapagci](https://huggingface.co/erayyapagci), the author of [balbal](https://github.com/erayyap/balbal).
## License
[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; credit is required in any project using this model:
"speakwrite-azerbaijani by LocalDoc" with a link to this page. For commercial use, contact [LocalDoc](https://huggingface.co/LocalDoc).
Based on `nvidia/stt_en_fastconformer_ctc_large` (CC BY 4.0, NVIDIA) and on code from balbal (Apache 2.0).
|