README.md

metadata

language:
  - az
license: cc-by-nc-4.0
library_name: nemo
pipeline_tag: automatic-speech-recognition
tags:
  - asr
  - ctc
  - fastconformer
  - kenlm
  - azerbaijani

speakwrite-azerbaijani

Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with a 3-pass loop refiner, product-key acoustic memory (700 x 700 sub-keys) and a 6-gram token KenLM. Input is 16 kHz mono; output has case and . , ? !, with numbers written as words.

Results

LocalDoc/fleurs-azerbaijani-asr, not used for training. Reference: text_norm (numbers as words). WER and CER ignore case and punctuation. Clips with norm_status = failed are excluded (11 of 3977).

split	clips	hours	no LM: WER / CER	KenLM: WER / CER
test	914	3.20	16.34 / 4.75	10.98 / 3.57
dev	397	1.34	16.52 / 4.75	11.05 / 3.50
train	2655	9.27	15.82 / 4.80	11.30 / 3.78
all	3966	13.81	16.01 / 4.79	11.20 / 3.70

• Decoder weights (alpha 1.0, beta 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote.
• Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not.
• Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01.

Speed, PyTorch float32 on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM:

threads	speed	measured on
1	12.7x	an earlier checkpoint of this model, 100 clips
8	~45x	this checkpoint, all FLEURS

The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time. ONNX runtimes were not measured.

Use

Needs balbal at commit 1be9f88b77c63fbafa7713c6e46e79847f6c390e (Python 3.11-3.12, Linux; install requirements/full.txt, then run inference/build_native.py). Copy this repository's refiner.json over balbal's configs/balbal-v2-manas/refiner.json: this model's memory size differs from balbal's default, and the weights will not load otherwise.

import json, sys
sys.path[:0] = ["balbal/inference", "balbal"]
from balbal.audio import read_audio
from runtime import Engine

engine = Engine("speakwrite-azerbaijani", runtime="torch", settings=json.load(open("speakwrite-azerbaijani/decoder.json")))
print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"])

Keep itn=False: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks.

To use the model without the KenLM, take the argmax of the CTC output instead of Engine: without an LM, WER on FLEURS test is 16.34. The lm/ folder is then not needed.

Training

About 2800 hours of Azerbaijani speech. Encoder initialized from nvidia/stt_en_fastconformer_ctc_large, then modified and trained for 4.5 epochs (127552 updates) with the balbal-v2-manas recipe. balbal's own weights were not used.

License

CC BY-NC 4.0. Non-commercial use only; credit is required in any project using this model: "speakwrite-azerbaijani by LocalDoc" with a link to this page. For commercial use, contact LocalDoc.

Based on nvidia/stt_en_fastconformer_ctc_large (CC BY 4.0, NVIDIA) and on code from balbal (Apache 2.0).