Update README.md

README.md

@@ -9,6 +9,7 @@ tags:
 - ctc
 - fastconformer
 - kenlm
+- onnx
 - azerbaijani
 ---
 
@@ -23,28 +24,54 @@ Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with
 [LocalDoc/fleurs-azerbaijani-asr](https://huggingface.co/datasets/LocalDoc/fleurs-azerbaijani-asr), not used for training. Reference: `text_norm`
 (numbers as words). WER and CER ignore case and punctuation. Clips with `norm_status = failed` are excluded (11 of 3977).
 
-| split | clips | hours | no LM: WER / CER | KenLM: WER / CER |
-|---|---|---|---|---|
-| test | 914 | 3.20 | 16.34 / 4.75 | **10.98 / 3.57** |
-| dev | 397 | 1.34 | 16.52 / 4.75 | 11.05 / 3.50 |
-| train | 2655 | 9.27 | 15.82 / 4.80 | 11.30 / 3.78 |
-| all | 3966 | 13.81 | 16.01 / 4.79 | 11.20 / 3.70 |
+WER / CER:
+
+| split | clips | hours | PyTorch, no LM | PyTorch + KenLM | ONNX fast + KenLM |
+|---|---|---|---|---|---|
+| test | 914 | 3.20 | 16.34 / 4.75 | **10.98 / 3.57** | **11.03 / 3.61** |
+| dev | 397 | 1.34 | 16.52 / 4.75 | 11.05 / 3.50 | 11.25 / 3.59 |
+| train | 2655 | 9.27 | 15.82 / 4.80 | 11.30 / 3.78 | 11.36 / 3.83 |
+| all | 3966 | 13.81 | 16.01 / 4.79 | 11.20 / 3.70 | 11.27 / 3.75 |
 
 - Decoder weights (`alpha` 1.0, `beta` 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote.
+- The ONNX plain graph also gives 11.03 / 3.61 on FLEURS test; it was not run on the other splits.
 - Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not.
 - Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01.
 
-Speed, PyTorch float32 on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM:
+Speed on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM:
 
-| threads | speed | measured on |
-|---|---|---|
-| 1 | 12.7x | an earlier checkpoint of this model, 100 clips |
-| 8 | ~45x | this checkpoint, all FLEURS |
+| runtime | threads | speed | weights | measured on |
+|---|---|---|---|---|
+| ONNX fast | 8 | 84-87x | 268 MB | this checkpoint, FLEURS test, dev and train |
+| ONNX plain | 8 | ~68x | 548 MB | this checkpoint, FLEURS test |
+| PyTorch float32 | 8 | ~45x | 865 MB | this checkpoint, all FLEURS |
+| PyTorch float32 | 1 | 12.7x | 865 MB | an earlier checkpoint of this model, 100 clips |
 
-The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time. ONNX runtimes were not measured.
+The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time in the PyTorch runs and adds 1 GB to the download for every runtime. ONNX was not measured on 1 thread.
 
 ## Use
 
+```bash
+pip install balbal
+```
+
+```python
+import balbal
+
+model = balbal.Model("LocalDoc/speakwrite-azerbaijani", threads=8)
+print(model.transcribe("speech.wav")["text"])
+```
+
+Tested with balbal 0.3.0 on Python 3.12 and Linux. The first run downloads about 1.3 GB: the fast graph and the KenLM. Pass `runtime="onnx"` for the plain graph, which uses standard ONNX operators only.
+
+`transcribe` takes a file path or a waveform array with its sample rate: `model.transcribe(wave, 16000)`. WAV, FLAC and OGG are read directly; other formats need FFmpeg. The result also has `words`, with `word`, `start`, `end` and `confidence` for each word.
+
+Audio longer than 30 s is cut into 30 s windows with 10 s overlap. `overlap_seconds=0` cuts fixed 30 s chunks instead, which is faster but gave a higher WER on our long recordings.
+
+### PyTorch
+
+`model.pt` is the original checkpoint, for further training and experiments.
+
 Needs [balbal](https://github.com/erayyap/balbal) at commit `1be9f88b77c63fbafa7713c6e46e79847f6c390e` (Python 3.11-3.12, Linux; install
 `requirements/full.txt`, then run `inference/build_native.py`). Copy this repository's `refiner.json` over balbal's
 `configs/balbal-v2-manas/refiner.json`: this model's memory size differs from balbal's default, and the weights will not load otherwise.
@@ -61,7 +88,7 @@ print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"])
 
 Keep `itn=False`: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks.
 
-To use the model without the KenLM, take the argmax of the CTC output instead of `Engine`: without an LM, WER on FLEURS test is 16.34. The `lm/` folder is then not needed.
+To use the PyTorch model without the KenLM, take the argmax of the CTC output instead of `Engine`: without an LM, WER on FLEURS test is 16.34. The `lm/` folder is then not needed.
 
 ## Training