lipi
Small, fast, on-device English speech recognition. Streaming-capable, 4-bit weights, 13.6 MB, built for CPU-only runtimes.
For English plus Hinglish (Hindi and English mixed), see hitl-systems/lipi-hn.
Results
Word error rate (%), lower is better. Greedy decoding, no language model, full context, English only.
Measured from the exported 4-bit weights with 8-bit activations. Test data: the hf-audio/open-asr-leaderboard test configurations, using the cleaned versions where the leaderboard provides them.
| Benchmark (test set) | WER |
|---|---|
| LibriSpeech test-clean | 4.55 |
| LibriSpeech test-other | 11.75 |
| Earnings-22 (cleaned, call level) | 17.50 |
| GigaSpeech (cleaned) | 14.80 |
| SPGISpeech | 3.82 |
| AMI (cleaned) | 14.78 |
| VoxPopuli (cleaned) | 6.39 |
| Mean of the seven | 10.51 |
Also: Earnings-22 raw test 20.49, Common Voice (English) 29.38. Streaming (640 ms chunks) costs accuracy: on LibriSpeech dev-clean 4.46 full context versus 5.86 streaming.
Scoring: our own scorer with the Whisper English text normaliser applied to the references; Common Voice audio is resampled from 48 kHz to 16 kHz. This is not the leaderboard's own harness, so an official run may differ slightly. Known weak spots: accented and noisy speech (Common Voice) and phone-quality business calls (Earnings-22).
Optional language model
lm.bin is a word 3-gram (KenLM, 12.2 MB) for CTC beam-search decoding (for example pyctcdecode with the SentencePiece pieces as labels). It is optional: the numbers above are greedy and do not use it.
Measured on the exported 4-bit model's outputs, beam search with and without the model on the same logits, one setting for every set (alpha 0.1, beta 0.0):
| Test set | WER, beam | WER, beam + LM | relative |
|---|---|---|---|
| LibriSpeech test-clean | 4.43 | 3.91 | -11.7 % |
| LibriSpeech test-other | 11.77 | 10.27 | -12.8 % |
On the first 1,500 utterances of the GigaSpeech, SPGISpeech, AMI and VoxPopuli development sets the same setting gave -4.5 %, -3.2 %, -3.4 % and -5.8 % relative. Gains on the other benchmark test sets above have not been measured. Beam search is slower than greedy decoding and needs a CTC beam decoder.
Files
lipi-w4.lipithe 4-bit model (13.6 MB)tokenizer.modelSentencePiece tokenizer (1024 pieces)lm.binoptional word language model (KenLM trie, 12.2 MB)
License
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0) for the model weights and these documents: attribution required, non-commercial use only, and adaptations must be shared under the same licence. Full text in LICENSE.