Tembra speech recognition for low-resource languages
Armenian, Georgian, Uzbek, Kazakh, Kyrgyz โ languages with little public speech data. Tembra AI builds dedicated recognizers for them with one shared approach. This page compares them with commercial speech-to-text APIs and open models on the public FLEURS and Common Voice test sets (November 2026).
Lowest average word error rate in every language on this page, and fewer errors than both ElevenLabs Scribe v2 and Google Chirp 3 in each of them.
- Armenian: 4.87 % average WER (17 % fewer errors than ElevenLabs, 29 % fewer errors than Google); ElevenLabs Scribe v2 5.87 %, Google Chirp 3 6.90 %; best open model 7.20 % (NVIDIA stt_hy FastConformer) โ the lowest in the table.
- Georgian: 6.57 % average WER (4 % fewer errors than ElevenLabs, 8 % fewer errors than Google); ElevenLabs Scribe v2 6.86 %, Google Chirp 3 7.11 %; best open model 7.71 % (NVIDIA stt_ka FastConformer) โ the lowest in the table (ElevenLabs Scribe v2 is within the confidence interval).
- Uzbek: 6.37 % average WER (50 % fewer errors than ElevenLabs, 28 % fewer errors than Google); ElevenLabs Scribe v2 12.69 %, Google Chirp 3 8.81 %; best open model 6.45 % (GigaAM Multilingual large CTC (Sber)) โ the lowest in the table (GigaAM Multilingual large CTC (Sber) is within the confidence interval).
- Kazakh: 5.73 % average WER (42 % fewer errors than ElevenLabs, 65 % fewer errors than Google); ElevenLabs Scribe v2 9.89 %, Google Chirp 3 16.24 %; best open model 6.81 % (GigaAM Multilingual large CTC (Sber)) โ the lowest in the table.
- Kyrgyz: 5.00 % average WER (62 % fewer errors than ElevenLabs, 63 % fewer errors than Google); ElevenLabs Scribe v2 13.10 %, Google Chirp 3 13.49 %; best open model 5.12 % (GigaAM Multilingual large CTC (Sber)) โ the lowest in the table (GigaAM Multilingual large CTC (Sber) is within the confidence interval).
Average word error rate over the public test sets
WER, %, lower is better: the mean of the FLEURS and Common Voice WER of each language. ยซBest open modelยป is the open model with the lowest average among those we ran for that language (NVIDIA FastConformer, GigaAM Multilingual, Meta Omnilingual ASR, Meta MMS); ยซโยป โ not evaluated on all test sets of the language (Whisper was run on FLEURS only for Armenian, Uzbek and Kazakh โ see the per-language tables โ and does not support Kyrgyz).
| language | test sets | Tembra | ElevenLabs Scribe v2 | Google Chirp 3 | best open model | OpenAI Whisper large-v3 |
|---|---|---|---|---|---|---|
| Armenian (ีีกีตีฅึีฅีถ) | FLEURS + CV | 4.87 | 5.87 | 6.90 | 7.20 (NVIDIA stt_hy FastConformer) | โ |
| Georgian (แฅแแ แแฃแแ) | FLEURS + CV | 6.57 | 6.86 | 7.11 | 7.71 (NVIDIA stt_ka FastConformer) | 67.76 |
| Uzbek (Oสปzbekcha) | FLEURS + CV | 6.37 | 12.69 | 8.81 | 6.45 (GigaAM Multilingual large CTC (Sber)) | โ |
| Kazakh (าะฐะทะฐาัะฐ) | FLEURS + CV | 5.73 | 9.89 | 16.24 | 6.81 (GigaAM Multilingual large CTC (Sber)) | โ |
| Kyrgyz (ะััะณัะทัะฐ) | FLEURS + CV | 5.00 | 13.10 | 13.49 | 5.12 (GigaAM Multilingual large CTC (Sber)) | โ |
Results per language
WER, %, 95 % bootstrap confidence intervals in parentheses (2,000 resamples over FLEURS sentences or Common Voice speakers).
Armenian
| system | FLEURS test | Common Voice 26 test |
|---|---|---|
| Tembra | 4.96 (4.3โ5.6) | 4.78 (4.4โ5.2) |
| ElevenLabs Scribe v2 | 4.45 (3.9โ5.1) | 7.29 (6.9โ7.7) |
| Google Chirp 3 | 5.28 (4.7โ6.0) | 8.51 (8.0โ9.1) |
| NVIDIA stt_hy FastConformer | 9.54 (8.7โ10.5) | 4.87 (4.5โ5.2) |
| OpenAI Whisper large-v3 | 39.88 (38.2โ41.7) | โ |
Test sets: FLEURS test, 418 utterances that are absent from our training data; Common Voice 26 test, 3,446 clips that are absent from our training data.
Georgian
| system | FLEURS test | Common Voice 22 test |
|---|---|---|
| Tembra | 8.66 (7.8โ9.5) | 4.48 (3.9โ5.1) |
| ElevenLabs Scribe v2 | 7.27 (6.6โ8.0) | 6.45 (5.7โ7.3) |
| Google Chirp 3 | 6.56 (5.8โ7.3) | 7.66 (6.7โ8.7) |
| NVIDIA stt_ka FastConformer | 11.12 (10.2โ12.1) | 4.29 (3.8โ4.9) |
| Meta Omnilingual ASR CTC 1B v2 | 17.00 (15.9โ18.1) | 17.33 (16.2โ18.6) |
| Meta MMS-1B-all | 23.57 (22.4โ24.8) | 22.78 (21.5โ24.1) |
| OpenAI Whisper large-v3 | 66.53 (64.8โ68.1) | 69.00 (67.4โ70.6) |
Test sets: FLEURS test, 725 utterances without digits; Common Voice 22 test, random 2,000 clips.
Uzbek
| system | FLEURS test | Common Voice 22 test |
|---|---|---|
| Tembra | 7.28 (6.4โ8.1) | 5.46 (4.8โ6.2) |
| GigaAM Multilingual large CTC (Sber) | 7.47 (6.6โ8.3) | 5.43 (4.7โ6.2) |
| Google Chirp 3 | 7.60 (6.7โ8.5) | 10.03 (9.0โ11.1) |
| ElevenLabs Scribe v2 | 11.60 (10.3โ13.0) | 13.78 (12.4โ15.2) |
| Meta MMS-1B-all | 23.96 (22.4โ25.5) | 20.07 (18.8โ21.4) |
| Meta Omnilingual ASR CTC 1B v2 | 24.41 (22.5โ26.3) | 37.50 (35.3โ39.8) |
| OpenAI Whisper large-v3 | 89.58 (85.5โ94.8) | โ |
Test sets: FLEURS test, 697 utterances without digits; Common Voice 22 test, 2,000 clips (random sample).
Kazakh
| system | FLEURS test | Common Voice 22 test |
|---|---|---|
| Tembra | 3.57 (3.0โ4.3) | 7.88 (6.5โ9.4) |
| GigaAM Multilingual large CTC (Sber) | 3.75 (3.1โ4.5) | 9.87 (7.9โ11.9) |
| ElevenLabs Scribe v2 | 4.40 (3.7โ5.2) | 15.39 (13.3โ17.5) |
| Google Chirp 3 | 5.18 (4.4โ6.0) | 27.30 (24.3โ30.5) |
| Meta Omnilingual ASR CTC 1B v2 | 11.71 (10.7โ12.8) | 24.85 (21.8โ28.0) |
| Meta MMS-1B-all | 13.02 (11.9โ14.2) | 26.75 (24.1โ29.5) |
| OpenAI Whisper large-v3 | 30.24 (28.9โ31.7) | โ |
Test sets: FLEURS test, 655 utterances without digits; Common Voice 22 test, 533 clips.
Kyrgyz
| system | FLEURS test | Common Voice 22 test |
|---|---|---|
| Tembra | 4.76 (4.1โ5.4) | 5.24 (3.9โ6.9) |
| GigaAM Multilingual large CTC (Sber) | 4.96 (4.3โ5.6) | 5.28 (4.0โ7.0) |
| ElevenLabs Scribe v2 | 9.37 (8.4โ10.4) | 16.83 (14.4โ19.7) |
| Google Chirp 3 | 8.69 (7.8โ9.6) | 18.29 (15.6โ21.5) |
| Meta MMS-1B-all | 15.30 (14.2โ16.5) | 23.22 (20.8โ26.2) |
| Meta Omnilingual ASR CTC 1B v2 | 12.65 (11.5โ13.7) | 29.98 (26.8โ33.4) |
Test sets: FLEURS test, 750 utterances without digits; Common Voice 22 test, 1,614 clips.
How we evaluated
- Every system transcribed the same audio files; references and hypotheses go through the same normalization (lowercase, punctuation removed, letters of the language's script only). Utterances whose reference contains digits or words in another script are not scored, for every system alike, because systems write numbers differently. Armenian is scored with the ArmBench-ASR normalization.
- Commercial APIs were called on 5โ6 November 2026 with the language set and every other option at its default (ElevenLabs
scribe_v2; Google Cloud Speech-to-Text v2chirp_3, locationus). Open models were run locally with their default decoding. - Model versions and decoding settings of the Tembra models were chosen on dev splits only, never on test data.
- No test audio or test text was used to train or tune any Tembra model; training text was deduplicated against every test and dev sentence. For Armenian only test utterances that do not occur in our training data (audio or text) are scored.
- The training data of the commercial APIs is not public. NVIDIA's Georgian model was trained on Common Voice 17 train, dev and ยซotherยป (per its model card), its Armenian model on Common Voice 17 and FLEURS; this can favour them on these tests.
Contact
Tembra AI โ https://tembra.ai