--- license: other license_name: anonymous-review-only license_link: LICENSE library_name: nemo pipeline_tag: automatic-speech-recognition language: - bg - hr - cs - da - nl - en - et - fi - fr - de - el - hu - it - lv - lt - mt - pl - pt - ro - ru - sk - sl - es - sv - uk tags: - automatic-speech-recognition - speech - FastConformer - TDT - knowledge-distillation - multilingual - anonymous-review --- # CAFE-25lang > **This repository is for review only.** It is provided solely for reference by ARR reviewers and > chairs, and no license is granted for any use of the released content, model, code, or results. > It will be deleted after the review process ends and replaced by the official open-source Hugging > Face model release under the Apache 2.0 license. **Anonymous model repository for the ARR submission *"CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition"*.** CAFE-25lang is the 25-language CAFE model of the paper (*CAFE (ours)* in its tables). It is a compact FastConformer-TDT transducer distilled from [Parakeet-TDT-0.6B-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3). CAFE (**C**onfined-**A**dapter **Fe**ature distillation) sends the CKA feature-distillation gradient only into language-routed adapters that English bypasses, so the shared encoder never receives it. - **Size:** 254.45M parameters: a 253.95M shared model plus 0.495M of adapters. The optional language gate adds 0.495M (0.19%). - **Architecture:** FastConformer encoder (24 layers, 640-dim) with a TDT decoder, initialized by structured pruning of the 627M teacher. - **Adapters:** residual bottleneck adapters (rank 64) after encoder layers 3, 7, 11, 15, 19 and 23, applied to non-English utterances only. - **Training data:** about 116.9M utterances (564k hours) in 25 languages from 10 corpora. - **Checkpoint:** weight average of the last 7 checkpoints (*CA* in the paper). **Languages (25):** Bulgarian (`bg`), Croatian (`hr`), Czech (`cs`), Danish (`da`), Dutch (`nl`), English (`en`), Estonian (`et`), Finnish (`fi`), French (`fr`), German (`de`), Greek (`el`), Hungarian (`hu`), Italian (`it`), Latvian (`lv`), Lithuanian (`lt`), Maltese (`mt`), Polish (`pl`), Portuguese (`pt`), Romanian (`ro`), Russian (`ru`), Slovak (`sk`), Slovenian (`sl`), Spanish (`es`), Swedish (`sv`), Ukrainian (`uk`). ## Files | File | Contents | | --- | --- | | `CAFE-25lang.nemo` | NeMo checkpoint of the shared model (253.95M); on its own it is the English path | | `adapters.safetensors` | The six language-routed adapters (0.495M), attached for non-English audio | | `language_gate.safetensors` | Optional language gate (0.495M), for audio whose language is unknown | | `inference.py` | Inference when the language is known (`--lang en` or `--lang other`) | | `inference_gated.py` | Inference with the language gate; no language flag needed | | `eval/` | Scripts for the Open ASR Leaderboard and Multilingual LibriSpeech numbers | | `SHA256SUMS` | Checksums of all files above | ## Usage Install the dependencies; audio must be 16 kHz mono WAV: ```bash pip install "nemo_toolkit[asr]" torch safetensors soundfile ``` CAFE routes by language: English runs on the shared model alone, every other language also needs the adapters. Routing the wrong way does not fail, it only costs accuracy (see [Routing and the language gate](#routing-and-the-language-gate)). There are two ways to run it. ### 1. Without the gate: language known The caller says whether the audio is English. This is the setting of the main results in the paper. ```bash python inference.py --lang en audio_en.wav # shared model only python inference.py --lang other audio_de.wav audio_pt.wav # adapters attached ``` The same from Python: ```python import torch import nemo.collections.asr as nemo_asr from inference import attach_adapters, safe_decoding model = nemo_asr.models.ASRModel.restore_from("CAFE-25lang.nemo", map_location="cpu").eval() safe_decoding(model) for h in model.transcribe(["audio_en.wav"]): # English: no adapters print(h.text) attach_adapters(model, "adapters.safetensors", torch.device("cpu"), torch.float32) for h in model.transcribe(["audio_de.wav", "audio_pt.wav"]): # other languages: adapters on print(h.text) ``` ### 2. With the gate: language unknown The gate decides per utterance. It reads encoder layer 11 with the adapters off and predicts whether the utterance is non-English. Utterances with p(non-English) ≥ 0.8582 are then transcribed with the adapters, the rest without: ```bash python inference_gated.py audio_en.wav audio_de.wav audio_pt.wav python inference_gated.py --json audio_en.wav audio_de.wav # one JSON object per line ``` Each output line gives the gate probability, the chosen path (`EN` or `ML`) and the transcript. `--threshold` overrides the threshold of 0.8582 stored in `language_gate.safetensors`. The gate uses at most the first 20 s of each file. Both scripts default to the files in this directory, use the GPU when one is available, and accept `--model`, `--adapters`, `--device` and `--batch_size`. ## Results All numbers are taken from the paper; accuracy is word error rate (WER, %, lower is better). Every system is decoded and scored with the same pipeline and the official Open ASR Leaderboard text normalizer. *EN-7* is the mean over the seven Open ASR Leaderboard English short-form sets and *ML-9* the mean of its multilingual sets (FLEURS mean and MLS mean, five and four languages). *Avg* is the mean of EN-7 and ML-9. Unless stated otherwise, the language of each test set decides whether the adapters are used. ### Comparison with external systems Mean WER per test suite on 48 test sets in 24 languages, with model size and inference cost on one NVIDIA H100 (efficiency protocol in the paper's appendix). Bold marks the best result and underline the second best within each size group: the large group (Whisper-medium to omniASR-CTC 1B v2) and the compact group (Whisper-small to CAFE-25lang). The teacher is shown for reference. | Metric | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | Params (M) | 627 | 764 | 1,632 | 1,011 | 782 | 976 | 242 | 326 | 370 | 254 | | Number of languages | 25 | ~100 | >1600 | 75 | 30 | >1600 | ~100 | >1600 | 75 | 25 | | RTFx bs1 ↑ | 198 | 21 | 10 | 590 | 18 | 513 | 35 | 848 | 39 | 213 | | RTFx bs32 ↑ | 2,520 | 386 | 117 | 1,282 | 310 | 2,142 | 546 | 3,405 | 37 | 2,602 | | P50 lat. bs1 (s) ↓ | 0.095 | 0.704 | 1.990 | 0.033 | 1.070 | 0.037 | 0.417 | 0.024 | 0.501 | 0.089 | | Peak mem (MB) ↓ | 3,310 | 6,932 | 17,310 | 25,803 | 6,266 | 16,033 | 2,617 | 14,802 | 2,505 | 2,593 | | EN short-form (7) | 6.83 | 11.21 | 11.28 | 7.90 | **6.50** | 12.21 | 13.91 | 17.94 | **8.73** | 9.03 | | Multilingual (9) | 5.14 | **6.36** | 7.58 | 8.74 | 8.29 | 7.16 | 10.76 | 12.68 | 10.71 | **7.00** | | EN long-form (4) | 10.77 | 16.28 | 23.50 | 12.22 | **10.91** | 54.82 | 16.71 | 42.45 | **13.01** | 13.35 | | MLS-8 (8) | 6.66 | 7.76 | 7.79 | 10.65 | 12.60 | **7.71** | 11.51 | 11.89 | 11.84 | **6.58** | | Common Voice v25 (24) | 16.48 | 26.94 | **17.72** | 32.87 | N/A | 20.07 | 38.17 | 32.23 | 37.74 | **21.56** | Qwen3-ASR-0.6B has no 24-language Common Voice mean because some languages are outside its supported set. ### Main comparison: Open ASR Leaderboard All students share the same data, schedule, and model size, and differ only in the adapters and the feature-distillation loss. *CAFE w/o sg* is CAFE without the stop-gradient. *mean ± std* is over the last 7 checkpoints and *CA* is their weight average; this repository contains the CA model of the *CAFE (ours)* row. The 8-language pair is trained on the 8 MLS languages only and is not part of this repository. | System | Languages | Adapters | Feature KD | EN-7 mean ± std | EN-7 CA | ML-9 mean ± std | ML-9 CA | Avg mean ± std | Avg CA | |---|---|---|---|---|---|---|---|---|---| | Baseline | 25 | – | – | 10.33 ± 0.82 | 9.85 | 7.59 ± 0.20 | 7.24 | 8.96 ± 0.39 | 8.55 | | CKA | 25 | – | CKA | 11.42 ± 0.67 | 11.25 | **7.22 ± 0.12** | **6.83** | 9.32 ± 0.33 | 9.04 | | Capacity | 25 | ✓ | – | 9.98 ± 0.53 | 9.67 | 7.77 ± 0.11 | 7.34 | 8.88 ± 0.27 | 8.51 | | CAFE w/o sg | 25 | ✓ | CKA (no sg) | 10.75 ± 0.69 | 10.44 | 7.37 ± 0.12 | 6.95 | 9.06 ± 0.32 | 8.70 | | **CAFE (ours)** | 25 | ✓ | CKA (confined) | **9.32 ± 0.32** | **9.03** | 7.43 ± 0.13 | 7.00 | **8.38 ± 0.19** | **8.02** | | Baseline-8lang | 8 | – | – | 10.32 ± 0.54 | 10.11 | 7.08 ± 0.15 | 6.62 | 8.70 ± 0.22 | 8.36 | | **CAFE-8lang (ours)** | 8 | ✓ | CKA (confined) | **8.93 ± 0.27** | **8.71** | **6.91 ± 0.12** | **6.53** | **7.92 ± 0.13** | **7.62** | ### Adapter overhead | | **CAFE-25lang** | Baseline (same model without adapters) | |---|---|---| | Params | 254.45M | 253.95M | | RTFx bs1 ↑ | 212.9 ± 6.9 | 221.3 ± 5.2 | | RTFx bs32 ↑ | 2602.3 ± 68.9 | 2656.6 ± 79.8 | | P50 lat. (s) ↓ | 0.089 ± 0.001 | 0.088 ± 0.000 | | Peak mem (MB) ↓ | 2593.4 | 2592.5 | RTFx is the inverse real-time factor, P50 the median latency per utterance at batch size 1, and peak memory is measured at batch size 32, all on one NVIDIA H100. ### Routing and the language gate Gate accuracy on 7,692 utterances, 500 sampled from each of the 16 EN-7 and ML-9 test sets (all 192 for MLS pt): | Overall | English | Non-English | |---|---|---| | 98.9% | 97.6% | 99.9% | WER when the English/non-English decision comes from the test set (language given) or from the gate (language unknown). The last two columns apply the adapters to every utterance or to none: | Open ASR Leaderboard | Baseline (no adapters, no CKA) | CAFE, language given | CAFE + gate, language unknown | Adapters always on | Adapters never on | |---|---|---|---|---|---| | EN-7 mean | 9.85 | 9.03 | 9.11 | 15.95 | 9.03 | | ML-9 mean | 7.24 | 7.00 | 7.03 | 7.00 | 8.04 | | Avg (EN-7, ML-9) | 8.55 | 8.02 | 8.07 | 11.48 | 8.54 | With the gate the average WER rises by 0.05 (8.02 to 8.07). Applying the adapters to English is costly (EN-7 15.95 against 9.03), while never applying them loses the multilingual gain (ML-9 8.04 against 7.00).
Per-dataset WER with and without the gate | Test set | Baseline (no adapters, no CKA) | CAFE, language given | CAFE + gate, language unknown | Adapters always on | Adapters never on | |---|---|---|---|---|---| | **EN-7 mean** | 9.85 | 9.03 | 9.11 | 15.95 | 9.03 | | **FLEURS mean** | 7.22 | 7.00 | 7.04 | 7.00 | 8.46 | | **MLS mean** | 7.27 | 7.01 | 7.01 | 7.01 | 7.62 | | **Multilingual mean** | 7.24 | 7.00 | 7.03 | 7.00 | 8.04 | | **Avg (EN, ML)** | 8.55 | 8.02 | 8.07 | 11.48 | 8.54 |
### Per-dataset results Every test set behind the summary table, against the same external systems. Long-form results for omniASR and Qwen3-ASR-0.6B use 40 s non-overlapping windows, the maximum input length of omniASR; Qwen3-ASR-0.6B degenerated with its native 1200 s windows.
Model size and inference cost | Metric | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | Number of languages | 25 | ~100 | >1600 | 75 | 30 | >1600 | ~100 | >1600 | 75 | 25 | | Params (M) ↓ | 627 | 764 | 1,632 | 1,011 | 782 | 976 | 242 | 326 | 370 | 254 | | RTFx bs1 ↑ | 198 | 21 | 10 | 590 | 18 | 513 | 35 | 848 | 39 | 213 | | RTFx bs32 ↑ | 2,520 | 386 | 117 | 1,282 | 310 | 2,142 | 546 | 3,405 | 37 | 2,602 | | P50 lat. bs1 (s) ↓ | 0.095 | 0.704 | 1.990 | 0.033 | 1.070 | 0.037 | 0.417 | 0.024 | 0.501 | 0.089 | | Peak mem (MB) ↓ | 3,310 | 6,932 | 17,310 | 25,803 | 6,266 | 16,033 | 2,617 | 14,802 | 2,505 | 2,593 |
Open ASR Leaderboard - English short-form (7) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | ami | 11.44 | 38.96 | 26.03 | 13.12 | 10.96 | 24.24 | 53.35 | 32.51 | 15.47 | 13.91 | | earnings22 | 11.19 | 11.49 | 16.90 | 13.89 | 10.76 | 19.79 | 13.21 | 26.33 | 14.48 | 13.55 | | gigaspeech | 9.57 | 10.05 | 12.83 | 10.87 | 8.37 | 15.80 | 10.56 | 22.84 | 11.17 | 11.72 | | LibriSpeech.clean | 1.93 | 2.58 | 2.16 | 2.60 | 1.75 | 2.83 | 2.59 | 4.71 | 2.56 | 2.48 | | LibriSpeech.other | 3.59 | 4.87 | 6.54 | 4.88 | 4.08 | 6.67 | 6.60 | 11.86 | 5.98 | 5.81 | | spgispeech | 3.98 | 2.92 | 5.82 | 2.57 | 2.79 | 7.37 | 3.24 | 14.05 | 2.79 | 7.98 | | voxpopuli | 6.14 | 7.63 | 8.71 | 7.35 | 6.82 | 8.79 | 7.85 | 13.28 | 8.65 | 7.76 | | **category mean (7)** | 6.83 | 11.21 | 11.28 | 7.90 | **6.50** | 12.21 | 13.91 | 17.94 | **8.73** | 9.03 |
Open ASR Leaderboard - English long-form (4 sets; CORAAL as 8 subsets) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | earnings21 | 10.21 | 12.52 | 21.30 | 9.82 | 9.20 | 52.02 | 14.07 | 31.14 | 10.80 | 12.31 | | earnings22-lf | 14.52 | 16.73 | 24.31 | 13.62 | 13.59 | 53.60 | 17.98 | 40.19 | 15.24 | 14.76 | | tedlium | 2.74 | 3.10 | 11.20 | 3.38 | 2.89 | 52.95 | 3.52 | 31.48 | 4.06 | 4.08 | | coraal avg (8) | 15.59 | 32.76 | 37.20 | 22.06 | 17.98 | 60.71 | 31.27 | 66.97 | 21.94 | 22.25 | |   CORAAL ATL | 16.69 | 39.45 | 40.75 | 24.11 | 18.20 | 55.15 | 34.57 | 72.58 | 23.63 | 22.77 | |   CORAAL DCA | 19.00 | 39.57 | 40.81 | 26.22 | 21.71 | 64.03 | 39.34 | 67.28 | 26.05 | 25.86 | |   CORAAL DCB | 11.72 | 23.40 | 34.54 | 16.93 | 13.70 | 60.98 | 25.31 | 63.50 | 17.74 | 18.35 | |   CORAAL DTA | 19.61 | 36.16 | 42.01 | 29.29 | 22.76 | 62.19 | 38.86 | 75.00 | 27.95 | 28.13 | |   CORAAL LES | 15.47 | 33.06 | 36.95 | 21.43 | 18.04 | 58.62 | 29.94 | 65.49 | 20.86 | 21.77 | |   CORAAL PRV | 22.36 | 43.39 | 46.82 | 31.41 | 26.92 | 72.71 | 41.13 | 79.68 | 30.64 | 31.01 | |   CORAAL ROC | 8.16 | 19.41 | 23.31 | 10.96 | 9.41 | 64.91 | 16.46 | 54.70 | 11.61 | 12.75 | |   CORAAL VLD | 11.71 | 27.64 | 32.38 | 16.11 | 13.11 | 47.10 | 24.51 | 57.57 | 17.06 | 17.38 | | **category mean (4)** | 10.77 | 16.28 | 23.50 | 12.22 | **10.91** | 54.82 | 16.71 | 42.45 | **13.01** | 13.35 |
Open ASR Leaderboard - Multilingual: FLEURS (5) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | de | 4.25 | 4.66 | 7.99 | 8.26 | 5.54 | 7.46 | 7.61 | 14.53 | 10.08 | 7.61 | | es | 3.27 | 3.12 | 5.39 | 5.59 | 4.47 | 5.33 | 4.67 | 9.56 | 8.21 | 5.38 | | fr | 4.72 | 6.89 | 11.25 | 9.79 | 7.07 | 10.84 | 11.52 | 21.23 | 13.16 | 9.83 | | it | 2.45 | 3.56 | 5.47 | 5.76 | 4.62 | 4.87 | 16.02 | 9.46 | 6.87 | 4.43 | | pt | 4.54 | 4.29 | 7.70 | 7.42 | 6.08 | 7.19 | 6.27 | 13.63 | 8.76 | 7.74 | | **category mean (5)** | 3.85 | **4.50** | 7.56 | 7.36 | 5.56 | 7.14 | 9.22 | 13.68 | 9.42 | **7.00** |
Open ASR Leaderboard - Multilingual: MLS (4) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | es | 4.40 | 4.77 | 4.80 | 6.44 | 6.80 | 4.20 | 6.89 | 7.25 | 7.61 | 4.74 | | fr | 5.13 | 7.17 | 6.82 | 8.31 | 8.54 | 6.94 | 10.93 | 12.95 | 11.05 | 4.85 | | it | 8.52 | 10.58 | 7.25 | 14.14 | 14.76 | 6.51 | 16.02 | 10.73 | 14.49 | 7.37 | | pt | 7.67 | 10.33 | 11.50 | 11.55 | 14.03 | 11.05 | 15.37 | 15.81 | 14.83 | 11.07 | | **category mean (4)** | 6.43 | 8.21 | 7.59 | 10.11 | 11.03 | **7.18** | 12.30 | 11.69 | 12.00 | **7.01** | | **EN short-form & Multilingual mean** | 5.99 | 8.79 | 9.43 | 8.32 | **7.40** | 9.68 | 12.34 | 15.31 | 9.72 | **8.02** |
Multilingual LibriSpeech (MLS-8) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | en | 4.82 | 6.21 | 8.68 | 6.75 | 5.77 | 9.20 | 7.66 | 15.10 | 7.96 | 6.48 | | de | 5.79 | 5.75 | 7.02 | 10.36 | 8.18 | 6.69 | 9.35 | 11.02 | 10.43 | 6.59 | | nl | 11.24 | 11.42 | 6.92 | 11.22 | 16.69 | 7.93 | 16.54 | 12.89 | 11.64 | 5.89 | | pl | 5.74 | 5.89 | 9.32 | 16.39 | 26.02 | 9.18 | 9.26 | 9.39 | 16.72 | 5.63 | | es | 4.39 | 4.76 | 4.80 | 6.44 | 6.80 | 4.20 | 6.90 | 7.25 | 7.61 | 4.74 | | fr | 5.07 | 7.18 | 6.82 | 8.31 | 8.54 | 6.94 | 10.94 | 12.95 | 11.05 | 4.87 | | it | 8.53 | 10.60 | 7.25 | 14.14 | 14.76 | 6.51 | 16.06 | 10.73 | 14.49 | 7.37 | | pt | 7.69 | 10.28 | 11.50 | 11.55 | 14.03 | 11.05 | 15.37 | 15.81 | 14.83 | 11.04 | | **mean (8)** | 6.66 | 7.76 | 7.79 | 10.65 | 12.60 | **7.71** | 11.51 | 11.89 | 11.84 | **6.58** |
Common Voice v25 (24) | Test set | Parakeet-TDT-0.6B-v3 (teacher) | Whisper-medium | omniASR-LLM 300M v2 | OWSM-CTC v4 | Qwen3-ASR-0.6B | omniASR-CTC 1B v2 | Whisper-small | omniASR-CTC 300M v2 | OWSM v4 small | **CAFE-25lang (this model)** | |---|---|---|---|---|---|---|---|---|---|---| | bg | 18.21 | 23.76 | 12.87 | 46.54 | N/A | 16.64 | 41.00 | 30.08 | 52.54 | 18.20 | | cs | 5.92 | 18.30 | 9.61 | 23.05 | 42.79 | 10.53 | 33.55 | 20.33 | 27.39 | 12.47 | | da | 40.38 | 44.38 | 40.56 | 66.55 | 55.37 | 42.77 | 52.21 | 52.46 | 74.82 | 43.70 | | de | 4.40 | 7.83 | 10.58 | 8.83 | 8.30 | 10.02 | 13.97 | 19.29 | 10.71 | 8.83 | | el | 18.23 | 22.76 | 13.51 | 62.36 | 52.11 | 25.40 | 29.88 | 37.23 | 65.56 | 32.80 | | en | 8.47 | 12.38 | 18.94 | 11.45 | 9.67 | 19.78 | 17.18 | 31.20 | 14.22 | 13.94 | | es | 3.76 | 7.15 | 7.53 | 7.20 | 6.58 | 7.83 | 11.63 | 14.23 | 9.34 | 7.22 | | et | 22.29 | 44.33 | 22.62 | 53.75 | N/A | 20.79 | 69.50 | 39.42 | 52.14 | 27.37 | | fi | 48.76 | 53.14 | 51.97 | 64.14 | 72.67 | 52.67 | 58.76 | 58.49 | 67.07 | 54.29 | | fr | 7.33 | 18.85 | 14.28 | 12.31 | 11.75 | 15.18 | 26.92 | 24.82 | 16.22 | 11.64 | | hu | 6.55 | 18.90 | 11.02 | 35.12 | 55.36 | 13.44 | 36.10 | 26.10 | 36.78 | 11.79 | | it | 3.59 | 8.62 | 8.97 | 8.18 | 8.70 | 8.78 | 16.81 | 16.28 | 10.24 | 7.15 | | lt | 16.60 | 50.51 | 22.56 | 60.22 | N/A | 26.23 | 73.12 | 45.25 | 65.21 | 25.68 | | lv | 36.08 | 33.94 | 18.87 | 55.13 | N/A | 21.05 | 55.98 | 36.61 | 58.51 | 17.79 | | mt | 12.16 | 93.16 | 11.79 | 62.61 | N/A | 17.77 | 109.40 | 31.01 | 73.66 | 46.73 | | nl | 4.21 | 6.53 | 6.98 | 8.07 | 10.11 | 7.70 | 12.66 | 16.74 | 12.11 | 6.04 | | pl | 8.53 | 15.31 | 15.81 | 20.25 | 30.20 | 15.53 | 20.74 | 25.53 | 24.85 | 15.91 | | pt | 8.04 | 9.85 | 12.16 | 8.91 | 10.92 | 12.87 | 22.75 | 23.69 | 13.01 | 10.94 | | ro | 10.79 | 22.13 | 13.35 | 20.10 | 42.23 | 12.97 | 31.82 | 23.80 | 34.09 | 22.46 | | ru | 5.54 | 9.44 | 12.93 | 9.06 | 12.96 | 14.86 | 16.42 | 32.45 | 14.68 | 14.62 | | sk | 8.36 | 40.31 | 15.70 | 33.79 | N/A | 25.43 | 52.97 | 34.31 | 41.75 | 21.68 | | sl | 49.60 | 33.42 | 17.71 | 34.86 | N/A | 25.05 | 45.02 | 48.08 | 42.90 | 26.51 | | sv-SE | 17.59 | 12.13 | 17.90 | 35.41 | 37.47 | 17.22 | 21.15 | 33.36 | 41.65 | 17.99 | | uk | 30.09 | 39.44 | 37.12 | 41.03 | N/A | 41.06 | 46.62 | 52.84 | 46.21 | 41.70 | | **mean all (24)** | 16.48 | 26.94 | **17.72** | 32.87 | N/A | 20.07 | 38.17 | 32.23 | 37.74 | **21.56** |
## Evaluation [`eval/README.md`](eval/README.md) shows how to reproduce the Open ASR Leaderboard and Multilingual LibriSpeech numbers with the official leaderboard runners and normalizer. Decoding is greedy, with the CUDA-graph decoder disabled.