Parakeet-Air

A 4-bit build of NVIDIA's Parakeet TDT 0.6B v3 speech recognition model for 25 European languages, in GGUF for transcribe.cpp. It has the same size (485,425,504 bytes) and the same tensor types as handy-computer's widely used parakeet-tdt-0.6b-v3-Q4_K_M.gguf, and it loads in unmodified transcribe.cpp. In a pre-registered test on 22,766 utterances (FLEURS in 25 languages plus LibriSpeech), its word error rate was within 0.5 points of full precision (+0.21 points) where handy-computer's Q4_K_M file is +0.47, and its transcripts were 26% closer to full precision's than that file's.

Model Released under the webAI Non-Commercial License ver. 1.0; see License and attribution.

At a glance

Passes all four rules of the pre-registered test.

F32 (full precision) handy-computer Q4_K_M Parakeet-Air
File size 2,508,435,616 B 485,425,504 B 485,425,504 B
Word error rate, all 22,766 utterances 11.59% 12.06% 11.80%
Word error rate, FLEURS test, 25 languages 12.80% 13.33% 13.04%
Word error rate, LibriSpeech test-clean 1.96% 1.98% 1.94%
Words that differ from F32's transcript, per 100 words 0 (itself) 4.81 3.54
Test sets where it beats handy-computer's Q4_K_M (95% interval below zero) 8 of 26, worse in none

transcribe.cpp 5700f127, CUDA backend on an NVIDIA H200, greedy decoding, timestamps and inverse text normalization off; Whisper text normalizers; one run per file. Word error rates are pooled (total errors over total words). See Evaluation protocol.

Word error rate for F32, handy-computer's Q4_K_M and Parakeet-Air

Highlights

  • Within half a point of full precision. Pooled word error rate 11.80% vs F32's 11.59%, +0.21 points (95% CI +0.16 to +0.26), inside the pre-registered 0.5-point margin.
  • Lower word error than handy-computer's Q4_K_M. −0.26 points (95% CI −0.32 to −0.20) overall, about 55% of that file's increase over F32 removed. It is significantly lower in 8 test sets (Czech, Danish, Croatian, Lithuanian, Latvian, Maltese, Dutch, Romanian) and significantly higher in none.
  • Closer to full precision's transcripts. 3.54 words per 100 differ from F32's transcript, against 4.81 for handy-computer's Q4_K_M: −26% (word edits per sentence −0.70, 95% CI −0.74 to −0.66), lower in all 26 test sets.
  • Same size, same runtime. Same bytes and tensor types as handy-computer's Q4_K_M; it loads in stock transcribe.cpp with no code changes.

It is not identical to full precision: its word error rate is still significantly higher than F32's overall and in 10 of the 26 test sets, and handy-computer's 740 MB Q8_0 shows no measurable loss. See Limitations.

Model Details

Property Value
Model ID webAI-Official/Parakeet-Air (proposed; not yet published)
Base model nvidia/parakeet-tdt-0.6b-v3, 0.6B parameters (FastConformer encoder, TDT decoder), via handy-computer's F32 GGUF conversion in handy-computer/parakeet-tdt-0.6b-v3-gguf
Model file Parakeet-Air-Q4_K_M.gguf, 485,425,504 bytes (sha256 14c12436640ccdd6…)
Format GGUF, file type Q4_K_M: 194 matrices in Q4_K, 31 in Q8_0 (attention output, prediction network, joint network), the 48 pointwise convolutions in F16 and the remaining small tensors in F32, exactly as in handy-computer's Q4_K_M. The file's metadata is the same as handy-computer's, so it reports its name as "Parakeet TDT 0.6B v3"
Runs on transcribe.cpp. Evaluated with the CUDA backend (commit 5700f127, NVIDIA H200); also run with the Metal backend on Apple silicon
Input / output 16 kHz mono audio in any of the 25 languages / text with punctuation and capitalization; language detected automatically
License webAI Non-Commercial License ver. 1.0; based on NVIDIA's CC-BY-4.0 weights

Results

Every result comes from one test, run once per file: each file transcribed the same 22,766 utterances (FLEURS test in 25 languages, 20,146 utterances, plus LibriSpeech test-clean, 2,620) with stock transcribe.cpp on the same GPU. Verdicts pair the same utterances and use a 95% bootstrap interval that resamples groups of related utterances (FLEURS by sentence, since several speakers read each sentence; LibriSpeech by speaker and chapter), 8,586 groups in all. "F32" is handy-computer's full-precision conversion of NVIDIA's weights; "handy-computer Q8_0" and "handy-computer Q4_K_M" are files from the same repository; "transcribe.cpp quantizer, Q4_K_M" is a second same-size file, made from the same F32 file with transcribe.cpp's standard quantizer. A second F32 run gave identical transcripts, so runtime noise is zero.

Pre-registered test

Parakeet-Air passes all four rules, which were written down before the test ran.

Check Rule Result (95% CI) Pass
Harness: F32 and handy-computer Q4_K_M on LibriSpeech test-clean within 0.10 points of handy-computer's published 1.95% and 1.98% 1.96% and 1.98% yes
Primary: Parakeet-Air vs F32, word error rate upper end of the 95% interval below +0.5 points 11.59% → 11.80%, +0.21 (+0.16 to +0.26) yes
Primary: Parakeet-Air vs handy-computer Q4_K_M, word edits from F32's transcript per sentence upper end of the 95% interval below 0 2.65 → 1.95, −0.70 (−0.74 to −0.66) yes
Guardrail: Parakeet-Air vs handy-computer Q4_K_M, word error rate not significantly worse 12.06% → 11.80%, −0.26 (−0.32 to −0.20) yes

These four results are confirmatory. Everything else on this page (per-language rows and the other files) is supporting detail and should be read as preliminary.

Word error rate

Parakeet-Air is between full precision and the other two 4-bit files; the 8-bit file matches full precision.

File Size Word error rate, all 22,766 Change from F32 (95% interval) FLEURS, 25 languages LibriSpeech test-clean
F32 (full precision) 2,508 MB 11.59% reference 12.80% 1.96%
handy-computer Q8_0 740 MB 11.58% −0.01 points (−0.03 to +0.01), no clear change 12.79% 1.94%
handy-computer Q4_K_M 485 MB 12.06% +0.47 points (+0.41 to +0.53) 13.33% 1.98%
transcribe.cpp quantizer, Q4_K_M 485 MB 12.07% +0.48 points (+0.42 to +0.54) 13.34% 1.94%
Parakeet-Air 485 MB 11.80% +0.21 points (+0.16 to +0.26) 13.04% 1.94%

Rates are pooled: total word errors over total reference words. On FLEURS alone, Parakeet-Air is +0.24 points from F32 (95% CI +0.19 to +0.29). The mean of the 25 per-language FLEURS rates is 12.96% for F32, 13.48% for handy-computer's Q4_K_M and 13.19% for Parakeet-Air.

Per language

Against handy-computer's Q4_K_M, Parakeet-Air is lower in 8 of 26 test sets and higher in none; the other 18 show no clear change.

Change in word error rate from F32 per test set, for handy-computer's Q4_K_M and Parakeet-Air

Test set F32 handy-computer Q4_K_M Parakeet-Air Parakeet-Air vs handy-computer Q4_K_M (95% interval)
FLEURS Bulgarian 12.77% 13.10% 13.11% +0.01 (−0.28 to +0.32), no clear change
FLEURS Czech 12.38% 13.09% 12.66% −0.43 (−0.79 to −0.07), lower
FLEURS Danish 18.61% 19.83% 19.26% −0.57 (−0.93 to −0.19), lower
FLEURS German 5.22% 5.25% 5.25% +0.00 (−0.19 to +0.20), no clear change
FLEURS Greek 35.32% 35.81% 35.52% −0.29 (−0.71 to +0.11), no clear change
FLEURS English 4.86% 4.90% 4.93% +0.03 (−0.14 to +0.21), no clear change
FLEURS Spanish 3.74% 3.84% 3.70% −0.14 (−0.30 to +0.00), no clear change
FLEURS Estonian 17.98% 18.33% 18.10% −0.23 (−0.68 to +0.20), no clear change
FLEURS Finnish 13.49% 14.03% 13.79% −0.24 (−0.62 to +0.11), no clear change
FLEURS French 5.26% 5.60% 5.50% −0.10 (−0.32 to +0.12), no clear change
FLEURS Croatian 12.60% 13.44% 12.92% −0.52 (−0.84 to −0.21), lower
FLEURS Hungarian 16.23% 16.93% 16.80% −0.13 (−0.52 to +0.26), no clear change
FLEURS Italian 3.01% 3.02% 3.03% +0.01 (−0.15 to +0.18), no clear change
FLEURS Lithuanian 22.27% 23.77% 23.04% −0.73 (−1.14 to −0.34), lower
FLEURS Latvian 23.94% 25.46% 24.24% −1.22 (−1.71 to −0.76), lower
FLEURS Maltese 20.49% 21.31% 20.77% −0.54 (−0.95 to −0.16), lower
FLEURS Dutch 7.65% 8.03% 7.54% −0.49 (−0.85 to −0.12), lower
FLEURS Polish 7.41% 7.67% 7.41% −0.26 (−0.51 to +0.01), no clear change
FLEURS Portuguese 4.90% 4.90% 4.90% +0.00 (−0.15 to +0.16), no clear change
FLEURS Romanian 12.67% 13.67% 13.03% −0.63 (−0.95 to −0.34), lower
FLEURS Russian 6.48% 6.59% 6.67% +0.09 (−0.13 to +0.28), no clear change
FLEURS Slovak 10.07% 10.73% 10.61% −0.12 (−0.46 to +0.24), no clear change
FLEURS Slovenian 24.44% 24.65% 24.44% −0.21 (−0.68 to +0.26), no clear change
FLEURS Swedish 15.23% 15.81% 15.57% −0.24 (−0.62 to +0.13), no clear change
FLEURS Ukrainian 6.88% 7.20% 6.95% −0.25 (−0.50 to +0.00), no clear change
LibriSpeech test-clean (English) 1.96% 1.98% 1.94% −0.04 (−0.09 to +0.02), no clear change

Differences are in percentage points. With 26 test sets, about one false "lower" or "higher" is expected by chance at this interval width. Against F32, Parakeet-Air is significantly higher in 10 test sets (Bulgarian, Danish, French, Croatian, Hungarian, Lithuanian, Romanian, Russian, Slovak, Swedish); handy-computer's Q4_K_M is significantly higher in 16.

Closeness to full precision's transcripts

Parakeet-Air's transcripts are 26% closer to F32's than handy-computer's Q4_K_M's, at the identical file size.

File Words that differ from F32's transcript, per 100 reference words
handy-computer Q8_0 0.91
handy-computer Q4_K_M 4.81
transcribe.cpp quantizer, Q4_K_M 4.78
Parakeet-Air 3.54

This counts word edits (substitutions, insertions, deletions) between each file's normalized transcript and F32's transcript of the same audio, so it measures change from full precision regardless of whether the reference text agrees. Parakeet-Air is lower than handy-computer's Q4_K_M in all 26 test sets.

Words that differ from F32's transcript per 100 words, for every quantized file

Files

File What it is
Parakeet-Air-Q4_K_M.gguf Parakeet-Air (485,425,504 bytes, sha256 14c12436640ccdd6f6890e791751454728897b7494ccbc0adb4a358a290e4fdb); modified from handy-computer's F32 GGUF of NVIDIA's weights
LICENSE.md webAI Non-Commercial License ver. 1.0
cc-by-4.0-LICENSE.txt Creative Commons Attribution 4.0 International, the license of NVIDIA's Parakeet TDT 0.6B v3 and of handy-computer's GGUF conversion
figures/ The charts in this card

Check the download with shasum -a 256 Parakeet-Air-Q4_K_M.gguf against the hash above.

Quick start

git clone https://github.com/handy-computer/transcribe.cpp && cd transcribe.cpp
git checkout 5700f127
cmake -B build && cmake --build build -j
hf download webAI-Official/Parakeet-Air Parakeet-Air-Q4_K_M.gguf --local-dir .
ffmpeg -i input.mp3 -ar 16000 -ac 1 input.wav
build/bin/transcribe-cli -m Parakeet-Air-Q4_K_M.gguf input.wav
  • transcribe.cpp expects 16 kHz mono audio; the ffmpeg line converts other formats.
  • The default build uses Metal on Apple silicon and the CPU elsewhere. For NVIDIA GPUs, the backend this file was evaluated on, configure with cmake -B build -DTRANSCRIBE_CUDA=ON and run with --backend cuda.
  • The evaluation used greedy decoding with timestamps and inverse text normalization off. The command line prints word and token timestamps by default; those were not evaluated.

Limitations

  • Not identical to full precision. Its word error rate is +0.21 points above F32's overall (significant) and significantly higher in 10 of the 26 test sets. If accuracy matters more than size, handy-computer's Q8_0 (740 MB) showed no measurable loss on this test.
  • Moderate gains. The improvement over handy-computer's Q4_K_M is −0.26 points of word error rate overall; in 18 of 26 test sets the difference was not significant.
  • One run. Each file was measured once on one test set. Greedy decoding on this backend is deterministic (a second F32 run gave identical transcripts), so the intervals reflect the choice of utterances, not runtime noise.
  • Scope. Read speech only: FLEURS (read Wikipedia sentences) and LibriSpeech test-clean (read audiobooks), one sentence or short passage per clip. Conversational, noisy, accented, long-form and streaming audio were not evaluated, nor were timestamps or inverse text normalization.
  • Backends. Results hold for transcribe.cpp's CUDA backend. Metal on Apple silicon was smoke-tested only; CPU, Vulkan and HIP were not tested with this file.
  • Speed was not measured under controlled conditions.One example on an Apple silicon Mac with the Metal backend: a 4.5 s clip transcribed in 218 ms of processing, 4.3 s in total including model loading.
  • Which file. Results hold for the included Parakeet-Air-Q4_K_M.gguf (sha256 above), byte-identical to the file that was evaluated.

Evaluation protocol

  • Runtime: stock transcribe.cpp at commit 5700f127, CUDA backend, one NVIDIA H200 (shared with another job), batch transcription through transcribe.cpp's own scripts/wer/run.py with --backend cuda --timestamps none; greedy decoding; inverse text normalization off (the default for this model). The harness passes each test set's language code, which transcribe.cpp's Parakeet implementation does not use: the model detects the language itself.
  • Files: F32, Q8_0 and Q4_K_M from handy-computer/parakeet-tdt-0.6b-v3-gguf (sha256 bbd36acd8c6fbb81…, 5859f77944efcd8e…, b68557be1e3c4020…), which handy-computer converted from NVIDIA's revision 6d590f7.
  • Test data: FLEURS test split from google/fleurs (revision 70bb2e84), 25 languages, 20,146 utterances, with FLEURS's normalized transcriptions as references; LibriSpeech test-clean, 2,620 utterances, from transcribe.cpp's WER manifest. Spanish and Portuguese use the FLEURS configurations es_419 and pt_br.
  • Word error rate: Whisper's English text normalizer for English (FLEURS English and LibriSpeech) and Whisper's basic normalizer for the other languages, applied to reference and transcript; word errors counted with jiwer; pooled over words. Utterances whose reference is empty after normalization are skipped (none in this test).
  • Closeness to F32: word edits between each file's normalized transcript and F32's normalized transcript of the same utterance.
  • Statistics: paired comparisons over the same 22,766 utterances, with a 95% interval from 2,000 bootstrap resamples of 8,586 groups (8,499 FLEURS sentences, each read by one or more speakers in one language, and 87 LibriSpeech speaker-chapters); differences in errors per group are divided by mean words per group to give word-error-rate points. "No clear change" means the interval includes zero.
  • Pre-registration: the question, primary measures, the 0.5-point margin against F32, the harness check and the pass rules were written down before the test ran.
  • Comparability: NVIDIA's and handy-computer's published word error rates use their own normalization and data versions. The harness check above reproduces handy-computer's LibriSpeech test-clean numbers within 0.01 points; other published numbers are not directly comparable with this page.

License and attribution

Released under the webAI Non-Commercial License ver. 1.0 (LICENSE.md). This distribution incorporates a modified version of NVIDIA’s Parakeet TDT 0.6B v3.

The original NVIDIA Parakeet TDT 0.6B v3 model is licensed under the Creative Commons Attribution 4.0 International License (CC-BY-4.0).

Original model: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 License: https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/cc-by-4.0.md

The F32 GGUF conversion of the original NVIDIA Parakeet TDT 0.6B v3 model by handy-computer is licensed under the Creative Commons Attribution 4.0 International License (CC-BY-4.0). Original https://huggingface.co/handy-computer/parakeet-tdt-0.6b-v3-gguf License: https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/cc-by-4.0.md

The F32 GGUF conversion of the original NVIDIA Parakeet TDT 0.6B v3 model by handy-computer has been modified using quantization technology developed by webAI to 4-bit (Q4_K_M) at reduced numeric precision. This modified version is made available under the webAI Non-Commercial License Version 1.0.

The webAI license does not replace or restrict rights granted by NVIDIA or handy-computer under CC-BY-4.0 with respect to NVIDIA’s or handy-computer’s original licensed material. No NVIDIA or handy-computer sponsorship, affiliation, or endorsement is implied.

Please follow NVIDIA’s model card for intended use and its notes on bias, privacy and safety.

Downloads last month
9
GGUF
Model size
0.6B params
Architecture
parakeet
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for webAI-Official/Parakeet-Air

Quantized
(155)
this model