File size: 7,745 Bytes
ed36312 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 | ---
license: cc-by-4.0
library_name: parakeet.cpp
tags:
- automatic-speech-recognition
- asr
- parakeet
- gguf
- ggml
- cpp-inference
- nemo
pipeline_tag: automatic-speech-recognition
base_model:
- nvidia/parakeet-tdt_ctc-110m
- nvidia/parakeet_realtime_eou_120m-v1
- nvidia/parakeet-ctc-0.6b
- nvidia/parakeet-rnnt-0.6b
- nvidia/parakeet-tdt-0.6b-v2
- nvidia/parakeet-tdt-0.6b-v3
- nvidia/parakeet-ctc-1.1b
- nvidia/parakeet-rnnt-1.1b
- nvidia/parakeet-tdt-1.1b
- nvidia/parakeet-tdt_ctc-1.1b
---
# Parakeet GGUF — models for parakeet.cpp
GGUF-format weights for [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++/ggml port of NVIDIA NeMo Parakeet that matches the upstream PyTorch models on CPU. This single repo collects **every supported model × quantization** as a flat set of `.gguf` files — download just the one you need.
**F16 is the recommended default** — same accuracy as F32, ~1.7× smaller, and typically the fastest on modern CPUs via ggml's F32×F16 matmul fast path.
## Models
### tdt_ctc-110m
Source: [nvidia/parakeet-tdt_ctc-110m](https://huggingface.co/nvidia/parakeet-tdt_ctc-110m) · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt_ctc-110m-f16.gguf` ← **recommended** | F16 | 267.5 MB | 0.0000 |
| `tdt_ctc-110m-q8_0.gguf` | Q8_0 | 177.8 MB | 0.0000 |
| `tdt_ctc-110m-q6_k.gguf` | Q6_K | 155.9 MB | not measured |
| `tdt_ctc-110m-q5_k.gguf` | Q5_K | 143.3 MB | not measured |
| `tdt_ctc-110m-q4_k.gguf` | Q4_K | 131.4 MB | 0.0000 |
### realtime_eou_120m-v1
Source: [nvidia/parakeet_realtime_eou_120m-v1](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) · Cache-aware streaming RNNT (FastConformer, EOU/EOB) · heads: RNNT (streaming)
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `realtime_eou_120m-v1-f16.gguf` ← **recommended** | F16 | 266.5 MB | not measured |
| `realtime_eou_120m-v1-q8_0.gguf` | Q8_0 | 176.0 MB | not measured |
| `realtime_eou_120m-v1-q6_k.gguf` | Q6_K | 153.9 MB | not measured |
| `realtime_eou_120m-v1-q5_k.gguf` | Q5_K | 141.2 MB | not measured |
| `realtime_eou_120m-v1-q4_k.gguf` | Q4_K | 129.1 MB | not measured |
### ctc-0.6b
Source: [nvidia/parakeet-ctc-0.6b](https://huggingface.co/nvidia/parakeet-ctc-0.6b) · CTC (FastConformer) · heads: CTC
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `ctc-0.6b-f16.gguf` ← **recommended** | F16 | 1373.4 MB | 0.0000 |
| `ctc-0.6b-q8_0.gguf` | Q8_0 | 875.4 MB | 0.0000 |
| `ctc-0.6b-q6_k.gguf` | Q6_K | 746.8 MB | not measured |
| `ctc-0.6b-q5_k.gguf` | Q5_K | 676.3 MB | not measured |
| `ctc-0.6b-q4_k.gguf` | Q4_K | 609.9 MB | not measured |
### rnnt-0.6b
Source: [nvidia/parakeet-rnnt-0.6b](https://huggingface.co/nvidia/parakeet-rnnt-0.6b) · RNNT transducer (FastConformer) · heads: RNNT
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `rnnt-0.6b-f16.gguf` ← **recommended** | F16 | 1402.8 MB | 0.0000 |
| `rnnt-0.6b-q8_0.gguf` | Q8_0 | 903.9 MB | 0.0000 |
| `rnnt-0.6b-q6_k.gguf` | Q6_K | 776.3 MB | not measured |
| `rnnt-0.6b-q5_k.gguf` | Q5_K | 705.7 MB | not measured |
| `rnnt-0.6b-q4_k.gguf` | Q4_K | 639.2 MB | not measured |
### tdt-0.6b-v2
Source: [nvidia/parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2) · TDT transducer (FastConformer) · heads: TDT
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-0.6b-v2-f16.gguf` ← **recommended** | F16 | 1404.2 MB | 0.0000 |
| `tdt-0.6b-v2-q8_0.gguf` | Q8_0 | 903.8 MB | 0.0000 |
| `tdt-0.6b-v2-q6_k.gguf` | Q6_K | 775.9 MB | not measured |
| `tdt-0.6b-v2-q5_k.gguf` | Q5_K | 705.0 MB | not measured |
| `tdt-0.6b-v2-q4_k.gguf` | Q4_K | 638.4 MB | not measured |
### tdt-0.6b-v3
Source: [nvidia/parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) · TDT transducer (FastConformer) · heads: TDT
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-0.6b-v3-f16.gguf` ← **recommended** | F16 | 1441.0 MB | 0.0000 |
| `tdt-0.6b-v3-q8_0.gguf` | Q8_0 | 940.7 MB | 0.0000 |
| `tdt-0.6b-v3-q6_k.gguf` | Q6_K | 812.7 MB | not measured |
| `tdt-0.6b-v3-q5_k.gguf` | Q5_K | 741.9 MB | not measured |
| `tdt-0.6b-v3-q4_k.gguf` | Q4_K | 675.2 MB | not measured |
### ctc-1.1b
Source: [nvidia/parakeet-ctc-1.1b](https://huggingface.co/nvidia/parakeet-ctc-1.1b) · CTC (FastConformer) · heads: CTC
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `ctc-1.1b-f16.gguf` ← **recommended** | F16 | 2395.8 MB | 0.0000 |
| `ctc-1.1b-q8_0.gguf` | Q8_0 | 1526.3 MB | 0.0000 |
| `ctc-1.1b-q6_k.gguf` | Q6_K | 1301.7 MB | not measured |
| `ctc-1.1b-q5_k.gguf` | Q5_K | 1178.5 MB | not measured |
| `ctc-1.1b-q4_k.gguf` | Q4_K | 1062.6 MB | not measured |
### rnnt-1.1b
Source: [nvidia/parakeet-rnnt-1.1b](https://huggingface.co/nvidia/parakeet-rnnt-1.1b) · RNNT transducer (FastConformer) · heads: RNNT
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `rnnt-1.1b-f16.gguf` ← **recommended** | F16 | 2425.2 MB | 0.0000 |
| `rnnt-1.1b-q8_0.gguf` | Q8_0 | 1554.7 MB | 0.0000 |
| `rnnt-1.1b-q6_k.gguf` | Q6_K | 1331.2 MB | not measured |
| `rnnt-1.1b-q5_k.gguf` | Q5_K | 1207.9 MB | not measured |
| `rnnt-1.1b-q4_k.gguf` | Q4_K | 1091.9 MB | not measured |
### tdt-1.1b
Source: [nvidia/parakeet-tdt-1.1b](https://huggingface.co/nvidia/parakeet-tdt-1.1b) · TDT transducer (FastConformer) · heads: TDT
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-1.1b-f16.gguf` ← **recommended** | F16 | 2425.3 MB | 0.0000 |
| `tdt-1.1b-q8_0.gguf` | Q8_0 | 1554.8 MB | 0.0000 |
| `tdt-1.1b-q6_k.gguf` | Q6_K | 1331.2 MB | not measured |
| `tdt-1.1b-q5_k.gguf` | Q5_K | 1207.9 MB | not measured |
| `tdt-1.1b-q4_k.gguf` | Q4_K | 1091.9 MB | not measured |
### tdt_ctc-1.1b
Source: [nvidia/parakeet-tdt_ctc-1.1b](https://huggingface.co/nvidia/parakeet-tdt_ctc-1.1b) · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC
| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt_ctc-1.1b-f16.gguf` ← **recommended** | F16 | 2429.5 MB | 0.0000 |
| `tdt_ctc-1.1b-q8_0.gguf` | Q8_0 | 1559.0 MB | 0.0000 |
| `tdt_ctc-1.1b-q6_k.gguf` | Q6_K | 1335.4 MB | not measured |
| `tdt_ctc-1.1b-q5_k.gguf` | Q5_K | 1212.1 MB | not measured |
| `tdt_ctc-1.1b-q4_k.gguf` | Q4_K | 1096.1 MB | not measured |
> WER (word error rate) is computed against the upstream NeMo reference on `tests/fixtures/speech.wav` (LibriSpeech `2086-149220-0033`, ~7.4 s, English). 0.0 = byte-for-byte identical transcript. See [parity.md](https://github.com/mudler/parakeet.cpp/blob/main/docs/parity.md) and [quantization.md](https://github.com/mudler/parakeet.cpp/blob/main/docs/quantization.md).
## Quantization notes
Quantization is applied **only** to the large linear weights fed directly into `ggml_mul_mat` (encoder FFN + attention projections, subsampling output projection, joint enc/pred projections). All other tensors (mel filterbank, LSTM prediction net, conv kernels, batch_norm stats, norms, biases, embeddings) stay F32.
## Usage
```bash
# 1. Clone + build parakeet.cpp
git clone https://github.com/mudler/parakeet.cpp
cd parakeet.cpp
cmake -B build -DPARAKEET_BUILD_CLI=ON && cmake --build build -j
# 2. Download one quant (F16 recommended)
huggingface-cli download mudler/parakeet-cpp-gguf tdt_ctc-110m-f16.gguf --local-dir models/
# 3. Transcribe
build/examples/cli/parakeet-cli transcribe \
--model models/tdt_ctc-110m-f16.gguf \
--input audio.wav
```
## License
The GGUF weights are derived from the NVIDIA NeMo Parakeet checkpoints, released under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/) license. The parakeet.cpp runtime is MIT-licensed.
|