| --- |
| license: apache-2.0 |
| language: [ar, de, el, en, es, fr, it, ja, ko, nl, pl, pt, vi, zh] |
| tags: [automatic-speech-recognition, onnx, onnx-asr, nemo-conformer-aed, cohere] |
| base_model: CohereLabs/cohere-transcribe-03-2026 |
| pipeline_tag: automatic-speech-recognition |
| library_name: onnx-asr |
| --- |
| |
| # Cohere Transcribe 2B — ONNX (nemo-conformer-aed) |
|
|
| ONNX export of [CohereLabs/cohere-transcribe-03-2026](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026) |
| for [onnx-asr](https://github.com/istupakov/onnx-asr). |
|
|
| **Stock onnx-asr loads this model. No patches and no new model family are needed.** |
|
|
| Despite the "2B ASR" framing, `cohere_asr` is not a speech-LLM. It is an attention |
| encoder-decoder built on a NeMo FastConformer: `CohereAsrConfig` declares |
| `sub_configs = {"encoder_config": ParakeetEncoderConfig}`, and its decoder prompt is |
| token-for-token the NVIDIA Canary prompt. It therefore drops straight into the existing |
| `nemo-conformer-aed` family that already serves Canary. |
|
|
| ## Usage |
|
|
| ```python |
| import onnx_asr |
| |
| # straight from the Hub; the model type is read from config.json |
| model = onnx_asr.load_model("OpenVoiceOS/cohere-transcribe-2b-onnx", quantization="int8") |
| |
| # or from a local copy |
| model = onnx_asr.load_model("nemo-conformer-aed", "path/to/this/repo") |
| |
| print(model.recognize("audio_16khz.wav", language="en")) |
| ``` |
|
|
| Drop `quantization="int8"` for the fp32 graphs. |
|
|
| `language` accepts any of the 14 supported codes: `ar de el en es fr it ja ko nl pl pt vi zh`. |
| `pnc=True|False` toggles punctuation and capitalisation. |
|
|
| ## Graph contract |
|
|
| Two graphs, drop-in compatible with `istupakov/canary-1b-v2-onnx`: |
|
|
| | graph | inputs | outputs | |
| |---|---|---| |
| | `encoder-model.onnx` | `audio_signal` [B,128,T] f32, `length` [B] i64 | `encoder_embeddings` [B,T/8,1024] f32, `encoder_mask` [B,T/8] i64 | |
| | `decoder-model.onnx` | `input_ids` [B,C] i64, `encoder_embeddings` [B,E,1024] f32, `encoder_mask` [B,E] i64, `decoder_mems` [9,B,P,1024] f32 | `logits` [B,C,16384] f32 (log-softmax), `decoder_hidden_states` [9,B,P+C,1024] f32 | |
|
|
| Two details make the zero-patch fit work: |
|
|
| - The 1280 -> 1024 `decoder.proj` linear is **folded into the encoder output**, so the decoder |
| graph sees 1024-dim memories exactly as the contract requires. |
| - The HF `CohereAsrDecoder` uses a standard transformers KV cache. The export re-expresses it as |
| NeMo-style `decoder_mems` (per-layer pre-layernorm hidden states, `num_layers + 1 = 9` entries), |
| which is what the onnx-asr AED decode loop drives. |
|
|
| No feature extractor is baked into the graph. `CohereAsrFeatureExtractor` turned out to be the |
| standard NeMo log-mel front end — dither 1e-5, preemphasis 0.97, n_fft 512 / win 400 / hop 160, |
| symmetric Hann, 128 slaney mels, `log(x + 2**-24)`, per-feature normalisation — so onnx-asr's |
| built-in `nemo128` preprocessor already matches it, down to the frame-count formula. |
| |
| ## Files |
| |
| | file | size | |
| |---|---| |
| | `encoder-model.onnx` + `.data` | 7.59 GB | |
| | `decoder-model.onnx` | 676 MB | |
| | `encoder-model.int8.onnx` + `.data` | 1.91 GB | |
| | `decoder-model.int8.onnx` | 170 MB | |
| |
| fp32 total 8.3 GB, int8 total 2.1 GB. |
| |
| ## Accuracy |
| |
| Four FLEURS clips (2 en, 2 pt), greedy decoding, against native `transformers` on the same clips. |
| |
| | build | token-identical to native | |
| |---|---| |
| | fp32 | **4 / 4** | |
| | int8 | 2 / 4 | |
| |
| fp32 is exact. int8 dynamic quantisation costs a little accuracy — observed drift is a spurious |
| comma and a mis-spelled rare proper noun. Session-swapping shows both graphs contribute (encoder |
| int8 alone: 3/4 drift; decoder int8 alone: 2/4 drift), so there is no single subgraph to exclude. |
| Use fp32 when accuracy matters and int8 when size matters. |
| |
| Note: onnx-asr's built-in detokeniser drops the space before an opening bracket or quote |
| (`sugar(especially`). That is upstream onnx-asr behaviour shared with Canary, not an export |
| defect — the token ids are identical. |
| |
| ## Speed |
| |
| AMD Ryzen 5 7600 (6 cores / 12 threads), CPU execution provider, `OMP_NUM_THREADS=6`, `nice -n 10`. |
| |
| | clip | duration | fp32 RTF | int8 RTF | |
| |---|---|---|---| |
| | en_1 | 6.5 s | 0.178 | 0.186 | |
| | en_2 | 16.4 s | 0.355 | 0.186 | |
| | pt_1 | 11.8 s | 0.348 | 0.214 | |
| | pt_2 | 14.6 s | 0.301 | 0.186 | |
| | **mean** | | **0.30** | **0.19** | |
| |
| ## Scope |
| |
| Single-clip transcription only. The source processor splits audio longer than 35 s at |
| low-energy boundaries and stitches the pieces back together; that chunking is not part of this |
| export. Feed clips under about 30 s, or segment them yourself. |
| |
| ## Attribution |
| |
| Source model and weights: **Cohere Labs**, [CohereLabs/cohere-transcribe-03-2026](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026), Apache-2.0. |
| This repository contains only an ONNX conversion; the license is inherited unchanged. |
| |
