File size: 2,763 Bytes
ce380b1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | ---
license: apache-2.0
base_model: Audio8/ARK-ASR-0.6B
base_model_relation: quantized
library_name: onnx-asr
pipeline_tag: automatic-speech-recognition
tags:
- onnx
- onnxruntime
- automatic-speech-recognition
- speech
- audio
- asr
- int8
- speech-llm
language:
- zh
- en
- de
- ja
- fr
- ko
- es
- pl
- it
- ro
- hu
- cs
- nl
- fi
- hr
- sk
- sl
- et
- lt
---
# ARK-ASR-0.6B ONNX
ONNX export of [Audio8/ARK-ASR-0.6B](https://huggingface.co/Audio8/ARK-ASR-0.6B) for
[onnx-asr](https://github.com/istupakov/onnx-asr). All credit for the model goes to
Audio8 (AutoArk AI). This repository only contains the converted graphs; the
weights are the original ones.
The model is a speech-LLM: a Whisper-large-v3-style audio encoder with rotary
position embeddings, an MLP adapter that merges four encoder frames into one
embedding, and a Qwen2 0.6B causal language model that writes the transcription.
## Usage
```sh
pip install onnx-asr[cpu,hub]
```
```py
import onnx_asr
model = onnx_asr.load_model("speech-llm", "OpenVoiceOS/ARK-ASR-0.6B-onnx")
print(model.recognize("audio.wav"))
# int8 weights
model = onnx_asr.load_model("speech-llm", "OpenVoiceOS/ARK-ASR-0.6B-onnx", quantization="int8")
```
## Files
| File | Contents |
| --- | --- |
| `encoder.onnx` | audio encoder and MLP adapter, log-mel features in, LM embeddings out |
| `embed_tokens.onnx` | token embedding table |
| `decoder.onnx` | Qwen2 decoder with KV cache, logits out |
| `*.int8.onnx` | dynamically quantized int8 weights |
| `config.json` | model type, prompt token ids, suppressed token ids |
| `vocab.json` | tokenizer vocabulary for detokenization |
## Graph contract
| Graph | Inputs | Outputs |
| --- | --- | --- |
| `encoder.onnx` | `input_features (1, 128, frames)` | `audio_embeds (1, frames/8, 896)` |
| `embed_tokens.onnx` | `input_ids (1, S)` | `inputs_embeds (1, S, 896)` |
| `decoder.onnx` | `inputs_embeds (1, S, 896)`, `attn_bias (1, 1, S, P+S)`, `position_ids (1, S)`, `past_key_values.{0..23}.{key,value} (1, 2, P, 64)` | `logits (1, S, 163958)`, `present.{0..23}.{key,value} (1, 2, P+S, 64)` |
## Accuracy
Four FLEURS clips (2 English, 2 Mandarin), greedy decoding, compared against the
PyTorch model in float32:
* fp32 ONNX: 4 of 4 transcriptions identical to PyTorch, character for character.
* int8 ONNX: 3 of 4 identical. One Mandarin clip stops early and loses the last
clause.
Speed on a 12-core CPU under load: RTF 0.35 to 0.61 (fp32) and 0.09 to 0.29
(int8).
## Licence
Apache 2.0, the same licence as the source model. The model was published by
Audio8; see the [source repository](https://huggingface.co/Audio8/ARK-ASR-0.6B)
and the paper [arXiv:2605.28139](https://arxiv.org/abs/2605.28139).
|