--- library_name: transformers tags: - automatic-speech-recognition - speech - audio - multilingual - transformers - pytorch - safetensors - hotword - audio8 pipeline_tag: automatic-speech-recognition language: - en - zh - fr - ja - yue - de - ko license: apache-2.0 repository: https://github.com/AutoArk/open-audio-opd ---
# Audio8-ASR-0.1B [![GitHub](https://img.shields.io/badge/GitHub-AutoArk%2Fopen--audio--opd-blue?logo=github)](https://github.com/AutoArk/open-audio-opd) [![arXiv](https://img.shields.io/badge/arXiv-2605.28139-b31b1b?logo=arxiv)](https://arxiv.org/abs/2605.28139) [![License](https://img.shields.io/badge/License-Apache--2.0-green)](https://www.apache.org/licenses/LICENSE-2.0)
`Audio8-ASR-0.1B` is a compact autoregressive ASR model whose language-model component has only 0.1B parameters. It supports multilingual speech recognition for languages including Chinese, English, French, German, Japanese, Korean, and Cantonese. We position it as one of the smallest usable performance ASR models in the LLM era. This base repository provides the Hugging Face Transformers checkpoint. We also provide deployment-focused releases: - [Audio8-ASR-0.1B-onnx-runtime](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-onnx-runtime) - [Audio8-ASR-0.1B-iOS-ANE](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-iOS-ANE) The ONNX Runtime release is designed for edge-device deployment and can run with roughly 1.1 GB peak memory footprint, depending on device, runtime configuration, and workload. The iOS release is designed for local iPhone transcription with roughly 200 MB peak runtime memory footprint, depending on device, iOS version, and workload. ## Evaluation Results | Evaluation suite | Dataset / split | Language | Metric | Score (%) | H200 RTFx | | --- | --- | :---: | :---: | ---: | ---: | | Open ASR Leaderboard | AMI Cleaned | EN | WER | 10.99 | 396.91 | | Open ASR Leaderboard | Earnings22 | EN | WER | 12.31 | 654.17 | | Open ASR Leaderboard | GigaSpeech Cleaned | EN | WER | 8.48 | 641.19 | | Open ASR Leaderboard | LibriSpeech test.clean | EN | WER | 2.70 | 687.84 | | Open ASR Leaderboard | LibriSpeech test.other | EN | WER | 6.59 | 610.52 | | Open ASR Leaderboard | SPGISpeech | EN | WER | 3.73 | 870.32 | | Open ASR Leaderboard | VoxPopuli Cleaned AA | EN | WER | 4.39 | 686.14 | | **Open ASR Leaderboard** | **Seven-split mean / composite** | **EN** | **WER / RTFx** | **7.03** | **741.15** | | Internal canonical ASR eval | WenetSpeech meeting | ZH | CER | 8.842 | - | | Internal canonical ASR eval | WenetSpeech net | ZH | CER | 7.976 | - | The Open ASR results use the seven current public splits from [`hf-audio/open-asr-leaderboard`](https://huggingface.co/datasets/hf-audio/open-asr-leaderboard) at dataset revision `b6bdcd0beb34f8975dc659796176d88f43aff502`. They were measured with the standalone Transformers package on standardized H200 Hugging Face Jobs using BF16, eager attention, greedy decoding, `max_new_tokens=256`, and the documented 30-second audio cap. Per-split batch sizes were 1152, 1024, 1408, 1024, 1024, 2048, and 628. Raw manifests are stored in `hf://buckets/AutoArk-AI/audio8-asr-open-asr-results`, and the corresponding machine-readable results are provided in `.eval_results/open_asr_leaderboard.yaml`. The internal canonical WenetSpeech results come from the reproducibility-checked `teacher0p6B-step3000` export with batch size 128. Its effective model tensors are byte-identical to this standalone release; the release only removes a redundant tied LM-head tensor and packages the same weights for standalone use. Chinese results are reported as character error rate. AISHELL is intentionally excluded from this table. ## Model Overview - **Task:** automatic speech recognition - **Checkpoint format:** `safetensors` - **Sampling rate:** 16 kHz - **Decoder:** 8-layer Qwen-style causal LM - **Audio front end:** Qwen3-ASR audio encoder plus MLP adapter/projector - **Language-model parameters:** 103,502,336 (about 0.104B) - **End-to-end unique parameters:** 323,990,528 (about 0.324B) - **Runtime:** Hugging Face Transformers - **Hotwords:** optional decode-time logit boosting, no fine-tuning required The model should be loaded with `trust_remote_code=True`. ## Files - `config.json`, tokenizer files, processor files, and `model.safetensors` - `configuration_arkasr.py`, `modeling_arkasr.py`, `processing_arkasr.py` - `qwen3_asr_audio_config.py`, `qwen3_asr_audio_model.py` - `hotword/`: backend-agnostic hotword trie - `examples/`: Transformers inference examples The root `config.json` is intentionally kept in this repository so Hugging Face can recognize the model package and count downloads through normal model-file queries. ## Transformers Inference ```python import torch from transformers import AutoModelForCausalLM, AutoProcessor model_path = "AutoArk-AI/Audio8-ASR-0.1B" audio_path = "path/to/audio.wav" device = "cuda" if torch.cuda.is_available() else "cpu" torch_dtype = torch.bfloat16 if device == "cuda" else torch.float32 processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, torch_dtype=torch_dtype, attn_implementation="eager", ).to(device) model.eval() conversation = [ { "role": "user", "content": [ {"type": "audio", "path": audio_path}, {"type": "text", "text": "Please transcribe this audio."}, ], } ] batch = processor.apply_chat_template( conversation, return_tensors="pt", sampling_rate=16000, audio_padding="longest", add_generation_prompt=True, audio_max_length=30 * 16000, text_kwargs={"padding": "longest", "truncation": True, "max_length": 1000}, ) batch = {key: value.to(device) if hasattr(value, "to") else value for key, value in dict(batch).items()} with torch.inference_mode(): output_ids = model.generate(**batch, max_new_tokens=128, do_sample=False) prompt_len = int(batch["input_ids"].shape[1]) text = processor.decode(output_ids[0, prompt_len:], skip_special_tokens=True).strip() print(text) ``` Equivalent script: ```bash python examples/transcribe.py path/to/audio.wav --model AutoArk-AI/Audio8-ASR-0.1B ``` For local staging before upload: ```bash python examples/transcribe.py path/to/audio.wav --model . ``` ## Hotword Boosting Hotwords are applied at decode time by nudging logits for tokenizer paths that match the requested words. This does not modify model weights and does not inject the hotwords into the prompt. ```bash python examples/transcribe_hotword.py path/to/audio.wav \ --model AutoArk-AI/Audio8-ASR-0.1B \ --hotwords "Audio8,AutoArk" ``` Main knobs: - `--hotword_topk`: only boost tokens already inside the current top-k logits. - `--hotword_start_boost`: boost for the first token of each hotword. - `--hotword_continuation_boost`: boost for continuation tokens after a matched prefix. ## Related Releases - [ONNX Runtime package](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-onnx-runtime) - [iOS ANE package](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-iOS-ANE) ## Limitations - The default examples target short-form ASR and truncate audio at 30 seconds. - Hotword boosting can help with near-miss terms but can also over-bias decoding when boost values are too high. - Some Transformers/tokenizers versions emit a Qwen tokenizer regex warning. The staged tokenizer config is kept in the loadable form used by this package; pass explicit tokenizer regex flags only after testing your local Transformers version. ## Acknowledgements The audio encoder backbone is based on [Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B), with the audio adapter and projector trained as part of Audio8-ASR. The language-model backbone is based on [Ref-Pretrain-Qwen-104M](https://huggingface.co/MiniLLM/Ref-Pretrain-Qwen-104M).