README.md

---
license: apache-2.0
language:
- zh
- en
- multilingual
pipeline_tag: automatic-speech-recognition
library_name: llama.cpp
tags:
- qwen3-asr
- spacemit
- k1
- k3
- gguf
- onnxruntime
---

# Qwen3-ASR-0.6B for SpacemiT K1/K3

This repository contains a SpacemiT edge-deployment version of
[Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B). The speech
encoder is split into dynamic-quantized ONNX frontend/backend models. This
package uses the multi-batch frontend artifact under the standard filename and
runs through the SpacemiT Execution Provider, while the Qwen text decoder is stored
as a Q4_1 GGUF model and runs through the SpacemiT `llama.cpp` backend.

## Model origin and acknowledgement

Qwen3-ASR was introduced by the Qwen Team in the
[Qwen3-ASR Technical Report](https://arxiv.org/abs/2601.21337). It is a
multilingual automatic speech recognition family built for language
identification and speech transcription. The original Qwen3-ASR family
supports 30 languages and 22 Chinese dialects, and is designed for both
offline and streaming inference.

This repository is a deployment conversion, not a new base model. We thank the
Qwen Team for releasing the original architecture, model weights, technical
report, and inference code:

- [Qwen3-ASR paper](https://arxiv.org/abs/2601.21337)
- [Official Qwen3-ASR repository](https://github.com/QwenLM/Qwen3-ASR)
- [Official Qwen3-ASR-0.6B model](https://huggingface.co/Qwen/Qwen3-ASR-0.6B)

The current SpacemiT package exposes offline WAV transcription through the
OpenAI-compatible `llama-server` chat completions API. Chinese transcription
has been validated on both K1 and K3 with the bundled `zh.wav` sample.

## Files

```text
Qwen3-ASR-0.6B-SpacemiT/
├── Qwen3-ASR-0.6B-text-q41.gguf
├── Qwen3-ASR-0.6B-encoder-frontend.dynq.onnx
├── Qwen3-ASR-0.6B-encoder-backend.dynq.onnx
├── configs/
│   ├── K1/config.json
│   └── K3/config.json
├── zh.wav
└── README.md
```

`Qwen3-ASR-0.6B-encoder-frontend.dynq.onnx` is the multi-batch frontend
converted from `Qwen3-ASR-0.6B-encoder-frontend.dynq.batched.onnx`. The filename
is unchanged so the existing K1/K3 configurations remain compatible.

## Supported platforms

| Platform | Accelerated cores | SMT config | `llama-server` threads |
|---|---:|---|---:|
| SpacemiT K1 / X60 | `0,1,2,3` | `configs/K1` | `-t 4` |
| SpacemiT K3 / A100 | `8,9,10,11,12,13,14,15` | `configs/K3` | `-t 8` |

The platform-specific `config.json` controls the ONNX encoder's SpaceMIT EP
thread count and affinity. The `-t` argument controls the GGUF text decoder.
Do not use the K3 config on K1: K1 has four accelerated cores at IDs `0-3`,
whereas K3 has eight accelerated cores at IDs `8-15`.

The SpacemiT `llama.cpp` runtime automatically detects the accelerated CPU
cores. Normal runs do not require manually setting `SPACEMIT_PERFER_CORE_ARCH`,
`SPACEMIT_PERFER_CORE_ID`, or `SPACEMIT_MEM_BACKEND`.

## Prerequisites

Two runtime components are required:

1. A SpacemiT ONNX Runtime package containing `libonnxruntime` and the
   SpaceMIT Execution Provider:
   [spacemit-com/onnxruntime releases](https://github.com/spacemit-com/onnxruntime/releases)
2. An SMT-enabled SpacemiT `llama.cpp` build containing `llama-server`:
   [spacemit-com/llama.cpp](https://github.com/spacemit-com/llama.cpp)

The validation recorded below used SpacemiT ORT `2.0.6` and an SMT-enabled
RISC-V `llama-server` from the SpacemiT fork. The official prebuilt
`spacemit-llama.cpp.riscv64.0.1.7` package was also separately validated on K3.

### Option A: use the prebuilt packages

Download and unpack the current RISC-V glibc releases. The following versions
were current when this model card was prepared:

```bash
wget https://github.com/spacemit-com/onnxruntime/releases/download/2.0.6/spacemit-ort.riscv64.2.0.6.tar.gz
tar -xf spacemit-ort.riscv64.2.0.6.tar.gz

wget https://github.com/spacemit-com/llama.cpp/releases/download/v0.1.7/spacemit-llama.cpp.riscv64.0.1.7.tar.gz
tar -xf spacemit-llama.cpp.riscv64.0.1.7.tar.gz
```

Check the release pages for newer mutually compatible packages when deploying
this model in another software image.

### Option B: build llama.cpp from source

Cross-compilation requires a SpacemiT RISC-V toolchain in addition to the
unpacked ORT package.

```bash
git clone --recursive https://github.com/spacemit-com/llama.cpp.git
cd llama.cpp

export RISCV_ROOT_PATH=/path/to/spacemit-riscv-toolchain
export SPACEMIT_ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6

bash build_spacemit.sh glibc
```

The installed runtime is generated under `build/installed/`. The equivalent
manual CMake build must enable at least
`GGML_CPU_RISCV64_SPACEMIT=ON` and `LLAMA_SERVER_SMT_MTMD=ON`, and pass
`SPACEMIT_ORT_DIR` to CMake.

## Run on a K1 or K3 board

Copy the model folder, the unpacked ORT package, and the prebuilt or locally
built `llama.cpp` installation to the board. Set the following paths to match
their actual locations:

```bash
export MODEL_DIR=/path/to/Qwen3-ASR-0.6B-SpacemiT
export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
export LLAMA_DIR=/path/to/spacemit-llama.cpp.riscv64.0.1.7

export LD_LIBRARY_PATH="${LLAMA_DIR}/lib:${ORT_DIR}/lib:${LD_LIBRARY_PATH:-}"
```

### K1

K1 uses four accelerated cores, `0-3`:

```bash
"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/Qwen3-ASR-0.6B-text-q41.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K1" \
  -t 4 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup
```

The K1 config applies:

```json
"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "4",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "0;1;2;3"
}
```

### K3

K3 uses eight accelerated cores, `8-15`:

```bash
"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/Qwen3-ASR-0.6B-text-q41.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K3" \
  -t 8 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup
```

The K3 config applies:

```json
"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "8",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "8;9;10;11;12;13;14;15"
}
```

Wait until the server prints a line similar to:

```text
llama_server: listening on http://0.0.0.0:8080
```

If the service is reachable outside a trusted local network, configure an API
key and appropriate network access controls instead of exposing an unauthenticated
`0.0.0.0` endpoint.

## Send a transcription request

The request can be sent from the board itself or another machine that can
reach the board. Set `SERVER_URL=http://127.0.0.1:8080` when testing locally,
or replace `BOARD_IP` when testing remotely.

```bash
export SERVER_URL=http://127.0.0.1:8080

base64 < "${MODEL_DIR}/zh.wav" | tr -d '\n' | jq -Rs '{
  model: "qwen3-asr",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "input_audio",
          input_audio: {
            data: .,
            format: "wav"
          }
        },
        {
          type: "text",
          text: "language Chinese<asr_text>"
        }
      ]
    }
  ],
  max_tokens: 128,
  temperature: 0
}' | curl "${SERVER_URL}/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data-binary @-
```

The transcription is returned in `choices[0].message.content`. Replace
`zh.wav` with another 16 kHz WAV file to transcribe your own audio. The
`language Chinese` prefix can be changed to another supported language name.

## Verified example

The bundled `zh.wav` file was tested on 2026-08-07 with `--warmup` enabled and
the multi-batch frontend described above.
Both boards returned HTTP 200 and the same content:

```text
language Chinese<asr_text>开放时间：早上9点至下午5点。
```

Observed single-request wall time after server warmup:

| Board | HTTP status | Wall time |
|---|---:|---:|
| K1 (`0-3`, `-t 4`) | 200 | 3.76 s |
| K3 (`8-15`, `-t 8`) | 200 | 0.85 s |

These figures are functional smoke-test observations rather than a formal
benchmark. Startup and first-request time can be longer while ONNX graphs are
compiled and caches are populated.

The official prebuilt `llama.cpp` v0.1.7 package was additionally tested on K3
with ORT 2.0.6. It returned HTTP 200, the same transcription, and a 0.86 s
single-request wall time after warmup.

## Citation

Please cite the original Qwen3-ASR work when using this converted model:

```bibtex
@article{Qwen3-ASR,
  title={Qwen3-ASR Technical Report},
  author={Xian Shi and Xiong Wang and Zhifang Guo and Yongqi Wang and Pei Zhang and Xinyu Zhang and Zishan Guo and Hongkun Hao and Yu Xi and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
  journal={arXiv preprint arXiv:2601.21337},
  year={2026}
}
```

## License

The original Qwen3-ASR model is released under the Apache License 2.0. The
SpacemiT `llama.cpp` and ONNX Runtime packages are separate dependencies and
remain subject to their respective repository licenses.