README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-ASR-0.6B
pipeline_tag: audio-text-to-text
language:
  - vi
library_name: transformers
tags:
  - qwen3-asr
  - speech-to-command
  - function-calling
  - tool-calling
  - vllm
---

# Vietnamese Speech-to-Command — route-v1 step 1250

This is a standalone Hugging Face export of a Qwen3-ASR-0.6B model fine-tuned
to map Vietnamese driver speech directly to structured vehicle function calls.
It does not run a separate ASR stage before selecting an action.

The output is exactly one of:

- one or more vehicle calls from the bundled catalog; or
- `non_tool(text=...)`, which routes unsupported or conversational speech and
  carries the recognized transcript.

The export was produced from `model_best.pt` at training step 1250. The main
weight file is `model.safetensors` (about 1.5 GB).

## Prompt contract

Do not write or paste a separate tool system prompt. The bundled
`chat_template.jinja` constructs the model's training-time system prompt from
the `tools` field of each Chat Completions request.

Every request must follow this contract:

1. Load and send all entries from `tools_openai.json` unchanged.
2. Put the audio in the current `user` message. No text transcription is sent
   alongside it.
3. Use `tool_choice="auto"`.
4. Set `chat_template_kwargs.enable_thinking=false`.
5. Use greedy decoding (`temperature=0`).
6. Read the result from `message.tool_calls`.

There was no additional natural-language system message during training.
Adding one, omitting tools, changing tool names/arguments, or using the audio
transcription endpoint changes the input distribution and is unsupported.

## Serve from Hugging Face with vLLM

The checkpoint was tested end to end with vLLM 0.19.0 and 0.23.0. Use a vLLM
build that natively registers `Qwen3ASRForConditionalGeneration`. The
`audio` extra is required for Chat Completions requests containing WAV data.

```bash
# Choose the version provided by your deployment image.
python -m pip install "vllm[audio]==0.23.0"
# or: python -m pip install "vllm[audio]==0.19.0"

# Required only when MODEL_ID is a private Hugging Face repository.
export HF_TOKEN=hf_your_read_token

MODEL_ID=YOUR_ORG/YOUR_MODEL_REPO

vllm serve "$MODEL_ID" \
  --served-model-name command-asr \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --generation-config auto \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --limit-mm-per-prompt '{"audio":1}'
```

The server listens on `http://127.0.0.1:8000` by default. Confirm readiness:

```bash
curl -fsS http://127.0.0.1:8000/v1/models
```

Do not add `--trust-remote-code` for vLLM: this architecture is loaded through
vLLM's native Qwen3-ASR implementation. The Python files in this repository
are retained for standalone Transformers loading.

The vLLM 0.19.0 compatibility test used its resolved Transformers 4.57.6
stack. The vLLM 0.23.0 compatibility test used Transformers 5.14.1. Both
versions returned matching tool names and arguments on the same command and
non-tool audio smoke tests. These software tests do not by themselves certify
platform-specific kernels in an ARM64/Tegra container.

## Send an audio request

Use a short, clear WAV file; mono 16 kHz PCM is recommended.

```bash
python example_client.py ./command.wav
```

Against a different URL or served model name:

```bash
python example_client.py ./command.wav \
  --base-url http://SERVER:8000/v1 \
  --model command-asr
```

The script prints the assistant message as JSON. A vehicle request should have
one or more entries in `tool_calls`, for example:

```json
{
  "tool_calls": [
    {
      "function": {
        "name": "set_auto_high_beam",
        "arguments": "{\"state\":\"on\"}"
      }
    }
  ]
}
```

Ordinary speech should route through `non_tool`:

```json
{
  "tool_calls": [
    {
      "function": {
        "name": "non_tool",
        "arguments": "{\"text\":\"hôm nay thời tiết thế nào\"}"
      }
    }
  ]
}
```

## Bundled smoke-test audio

The `samples/` directory contains nine WAV files covering complete input
durations from 1 through 9 seconds. Seven exercise vehicle tool calls and two
exercise the `non_tool` route. Machine-readable transcripts, durations, and
expected results are included:

```bash
python example_client.py samples/tool_call_set_fog_lights.wav
python example_client.py samples/non_tool_news.wav
```

See `samples/README.md` for the complete duration table and validation rules.
All nine files were exercised end to end with vLLM 0.19.0 and 0.23.0.

## Known scope

- Primary language: Vietnamese.
- Intended input: short in-car commands and short non-tool utterances.
- Tool schemas are part of the learned contract; changing the catalog requires
  evaluation and may require further fine-tuning.
- This is an experimental checkpoint, not a safety-certified vehicle control
  system. Validate every returned call against the supplied schema before
  execution.

## License and attribution

This model is derived from `Qwen/Qwen3-ASR-0.6B`, released under Apache-2.0.
Retain the upstream license and attribution when redistributing the model.