thangylvp/stcc

Vietnamese Speech-to-Command — route-v1 step 1250

This is a standalone Hugging Face export of a Qwen3-ASR-0.6B model fine-tuned to map Vietnamese driver speech directly to structured vehicle function calls. It does not run a separate ASR stage before selecting an action.

The output is exactly one of:

• one or more vehicle calls from the bundled catalog; or
• non_tool(text=...), which routes unsupported or conversational speech and carries the recognized transcript.

The export was produced from model_best.pt at training step 1250. The main weight file is model.safetensors (about 1.5 GB).

Prompt contract

Do not write or paste a separate tool system prompt. The bundled chat_template.jinja constructs the model's training-time system prompt from the tools field of each Chat Completions request.

Every request must follow this contract:

1. Load and send all entries from tools_openai.json unchanged.
2. Put the audio in the current user message. No text transcription is sent alongside it.
3. Use tool_choice="auto".
4. Set chat_template_kwargs.enable_thinking=false.
5. Use greedy decoding (temperature=0).
6. Read the result from message.tool_calls.

There was no additional natural-language system message during training. Adding one, omitting tools, changing tool names/arguments, or using the audio transcription endpoint changes the input distribution and is unsupported.

Serve from Hugging Face with vLLM

The checkpoint was tested end to end with vLLM 0.19.0 and 0.23.0. Use a vLLM build that natively registers Qwen3ASRForConditionalGeneration. The audio extra is required for Chat Completions requests containing WAV data.

# Choose the version provided by your deployment image.
python -m pip install "vllm[audio]==0.23.0"
# or: python -m pip install "vllm[audio]==0.19.0"

# Required only when MODEL_ID is a private Hugging Face repository.
export HF_TOKEN=hf_your_read_token

MODEL_ID=YOUR_ORG/YOUR_MODEL_REPO

vllm serve "$MODEL_ID" \
  --served-model-name command-asr \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --generation-config auto \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --limit-mm-per-prompt '{"audio":1}'

The server listens on http://127.0.0.1:8000 by default. Confirm readiness:

curl -fsS http://127.0.0.1:8000/v1/models

Do not add --trust-remote-code for vLLM: this architecture is loaded through vLLM's native Qwen3-ASR implementation. The Python files in this repository are retained for standalone Transformers loading.

The vLLM 0.19.0 compatibility test used its resolved Transformers 4.57.6 stack. The vLLM 0.23.0 compatibility test used Transformers 5.14.1. Both versions returned matching tool names and arguments on the same command and non-tool audio smoke tests. These software tests do not by themselves certify platform-specific kernels in an ARM64/Tegra container.

Send an audio request

Use a short, clear WAV file; mono 16 kHz PCM is recommended.

python example_client.py ./command.wav

Against a different URL or served model name:

python example_client.py ./command.wav \
  --base-url http://SERVER:8000/v1 \
  --model command-asr

The script prints the assistant message as JSON. A vehicle request should have one or more entries in tool_calls, for example:

{
  "tool_calls": [
    {
      "function": {
        "name": "set_auto_high_beam",
        "arguments": "{\"state\":\"on\"}"
      }
    }
  ]
}

Ordinary speech should route through non_tool:

{
  "tool_calls": [
    {
      "function": {
        "name": "non_tool",
        "arguments": "{\"text\":\"hôm nay thời tiết thế nào\"}"
      }
    }
  ]
}

Bundled smoke-test audio

The samples/ directory contains nine WAV files covering complete input durations from 1 through 9 seconds. Seven exercise vehicle tool calls and two exercise the non_tool route. Machine-readable transcripts, durations, and expected results are included:

python example_client.py samples/tool_call_set_fog_lights.wav
python example_client.py samples/non_tool_news.wav

See samples/README.md for the complete duration table and validation rules. All nine files were exercised end to end with vLLM 0.19.0 and 0.23.0.

Known scope

• Primary language: Vietnamese.
• Intended input: short in-car commands and short non-tool utterances.
• Tool schemas are part of the learned contract; changing the catalog requires evaluation and may require further fine-tuning.
• This is an experimental checkpoint, not a safety-certified vehicle control system. Validate every returned call against the supplied schema before execution.

License and attribution

This model is derived from Qwen/Qwen3-ASR-0.6B, released under Apache-2.0. Retain the upstream license and attribution when redistributing the model.