| --- |
| pipeline_tag: automatic-speech-recognition |
| library_name: transformers |
| tags: |
| - qwen3-asr |
| - vllm |
| --- |
| |
| # Serval-2-Flash |
|
|
| Speech recognition model with native Hugging Face Transformers and vLLM support. |
|
|
| ## Usage (Transformers) |
|
|
| ```bash |
| uv pip install torch librosa transformers |
| ``` |
|
|
| ```python |
| import librosa |
| import torch |
| from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration |
| |
| model_id = "Reson8/Serval-2-Flash" |
| processor = AutoProcessor.from_pretrained(model_id) |
| model = Qwen3ASRForConditionalGeneration.from_pretrained( |
| model_id, dtype=torch.bfloat16 |
| ).to("cuda") |
| |
| audio, _ = librosa.load("audio.wav", sr=16000) |
| inputs = processor.apply_transcription_request( |
| audio=audio, language="English", return_tensors="pt" |
| ).to(model.device, dtype=model.dtype) |
| |
| with torch.inference_mode(): |
| output = model.generate(**inputs, max_new_tokens=512) |
| |
| generated = output[:, inputs["input_ids"].shape[-1]:] |
| print(processor.batch_decode(generated, skip_special_tokens=True)[0]) |
| ``` |
|
|
| Omit `language` to detect it automatically. |
|
|
| ## Usage (vLLM) |
|
|
| ```bash |
| uv pip install "vllm[audio]" openai |
| vllm serve Reson8/Serval-2-Flash --max-model-len 4096 |
| ``` |
|
|
| ```python |
| import base64 |
| from openai import OpenAI |
| |
| client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") |
| audio = base64.b64encode(open("audio.wav", "rb").read()).decode() |
| |
| response = client.chat.completions.create( |
| model="Reson8/Serval-2-Flash", |
| messages=[{ |
| "role": "user", |
| "content": [{ |
| "type": "audio_url", |
| "audio_url": {"url": f"data:audio/wav;base64,{audio}"}, |
| }], |
| }], |
| temperature=0, |
| extra_body={"chat_template_kwargs": {"language": "English"}}, |
| ) |
| print(response.choices[0].message.content) |
| ``` |
|
|
| ## Transcription options |
|
|
| Pass these options through vLLM's `chat_template_kwargs`, or directly to |
| `processor.apply_chat_template` in Transformers. |
|
|
| | Option | Default | Effect | |
| |---|---|---| |
| | `language` | unset | Forces the language using its English name, such as `"Dutch"` or `"English"`. If unset, output starts with `language <Name><asr_text>`. | |
| | `digits` | `true` | Numbers as digits; `false` writes them as words. | |
| | `filler` | `false` | `true` keeps fillers such as "um" and "uh". | |
| | `verbatim` | `true` | `false` collapses repeated words. | |
|
|
