README.md

metadata

pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
  - qwen3-asr
  - vllm

Serval-2-Flash

Speech recognition model with native Hugging Face Transformers and vLLM support.

Usage (Transformers)

uv pip install torch librosa transformers

import librosa
import torch
from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration

model_id = "Reson8/Serval-2-Flash"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3ASRForConditionalGeneration.from_pretrained(
    model_id, dtype=torch.bfloat16
).to("cuda")

audio, _ = librosa.load("audio.wav", sr=16000)
inputs = processor.apply_transcription_request(
    audio=audio, language="English", return_tensors="pt"
).to(model.device, dtype=model.dtype)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=512)

generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Omit language to detect it automatically.

Usage (vLLM)

uv pip install "vllm[audio]" openai
vllm serve Reson8/Serval-2-Flash --max-model-len 4096

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
audio = base64.b64encode(open("audio.wav", "rb").read()).decode()

response = client.chat.completions.create(
    model="Reson8/Serval-2-Flash",
    messages=[{
        "role": "user",
        "content": [{
            "type": "audio_url",
            "audio_url": {"url": f"data:audio/wav;base64,{audio}"},
        }],
    }],
    temperature=0,
    extra_body={"chat_template_kwargs": {"language": "English"}},
)
print(response.choices[0].message.content)

Transcription options

Pass these options through vLLM's chat_template_kwargs, or directly to processor.apply_chat_template in Transformers.

Option	Default	Effect
language	unset	Forces the language using its English name, such as "Dutch" or "English". If unset, output starts with language <Name><asr_text>.
digits	true	Numbers as digits; false writes them as words.
filler	false	true keeps fillers such as "um" and "uh".
verbatim	true	false collapses repeated words.