hviske-v5-tiny

hviske-v5-tiny is a 263M-parameter Danish ASR model distilled from the syv-transcribe ensemble (~2.1B). About 8× smaller than its teachers while staying competitive with them, and roughly 2× faster than any 2B model on the Danish ASR leaderboard.

Audio is expected at 16 kHz mono, clips up to 35 s. Danish only, offline transcription (no timestamps, diarization, or streaming).

Accuracy

Measured with the leaderboard harness, unmodified (--backend cohere-asr), on a single RTX 3090.

Dataset	WER	CER
CoRal conversation	26.07	15.97
CoRal read-aloud	14.76	6.00
Common Voice 17 (da)	9.89	3.57
FLEURS (da)	11.32	4.49
FTSpeech	7.15	3.84
Mean	13.84	6.77

FTSpeech 7.15 is the best score on the leaderboard at the time of submission, ahead of the 2.1B syv-transcribe ensemble (8.02).

Speed

RTFx = audio seconds transcribed per wall-clock second. "Single" is one clip at a time (the interactive case); "batched" is the best measured batched/concurrent configuration for that runtime. Where clip length matters the batched cell shows both: longer clips amortise per-request cost, so 30 s audio yields a higher RTFx than the 11 s FLEURS average.

Hardware	Runtime	Single	Batched	Batch config	WER
RTX 3090	custom CUDA kernels (cuda/)	320×	3833× (11 s) · 4341× (30 s)	batch 64	11.47
RTX 3090	vLLM 0.19.0 + CUDA kernels	182×	2646× (11 s) · 3587× (30 s)	512 concurrent	11.21
RTX 3090	vLLM 0.19.0 stock	176×	1783× (11 s) · 2936× (30 s)	256–384 concurrent	11.27
RTX 3090	PyTorch bf16 (leaderboard harness)	158×	323×	batch 8	11.32
Apple M4 (base)	MLX int4	87×	93×	batch 8	10.60†
Apple M4 (base)	MLX int8	63×	110×	batch 8	10.60†
Apple M4 (base)	MLX fp16	39×	78×	batch 8	10.46†
Apple M4 (base)	GGUF q4_k via CrispASR (Metal)	56×	—	sequential CLI	10.51†
Apple M4 (base)	GGUF q8_0 via CrispASR (Metal)	45×	—	sequential CLI	10.44†
Apple M4 (base)	ONNX Runtime CPU, 2 threads	31×	—	batch axis fixed at 1	10.55†
Apple M4 (base)	GGUF q4_k via CrispASR (CPU, 8 threads)	24×	—	sequential CLI	10.88†
Apple M4 (base)	PyTorch CPU fp32	11×	—		11.03†
x86 CPU, 4 threads	PyTorch fp32	3.2×	—		—

WER is FLEURS-da (lowercase, punctuation-stripped, same normaliser as the leaderboard). RTX rows are the full 930-clip test set; † rows are the 200-clip subset used in the per-build sections below, which skews ~0.8 lower because its clips are shorter — compare within a group, not across. The spread inside each group (±0.1–0.2) is quantisation/bf16 noise, not a real accuracy difference between runtimes.

Notes on the dashes: the CrispASR CLI processes files sequentially (passing many files in one invocation only amortises model load), and the published ONNX graphs were exported with the batch axis fixed at 1, so neither supports true batching today. Both are fixable — the ONNX one just needs a re-export with a dynamic batch axis.

Batching pays off very differently per backend: 12× on the 3090 with the custom kernels, 9–15× under vLLM, but only 1.1–2× on a base M4, where a single stream already keeps the GPU busy.

The Apple rows were measured back-to-back within each build, but the M4 had other work running and its throughput moves by roughly ±25% with machine load (the MLX int4 single figure measured as high as 139× on an idle machine, 73–87× loaded). Treat all Apple numbers as a band, not a point; per-build details are in the MLX, ONNX and GGUF sections below.

Usage

PyTorch

import soundfile as sf
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    "syvai/hviske-v5-tiny", trust_remote_code=True).eval()
processor = AutoProcessor.from_pretrained("syvai/hviske-v5-tiny", trust_remote_code=True)

audio, sr = sf.read("clip.wav", dtype="float32")
text = model.transcribe(processor=processor, language="da",
                        audio_arrays=[audio], sample_rates=[sr])[0]
print(text)

• Requires trust_remote_code=True and transformers 4.57.x — 5.x regressed remote-model loading for this architecture.
• Use bf16 on GPU, not fp16: the architecture's masking values overflow in fp16.

CUDA kernels (NVIDIA GPUs)

The fastest GPU path is the custom Triton kernel stack in cuda/: flash-style relative-position attention (no rel_shift copies or score-tensor materialization), single-query decode attention, fused depthwise-conv+BatchNorm+SiLU, channels-last subsampling convs, a GPU log-mel frontend, and a CUDA-graphed greedy decode loop. Same weights, same math: WER on the full FLEURS-da test set is 11.47 vs 11.45 through the reference path on identical audio. On an RTX 3090 it measures 3833× realtime on the full test set (vs 323× for the reference batched harness) and 4341× on 30 s clips — about 48% MFU, so the remaining headroom on this card is small.

cuda/hviske_enc_kernels.py also patches the encoder of vLLM 0.19.0 in place (instructions in cuda/README.md), which takes vLLM serving from 1952× to 2646× on 11 s clips and 2936× to 3587× on 30 s clips at unchanged WER. After the patch, vLLM is CPU-frontend-bound: use concurrency ≥512 to saturate the GPU.

Server (vLLM)

A bf16 checkpoint with the serving fixes already applied lives in vllm/. Tested with vLLM 0.19.0.

pip install vllm==0.19.0
huggingface-cli download syvai/hviske-v5-tiny --include 'vllm/*' --local-dir ./hviske

vllm serve ./hviske/vllm --served-model-name hviske-v5-tiny \
  --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.90 \
  --max-model-len 1024 --max-num-seqs 512 --api-server-count 4 --port 18010

curl http://127.0.0.1:18010/v1/audio/transcriptions \
  -F file=@clip.wav -F model=hviske-v5-tiny -F language=da

On a single RTX 3090: 176× realtime for one request (62 ms p50) and 1783× batched at 256 concurrent requests, rising to 2936× on 30 s clips — at unchanged WER (11.27). See vllm/README.md for tuning notes and the reasons that directory differs from the root checkpoint.

Apple silicon (MLX)

Native MLX builds live in mlx/ — fp16, int8 and int4 weights plus a small pure-MLX runtime, so the model runs on a Mac with no PyTorch installed.

pip install mlx numpy sentencepiece soundfile huggingface_hub

import sys
import mlx.core as mx
import soundfile as sf
from huggingface_hub import snapshot_download

path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["mlx/int4/*", "mlx/hviske_mlx/*"])
sys.path.insert(0, f"{path}/mlx")
from hviske_mlx.transcribe import Hviske

model = Hviske(f"{path}/mlx/int4")
audio, sr = sf.read("clip.wav", dtype="float32")   # 16 kHz mono
print(model.generate(mx.array(audio))["text"])

Batched greedy decoding is available via model.generate_batch([a1, a2, ...]).

On a base M4, 200 FLEURS-da clips, greedy, batch 1:

Build	Size	RTFx	ms/clip	WER
mlx/fp16	526 MB	47.6×	237	10.46
mlx/int8	300 MB	63.5×	178	10.60
mlx/int4	179 MB	73.4×	154	10.60

int4 matches int8 on accuracy while being 41% smaller and faster, so it is the recommended default; fp16 tracks the PyTorch model most closely. The port is verified against PyTorch stage by stage (encoder output agrees to 1.5e-06; 38 of 40 clips decode byte-identically) — see mlx/README.md for details, limitations, and the alternative mlx-speech runtime.

CPU (ONNX Runtime)

ONNX graphs live in onnx/ — portable CPU inference with no PyTorch, on macOS, Linux or Windows.

pip install onnxruntime numpy sentencepiece soundfile huggingface_hub

import sys
import soundfile as sf
from huggingface_hub import snapshot_download

path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["onnx/*"])
sys.path.insert(0, f"{path}/onnx")
from hviske_onnx.runtime import HviskeOnnx

model = HviskeOnnx(f"{path}/onnx", encoder_int8=False, decoder_int8=True, threads=2)
audio, sr = sf.read("clip.wav", dtype="float32")   # 16 kHz mono
print(model.generate(audio)["text"])

On a base M4, 200 FLEURS-da clips: 30.7× realtime at 10.55 WER with 2 threads — 2.8× faster than the PyTorch CPU path at equal accuracy. The encoder is fp32 and the decoder int8: dynamic int8 helps the decoder (3× faster, no WER cost) but hurts the conv-heavy encoder on ARM (2× slower, +1.9 WER). See onnx/README.md.

GGUF (CrispASR)

GGUF builds for the CrispASR C++/ggml runtime live in gguf/ — no Python at inference, with Metal/CUDA/Vulkan/CPU backends.

./build/bin/crispasr --backend cohere -m hviske-v5-tiny-q4_k.gguf -f clip.wav -l da -t 4

Build	Size	RTFx (Metal)	WER
q4_k	160 MB	56.4×	10.51
q5_0	190 MB	52.5×	10.57
q6_k	243 MB	50.5×	10.53
q8_0	281 MB	45.0×	10.44
f16	527 MB	39.1×	10.51

q4_k is the recommended build — smallest, fastest, and no measurable accuracy cost against f16. CPU-only it runs at 23.5× realtime on 8 threads. See gguf/README.md.

Limitations

• Danish only, despite the multilingual tokenizer inherited from the teacher.
• Conversational/spontaneous speech is the weakest domain (CoRal conversation 26.07 WER), consistent with the teacher family.
• Distilled from teacher pseudo-labels, so it inherits the teacher's biases and cannot exceed it on material where the teacher is wrong.
• No timestamps, diarization, or streaming.

License

CC BY-NC 4.0, inherited from the teacher syvai/hviske-v5.3.