wav2vec2.cpp — Run Any wav2vec2 Model Locally, No Python Required
Why wav2vec2.cpp?
Whisper gets a lot of love for local ASR. But wav2vec2 models — especially the XLSR-53 family from jonatasgrosman — are often better for non-English languages. They're smaller, faster, and purpose-trained per language on Common Voice and FLEURS.
The problem: running them requires Python, PyTorch, and a multi-GB transformers install. There was no whisper.cpp equivalent.
So we built one.
What's in the release
15 GGUF repos on liodon-ai, covering:
| Language | HuggingFace Repo |
|---|---|
| English | wav2vec2-base-960h-GGUF, wav2vec2-large-960h-GGUF |
| Japanese | wav2vec2-large-xlsr-53-japanese-GGUF |
| Polish | wav2vec2-large-xlsr-53-polish-GGUF |
| Indonesian | wav2vec2-indonesian-javanese-sundanese-GGUF |
| Dutch | wav2vec2-large-xlsr-53-dutch-GGUF |
| Greek | wav2vec2-large-xlsr-53-greek-GGUF |
| Arabic | wav2vec2-large-xlsr-53-arabic-GGUF |
| Hungarian | wav2vec2-large-xlsr-53-hungarian-GGUF |
| Portuguese | wav2vec2-large-xlsr-53-portuguese-GGUF |
| Russian | wav2vec2-large-xlsr-53-russian-GGUF |
| Telugu | wav2vec2-large-xlsr-53-telugu-GGUF |
| Tamil | vakyansh-wav2vec2-tamil-tam-250-GGUF |
| Hindi | Wav2Vec2-large-xlsr-hindi-GGUF |
| Chinese | wav2vec2-large-xlsr-53-chinese-zh-cn-GGUF |
Each repo ships three quant levels: F16 (near-lossless), Q8_0 (~40% smaller, virtually identical output), and Q4_0 (~65% smaller, slight degradation on large-vocab scripts).
Usage
# Clone and build (no GPU required)
git clone --recursive https://github.com/liodon-ai/wav2vec2.cpp
cd wav2vec2.cpp && mkdir build && cd build && cmake .. && make -j
# Download a model
huggingface-cli download liodon-ai/wav2vec2-large-xlsr-53-arabic-GGUF \
model_q8_0.gguf --local-dir .
# Transcribe
./wav2vec2-cli -m model_q8_0.gguf -f audio.wav
# Word timestamps
./wav2vec2-cli -m model_q8_0.gguf -f audio.wav -w
# SRT subtitles
./wav2vec2-cli -m model_q8_0.gguf -f audio.wav --format srt > output.srt
# JSON output
./wav2vec2-cli -m model_q8_0.gguf -f audio.wav --format json
No Python. No PyTorch. The binary handles resampling internally — pass any 16kHz+ WAV and it does the right thing.
Parity vs HuggingFace
Every model card includes a parity table: CER measured between the GGUF output and the HuggingFace reference model on 8 FLEURS test samples. This isolates conversion error from model quality.
Representative results:
| Model | F16 CER | Q8_0 CER | Q4_0 CER |
|---|---|---|---|
| wav2vec2-base-960h (EN) | 0.00% | 0.32% | ~0% |
| xlsr-53-arabic | 0.00% | 0.00% | 2.21% |
| xlsr-53-russian | 0.00% | 0.25% | 1.53% |
| xlsr-53-telugu | 0.00% | 0.06% | 2.66% |
| vakyansh-tamil | 0.17% | 0.23% | 1.64% |
| xlsr-53-japanese | ~0% | ~0% | 8.26%* |
*Q4_0 degrades on large-vocab scripts (Japanese ~5K chars, Chinese ~3.5K chars). Use Q8_0 for those.
What wav2vec2.cpp supports
- Architectures: base/large post-norm (960h) and pre-norm/stable_layer_norm (XLSR-53, XLS-R)
- CTC decoding: greedy and beam search
- Word timestamps: frame-accurate (frame × 320 / 16000 = seconds)
- Output formats:
txt,json,srt,vtt - Quantization: F16, Q8_0, Q4_0 via ggml
- Resampling: linear interpolation, handles any input sample rate
- Non-standard pad tokens:
pad_token_idread from GGUF KV — works for models where the CTC blank isn't at index zero
What's next
We're treating wav2vec2.cpp as infrastructure for the ASR lane of liodon-ai. Next up:
- Python bindings (pybind11) so you can call it from a pipeline without subprocess
- Whisper.cpp-style server with HTTP endpoints
- More languages — Kannada, Bengali, Swahili, Marathi are underserved and high-download targets
- Whisper fine-tunes on Common Voice for the same language set — wav2vec2 + Whisper coverage per language
If you use this and hit issues, open a GitHub issue or ping us on the HuggingFace community tab of any of the repos.
Built by Liodon AI. Source: github.com/liodon-ai/wav2vec2.cpp