cstr/vibevoice-asr-bitnet-GGUF

VibeVoice-ASR-BitNet GGUF

GGUF conversion of microsoft/VibeVoice-ASR-BitNet for use with CrispASR.

Available files

All variants use TQ2_0 (ternary 2-bit) for the LM projection weights. They differ in how aggressively the VAE encoder and LM embedding are quantized. All produce identical transcription output on the JFK benchmark.

File	VAE	Embed	Size	Recommended
vibevoice-asr-bitnet-tq2.gguf	Q8_0	F16	1.55 GB	default
vibevoice-asr-bitnet-embed-q8.gguf	Q8_0	Q8_0	1.34 GB
vibevoice-asr-bitnet-vae-q5.gguf	Q5_0	F16	1.33 GB
vibevoice-asr-bitnet-both-q5.gguf	Q5_0	Q8_0	1.12 GB
vibevoice-asr-bitnet-vae-q4.gguf	Q4_0	F16	1.26 GB
vibevoice-asr-bitnet-aggro.gguf	Q4_0	Q8_0	1.05 GB	smallest

Pick any file -- quality is identical across all variants (verified via transcription roundtrip on JFK audio). The aggro variant is 35% smaller than the default with zero degradation.

Model Details

Property	Value
Base model	microsoft/VibeVoice-ASR-BitNet (1.5B params)
Architecture	Dual VAE encoder + Qwen2 LM decoder
LM	28 layers, 1536 hidden, 12 heads, 2 KV heads
VAE	ConvNeXt-style 1D CNN, stride 3200, acoustic (64d) + semantic (128d)
Input	24 kHz mono PCM
Languages	English, Chinese, French, Italian, Korean, Portuguese, Vietnamese
License	MIT

Quantization strategy

The LM projection weights (q/k/v/o/gate/up/down) are BitNet-trained with ternary values {-1, 0, 1}. The converter applies the ternary quantization (s = 1/mean(|w|), round, clamp) and packs into ggml's native TQ2_0 format (2.06 bpw). No custom SIMD kernels needed -- standard ggml mul_mat handles TQ2_0 transparently.

Component	Quant options	Notes
LM projections	TQ2_0 (all variants)	BitNet ternary, 2 bits/weight
VAE encoders	Q8_0 / Q5_0 / Q4_0	ConvNeXt conv + FFN weights
LM embedding	F16 / Q8_0	Lookup table, Q8_0 is lossless
Norms / biases	F32 (all variants)	Numerical stability
lm_head	tied to tok_emb (not stored)	Backend falls back automatically

Quality verification

All 6 variants were tested on JFK audio (11s, 24 kHz). Every variant produces identical output:

ANd so, my fellow Americans, ask not what your country can do for you -- ask what you can do for your country..

Sweep ran on Kaggle (30 GB RAM, CPU). Conversion + transcription took ~20 min total for all 6 variants.

Usage

CLI

# Auto-download (uses the default tq2 variant)
crispasr -m auto --backend vibevoice-bitnet -f audio.wav

# Or with a specific file
crispasr -m vibevoice-asr-bitnet-aggro.gguf --backend vibevoice -f audio.wav

C API

#include "vibevoice.h"

vibevoice_context_params params = vibevoice_context_default_params();
params.n_threads = 4;
vibevoice_context *ctx = vibevoice_init_from_file("vibevoice-asr-bitnet-aggro.gguf", params);
char *text = vibevoice_transcribe(ctx, samples_24khz, n_samples);
printf("%s\n", text);
free(text);
vibevoice_free(ctx);

Python

import crispasr
s = crispasr.Session("vibevoice-asr-bitnet-aggro.gguf", backend="vibevoice")
result = s.transcribe("audio.wav")
print(result["text"])

Conversion

# Default (Q8_0 VAE + F16 embed)
python models/convert-vibevoice-bitnet-to-gguf.py \
    --input microsoft/VibeVoice-ASR-BitNet \
    --output vibevoice-asr-bitnet-tq2.gguf

# Aggressive (Q4_0 VAE + Q8_0 embed, 1.05 GB)
python models/convert-vibevoice-bitnet-to-gguf.py \
    --input microsoft/VibeVoice-ASR-BitNet \
    --output vibevoice-asr-bitnet-aggro.gguf \
    --vae-quant q4_0 --embed-quant q8_0

Acknowledgements

• microsoft/VibeVoice for the model architecture and training
• microsoft/VibeASR.cpp for the BitNet quantization approach