README.md

---
language:
- en
- fr
- it
- pt
- es
- de
license: mit
pipeline_tag: automatic-speech-recognition
tags:
- ASR
- quantization
- cpu-inference
- gguf
- bitnet
library_name: ggml
base_model: microsoft/VibeVoice-ASR-BitNet
---

# VibeVoice-ASR-BitNet-slim

A repack of [microsoft/VibeVoice-ASR-BitNet](https://huggingface.co/microsoft/VibeVoice-ASR-BitNet)
with a redundant tensor removed. **No retraining, no re-quantisation of the ternary
weights** — the transformer body is byte-for-byte the original.

| | LM | VAE | Total |
|:--|--:|--:|--:|
| microsoft/VibeVoice-ASR-BitNet | 992.9 MB | 703.1 MB | 1.70 GB |
| this repo | **526.1 MB** | 703.1 MB | **1.23 GB** |

## What changed

The released LM GGUF carries `output.weight` as **F16, 466.7 MB — 47% of the file**,
next to `token_embd.weight` already stored as Q6_K at 191.4 MB.

In the source checkpoint `tie_word_embeddings` is `true`, and `lm_head.weight` is
**bit-identical** to `embed_tokens.weight`. So the F16 tensor is the same matrix a
second time, at higher precision than the copy the file already holds.

llama.cpp loads `LLM_TENSOR_OUTPUT` as `TENSOR_NOT_REQUIRED` and falls back to
`token_embd` when it is absent, so the tensor can simply be dropped. The output
projection then runs through the Q6_K copy instead of the F16 one — the only
numerical change in this repack.

Removing it also takes 466.7 MB off the memory read on **every decoded token**, which
was roughly half the LM's per-token bandwidth.

## What it costs

Not nothing. Moving the output projection from F16 to Q6_K is measurable.
FLEURS, 24 clips per language, greedy decoding, 2 threads, numbers spelled out on
both sides before scoring:

| Language | microsoft/VibeVoice-ASR-BitNet | this repo | Δ |
|:--|--:|--:|--:|
| Spanish | 6.47 | 6.47 | +0.00 |
| English | 8.23 | 8.58 | +0.34 |
| Portuguese | 8.90 | 8.57 | −0.33 |
| Italian | 9.67 | 9.52 | −0.16 |
| German | 14.63 | 14.98 | +0.35 |
| French | 34.08 | 35.88 | +1.81 |
| **corpus** | **14.31** | **14.69** | **+0.38** |

So: **about +0.4 WER for −47% LM size**. Two languages improve, one is unchanged,
three get worse. At ~400 reference words per language a ±0.3 swing is inside the
noise; French's +1.81 is roughly seven word errors and sits at the edge of it.

Take the trade if size or decode bandwidth matters to you, and don't if you need
every last point of accuracy.

### Bit budget

| Component | Type | Weights | MB | bits/wt |
|:--|:--|--:|--:|--:|
| transformer body | I2_S | 1,310,195,712 | 327.6 | 2.00 |
| token embedding | Q6_K | 233,373,696 | 191.4 | 6.56 |
| norms / biases | F32 | 144,896 | 0.6 | 32.00 |
| **total** | | 1,543,714,304 | **519.6** | **2.69** |

The released file is 4.44 bits/weight overall; this one is 2.69. Note that the
ternary body is packed at exactly **2.000** bits/weight, not log₂3 = 1.585 — I2_S
stores four ternary values per byte and leaves one of four codes unused, which is
68 MB of padding (20.8% of the body).

## Usage

Drop-in for the released model — same runtime, same flags:

```bash
./build/bin/asr_infer \
    --vae-model vibeasr-vae-encoder-i8_s.gguf \
    --lm-model  vibeasr-lm-i2_s-tied.gguf \
    --audio input.wav -t 4 --greedy
```

## Languages

VibeVoice-ASR was trained on **en, zh, fr, it, ko, pt, vi**. Among EU official
languages that means English, French, Italian and Portuguese are in-distribution;
Spanish and German are not in the training mix but generalise usably. The other EU
languages degrade sharply and this repack does not change that — it is a packaging
fix, not a capability change.

## Speed

Not covered here by design. This card documents the **model artifact** — what
changed in the weights and what it costs in accuracy. CPU inference speed is a
property of the runtime, and the fork this model ships with carries substantial
kernel work (AVX-512/VNNI dispatch, a register-tiled INT8 GEMM, vectorised
quantisation epilogues — RTF well under real time on 4 modest cores). The measured
speed tables, the per-stage breakdowns, and the scripts that regenerate them live in
the GitHub README:

➡️ **[martinobettucci/VibeASR-bitnet.cpp](https://github.com/martinobettucci/VibeASR-bitnet.cpp)** — "CPU optimisation on AVX-512"

The WER tables above were produced by that repo's benchmark harness
(`bench/run_asr.py`, methodology in `bench/README.md`); the harness documentation is
the reference for how they were scored (FLEURS slices, corpus-level WER, digit runs
spelled out in the clip's language on both sides).

## Provenance

Produced with `tools/requant_lm_head.cpp --drop` from
[martinobettucci/VibeASR-bitnet.cpp](https://github.com/martinobettucci/VibeASR-bitnet.cpp),
branch `claude/asr-cpu-optimization-cztnh9`. The VAE encoder and tokenizer files are
copied unmodified from the upstream repo.

Licensed MIT, as upstream.