---
license: other
language:
  - zh
  - yue
  - en
  - ja
  - ko
base_model: FunAudioLLM/SenseVoiceSmall
pipeline_tag: automatic-speech-recognition
tags:
  - sensevoice
  - funasr
  - gguf
  - quantized
---

# SenseVoiceSmall GGUF low-memory quantizations

Experimental Q5_0 and Q4_0 GGUF exports of
[`FunAudioLLM/SenseVoiceSmall`](https://huggingface.co/FunAudioLLM/SenseVoiceSmall).
The vocabulary and CMVN tensors are embedded in each file.

The upstream Q8_0 baseline is available from
[`FunAudioLLM/SenseVoiceSmall-GGUF`](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF).

## Files

| File | Size | Intended use |
| --- | ---: | --- |
| `sensevoice-small-q5_0.gguf` | 159.4 MiB | Optional low-memory balance |
| `sensevoice-small-q4_0.gguf` | 131.7 MiB | Experimental only; substantial output drift |

These files were exported directly from the official FP32 checkpoint. They
were not requantized from Q8.

## Measured behavior

Measurements used the persistent CPU FunASR GGUF runtime on an Intel Core
i7-14700KF and 59 Mandarin/English mixed-language clips.

| Quantization | Model size | Resident peak working set | 30-second recognition |
| --- | ---: | ---: | ---: |
| Q8_0 baseline | 242.4 MiB | 358.8 MiB | 993 ms |
| Q5_0 | 159.4 MiB | 275.8 MiB | 1,179 ms |
| Q4_0 | 131.7 MiB | 248.1 MiB | 955 ms |

Q5_0 matched the Q8 token sequence exactly on 15 of 59 clips and had 9.2%
aggregate token edit distance from Q8. Q4_0 matched 4 clips and had 22.3%
token edit distance. These are output-drift measurements relative to Q8, not
labeled word-error-rate measurements. Keep Q8 for quality-sensitive use;
validate Q5_0 with your own languages and microphones before deployment.

## Compatibility

These files use the custom `sensevoice-small` GGUF architecture implemented
by FunASR's `runtime/llama.cpp/sensevoice` runtime. They are not language
models and cannot be loaded by general-purpose llama.cpp applications.

The extended exporter used for Q5_0 and Q4_0 is included in this repository.

## Integrity

```text
source model.pt
size:   936291369 bytes
sha256: 833ca2dcfdf8ec91bd4f31cfac36d6124e0c459074d5e909aec9cabe6204a3ea

sensevoice-small-q5_0.gguf
size:   167117312 bytes
sha256: 24114cc2663de19da1f8c53c2232d9c98f8a6d9e663b2ab4766b414e691b6818

sensevoice-small-q4_0.gguf
size:   138086976 bytes
sha256: 22ea2934da85d1abc07bef5b6b6fb34ff0f45704467d02d78df5ba11df40d537
```

## License

Use of these derived weights is subject to the license and terms of the base
SenseVoiceSmall model. The exporter/runtime source follows its upstream FunASR
license.