| --- |
| license: other |
| language: |
| - zh |
| - yue |
| - en |
| - ja |
| - ko |
| base_model: FunAudioLLM/SenseVoiceSmall |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - sensevoice |
| - funasr |
| - gguf |
| - quantized |
| --- |
| |
| # SenseVoiceSmall GGUF low-memory quantizations |
|
|
| Experimental Q5_0 and Q4_0 GGUF exports of |
| [`FunAudioLLM/SenseVoiceSmall`](https://huggingface.co/FunAudioLLM/SenseVoiceSmall). |
| The vocabulary and CMVN tensors are embedded in each file. |
|
|
| The upstream Q8_0 baseline is available from |
| [`FunAudioLLM/SenseVoiceSmall-GGUF`](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF). |
| |
| ## Files |
| |
| | File | Size | Intended use | |
| | --- | ---: | --- | |
| | `sensevoice-small-q5_0.gguf` | 159.4 MiB | Optional low-memory balance | |
| | `sensevoice-small-q4_0.gguf` | 131.7 MiB | Experimental only; substantial output drift | |
|
|
| These files were exported directly from the official FP32 checkpoint. They |
| were not requantized from Q8. |
|
|
| ## Measured behavior |
|
|
| Measurements used the persistent CPU FunASR GGUF runtime on an Intel Core |
| i7-14700KF and 59 Mandarin/English mixed-language clips. |
|
|
| | Quantization | Model size | Resident peak working set | 30-second recognition | |
| | --- | ---: | ---: | ---: | |
| | Q8_0 baseline | 242.4 MiB | 358.8 MiB | 993 ms | |
| | Q5_0 | 159.4 MiB | 275.8 MiB | 1,179 ms | |
| | Q4_0 | 131.7 MiB | 248.1 MiB | 955 ms | |
| |
| Q5_0 matched the Q8 token sequence exactly on 15 of 59 clips and had 9.2% |
| aggregate token edit distance from Q8. Q4_0 matched 4 clips and had 22.3% |
| token edit distance. These are output-drift measurements relative to Q8, not |
| labeled word-error-rate measurements. Keep Q8 for quality-sensitive use; |
| validate Q5_0 with your own languages and microphones before deployment. |
|
|
| ## Compatibility |
|
|
| These files use the custom `sensevoice-small` GGUF architecture implemented |
| by FunASR's `runtime/llama.cpp/sensevoice` runtime. They are not language |
| models and cannot be loaded by general-purpose llama.cpp applications. |
|
|
| The extended exporter used for Q5_0 and Q4_0 is included in this repository. |
|
|
| ## Integrity |
|
|
| ```text |
| source model.pt |
| size: 936291369 bytes |
| sha256: 833ca2dcfdf8ec91bd4f31cfac36d6124e0c459074d5e909aec9cabe6204a3ea |
| |
| sensevoice-small-q5_0.gguf |
| size: 167117312 bytes |
| sha256: 24114cc2663de19da1f8c53c2232d9c98f8a6d9e663b2ab4766b414e691b6818 |
| |
| sensevoice-small-q4_0.gguf |
| size: 138086976 bytes |
| sha256: 22ea2934da85d1abc07bef5b6b6fb34ff0f45704467d02d78df5ba11df40d537 |
| ``` |
|
|
| ## License |
|
|
| Use of these derived weights is subject to the license and terms of the base |
| SenseVoiceSmall model. The exporter/runtime source follows its upstream FunASR |
| license. |
|
|
