| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: latishab/turnsense |
| base_model_relation: quantized |
| pipeline_tag: text-classification |
| library_name: ggml |
| tags: |
| - gguf |
| - ggml |
| - turn-taking |
| - end-of-utterance |
| - voice-agents |
| - text-classification |
| --- |
| |
| # TurnSense.cpp GGUF |
|
|
| Canonical F32 and selective Q8_0 GGUF artifacts for |
| [TurnSense.cpp](https://github.com/simonfxr/turnsense.cpp), a native C/C++ |
| end-of-utterance classifier built on official upstream ggml. |
| |
| These files are converted from |
| [`latishab/turnsense`](https://huggingface.co/latishab/turnsense) at immutable |
| revision `1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63`. The model architecture and |
| weights are unchanged except for deterministic LoRA merging and, for the Q8_0 |
| artifact, weight quantization. The production runtime has no ONNX Runtime or |
| Python dependency. |
|
|
| ## Files |
|
|
| | File | Purpose | Size | SHA-256 | |
| |---|---|---:|---| |
| | `turnsense-q8_0.gguf` | Recommended CPU/Vulkan runtime model | 145,012,704 bytes | `e978f2462e1887c2959173066e2f534f3c02fea686323bfc454aa37842970966` | |
| | `turnsense-f32.gguf` | Canonical conversion and quantization source | 540,047,168 bytes | `0eb7d0ea7bccbcd2f9aa4b722ee202efbf53b4e430ba90c527548f8578d2460e` | |
|
|
| `SHA256SUMS` contains the same checksums in machine-readable form. |
|
|
| ## Q8_0 policy |
| |
| The Q8_0 artifact is produced from the canonical F32 GGUF with the native |
| `turnsense_quantize` tool and upstream `ggml_quantize_chunk`: |
|
|
| - **Q8_0:** `token_embd.weight` and all 210 transformer attention/FFN |
| projection matrices. |
| - **F32:** all 61 RMS normalization vectors and `classifier.weight`. |
|
|
| This quantizes 211 tensors and retains 62 tensors as F32. Tensor payload drops |
| from 513.14 MiB to 136.40 MiB, a 3.76x reduction. Repeated conversion produces |
| a byte-identical Q8_0 file. The runtime validates this policy exactly and |
| rejects incompatible layouts. |
| |
| The dynamically quantized upstream ONNX model is **not** the source of this |
| Q8_0 artifact. |
|
|
| ## Usage |
|
|
| Download the recommended model: |
|
|
| ```bash |
| hf download simonfxr/turnsense.cpp-GGUF turnsense-q8_0.gguf \ |
| --local-dir models |
| ``` |
|
|
| Build and run the native runtime: |
|
|
| ```bash |
| git clone --recurse-submodules https://github.com/simonfxr/turnsense.cpp.git |
| cd turnsense.cpp |
| cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release |
| cmake --build build -j |
| |
| ./build/turnsense_cli \ |
| --model models/turnsense-q8_0.gguf \ |
| --backend cpu \ |
| --json "Could you send that report tomorrow?" |
| ``` |
|
|
| Use `--backend vulkan` for the Vulkan backend. The stable C API is documented |
| in the [source repository README](https://github.com/simonfxr/turnsense.cpp#c-api). |
|
|
| TurnSense operates on text, not audio. A voice agent should evaluate the latest |
| punctuated STT transcript at candidate pauses and combine `prob_eou` with VAD, |
| latency, and product policy. |
|
|
| ## Validation |
|
|
| The runtime graph uses official ggml operations and optimized backend kernels. |
| For Q8_0, it verifies that every quantized `get_rows` and `mul_mat` node remains |
| on the selected CPU or Vulkan backend rather than unexpectedly falling back. |
| |
| Measured maximum absolute probability differences on the bundled seven-case |
| fixture suite: |
| |
| | Backend / artifact | Reference | Maximum delta | Classifications | |
| |---|---|---:|---| |
| | CPU F32 | FP32 ONNX Runtime | `9.84e-7` | identical | |
| | Vulkan F32 | FP32 ONNX Runtime | `5.69e-4` | identical | |
| | CPU Q8_0 | native CPU F32 | `0.04161` | identical | |
| | Vulkan Q8_0 | native Vulkan F32 | `0.01413` | identical | |
| |
| Vulkan was validated on an AMD Radeon RX 7900 XTX. Backend floating-point |
| accumulation is not expected to be bit-identical. |
| |
| ## Prompt and labels |
| |
| The runtime prepends the literal prompt prefix `<|user|> ` and applies the |
| embedded GPT-2 byte-level BPE tokenizer. Do not append `<|im_end|>`. |
|
|
| - Label `0`: `NON_EOU` |
| - Label `1`: `EOU` |
|
|
| Applications should normally use `prob_eou` with a product-specific threshold |
| rather than treating argmax as a fixed policy. |
|
|
| ## Limitations |
|
|
| - The upstream model is English-focused. |
| - Predictions are sensitive to punctuation and STT transcript quality. |
| - Turn-taking decisions should also incorporate VAD, timing, and application |
| context. |
| - The fixture suite validates conversion parity; it is not a broad task-quality |
| benchmark. |
| - This model is not intended for safety-critical decisions. |
|
|
| ## Provenance |
|
|
| - Upstream source: <https://github.com/latishab/turnsense> |
| - Upstream source revision: `b40a25b4da94c961b64393752e507f59295061ad` |
| - Upstream model: <https://huggingface.co/latishab/turnsense> |
| - Upstream model revision: `1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63` |
| - Runtime source: <https://github.com/simonfxr/turnsense.cpp> |
|
|
| Full source artifact hashes and deterministic conversion instructions are in |
| [`docs/model-provenance.md`](https://github.com/simonfxr/turnsense.cpp/blob/main/docs/model-provenance.md). |
|
|
| ## License |
|
|
| The model artifacts are distributed under the Apache License 2.0, matching the |
| upstream TurnSense model. See `LICENSE`. The TurnSense.cpp runtime source is |
| separately distributed under the MIT License. |
|
|
