README.md

---
license: apache-2.0
language:
  - en
base_model: latishab/turnsense
base_model_relation: quantized
pipeline_tag: text-classification
library_name: ggml
tags:
  - gguf
  - ggml
  - turn-taking
  - end-of-utterance
  - voice-agents
  - text-classification
---

# TurnSense.cpp GGUF

Canonical F32 and selective Q8_0 GGUF artifacts for
[TurnSense.cpp](https://github.com/simonfxr/turnsense.cpp), a native C/C++
end-of-utterance classifier built on official upstream ggml.

These files are converted from
[`latishab/turnsense`](https://huggingface.co/latishab/turnsense) at immutable
revision `1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63`. The model architecture and
weights are unchanged except for deterministic LoRA merging and, for the Q8_0
artifact, weight quantization. The production runtime has no ONNX Runtime or
Python dependency.

## Files

| File | Purpose | Size | SHA-256 |
|---|---|---:|---|
| `turnsense-q8_0.gguf` | Recommended CPU/Vulkan runtime model | 145,012,704 bytes | `e978f2462e1887c2959173066e2f534f3c02fea686323bfc454aa37842970966` |
| `turnsense-f32.gguf` | Canonical conversion and quantization source | 540,047,168 bytes | `0eb7d0ea7bccbcd2f9aa4b722ee202efbf53b4e430ba90c527548f8578d2460e` |

`SHA256SUMS` contains the same checksums in machine-readable form.

## Q8_0 policy

The Q8_0 artifact is produced from the canonical F32 GGUF with the native
`turnsense_quantize` tool and upstream `ggml_quantize_chunk`:

- **Q8_0:** `token_embd.weight` and all 210 transformer attention/FFN
  projection matrices.
- **F32:** all 61 RMS normalization vectors and `classifier.weight`.

This quantizes 211 tensors and retains 62 tensors as F32. Tensor payload drops
from 513.14 MiB to 136.40 MiB, a 3.76x reduction. Repeated conversion produces
a byte-identical Q8_0 file. The runtime validates this policy exactly and
rejects incompatible layouts.

The dynamically quantized upstream ONNX model is **not** the source of this
Q8_0 artifact.

## Usage

Download the recommended model:

```bash
hf download simonfxr/turnsense.cpp-GGUF turnsense-q8_0.gguf \
  --local-dir models
```

Build and run the native runtime:

```bash
git clone --recurse-submodules https://github.com/simonfxr/turnsense.cpp.git
cd turnsense.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

./build/turnsense_cli \
  --model models/turnsense-q8_0.gguf \
  --backend cpu \
  --json "Could you send that report tomorrow?"
```

Use `--backend vulkan` for the Vulkan backend. The stable C API is documented
in the [source repository README](https://github.com/simonfxr/turnsense.cpp#c-api).

TurnSense operates on text, not audio. A voice agent should evaluate the latest
punctuated STT transcript at candidate pauses and combine `prob_eou` with VAD,
latency, and product policy.

## Validation

The runtime graph uses official ggml operations and optimized backend kernels.
For Q8_0, it verifies that every quantized `get_rows` and `mul_mat` node remains
on the selected CPU or Vulkan backend rather than unexpectedly falling back.

Measured maximum absolute probability differences on the bundled seven-case
fixture suite:

| Backend / artifact | Reference | Maximum delta | Classifications |
|---|---|---:|---|
| CPU F32 | FP32 ONNX Runtime | `9.84e-7` | identical |
| Vulkan F32 | FP32 ONNX Runtime | `5.69e-4` | identical |
| CPU Q8_0 | native CPU F32 | `0.04161` | identical |
| Vulkan Q8_0 | native Vulkan F32 | `0.01413` | identical |

Vulkan was validated on an AMD Radeon RX 7900 XTX. Backend floating-point
accumulation is not expected to be bit-identical.

## Prompt and labels

The runtime prepends the literal prompt prefix `<|user|> ` and applies the
embedded GPT-2 byte-level BPE tokenizer. Do not append `<|im_end|>`.

- Label `0`: `NON_EOU`
- Label `1`: `EOU`

Applications should normally use `prob_eou` with a product-specific threshold
rather than treating argmax as a fixed policy.

## Limitations

- The upstream model is English-focused.
- Predictions are sensitive to punctuation and STT transcript quality.
- Turn-taking decisions should also incorporate VAD, timing, and application
  context.
- The fixture suite validates conversion parity; it is not a broad task-quality
  benchmark.
- This model is not intended for safety-critical decisions.

## Provenance

- Upstream source: <https://github.com/latishab/turnsense>
- Upstream source revision: `b40a25b4da94c961b64393752e507f59295061ad`
- Upstream model: <https://huggingface.co/latishab/turnsense>
- Upstream model revision: `1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63`
- Runtime source: <https://github.com/simonfxr/turnsense.cpp>

Full source artifact hashes and deterministic conversion instructions are in
[`docs/model-provenance.md`](https://github.com/simonfxr/turnsense.cpp/blob/main/docs/model-provenance.md).

## License

The model artifacts are distributed under the Apache License 2.0, matching the
upstream TurnSense model. See `LICENSE`. The TurnSense.cpp runtime source is
separately distributed under the MIT License.