Hugging Face's logo

simonfxr
/
turnsense.cpp-GGUF

Text Classification
GGUF
English
ggml
turn-taking
end-of-utterance
voice-agents
conversational
Model card Files
xet
 Community
turnsense.cpp-GGUF / README.md
simonfxr's picture
simonfxr
Add TurnSense.cpp GGUF model card and licensing
20e5f36 verified
preview code
|
Raw
History Blame Contribute Delete
5.04 kB
metadata 
license: apache-2.0
language:
  - en
base_model: latishab/turnsense
base_model_relation: quantized
pipeline_tag: text-classification
library_name: ggml
tags:
  - gguf
  - ggml
  - turn-taking
  - end-of-utterance
  - voice-agents
  - text-classification

TurnSense.cpp GGUF

Canonical F32 and selective Q8_0 GGUF artifacts for TurnSense.cpp, a native C/C++ end-of-utterance classifier built on official upstream ggml.

These files are converted from latishab/turnsense at immutable revision 1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63. The model architecture and weights are unchanged except for deterministic LoRA merging and, for the Q8_0 artifact, weight quantization. The production runtime has no ONNX Runtime or Python dependency.

Files

File Purpose Size SHA-256
turnsense-q8_0.gguf Recommended CPU/Vulkan runtime model 145,012,704 bytes e978f2462e1887c2959173066e2f534f3c02fea686323bfc454aa37842970966
turnsense-f32.gguf Canonical conversion and quantization source 540,047,168 bytes 0eb7d0ea7bccbcd2f9aa4b722ee202efbf53b4e430ba90c527548f8578d2460e

SHA256SUMS contains the same checksums in machine-readable form.

Q8_0 policy

The Q8_0 artifact is produced from the canonical F32 GGUF with the native turnsense_quantize tool and upstream ggml_quantize_chunk:

  • Q8_0: token_embd.weight and all 210 transformer attention/FFN projection matrices.
  • F32: all 61 RMS normalization vectors and classifier.weight.

This quantizes 211 tensors and retains 62 tensors as F32. Tensor payload drops from 513.14 MiB to 136.40 MiB, a 3.76x reduction. Repeated conversion produces a byte-identical Q8_0 file. The runtime validates this policy exactly and rejects incompatible layouts.

The dynamically quantized upstream ONNX model is not the source of this Q8_0 artifact.

Usage

Download the recommended model:

hf download simonfxr/turnsense.cpp-GGUF turnsense-q8_0.gguf \
  --local-dir models

Build and run the native runtime:

git clone --recurse-submodules https://github.com/simonfxr/turnsense.cpp.git
cd turnsense.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

./build/turnsense_cli \
  --model models/turnsense-q8_0.gguf \
  --backend cpu \
  --json "Could you send that report tomorrow?"

Use --backend vulkan for the Vulkan backend. The stable C API is documented in the source repository README.

TurnSense operates on text, not audio. A voice agent should evaluate the latest punctuated STT transcript at candidate pauses and combine prob_eou with VAD, latency, and product policy.

Validation

The runtime graph uses official ggml operations and optimized backend kernels. For Q8_0, it verifies that every quantized get_rows and mul_mat node remains on the selected CPU or Vulkan backend rather than unexpectedly falling back.

Measured maximum absolute probability differences on the bundled seven-case fixture suite:

Backend / artifact Reference Maximum delta Classifications
CPU F32 FP32 ONNX Runtime 9.84e-7 identical
Vulkan F32 FP32 ONNX Runtime 5.69e-4 identical
CPU Q8_0 native CPU F32 0.04161 identical
Vulkan Q8_0 native Vulkan F32 0.01413 identical

Vulkan was validated on an AMD Radeon RX 7900 XTX. Backend floating-point accumulation is not expected to be bit-identical.

Prompt and labels

The runtime prepends the literal prompt prefix <|user|> and applies the embedded GPT-2 byte-level BPE tokenizer. Do not append <|im_end|>.

  • Label 0: NON_EOU
  • Label 1: EOU

Applications should normally use prob_eou with a product-specific threshold rather than treating argmax as a fixed policy.

Limitations

  • The upstream model is English-focused.
  • Predictions are sensitive to punctuation and STT transcript quality.
  • Turn-taking decisions should also incorporate VAD, timing, and application context.
  • The fixture suite validates conversion parity; it is not a broad task-quality benchmark.
  • This model is not intended for safety-critical decisions.

Provenance

Full source artifact hashes and deterministic conversion instructions are in docs/model-provenance.md.

License

The model artifacts are distributed under the Apache License 2.0, matching the upstream TurnSense model. See LICENSE. The TurnSense.cpp runtime source is separately distributed under the MIT License.