add: q8 GGUF for local inference with NeMo-Speech.cpp

.gitattributes

@@ -33,6 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
 fifo.png filter=lfs diff=lfs merge=lfs -text
 streaming_steps.png filter=lfs diff=lfs merge=lfs -text
 sortformer_intro.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -267,6 +267,37 @@ for segment in predicted_segments[0]:
 ```
 ## How to Use this Model
 
+There are several ways to use this model. Choose the one that fits your needs.
+
+### Run locally with NeMo-Speech.cpp
+
+[NeMo-Speech.cpp](https://github.com/NVIDIA/NeMo-Speech.cpp) provides a
+lightweight native C++ runtime for local speaker diarization. After
+[installing the runtime](https://github.com/NVIDIA/NeMo-Speech.cpp#installation):
+
+```bash
+hf download nvidia/diar_streaming_sortformer_4spk-v2 \
+  diar_streaming_sortformer_4spk-v2.q8_0.gguf \
+  --local-dir models
+
+nemo-speech diarize meeting.wav \
+  --model models/diar_streaming_sortformer_4spk-v2.q8_0.gguf
+```
+
+The same model can add word-level speaker tags to a transcription:
+
+```bash
+nemo-speech transcribe meeting.wav \
+  --model models/asr-model.gguf \
+  --diar-model models/diar_streaming_sortformer_4spk-v2.q8_0.gguf \
+  --json
+```
+
+See the [NeMo-Speech.cpp diarization guide](https://github.com/NVIDIA/NeMo-Speech.cpp/blob/main/docs/cli.md#diarize-audio)
+for more usage examples.
+
+### NVIDIA NeMo
+
 The model is available for use in the NeMo Framework[6], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
 
 ### Loading the Model
@@ -506,4 +537,4 @@ Also check out the [Riva live demo](https://developer.nvidia.com/riva#demos).
 
 ## Licence
 
-License to use this model is covered by the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode). By downloading the public and release version of the model, you accept the terms and conditions of the CC-BY-4.0 license.
+License to use this model is covered by the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode). By downloading the public and release version of the model, you accept the terms and conditions of the CC-BY-4.0 license.

diar_streaming_sortformer_4spk-v2.q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0679cfeb1ce356d0dea9470b31274f4bfc7eb927497d82005483770666da998a
+size 147075776