Add TF-GridNet F32 model
Browse files- .gitattributes +1 -0
- TF-GridNet-GGUF/README.md +71 -0
- TF-GridNet-GGUF/tf-gridnet-wsj0-2mix-f32.gguf +3 -0
.gitattributes
CHANGED
|
@@ -133,3 +133,4 @@ OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
|
| 133 |
OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 134 |
OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 135 |
Breeze-TTS-2-GGUF/breeze-tts-2-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 133 |
OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 134 |
OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 135 |
Breeze-TTS-2-GGUF/breeze-tts-2-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 136 |
+
TF-GridNet-GGUF/tf-gridnet-wsj0-2mix-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
TF-GridNet-GGUF/README.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TF-GridNet GGUF
|
| 2 |
+
|
| 3 |
+
Use with [audio.cpp](https://github.com/0xShug0/audio.cpp) for offline speech
|
| 4 |
+
separation. This package contains the ESPnet TF-GridNet WSJ0-2mix checkpoint,
|
| 5 |
+
its configuration, STFT window, and audio.cpp model spec in one GGUF.
|
| 6 |
+
|
| 7 |
+
## Source
|
| 8 |
+
|
| 9 |
+
- [Original checkpoint archive](https://drive.google.com/file/d/1TasZxZSnbSPsk_Wf7ZDhBAigS6zN8G9G/view): `98epoch.pth` and its matching `config.yaml`.
|
| 10 |
+
- [Official ESPnet example](https://kan-bayashi.github.io/espnet/notebook/SpeechEnhancement_CMU_11492_692_Spring2023%28Assignment7%29.html#model-selection).
|
| 11 |
+
- [TF-GridNet paper](https://arxiv.org/abs/2211.12433).
|
| 12 |
+
|
| 13 |
+
The checkpoint has 8,175,874 parameters stored as F32. The F32 conversion verifies
|
| 14 |
+
all 541 packaged tensors byte-for-byte, including the generated STFT window.
|
| 15 |
+
Only F32 is provided. The 16-bit candidates did not improve speed or peak VRAM
|
| 16 |
+
enough to justify their long-form speaker-track assignment differences.
|
| 17 |
+
It produces two speaker tracks at 8 kHz. The runtime supports other configured
|
| 18 |
+
speaker counts, but that requires a checkpoint trained for that count.
|
| 19 |
+
|
| 20 |
+
**License:** the downloaded checkpoint archive does not state a weight license.
|
| 21 |
+
ESPnet's source-code license does not establish the checkpoint's license.
|
| 22 |
+
Confirm permission with the checkpoint authors before redistribution or use.
|
| 23 |
+
|
| 24 |
+
## Usage
|
| 25 |
+
|
| 26 |
+
### Continuous Dialogue
|
| 27 |
+
|
| 28 |
+
A CUDA comparison used a continuous 167.333-second two-speaker conversation
|
| 29 |
+
generated by VibeVoice 1.5B. RTX 5090, eight threads, Debug build, logging enabled,
|
| 30 |
+
default separation settings, no precision overrides. Each session ran a short
|
| 31 |
+
warmup, the conversation, then the same short request. Peak VRAM was sampled
|
| 32 |
+
per process every 20 ms. Python was not rerun for this comparison.
|
| 33 |
+
|
| 34 |
+
| C++ package | Warm time | RTF | Peak VRAM | File size |
|
| 35 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 36 |
+
| F32 | 24.614 s | 0.14710 | 850 MiB | 31.27 MiB |
|
| 37 |
+
|
| 38 |
+
The short request repeated after long-form inference was byte-identical,
|
| 39 |
+
so this test found no request-state pollution.
|
| 40 |
+
|
| 41 |
+
### Command
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
audiocpp_cli --task sep --family tf_gridnet \
|
| 45 |
+
--model /path/to/tf-gridnet-wsj0-2mix-f32.gguf \
|
| 46 |
+
--backend cuda --audio mixture.wav --out-dir separated --log
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
The outputs are `speaker_1.wav` and `speaker_2.wav`. These labels are output
|
| 50 |
+
lanes, not identified people. The default uses 2.4-second windows with 1.6-second
|
| 51 |
+
overlap, matches speakers across overlaps, and normalizes each final track to
|
| 52 |
+
a peak of 0.9. Long silent gaps can make speaker matching ambiguous.
|
| 53 |
+
|
| 54 |
+
Use `--request-option audio_chunk_duration_sec=0` for whole-file inference.
|
| 55 |
+
Memory then grows with recording length. See the
|
| 56 |
+
[model documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/tf_gridnet.md)
|
| 57 |
+
for options and conversion instructions.
|
| 58 |
+
|
| 59 |
+
## Validation
|
| 60 |
+
|
| 61 |
+
On the official 11.49-second example mixture, C++ CUDA versus ESPnet's chunked
|
| 62 |
+
pipeline produced waveform cosine similarities of **0.99999991 / 0.99999998**.
|
| 63 |
+
Whole-file inference was also checked on CPU and Vulkan against Python, with
|
| 64 |
+
both speaker tracks above **0.9999999** cosine similarity.
|
| 65 |
+
|
| 66 |
+
The 327.6-second chunked CUDA comparison produced **0.99999841 / 0.99999778**.
|
| 67 |
+
Both tracks had exactly the same length as Python and only finite samples.
|
| 68 |
+
The short C++ request before and after the long request was byte-identical for
|
| 69 |
+
both speakers. These are numerical output comparisons, not claims about speech
|
| 70 |
+
separation quality on every recording.
|
| 71 |
+
|
TF-GridNet-GGUF/tf-gridnet-wsj0-2mix-f32.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:924f217010be282f01f3cbd31e74286fb8225fe6ce82441b67241130d33bf4de
|
| 3 |
+
size 32792096
|