audio-cpp commited on
Commit
f96f562
·
verified ·
1 Parent(s): e36610a

Add TF-GridNet F32 model

Browse files
.gitattributes CHANGED
@@ -133,3 +133,4 @@ OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
133
  OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
134
  OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
135
  Breeze-TTS-2-GGUF/breeze-tts-2-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
 
 
133
  OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
134
  OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
135
  Breeze-TTS-2-GGUF/breeze-tts-2-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
136
+ TF-GridNet-GGUF/tf-gridnet-wsj0-2mix-f32.gguf filter=lfs diff=lfs merge=lfs -text
TF-GridNet-GGUF/README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # TF-GridNet GGUF
2
+
3
+ Use with [audio.cpp](https://github.com/0xShug0/audio.cpp) for offline speech
4
+ separation. This package contains the ESPnet TF-GridNet WSJ0-2mix checkpoint,
5
+ its configuration, STFT window, and audio.cpp model spec in one GGUF.
6
+
7
+ ## Source
8
+
9
+ - [Original checkpoint archive](https://drive.google.com/file/d/1TasZxZSnbSPsk_Wf7ZDhBAigS6zN8G9G/view): `98epoch.pth` and its matching `config.yaml`.
10
+ - [Official ESPnet example](https://kan-bayashi.github.io/espnet/notebook/SpeechEnhancement_CMU_11492_692_Spring2023%28Assignment7%29.html#model-selection).
11
+ - [TF-GridNet paper](https://arxiv.org/abs/2211.12433).
12
+
13
+ The checkpoint has 8,175,874 parameters stored as F32. The F32 conversion verifies
14
+ all 541 packaged tensors byte-for-byte, including the generated STFT window.
15
+ Only F32 is provided. The 16-bit candidates did not improve speed or peak VRAM
16
+ enough to justify their long-form speaker-track assignment differences.
17
+ It produces two speaker tracks at 8 kHz. The runtime supports other configured
18
+ speaker counts, but that requires a checkpoint trained for that count.
19
+
20
+ **License:** the downloaded checkpoint archive does not state a weight license.
21
+ ESPnet's source-code license does not establish the checkpoint's license.
22
+ Confirm permission with the checkpoint authors before redistribution or use.
23
+
24
+ ## Usage
25
+
26
+ ### Continuous Dialogue
27
+
28
+ A CUDA comparison used a continuous 167.333-second two-speaker conversation
29
+ generated by VibeVoice 1.5B. RTX 5090, eight threads, Debug build, logging enabled,
30
+ default separation settings, no precision overrides. Each session ran a short
31
+ warmup, the conversation, then the same short request. Peak VRAM was sampled
32
+ per process every 20 ms. Python was not rerun for this comparison.
33
+
34
+ | C++ package | Warm time | RTF | Peak VRAM | File size |
35
+ | --- | ---: | ---: | ---: | ---: |
36
+ | F32 | 24.614 s | 0.14710 | 850 MiB | 31.27 MiB |
37
+
38
+ The short request repeated after long-form inference was byte-identical,
39
+ so this test found no request-state pollution.
40
+
41
+ ### Command
42
+
43
+ ```bash
44
+ audiocpp_cli --task sep --family tf_gridnet \
45
+ --model /path/to/tf-gridnet-wsj0-2mix-f32.gguf \
46
+ --backend cuda --audio mixture.wav --out-dir separated --log
47
+ ```
48
+
49
+ The outputs are `speaker_1.wav` and `speaker_2.wav`. These labels are output
50
+ lanes, not identified people. The default uses 2.4-second windows with 1.6-second
51
+ overlap, matches speakers across overlaps, and normalizes each final track to
52
+ a peak of 0.9. Long silent gaps can make speaker matching ambiguous.
53
+
54
+ Use `--request-option audio_chunk_duration_sec=0` for whole-file inference.
55
+ Memory then grows with recording length. See the
56
+ [model documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/tf_gridnet.md)
57
+ for options and conversion instructions.
58
+
59
+ ## Validation
60
+
61
+ On the official 11.49-second example mixture, C++ CUDA versus ESPnet's chunked
62
+ pipeline produced waveform cosine similarities of **0.99999991 / 0.99999998**.
63
+ Whole-file inference was also checked on CPU and Vulkan against Python, with
64
+ both speaker tracks above **0.9999999** cosine similarity.
65
+
66
+ The 327.6-second chunked CUDA comparison produced **0.99999841 / 0.99999778**.
67
+ Both tracks had exactly the same length as Python and only finite samples.
68
+ The short C++ request before and after the long request was byte-identical for
69
+ both speakers. These are numerical output comparisons, not claims about speech
70
+ separation quality on every recording.
71
+
TF-GridNet-GGUF/tf-gridnet-wsj0-2mix-f32.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:924f217010be282f01f3cbd31e74286fb8225fe6ce82441b67241130d33bf4de
3
+ size 32792096