card: drop PESQ (favors waveform codecs); note multilingual set
Browse files
README.md
CHANGED
|
@@ -42,7 +42,7 @@ Load instructions (canonical):
|
|
| 42 |
|
| 43 |
## How WideCodec compares (44 kHz-class, by bitrate)
|
| 44 |
|
| 45 |
-
Offline benchmark on a **9,291-clip wideband set** (188 clean β₯44.1 kHz datasets),
|
| 46 |
faster-UTMOSv2 + spectral vs ground truth. **Read this by bitrate, not raw MOS** β these
|
| 47 |
codecs span a ~30Γ bitrate range. DAC/NeMo/EnCodec are high-bitrate multi-codebook RVQ
|
| 48 |
*acoustic* codecs; WideCodec is a **single-codebook, 50-tokens/s** code at **~0.8 kbps** β
|
|
@@ -50,20 +50,20 @@ the only one here whose token stream an autoregressive LLM/TTS model can practic
|
|
| 50 |
|
| 51 |

|
| 52 |
|
| 53 |
-
| codec | native SR | tokens/s | codebooks | ~bitrate | UTMOSv2 |
|
| 54 |
-
|---|--:|--:|--:|--:|--:|--:|--:|--:|
|
| 55 |
-
| ground truth | β | β | β | β | 2.822 | β |
|
| 56 |
-
| nvidia nemo44k | 44.1k | ~86 | RVQ (many) | ~6β9 kbps | **2.903** |
|
| 57 |
-
| **WideCodec (ours, gs-2.0M)** | 44.1k | **50** | **1** | **~0.8 kbps** | **2.788** |
|
| 58 |
-
| dac | 44.1k | 86 | 9 | ~8 kbps | 2.672 |
|
| 59 |
-
| snac44k | 44.1k | multi-scale | 3β4 | ~2.6 kbps | 2.340 |
|
| 60 |
-
| encodec48k | 48k | 150 | RVQ | 24 kbps | 2.042 |
|
| 61 |
|
| 62 |
**Bitrate = frame-rate Γ Ξ£ logβ(codebook size).** WideCodec's FSQ is `levels=[4]Γ8,
|
| 63 |
num_quantizers=1` β 8Β·logβ4 = **16 bits/frame** Γ 50 frames/s = **800 bps = 0.8 kbps**
|
| 64 |
(vs DAC's 9 codebooks Γ 10 bits Γ 86 fps β 8 kbps).
|
| 65 |
|
| 66 |
-
The **WideCodec row is the final gs-2.0M checkpoint** (9,291-clip set). At ~0.8 kbps / 1 codebook it **beats DAC on naturalness** (UTMOSv2 2.79 vs 2.67) using ~10x fewer bits, and lands within ~0.12 of NeMo's ~6-9 kbps RVQ. It carries genuine high-band content (rolloff 9.0 kHz; HF energy ~91% of ground truth), and
|
| 67 |
[evaluation/REPORT_44k_class_codecs.md](https://github.com/Scicom-AI-Enterprise-Organization/WideCodec/blob/main/evaluation/REPORT_44k_class_codecs.md).
|
| 68 |
|
| 69 |
## Training stages / which revision to pull
|
|
|
|
| 42 |
|
| 43 |
## How WideCodec compares (44 kHz-class, by bitrate)
|
| 44 |
|
| 45 |
+
Offline benchmark on a **heavily multilingual 9,291-clip wideband set** (50 clips Γ 188 clean β₯44.1 kHz datasets, **400+ language/dialect tags**),
|
| 46 |
faster-UTMOSv2 + spectral vs ground truth. **Read this by bitrate, not raw MOS** β these
|
| 47 |
codecs span a ~30Γ bitrate range. DAC/NeMo/EnCodec are high-bitrate multi-codebook RVQ
|
| 48 |
*acoustic* codecs; WideCodec is a **single-codebook, 50-tokens/s** code at **~0.8 kbps** β
|
|
|
|
| 50 |
|
| 51 |

|
| 52 |
|
| 53 |
+
| codec | native SR | tokens/s | codebooks | ~bitrate | UTMOSv2 | mel-L1 β | HFβ₯11k | rolloff |
|
| 54 |
+
|---|--:|--:|--:|--:|--:|--:|--:|--:|
|
| 55 |
+
| ground truth | β | β | β | β | 2.822 | β | 0.00244 | 8263 |
|
| 56 |
+
| nvidia nemo44k | 44.1k | ~86 | RVQ (many) | ~6β9 kbps | **2.903** | 0.379 | 0.00196 | 8372 |
|
| 57 |
+
| **WideCodec (ours, gs-2.0M)** | 44.1k | **50** | **1** | **~0.8 kbps** | **2.788** | 0.571 | 0.00223 | 9011 |
|
| 58 |
+
| dac | 44.1k | 86 | 9 | ~8 kbps | 2.672 | **0.341** | 0.00156 | 8185 |
|
| 59 |
+
| snac44k | 44.1k | multi-scale | 3β4 | ~2.6 kbps | 2.340 | 0.493 | 0.00158 | 8402 |
|
| 60 |
+
| encodec48k | 48k | 150 | RVQ | 24 kbps | 2.042 | 0.458 | 0.00169 | 8800 |
|
| 61 |
|
| 62 |
**Bitrate = frame-rate Γ Ξ£ logβ(codebook size).** WideCodec's FSQ is `levels=[4]Γ8,
|
| 63 |
num_quantizers=1` β 8Β·logβ4 = **16 bits/frame** Γ 50 frames/s = **800 bps = 0.8 kbps**
|
| 64 |
(vs DAC's 9 codebooks Γ 10 bits Γ 86 fps β 8 kbps).
|
| 65 |
|
| 66 |
+
The **WideCodec row is the final gs-2.0M checkpoint** (9,291-clip set). At ~0.8 kbps / 1 codebook it **beats DAC on naturalness** (UTMOSv2 2.79 vs 2.67) using ~10x fewer bits, and lands within ~0.12 of NeMo's ~6-9 kbps RVQ. It carries genuine high-band content (rolloff 9.0 kHz; HF energy ~91% of ground truth), and genuine high-band content (reference-based LSD-HF 0.96, on par with the 8-kbps RVQ codecs), trading in-band fidelity for a ~10x bitrate advantage. (PESQ/SI-SDR are excluded β they structurally penalize a low-bitrate generative decoder regardless of naturalness.) Full report:
|
| 67 |
[evaluation/REPORT_44k_class_codecs.md](https://github.com/Scicom-AI-Enterprise-Organization/WideCodec/blob/main/evaluation/REPORT_44k_class_codecs.md).
|
| 68 |
|
| 69 |
## Training stages / which revision to pull
|