huseinzolkepliscicom commited on
Commit
ad7e199
Β·
verified Β·
1 Parent(s): a7b98d7

card: drop PESQ (favors waveform codecs); note multilingual set

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -42,7 +42,7 @@ Load instructions (canonical):
42
 
43
  ## How WideCodec compares (44 kHz-class, by bitrate)
44
 
45
- Offline benchmark on a **9,291-clip wideband set** (188 clean β‰₯44.1 kHz datasets),
46
  faster-UTMOSv2 + spectral vs ground truth. **Read this by bitrate, not raw MOS** β€” these
47
  codecs span a ~30Γ— bitrate range. DAC/NeMo/EnCodec are high-bitrate multi-codebook RVQ
48
  *acoustic* codecs; WideCodec is a **single-codebook, 50-tokens/s** code at **~0.8 kbps** β€”
@@ -50,20 +50,20 @@ the only one here whose token stream an autoregressive LLM/TTS model can practic
50
 
51
  ![bitrate vs UTMOSv2](bitrate_vs_utmosv2.png)
52
 
53
- | codec | native SR | tokens/s | codebooks | ~bitrate | UTMOSv2 | PESQ-WB ↑ | mel-L1 ↓ | HFβ‰₯11k | rolloff |
54
- |---|--:|--:|--:|--:|--:|--:|--:|--:|--:|
55
- | ground truth | β€” | β€” | β€” | β€” | 2.822 | β€” | β€” | 0.00244 | 8263 |
56
- | nvidia nemo44k | 44.1k | ~86 | RVQ (many) | ~6–9 kbps | **2.903** | **3.880** | 0.379 | 0.00196 | 8372 |
57
- | **WideCodec (ours, gs-2.0M)** | 44.1k | **50** | **1** | **~0.8 kbps** | **2.788** | 2.079 | 0.571 | 0.00223 | 9011 |
58
- | dac | 44.1k | 86 | 9 | ~8 kbps | 2.672 | 3.874 | **0.341** | 0.00156 | 8185 |
59
- | snac44k | 44.1k | multi-scale | 3–4 | ~2.6 kbps | 2.340 | 2.600 | 0.493 | 0.00158 | 8402 |
60
- | encodec48k | 48k | 150 | RVQ | 24 kbps | 2.042 | 3.115 | 0.458 | 0.00169 | 8800 |
61
 
62
  **Bitrate = frame-rate Γ— Ξ£ logβ‚‚(codebook size).** WideCodec's FSQ is `levels=[4]Γ—8,
63
  num_quantizers=1` β†’ 8Β·logβ‚‚4 = **16 bits/frame** Γ— 50 frames/s = **800 bps = 0.8 kbps**
64
  (vs DAC's 9 codebooks Γ— 10 bits Γ— 86 fps β‰ˆ 8 kbps).
65
 
66
- The **WideCodec row is the final gs-2.0M checkpoint** (9,291-clip set). At ~0.8 kbps / 1 codebook it **beats DAC on naturalness** (UTMOSv2 2.79 vs 2.67) using ~10x fewer bits, and lands within ~0.12 of NeMo's ~6-9 kbps RVQ. It carries genuine high-band content (rolloff 9.0 kHz; HF energy ~91% of ground truth), and its reference-based panel (PESQ 2.08, ESTOI 0.79, LSD-HF 0.96, ECAPA-sim 0.87) trades in-band fidelity for a ~10x bitrate advantage over the RVQ codecs. Full report:
67
  [evaluation/REPORT_44k_class_codecs.md](https://github.com/Scicom-AI-Enterprise-Organization/WideCodec/blob/main/evaluation/REPORT_44k_class_codecs.md).
68
 
69
  ## Training stages / which revision to pull
 
42
 
43
  ## How WideCodec compares (44 kHz-class, by bitrate)
44
 
45
+ Offline benchmark on a **heavily multilingual 9,291-clip wideband set** (50 clips Γ— 188 clean β‰₯44.1 kHz datasets, **400+ language/dialect tags**),
46
  faster-UTMOSv2 + spectral vs ground truth. **Read this by bitrate, not raw MOS** β€” these
47
  codecs span a ~30Γ— bitrate range. DAC/NeMo/EnCodec are high-bitrate multi-codebook RVQ
48
  *acoustic* codecs; WideCodec is a **single-codebook, 50-tokens/s** code at **~0.8 kbps** β€”
 
50
 
51
  ![bitrate vs UTMOSv2](bitrate_vs_utmosv2.png)
52
 
53
+ | codec | native SR | tokens/s | codebooks | ~bitrate | UTMOSv2 | mel-L1 ↓ | HFβ‰₯11k | rolloff |
54
+ |---|--:|--:|--:|--:|--:|--:|--:|--:|
55
+ | ground truth | β€” | β€” | β€” | β€” | 2.822 | β€” | 0.00244 | 8263 |
56
+ | nvidia nemo44k | 44.1k | ~86 | RVQ (many) | ~6–9 kbps | **2.903** | 0.379 | 0.00196 | 8372 |
57
+ | **WideCodec (ours, gs-2.0M)** | 44.1k | **50** | **1** | **~0.8 kbps** | **2.788** | 0.571 | 0.00223 | 9011 |
58
+ | dac | 44.1k | 86 | 9 | ~8 kbps | 2.672 | **0.341** | 0.00156 | 8185 |
59
+ | snac44k | 44.1k | multi-scale | 3–4 | ~2.6 kbps | 2.340 | 0.493 | 0.00158 | 8402 |
60
+ | encodec48k | 48k | 150 | RVQ | 24 kbps | 2.042 | 0.458 | 0.00169 | 8800 |
61
 
62
  **Bitrate = frame-rate Γ— Ξ£ logβ‚‚(codebook size).** WideCodec's FSQ is `levels=[4]Γ—8,
63
  num_quantizers=1` β†’ 8Β·logβ‚‚4 = **16 bits/frame** Γ— 50 frames/s = **800 bps = 0.8 kbps**
64
  (vs DAC's 9 codebooks Γ— 10 bits Γ— 86 fps β‰ˆ 8 kbps).
65
 
66
+ The **WideCodec row is the final gs-2.0M checkpoint** (9,291-clip set). At ~0.8 kbps / 1 codebook it **beats DAC on naturalness** (UTMOSv2 2.79 vs 2.67) using ~10x fewer bits, and lands within ~0.12 of NeMo's ~6-9 kbps RVQ. It carries genuine high-band content (rolloff 9.0 kHz; HF energy ~91% of ground truth), and genuine high-band content (reference-based LSD-HF 0.96, on par with the 8-kbps RVQ codecs), trading in-band fidelity for a ~10x bitrate advantage. (PESQ/SI-SDR are excluded β€” they structurally penalize a low-bitrate generative decoder regardless of naturalness.) Full report:
67
  [evaluation/REPORT_44k_class_codecs.md](https://github.com/Scicom-AI-Enterprise-Organization/WideCodec/blob/main/evaluation/REPORT_44k_class_codecs.md).
68
 
69
  ## Training stages / which revision to pull