| --- |
| license: apache-2.0 |
| pipeline_tag: audio-to-audio |
| tags: |
| - audio |
| - speech |
| - codec |
| - neural-codec |
| --- |
| |
| # LILAC: An Idempotent Neural Speech Codec (0.75 kb/s) |
|
|
| **Anonymous review copy — AAAI-27 submission.** |
|
|
| LILAC is a neural speech codec for 24 kHz speech at **0.75 kb/s** |
| (9.375 Hz frame rate, 20 FSQ coordinates × 4 bits per frame) that is |
| **idempotent by construction**: re-encoding its own decoded output |
| reproduces the token stream exactly — `encode(decode(codes)) == codes` — |
| for any number of decode/re-encode cycles. Quality, intelligibility, and |
| downstream metrics are therefore invariant under repeated cycles, where |
| comparison codecs degrade. |
|
|
| The codec is convolution-only: an invertible analysis chart (orthogonal |
| 1×1 mixes + additive couplings), finite scalar quantization on a selected |
| coordinate subset, a deterministic fill network for the discarded |
| coordinates, and an invertible anti-imaging stem acting on the five |
| waveform phases. |
|
|
| ## Files |
|
|
| - `lilac_swa10.pt` — the shipping checkpoint (58.5 M generator |
| parameters, stochastic weight averaging over the final ten |
| 1k-spaced training checkpoints). |
|
|
| ## Usage |
|
|
| With the companion code repository (see the paper's code link): |
|
|
| ```bash |
| uv run python infer.py --checkpoint lilac_swa10.pt \ |
| --input in.wav --output out.wav |
| # idempotence demo: 100 decode->re-encode cycles, codes bit-exact |
| uv run python infer.py --checkpoint lilac_swa10.pt \ |
| --input in.wav --output out100.wav --cycles 100 |
| ``` |
|
|
| Input is resampled to 24 kHz mono; output is 24 kHz PCM16. |
|
|
| ## Evaluation (full test splits, matched-rate comparison in the paper) |
|
|
| | Corpus | UTMOS ↑ | dWER ↓ | PESQ ↑ | STOI ↑ | |
| |---|---|---|---|---| |
| | LibriSpeech test-clean (out-of-domain) | 4.14 | 0.101 | 2.60 | 0.935 | |
| | LibriTTS-R test (out-of-domain) | 4.24 | 0.086 | 2.60 | 0.944 | |
|
|
| Token agreement across 100 decode/re-encode cycles: 1.000 (bit-exact). |
|
|
| ## Training data |
|
|
| Trained on the 44.1 kHz subset of HiFiTTS-2 (LibriVox-derived, |
| ~31,700 h, 4,629 speakers), resampled to 24 kHz. LibriSpeech and |
| LibriTTS(-R) readers are excluded by ID filter; evaluation corpora are |
| out-of-domain. |
|
|
| ## License |
|
|
| Apache-2.0. Copyright 2026 LILAC authors. |
|
|
