File size: 21,253 Bytes
a2c6e79
8ce71a5
a2c6e79
 
 
8ce71a5
a2c6e79
8ce71a5
 
a2c6e79
8ce71a5
 
 
 
 
 
 
 
 
 
 
 
 
a2c6e79
 
 
8ce71a5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0e51189
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8ce71a5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
---
library_name: transformers
license: other
license_name: lfm1.0
license_link: LICENSE
pipeline_tag: feature-extraction
tags:
- liquid
- lfm3
- audio
- audio-codec
- speech
- streaming
- edge
- multilingual
- custom_code
language:
- en
- zh
- de
- fr
- ja
- ko
---


<div align="center">
  <img
    src="https://huggingface.co/LiquidAI/LFM3-1.7B/resolve/main/assets/liquid-ai-banner.webp"
    alt="Liquid AI"
    style="width: 100%; max-width: 100%; height: auto; display: block; margin: 0;"
  />
  <div align="center" style="display: flex; justify-content: center; align-items: center; gap: 1.25em; margin: 8px 0 1.7rem; padding: 0; line-height: 1.3;">
    <a href="https://console.liquid.ai/dashboard" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath d=%22m8 6-6 6 6 6m8-12 6 6-6 6m-2-14-4 16%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>API</strong></a>
    <a href="https://playground.liquid.ai/talk" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath transform=%22translate(3 0)%22 d=%22M8 5.5v13L19 12 8 5.5Z%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Try</strong></a>
    <a href="https://docs.liquid.ai/lfm/getting-started/welcome" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath d=%22M12 6c-3-2-6-2.5-10-2v15c4-.5 7 0 10 2m0-15c3-2 6-2.5 10-2v15c-4-.5-7 0-10 2V6Z%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Docs</strong></a>
    <a href="https://www.liquid.ai/blog/liquidcodec" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath transform=%22translate(1 0)%22 d=%22m16 3 5 5L8 21H3v-5L16 3Zm-3 3 5 5%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Blog</strong></a>
  </div>
</div>

# LiquidCodec

LiquidCodec is a **compact, streaming audio codec** that compresses speech into discrete tokens and converts them back into audio. It is the audio tokenizer behind [LFM3-Audio-2B](https://huggingface.co/LiquidAI/LFM3-Audio-2B) and [LFM3-Omni-2B](https://huggingface.co/LiquidAI/LFM3-Omni-2B).


- **Easy to predict:** LiquidCodec’s grouped finite scalar quantization (GFSQ) produces codebooks with no sequential dependencies, so speech models can predict all codebooks of a frame in parallel.
- **Fast:** at FP32 on a single CPU thread of an Apple M4 Max, LiquidCodec encodes at about **361× real time** and decodes at about **87× real time**.
- **High fidelity:** best ViSQOL and lowest mel distance of all compared codecs tested on eight reconstruction test sets.
- **Better speech generation:** in matched TTS experiments, models trained on LiquidCodec tokens reach the highest naturalness (UTMOSv2) on all eight test sets.
- **Edge-ready:** decoder-only quantization-aware training lets the decoder run in Q4_0 with a 3.5× smaller footprint and minimal quality loss.

Find more information in [our blog post](https://www.liquid.ai/blog/liquidcodec).

## Model details
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />

<table style="width: 100%; display: table; border: 0; border-radius: 0; border-collapse: collapse;">
<thead style="border: 0;">
<tr style="border: 0;">
<th scope="col" style="text-align: left; padding: 0.55em 0.75em; border: 0; border-bottom: 1px solid rgba(127, 127, 127, 0.55);">Property</th>
<th scope="col" style="text-align: left; padding: 0.55em 0.75em; border: 0; border-bottom: 1px solid rgba(127, 127, 127, 0.55);">Specification</th>
</tr>
</thead>
<tbody style="border: 0;">
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Parameters</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">~74M</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Architecture</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">Symmetric, fully causal LFM3 encoder and decoder with grouped finite scalar quantization (GFSQ)</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Input / output</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">24 kHz mono speech</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Frame rate</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">12.5 Hz</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Tokens per frame</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">12 (one per codebook)</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Bitrate</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">≈ 1.65 kbps</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Streaming</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">Streaming encoding and decoding</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Training data</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">264k hours across 107 languages</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>License</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;"><a href="https://huggingface.co/LiquidAI/Liquid-Codec/blob/main/LICENSE">LFM Open License v1.0</a></td>
</tr>
</tbody>
</table>

LiquidCodec is trained and evaluated on speech. Its quality on music, singing or environmental sound has not been evaluated.

## Inference
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />

This repository contains the BF16 codec weights for use with Transformers. Quantized versions of the decoder (Q4_0 / Q4_K_M / Q8_0) are not published separately; they ship inside [LFM3-Audio-2B-GGUF](https://huggingface.co/LiquidAI/LFM3-Audio-2B-GGUF) and [LFM3-Omni-2B-GGUF](https://huggingface.co/LiquidAI/LFM3-Omni-2B-GGUF) for on-device inference with [llama.cpp](https://docs.liquid.ai/deployment/on-device/llama-cpp).

### Remote-code loading

The repository includes the codec configuration and model implementation, so it can be loaded with `AutoModel`:

```python
import torch
from transformers import AutoModel

codec = AutoModel.from_pretrained(
    "LiquidAI/Liquid-Codec",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda",
)
waveform = torch.randn(1, 24_000 * 3, device="cuda")  # 24 kHz
codes = codec.encode(waveform).audio_codes
reconstructed = codec.decode(codes).audio_values
```

### Native loading and streaming

```python
import torch
from transformers import LiquidCodecModel, LiquidCodecStreamingCache

codec = LiquidCodecModel.from_pretrained("LiquidAI/Liquid-Codec", dtype=torch.bfloat16, device_map="cuda")

waveform = torch.randn(1, 24_000 * 3, device="cuda")  # 24 kHz
codes = codec.encode(waveform).audio_codes  # (1, 12, 38)
reconstructed = codec.decode(codes).audio_values  # (1, 38 * 1920), float32

# streaming synthesis: every chunk returns the samples that no later frame changes any more
cache = LiquidCodecStreamingCache(codec.config)
chunks = [codec.decode(codes[..., i : i + 1], cache, finalize=False).audio_values for i in range(codes.shape[-1])]
chunks.append(codec.decode(codes[..., :0], cache, finalize=True).audio_values)
streamed = torch.cat(chunks, dim=-1)
```

### Transformers usage 🤗

Liquid Codec is supported natively in 🤗 Transformers, without `trust_remote_code`. Install 🤗 Transformers from source:

```sh
pip install git+https://github.com/huggingface/transformers
```

<details>
<summary>Encoding and decoding</summary>

```python
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio

model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

url = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/librispeech_mr_quilter.wav"
audio = load_audio(url, sampling_rate=feature_extractor.sampling_rate)
inputs = feature_extractor(
    audio=audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)

audio_codes = model.encode(**inputs).audio_codes
print("Audio codes shape:", audio_codes.shape)
# Audio codes shape: torch.Size([1, 12, 74])
audio_values = model.decode(audio_codes).audio_values
print("Audio values shape:", audio_values.shape)
# Audio values shape: torch.Size([1, 142080])
```

</details>

<details>
<summary>Batch processing</summary>

Clips of different lengths are padded to the longest one. Pass the masks along: `encode` returns the
`audio_codes_mask` of the valid frames of each clip, and `decode` the `audio_values_mask` of its valid samples.

```python
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio

model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

samples = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/"
audios = [
    load_audio(samples + name, sampling_rate=feature_extractor.sampling_rate)
    for name in ("librispeech_mr_quilter.wav", "dude_where_is_my_car.wav")
]
inputs = feature_extractor(
    audio=audios, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)

encoder_output = model.encode(**inputs)
print("Frames per clip:", encoder_output.audio_codes_mask.sum(-1).tolist())
# Frames per clip: [74, 814]

decoder_output = model.decode(encoder_output.audio_codes, encoder_output.audio_codes_mask)
print("Samples per clip:", decoder_output.audio_values_mask.sum(-1).tolist())
# Samples per clip: [142080, 1562880]

# drop the padding of each clip
audio_values = [
    values[mask] for values, mask in zip(decoder_output.audio_values, decoder_output.audio_values_mask)
]
```

</details>

<details>
<summary>Streaming decoding</summary>

The decoder is causal, so audio can be synthesized chunk by chunk as the codes arrive.

```python
import torch
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio

model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

url = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/librispeech_mr_quilter.wav"
audio = load_audio(url, sampling_rate=feature_extractor.sampling_rate)
inputs = feature_extractor(
    audio=audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)
audio_codes = model.encode(**inputs).audio_codes

chunk_size = 4  # codec frames per chunk
decoder_cache, overlap_add_cache = None, None
chunks = []
for start in range(0, audio_codes.shape[-1], chunk_size):
    outputs = model.decode(
        audio_codes[..., start : start + chunk_size],
        decoder_cache=decoder_cache,
        overlap_add_cache=overlap_add_cache,
        use_streaming=True,
    )
    decoder_cache, overlap_add_cache = outputs.decoder_cache, outputs.overlap_add_cache
    chunks.append(outputs.audio_values)  # play or send this chunk

# end of the stream
chunks.append(overlap_add_cache.flush().audio_values)

audio_values = torch.cat(chunks, dim=-1)
print("Audio values shape:", audio_values.shape)
# Audio values shape: torch.Size([1, 142080])
```

</details>

See the [Transformers documentation](https://huggingface.co/docs/transformers/main/en/model_doc/liquid_codec) for more.

## Performance
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />

### Speed

FP32, single-threaded CPU inference on a MacBook Pro with an Apple M4 Max. RTF = processing time ÷ audio duration (lower is faster).

| Tokenizer | Codebooks | Enc. RTF ↓ | Dec. RTF ↓ |
|---|---|---|---|
| **LiquidCodec** | 12 | **0.0028 ± 0.0001** | **0.0115 ± 0.0001** |
| X-Codec 2 | 1 | 0.7359 ± 0.0103 | 0.0379 ± 0.0009 |
| NanoCodec | 13 | 0.2145 ± 0.0057 | 0.2543 ± 0.0061 |
| Mimi | 12 (truncated) | 0.0361 ± 0.0004 | 0.0329 ± 0.0005 |

### Reconstruction and zero-shot TTS

LiquidCodec achieves the best ViSQOL and lowest mel distance on all eight reconstruction test sets, and remains the best on both with a Q4_0 decoder. TTS models trained on LiquidCodec tokens reach the highest UTMOSv2 on all eight test sets and the best WER/CER on four of eight.

| Test set | Tokenizer | SQMOS ↑ | PESQ (WB) ↑ | Mel dist. ↓ | SECS (ECAPA2) ↑ | ViSQOL ↑ | Recon. CER (%) ↓ | TTS WER/CER (%) ↓ | TTS UTMOSv2 ↑ |
|---|---|---|---|---|---|---|---|---|---|
| LibriSpeech test-clean (EN) | LiquidCodec BF16 | 4.442 | 2.640 | 0.120 | 0.785 | 4.319 | 0.89 | 3.15 | 3.31 |
| | LiquidCodec Q4_0 | 4.445 | 2.577 | 0.121 | 0.781 | 4.292 | 0.89 | 3.18 | 3.22 |
| | X-Codec 2 | 4.436 | 2.434 | 0.123 | 0.754 | 4.119 | 0.88 | 4.19 | 3.27 |
| | NanoCodec | 4.486 | 2.942 | 0.122 | 0.836 | 4.219 | 0.76 | 3.32 | 3.19 |
| | Mimi (12 cb) | 4.364 | 2.615 | 0.160 | 0.714 | 4.031 | 0.87 | 6.27 | 2.89 |
| Seed-TTS (EN) | LiquidCodec BF16 | 4.463 | 2.326 | 0.130 | 0.751 | 4.369 | 0.86 | 3.50 | 3.39 |
| | LiquidCodec Q4_0 | 4.460 | 2.271 | 0.131 | 0.744 | 4.341 | 0.86 | 3.54 | 3.31 |
| | X-Codec 2 | 4.436 | 2.099 | 0.140 | 0.745 | 4.105 | 0.86 | 5.65 | 3.24 |
| | NanoCodec | 4.467 | 2.599 | 0.136 | 0.799 | 4.304 | 0.73 | 3.95 | 3.29 |
| | Mimi (12 cb) | 4.347 | 2.396 | 0.163 | 0.722 | 4.124 | 1.01 | 6.15 | 2.98 |
| Seed-TTS (ZH) | LiquidCodec BF16 | 4.407 | 2.429 | 0.118 | 0.776 | 4.313 | 6.93 | 1.19 | 3.01 |
| | LiquidCodec Q4_0 | 4.393 | 2.375 | 0.119 | 0.768 | 4.285 | 7.04 | 1.20 | 2.92 |
| | X-Codec 2 | 4.387 | 2.237 | 0.126 | 0.823 | 4.064 | 7.30 | 1.44 | 2.85 |
| | NanoCodec | 4.407 | 2.606 | 0.122 | 0.811 | 4.189 | 6.41 | 1.27 | 2.88 |
| | Mimi (12 cb) | 4.198 | 2.401 | 0.152 | 0.702 | 3.992 | 7.59 | 2.12 | 2.53 |
| FLEURS (DE) | LiquidCodec BF16 | 4.237 | 2.434 | 0.120 | 0.807 | 4.207 | 3.65 | 5.30 | 3.20 |
| | LiquidCodec Q4_0 | 4.224 | 2.391 | 0.121 | 0.818 | 4.162 | 3.67 | 5.38 | 3.12 |
| | X-Codec 2 | 4.256 | 2.085 | 0.125 | 0.661 | 3.942 | 4.39 | 5.44 | 3.08 |
| | NanoCodec | 4.202 | 2.597 | 0.129 | 0.822 | 4.009 | 3.74 | 5.36 | 3.01 |
| | Mimi (12 cb) | 4.087 | 2.393 | 0.170 | 0.698 | 3.627 | 5.36 | 6.14 | 2.78 |
| FLEURS (FR) | LiquidCodec BF16 | 4.301 | 2.315 | 0.127 | 0.825 | 4.242 | 5.27 | 6.91 | 3.12 |
| | LiquidCodec Q4_0 | 4.302 | 2.262 | 0.128 | 0.820 | 4.212 | 5.27 | 5.26 | 3.04 |
| | X-Codec 2 | 4.317 | 2.019 | 0.134 | 0.606 | 3.914 | 6.84 | 6.37 | 3.01 |
| | NanoCodec | 4.305 | 2.500 | 0.137 | 0.824 | 4.072 | 5.06 | 7.72 | 3.02 |
| | Mimi (12 cb) | 3.876 | 2.207 | 0.192 | 0.710 | 3.758 | 15.95 | 8.73 | 2.67 |
| FLEURS (ZH) | LiquidCodec BF16 | 4.343 | 2.479 | 0.121 | 0.794 | 4.273 | 7.60 | 8.82 | 3.02 |
| | LiquidCodec Q4_0 | 4.339 | 2.436 | 0.123 | 0.782 | 4.249 | 7.56 | 7.84 | 2.94 |
| | X-Codec 2 | 4.335 | 2.307 | 0.127 | 0.559 | 4.047 | 8.24 | 7.86 | 2.86 |
| | NanoCodec | 4.334 | 2.679 | 0.133 | 0.845 | 4.164 | 6.68 | 7.45 | 2.91 |
| | Mimi (12 cb) | 3.916 | 2.179 | 0.191 | 0.616 | 3.837 | 9.51 | 8.35 | 2.53 |
| FLEURS (JA) | LiquidCodec BF16 | 4.347 | 2.233 | 0.122 | 0.766 | 4.162 | 6.80 | 7.29 | 3.05 |
| | LiquidCodec Q4_0 | 4.327 | 2.188 | 0.123 | 0.756 | 4.120 | 6.83 | 7.32 | 2.98 |
| | X-Codec 2 | 4.354 | 2.045 | 0.130 | 0.497 | 3.805 | 7.02 | 5.40 | 2.94 |
| | NanoCodec | 4.378 | 2.415 | 0.129 | 0.695 | 4.089 | 6.02 | 5.58 | 2.84 |
| | Mimi (12 cb) | 4.332 | 2.232 | 0.166 | 0.658 | 3.982 | 12.13 | 6.27 | 2.49 |
| FLEURS (KO) | LiquidCodec BF16 | 4.217 | 2.637 | 0.117 | 0.803 | 4.176 | 4.45 | 8.36 | 2.97 |
| | LiquidCodec Q4_0 | 4.214 | 2.599 | 0.118 | 0.800 | 4.146 | 4.30 | 6.71 | 2.88 |
| | X-Codec 2 | 4.214 | 2.299 | 0.128 | 0.591 | 3.870 | 5.08 | 6.22 | 2.87 |
| | NanoCodec | 4.212 | 2.589 | 0.137 | 0.769 | 4.018 | 3.41 | 5.96 | 2.86 |
| | Mimi (12 cb) | 3.843 | 2.165 | 0.178 | 0.543 | 3.530 | 9.37 | 9.81 | 2.54 |

<details>
<summary><small>Evaluation details</small></summary>

<p><small>
<strong>Reconstruction</strong>: each recording is encoded into tokens and decoded back into speech; the reconstruction is compared against the original. CER is measured on the reconstructed audio.<br>
<strong>Zero-shot TTS</strong>: for each tokenizer, a TTS model is trained with identical data (101k hours across six languages), architecture and compute. The backbone is initialized from LFM3-1.7B, with a six-layer, 1,024-dim autoregressive depth decoder predicting the codebooks within each frame. Each model is trained with identical batches in identical order. Only the token-specific embeddings and output heads change. At evaluation, models receive text without a reference speech prompt; scores aggregate three generation seeds.<br>
<strong>Backbone check</strong>: repeating the TTS experiment with a Qwen3-1.7B backbone gives consistent results.<br>
<strong>Q4_0</strong>: decoder quantized to 4 bits after decoder-only quantization-aware training.
</small></p>

</details>

## Architecture
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />

**Symmetric LFM3 encoder and decoder:** both are built from blocks of the LFM3 architecture, which is optimized for performance on the edge. The encoder mirrors the decoder exactly and is fully causal, so tokenization is as lightweight as detokenization. This suits real-time, low-latency streaming.

**Spectral processing:** the decoder predicts spectral coefficients directly. This avoids progressive upsampling with learned transposed convolutions, a costly step in similar codecs.

**Grouped finite scalar quantization:** FSQ rounds each dimension of a feature vector to one of a fixed number of values. GFSQ splits the features into groups that are quantized independently, and each group yields one token. Without RVQ's chain of dependencies between codebooks, multiple audio tokens can be predicted in parallel with MaskGIT-based methods.

## Citation
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />

```bibtex
@article{liquidAI2026liquidcodec,
  author  = {Liquid AI},
  title   = {LiquidCodec: High-Fidelity Audio Tokenization Optimized for Speed},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {www.liquid.ai/blog/liquidcodec},
}
```

For questions and feedback, join [Discord](https://discord.com/invite/liquid-ai). For custom deployment and fine-tuning support, [contact Liquid AI](https://www.liquid.ai/connect?utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface).