Feature Extraction
Transformers
Safetensors
liquid_codec
liquid
lfm3
audio
audio-codec
speech
streaming
edge
multilingual
custom_code
Instructions to use eustlb/Liquid-Codec with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use eustlb/Liquid-Codec with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="eustlb/Liquid-Codec", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("eustlb/Liquid-Codec", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 21,253 Bytes
a2c6e79 8ce71a5 a2c6e79 8ce71a5 a2c6e79 8ce71a5 a2c6e79 8ce71a5 a2c6e79 8ce71a5 0e51189 8ce71a5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 | ---
library_name: transformers
license: other
license_name: lfm1.0
license_link: LICENSE
pipeline_tag: feature-extraction
tags:
- liquid
- lfm3
- audio
- audio-codec
- speech
- streaming
- edge
- multilingual
- custom_code
language:
- en
- zh
- de
- fr
- ja
- ko
---
<div align="center">
<img
src="https://huggingface.co/LiquidAI/LFM3-1.7B/resolve/main/assets/liquid-ai-banner.webp"
alt="Liquid AI"
style="width: 100%; max-width: 100%; height: auto; display: block; margin: 0;"
/>
<div align="center" style="display: flex; justify-content: center; align-items: center; gap: 1.25em; margin: 8px 0 1.7rem; padding: 0; line-height: 1.3;">
<a href="https://console.liquid.ai/dashboard" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath d=%22m8 6-6 6 6 6m8-12 6 6-6 6m-2-14-4 16%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>API</strong></a>
<a href="https://playground.liquid.ai/talk" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath transform=%22translate(3 0)%22 d=%22M8 5.5v13L19 12 8 5.5Z%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Try</strong></a>
<a href="https://docs.liquid.ai/lfm/getting-started/welcome" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath d=%22M12 6c-3-2-6-2.5-10-2v15c4-.5 7 0 10 2m0-15c3-2 6-2.5 10-2v15c-4-.5-7 0-10 2V6Z%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Docs</strong></a>
<a href="https://www.liquid.ai/blog/liquidcodec" style="display: inline-flex; align-items: center; gap: 6px;"><img src="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 width=%2216%22 height=%2216%22 viewBox=%220 0 24 24%22 fill=%22none%22 stroke-width=%222.2%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22%3E%3Cstyle%3Epath%7Bstroke:%23000%7D%40media (prefers-color-scheme:dark)%7Bpath%7Bstroke:%23d1d5dc%7D%7D%3C/style%3E%3Cpath transform=%22translate(1 0)%22 d=%22m16 3 5 5L8 21H3v-5L16 3Zm-3 3 5 5%22/%3E%3C/svg%3E" width="16" height="16" alt="" aria-hidden="true" style="display: block; flex: 0 0 16px; width: 16px; height: 16px; margin: 0;" /><strong>Blog</strong></a>
</div>
</div>
# LiquidCodec
LiquidCodec is a **compact, streaming audio codec** that compresses speech into discrete tokens and converts them back into audio. It is the audio tokenizer behind [LFM3-Audio-2B](https://huggingface.co/LiquidAI/LFM3-Audio-2B) and [LFM3-Omni-2B](https://huggingface.co/LiquidAI/LFM3-Omni-2B).
- **Easy to predict:** LiquidCodec’s grouped finite scalar quantization (GFSQ) produces codebooks with no sequential dependencies, so speech models can predict all codebooks of a frame in parallel.
- **Fast:** at FP32 on a single CPU thread of an Apple M4 Max, LiquidCodec encodes at about **361× real time** and decodes at about **87× real time**.
- **High fidelity:** best ViSQOL and lowest mel distance of all compared codecs tested on eight reconstruction test sets.
- **Better speech generation:** in matched TTS experiments, models trained on LiquidCodec tokens reach the highest naturalness (UTMOSv2) on all eight test sets.
- **Edge-ready:** decoder-only quantization-aware training lets the decoder run in Q4_0 with a 3.5× smaller footprint and minimal quality loss.
Find more information in [our blog post](https://www.liquid.ai/blog/liquidcodec).
## Model details
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />
<table style="width: 100%; display: table; border: 0; border-radius: 0; border-collapse: collapse;">
<thead style="border: 0;">
<tr style="border: 0;">
<th scope="col" style="text-align: left; padding: 0.55em 0.75em; border: 0; border-bottom: 1px solid rgba(127, 127, 127, 0.55);">Property</th>
<th scope="col" style="text-align: left; padding: 0.55em 0.75em; border: 0; border-bottom: 1px solid rgba(127, 127, 127, 0.55);">Specification</th>
</tr>
</thead>
<tbody style="border: 0;">
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Parameters</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">~74M</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Architecture</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">Symmetric, fully causal LFM3 encoder and decoder with grouped finite scalar quantization (GFSQ)</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Input / output</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">24 kHz mono speech</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Frame rate</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">12.5 Hz</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Tokens per frame</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">12 (one per codebook)</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Bitrate</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">≈ 1.65 kbps</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Streaming</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">Streaming encoding and decoding</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>Training data</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;">264k hours across 107 languages</td>
</tr>
<tr style="border: 0;">
<th scope="row" style="text-align: left; background-color: transparent; font-weight: 400; border: 0; padding: 0.55em 0.75em;"><strong>License</strong></th>
<td style="border: 0; padding: 0.55em 0.75em;"><a href="https://huggingface.co/LiquidAI/Liquid-Codec/blob/main/LICENSE">LFM Open License v1.0</a></td>
</tr>
</tbody>
</table>
LiquidCodec is trained and evaluated on speech. Its quality on music, singing or environmental sound has not been evaluated.
## Inference
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />
This repository contains the BF16 codec weights for use with Transformers. Quantized versions of the decoder (Q4_0 / Q4_K_M / Q8_0) are not published separately; they ship inside [LFM3-Audio-2B-GGUF](https://huggingface.co/LiquidAI/LFM3-Audio-2B-GGUF) and [LFM3-Omni-2B-GGUF](https://huggingface.co/LiquidAI/LFM3-Omni-2B-GGUF) for on-device inference with [llama.cpp](https://docs.liquid.ai/deployment/on-device/llama-cpp).
### Remote-code loading
The repository includes the codec configuration and model implementation, so it can be loaded with `AutoModel`:
```python
import torch
from transformers import AutoModel
codec = AutoModel.from_pretrained(
"LiquidAI/Liquid-Codec",
trust_remote_code=True,
dtype=torch.bfloat16,
device_map="cuda",
)
waveform = torch.randn(1, 24_000 * 3, device="cuda") # 24 kHz
codes = codec.encode(waveform).audio_codes
reconstructed = codec.decode(codes).audio_values
```
### Native loading and streaming
```python
import torch
from transformers import LiquidCodecModel, LiquidCodecStreamingCache
codec = LiquidCodecModel.from_pretrained("LiquidAI/Liquid-Codec", dtype=torch.bfloat16, device_map="cuda")
waveform = torch.randn(1, 24_000 * 3, device="cuda") # 24 kHz
codes = codec.encode(waveform).audio_codes # (1, 12, 38)
reconstructed = codec.decode(codes).audio_values # (1, 38 * 1920), float32
# streaming synthesis: every chunk returns the samples that no later frame changes any more
cache = LiquidCodecStreamingCache(codec.config)
chunks = [codec.decode(codes[..., i : i + 1], cache, finalize=False).audio_values for i in range(codes.shape[-1])]
chunks.append(codec.decode(codes[..., :0], cache, finalize=True).audio_values)
streamed = torch.cat(chunks, dim=-1)
```
### Transformers usage 🤗
Liquid Codec is supported natively in 🤗 Transformers, without `trust_remote_code`. Install 🤗 Transformers from source:
```sh
pip install git+https://github.com/huggingface/transformers
```
<details>
<summary>Encoding and decoding</summary>
```python
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio
model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
url = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/librispeech_mr_quilter.wav"
audio = load_audio(url, sampling_rate=feature_extractor.sampling_rate)
inputs = feature_extractor(
audio=audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)
audio_codes = model.encode(**inputs).audio_codes
print("Audio codes shape:", audio_codes.shape)
# Audio codes shape: torch.Size([1, 12, 74])
audio_values = model.decode(audio_codes).audio_values
print("Audio values shape:", audio_values.shape)
# Audio values shape: torch.Size([1, 142080])
```
</details>
<details>
<summary>Batch processing</summary>
Clips of different lengths are padded to the longest one. Pass the masks along: `encode` returns the
`audio_codes_mask` of the valid frames of each clip, and `decode` the `audio_values_mask` of its valid samples.
```python
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio
model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
samples = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/"
audios = [
load_audio(samples + name, sampling_rate=feature_extractor.sampling_rate)
for name in ("librispeech_mr_quilter.wav", "dude_where_is_my_car.wav")
]
inputs = feature_extractor(
audio=audios, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)
encoder_output = model.encode(**inputs)
print("Frames per clip:", encoder_output.audio_codes_mask.sum(-1).tolist())
# Frames per clip: [74, 814]
decoder_output = model.decode(encoder_output.audio_codes, encoder_output.audio_codes_mask)
print("Samples per clip:", decoder_output.audio_values_mask.sum(-1).tolist())
# Samples per clip: [142080, 1562880]
# drop the padding of each clip
audio_values = [
values[mask] for values, mask in zip(decoder_output.audio_values, decoder_output.audio_values_mask)
]
```
</details>
<details>
<summary>Streaming decoding</summary>
The decoder is causal, so audio can be synthesized chunk by chunk as the codes arrive.
```python
import torch
from transformers import AutoFeatureExtractor, AutoModel
from transformers.audio_utils import load_audio
model_id = "LiquidAI/Liquid-Codec"
model = AutoModel.from_pretrained(model_id, device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
url = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/librispeech_mr_quilter.wav"
audio = load_audio(url, sampling_rate=feature_extractor.sampling_rate)
inputs = feature_extractor(
audio=audio, sampling_rate=feature_extractor.sampling_rate, return_tensors="pt", device=model.device
).to(model.device, model.dtype)
audio_codes = model.encode(**inputs).audio_codes
chunk_size = 4 # codec frames per chunk
decoder_cache, overlap_add_cache = None, None
chunks = []
for start in range(0, audio_codes.shape[-1], chunk_size):
outputs = model.decode(
audio_codes[..., start : start + chunk_size],
decoder_cache=decoder_cache,
overlap_add_cache=overlap_add_cache,
use_streaming=True,
)
decoder_cache, overlap_add_cache = outputs.decoder_cache, outputs.overlap_add_cache
chunks.append(outputs.audio_values) # play or send this chunk
# end of the stream
chunks.append(overlap_add_cache.flush().audio_values)
audio_values = torch.cat(chunks, dim=-1)
print("Audio values shape:", audio_values.shape)
# Audio values shape: torch.Size([1, 142080])
```
</details>
See the [Transformers documentation](https://huggingface.co/docs/transformers/main/en/model_doc/liquid_codec) for more.
## Performance
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />
### Speed
FP32, single-threaded CPU inference on a MacBook Pro with an Apple M4 Max. RTF = processing time ÷ audio duration (lower is faster).
| Tokenizer | Codebooks | Enc. RTF ↓ | Dec. RTF ↓ |
|---|---|---|---|
| **LiquidCodec** | 12 | **0.0028 ± 0.0001** | **0.0115 ± 0.0001** |
| X-Codec 2 | 1 | 0.7359 ± 0.0103 | 0.0379 ± 0.0009 |
| NanoCodec | 13 | 0.2145 ± 0.0057 | 0.2543 ± 0.0061 |
| Mimi | 12 (truncated) | 0.0361 ± 0.0004 | 0.0329 ± 0.0005 |
### Reconstruction and zero-shot TTS
LiquidCodec achieves the best ViSQOL and lowest mel distance on all eight reconstruction test sets, and remains the best on both with a Q4_0 decoder. TTS models trained on LiquidCodec tokens reach the highest UTMOSv2 on all eight test sets and the best WER/CER on four of eight.
| Test set | Tokenizer | SQMOS ↑ | PESQ (WB) ↑ | Mel dist. ↓ | SECS (ECAPA2) ↑ | ViSQOL ↑ | Recon. CER (%) ↓ | TTS WER/CER (%) ↓ | TTS UTMOSv2 ↑ |
|---|---|---|---|---|---|---|---|---|---|
| LibriSpeech test-clean (EN) | LiquidCodec BF16 | 4.442 | 2.640 | 0.120 | 0.785 | 4.319 | 0.89 | 3.15 | 3.31 |
| | LiquidCodec Q4_0 | 4.445 | 2.577 | 0.121 | 0.781 | 4.292 | 0.89 | 3.18 | 3.22 |
| | X-Codec 2 | 4.436 | 2.434 | 0.123 | 0.754 | 4.119 | 0.88 | 4.19 | 3.27 |
| | NanoCodec | 4.486 | 2.942 | 0.122 | 0.836 | 4.219 | 0.76 | 3.32 | 3.19 |
| | Mimi (12 cb) | 4.364 | 2.615 | 0.160 | 0.714 | 4.031 | 0.87 | 6.27 | 2.89 |
| Seed-TTS (EN) | LiquidCodec BF16 | 4.463 | 2.326 | 0.130 | 0.751 | 4.369 | 0.86 | 3.50 | 3.39 |
| | LiquidCodec Q4_0 | 4.460 | 2.271 | 0.131 | 0.744 | 4.341 | 0.86 | 3.54 | 3.31 |
| | X-Codec 2 | 4.436 | 2.099 | 0.140 | 0.745 | 4.105 | 0.86 | 5.65 | 3.24 |
| | NanoCodec | 4.467 | 2.599 | 0.136 | 0.799 | 4.304 | 0.73 | 3.95 | 3.29 |
| | Mimi (12 cb) | 4.347 | 2.396 | 0.163 | 0.722 | 4.124 | 1.01 | 6.15 | 2.98 |
| Seed-TTS (ZH) | LiquidCodec BF16 | 4.407 | 2.429 | 0.118 | 0.776 | 4.313 | 6.93 | 1.19 | 3.01 |
| | LiquidCodec Q4_0 | 4.393 | 2.375 | 0.119 | 0.768 | 4.285 | 7.04 | 1.20 | 2.92 |
| | X-Codec 2 | 4.387 | 2.237 | 0.126 | 0.823 | 4.064 | 7.30 | 1.44 | 2.85 |
| | NanoCodec | 4.407 | 2.606 | 0.122 | 0.811 | 4.189 | 6.41 | 1.27 | 2.88 |
| | Mimi (12 cb) | 4.198 | 2.401 | 0.152 | 0.702 | 3.992 | 7.59 | 2.12 | 2.53 |
| FLEURS (DE) | LiquidCodec BF16 | 4.237 | 2.434 | 0.120 | 0.807 | 4.207 | 3.65 | 5.30 | 3.20 |
| | LiquidCodec Q4_0 | 4.224 | 2.391 | 0.121 | 0.818 | 4.162 | 3.67 | 5.38 | 3.12 |
| | X-Codec 2 | 4.256 | 2.085 | 0.125 | 0.661 | 3.942 | 4.39 | 5.44 | 3.08 |
| | NanoCodec | 4.202 | 2.597 | 0.129 | 0.822 | 4.009 | 3.74 | 5.36 | 3.01 |
| | Mimi (12 cb) | 4.087 | 2.393 | 0.170 | 0.698 | 3.627 | 5.36 | 6.14 | 2.78 |
| FLEURS (FR) | LiquidCodec BF16 | 4.301 | 2.315 | 0.127 | 0.825 | 4.242 | 5.27 | 6.91 | 3.12 |
| | LiquidCodec Q4_0 | 4.302 | 2.262 | 0.128 | 0.820 | 4.212 | 5.27 | 5.26 | 3.04 |
| | X-Codec 2 | 4.317 | 2.019 | 0.134 | 0.606 | 3.914 | 6.84 | 6.37 | 3.01 |
| | NanoCodec | 4.305 | 2.500 | 0.137 | 0.824 | 4.072 | 5.06 | 7.72 | 3.02 |
| | Mimi (12 cb) | 3.876 | 2.207 | 0.192 | 0.710 | 3.758 | 15.95 | 8.73 | 2.67 |
| FLEURS (ZH) | LiquidCodec BF16 | 4.343 | 2.479 | 0.121 | 0.794 | 4.273 | 7.60 | 8.82 | 3.02 |
| | LiquidCodec Q4_0 | 4.339 | 2.436 | 0.123 | 0.782 | 4.249 | 7.56 | 7.84 | 2.94 |
| | X-Codec 2 | 4.335 | 2.307 | 0.127 | 0.559 | 4.047 | 8.24 | 7.86 | 2.86 |
| | NanoCodec | 4.334 | 2.679 | 0.133 | 0.845 | 4.164 | 6.68 | 7.45 | 2.91 |
| | Mimi (12 cb) | 3.916 | 2.179 | 0.191 | 0.616 | 3.837 | 9.51 | 8.35 | 2.53 |
| FLEURS (JA) | LiquidCodec BF16 | 4.347 | 2.233 | 0.122 | 0.766 | 4.162 | 6.80 | 7.29 | 3.05 |
| | LiquidCodec Q4_0 | 4.327 | 2.188 | 0.123 | 0.756 | 4.120 | 6.83 | 7.32 | 2.98 |
| | X-Codec 2 | 4.354 | 2.045 | 0.130 | 0.497 | 3.805 | 7.02 | 5.40 | 2.94 |
| | NanoCodec | 4.378 | 2.415 | 0.129 | 0.695 | 4.089 | 6.02 | 5.58 | 2.84 |
| | Mimi (12 cb) | 4.332 | 2.232 | 0.166 | 0.658 | 3.982 | 12.13 | 6.27 | 2.49 |
| FLEURS (KO) | LiquidCodec BF16 | 4.217 | 2.637 | 0.117 | 0.803 | 4.176 | 4.45 | 8.36 | 2.97 |
| | LiquidCodec Q4_0 | 4.214 | 2.599 | 0.118 | 0.800 | 4.146 | 4.30 | 6.71 | 2.88 |
| | X-Codec 2 | 4.214 | 2.299 | 0.128 | 0.591 | 3.870 | 5.08 | 6.22 | 2.87 |
| | NanoCodec | 4.212 | 2.589 | 0.137 | 0.769 | 4.018 | 3.41 | 5.96 | 2.86 |
| | Mimi (12 cb) | 3.843 | 2.165 | 0.178 | 0.543 | 3.530 | 9.37 | 9.81 | 2.54 |
<details>
<summary><small>Evaluation details</small></summary>
<p><small>
<strong>Reconstruction</strong>: each recording is encoded into tokens and decoded back into speech; the reconstruction is compared against the original. CER is measured on the reconstructed audio.<br>
<strong>Zero-shot TTS</strong>: for each tokenizer, a TTS model is trained with identical data (101k hours across six languages), architecture and compute. The backbone is initialized from LFM3-1.7B, with a six-layer, 1,024-dim autoregressive depth decoder predicting the codebooks within each frame. Each model is trained with identical batches in identical order. Only the token-specific embeddings and output heads change. At evaluation, models receive text without a reference speech prompt; scores aggregate three generation seeds.<br>
<strong>Backbone check</strong>: repeating the TTS experiment with a Qwen3-1.7B backbone gives consistent results.<br>
<strong>Q4_0</strong>: decoder quantized to 4 bits after decoder-only quantization-aware training.
</small></p>
</details>
## Architecture
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />
**Symmetric LFM3 encoder and decoder:** both are built from blocks of the LFM3 architecture, which is optimized for performance on the edge. The encoder mirrors the decoder exactly and is fully causal, so tokenization is as lightweight as detokenization. This suits real-time, low-latency streaming.
**Spectral processing:** the decoder predicts spectral coefficients directly. This avoids progressive upsampling with learned transposed convolutions, a costly step in similar codecs.
**Grouped finite scalar quantization:** FSQ rounds each dimension of a feature vector to one of a fixed number of values. GFSQ splits the features into groups that are quantized independently, and each group yields one token. Without RVQ's chain of dependencies between codebooks, multiple audio tokens can be predicted in parallel with MaskGIT-based methods.
## Citation
<hr style="margin: -0.7em 0 1.2em 0; border: 0; border-top: 1px solid rgba(127, 127, 127, 0.3);" />
```bibtex
@article{liquidAI2026liquidcodec,
author = {Liquid AI},
title = {LiquidCodec: High-Fidelity Audio Tokenization Optimized for Speed},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/liquidcodec},
}
```
For questions and feedback, join [Discord](https://discord.com/invite/liquid-ai). For custom deployment and fine-tuning support, [contact Liquid AI](https://www.liquid.ai/connect?utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface).
|