Add comprehensive model card for L3AC

README.md

@@ -0,0 +1,78 @@
+---
+pipeline_tag: audio-to-audio
+library_name: l3ac
+---
+
+# L3AC: Towards a Lightweight and Lossless Audio Codec
+
+This repository contains the implementation of L3AC, a lightweight neural audio codec introduced in the paper titled "[L3AC: Towards a Lightweight and Lossless Audio Codec](https://huggingface.co/papers/2504.04949)".
+
+Neural audio codecs have recently gained traction for their ability to compress high-fidelity audio and provide discrete tokens for generative modeling. However, leading approaches often rely on resource-intensive models and complex multi-quantizer architectures, limiting their practicality in real-world applications. In this work, we introduce L3AC, a lightweight neural audio codec that addresses these challenges by leveraging a single quantizer and a highly efficient architecture. To enhance reconstruction fidelity while minimizing model complexity, L3AC explores streamlined convolutional networks and local Transformer modules, alongside TConv--a novel structure designed to capture acoustic variations across multiple temporal scales. Despite its compact design, extensive experiments across diverse datasets demonstrate that L3AC matches or exceeds the reconstruction quality of leading codecs while reducing computational overhead by an order of magnitude. The single-quantizer design further enhances its adaptability for downstream tasks.
+
+<figure class="image">
+  <img src="https://github.com/zhai-lw/L3AC/raw/main/bubble_chart.svg" alt="Comparison of various audio codec">
+  <figcaption>Comparison of various audio codec</figcaption>
+</figure>
+
+**Paper:** [L3AC: Towards a Lightweight and Lossless Audio Codec](https://huggingface.co/papers/2504.04949)
+**Official GitHub Repository:** [https://github.com/zhai-lw/L3AC](https://github.com/zhai-lw/L3AC)
+
+## Installation
+
+You can install the `l3ac` library using pip:
+
+```bash
+pip install l3ac
+```
+
+### Demo
+
+Firstly, make sure you have installed the `librosa` package to load the example audio file. You can install it using pip:
+
+```bash
+pip install librosa
+```
+
+Then, you can use the following code to load a sample audio file, encode it using the L3AC model, and decode it back to audio. The code also calculates the mean squared error (MSE) between the original and generated audio.
+
+```python
+import librosa
+import torch
+import l3ac
+
+all_models = l3ac.list_models()
+print(f"Available models: {all_models}")
+
+MODEL_USED = '1kbps'
+codec = l3ac.get_model(MODEL_USED)
+print(f"loaded codec({MODEL_USED}) and codec sample rate: {codec.config.sample_rate}")
+
+sample_audio, sample_rate = librosa.load(librosa.example("libri1"))
+sample_audio = sample_audio[None, :]
+print(f"loaded sample audio and audio sample_rate :{sample_rate}")
+
+sample_audio = librosa.resample(sample_audio, orig_sr=sample_rate, target_sr=codec.config.sample_rate)
+
+codec.network.cuda()
+codec.network.eval()
+with torch.inference_mode():
+    audio_in = torch.tensor(sample_audio, dtype=torch.float32, device='cuda')
+    _, audio_length = audio_in.shape
+    print(f"{audio_in.shape=}")
+    q_feature, indices = codec.encode_audio(audio_in)
+    audio_out = codec.decode_audio(q_feature)  # or
+    # audio_out = codec.decode_audio(indices=indices['indices'])
+    generated_audio = audio_out[:, :audio_length].detach().cpu().numpy()
+
+mse = ((sample_audio - generated_audio) ** 2).mean().item()
+print(f"codec({MODEL_USED}) mse: {mse}")
+```
+
+### Available Models
+
+| config_name | Sample rate(Hz) | tokens/s | Codebook size | Bitrate(bps) |
+|-------------|-----------------|----------|---------------|--------------|
+| 0k75bps     | 16,000          | 44.44    | 117,649       | 748.6        |
+| 1kbps       | 16,000          | 59.26    | 117,649       | 998.2        |
+| 1k5bps      | 16,000          | 88.89    | 117,649       | 1497.3       |
+| 3kbps       | 16,000          | 166.67   | 250,047       | 2988.6       |