Feature Extraction
Transformers
Safetensors
liquid_codec
liquid
lfm3
audio
audio-codec
speech
streaming
edge
multilingual
custom_code
Instructions to use eustlb/Liquid-Codec with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use eustlb/Liquid-Codec with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="eustlb/Liquid-Codec", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("eustlb/Liquid-Codec", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 4,371 Bytes
8ce71a5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | # π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨
# This file was automatically generated from src/transformers/models/lfm3/modular_lfm3.py.
# Do NOT edit this file manually as any edits will be overwritten by the generation of
# the file from the modular. If any change should be done, please apply the change to the
# modular_lfm3.py file directly. One of our CI enforces this.
# π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨π¨
# Copyright 2026 The HuggingFace Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from huggingface_hub.dataclasses import strict
from transformers.configuration_utils import PreTrainedConfig
from transformers.utils import auto_docstring
@auto_docstring(checkpoint="LiquidAI/LFM3-1.7B")
@strict
class Lfm3Config(PreTrainedConfig):
r"""
conv_bias (`bool`, *optional*, defaults to `False`):
Whether to use bias in the conv layers.
conv_L_cache (`int`, *optional*, defaults to 7):
L_cache dim in the conv layers.
use_sandwich_norm (`bool`, *optional*, defaults to `True`):
Whether to apply an RMSNorm to the operator and feed-forward outputs before they enter the residual stream.
use_attention_gate (`bool`, *optional*, defaults to `True`):
Whether the attention layers scale each head's output by a sigmoid gate computed from the layer input.
```python
>>> from transformers import Lfm3Model, Lfm3Config
>>> # Initializing a LFM3 model
>>> configuration = Lfm3Config()
>>> # Initializing a model from the LFM3-1.7B style configuration
>>> model = Lfm3Model(configuration)
>>> # Accessing the model configuration
>>> configuration = model.config
```
"""
model_type = "lfm3"
keys_to_ignore_at_inference = ["past_key_values"]
default_theta = 1000000.0
vocab_size: int = 128_000
hidden_size: int = 2048
intermediate_size: int = 7680
num_hidden_layers: int = 24
num_attention_heads: int = 16
num_key_value_heads: int = 4
max_position_embeddings: int = 128_000
initializer_range: float = 0.02
norm_eps: float = 0.00001
use_cache: bool = True
pad_token_id: int | None = 124_893
bos_token_id: int | None = 124_894
eos_token_id: int | list[int] | None = 124_900
tie_word_embeddings: bool = True
rope_parameters: dict | None = None
conv_bias: bool = False
conv_L_cache: int = 7
layer_types: list[str] | None = None
attribute_map = {"rms_norm_eps": "norm_eps"}
_rope_type_labels = ("full_attention", "sliding_attention")
head_dim: int = 128
sliding_window: int | None = 256
use_sandwich_norm: bool = True
use_attention_gate: bool = True
attention_bias: bool = False
attention_dropout: float = 0.0
def __post_init__(self, **kwargs):
if self.rope_parameters is None:
self.rope_parameters = {
"sliding_attention": {"rope_type": "default", "rope_theta": 10_000.0},
"full_attention": {
"rope_type": "proportional",
"rope_theta": 1_000_000.0,
"partial_rotary_factor": 0.5,
},
}
if self.layer_types is None:
self.layer_types = [
"full_attention" if i % 4 == 2 else "sliding_attention" if i % 4 == 0 and i > 0 else "conv"
for i in range(self.num_hidden_layers)
]
super().__post_init__(**kwargs)
__all__ = ["Lfm3Config"]
|