Instructions to use mlx-community/mimi-encoder-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/mimi-encoder-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir mimi-encoder-mlx mlx-community/mimi-encoder-mlx
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
Add Mimi encoder MLX weights (fp32) + model card
Browse files- LICENSE +21 -0
- README.md +53 -0
- encoder.safetensors +3 -0
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 xocialize
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
library_name: mlx
|
| 4 |
+
base_model: kyutai/mimi
|
| 5 |
+
pipeline_tag: feature-extraction
|
| 6 |
+
tags:
|
| 7 |
+
- mlx
|
| 8 |
+
- audio
|
| 9 |
+
- audio-codec
|
| 10 |
+
- neural-codec
|
| 11 |
+
- mimi
|
| 12 |
+
- rvq
|
| 13 |
+
- apple-silicon
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# mlx-community/mimi-encoder-mlx
|
| 17 |
+
|
| 18 |
+
The **encoder** half of Kyutai's [Mimi](https://huggingface.co/kyutai/mimi) neural audio codec,
|
| 19 |
+
converted to MLX format for native inference on Apple Silicon and consumed by the
|
| 20 |
+
[`xocialize/mimi-encoder-mlx-swift`](https://github.com/xocialize/mimi-encoder-mlx-swift) Swift
|
| 21 |
+
port. Refer to the [original model card](https://huggingface.co/kyutai/mimi) for full details.
|
| 22 |
+
|
| 23 |
+
## Model
|
| 24 |
+
|
| 25 |
+
- **Family:** Mimi neural audio codec (Kyutai / Moshi — Défossez et al., [arXiv:2410.00037](https://arxiv.org/abs/2410.00037))
|
| 26 |
+
- **This artifact:** the **encoder** only (SEANet conv encoder → causal transformer → stride-2 downsample → split RVQ)
|
| 27 |
+
- **Input:** 24000 Hz, mono
|
| 28 |
+
- **Output:** `[16, T]` codebook-index grid at 12.5 Hz (1 semantic + 15 acoustic codebooks)
|
| 29 |
+
- **Precision:** fp32 (145 tensors)
|
| 30 |
+
|
| 31 |
+
## Files
|
| 32 |
+
|
| 33 |
+
- `encoder.safetensors` — the MLX encoder weights (fp32), extracted/converted from `kyutai/mimi`.
|
| 34 |
+
|
| 35 |
+
## Usage (Swift / MLX)
|
| 36 |
+
|
| 37 |
+
```swift
|
| 38 |
+
import MimiCodecEncoder
|
| 39 |
+
|
| 40 |
+
let encoder = MimiEncoder(config: .qwen3TTS12Hz)
|
| 41 |
+
try encoder.loadWeights(from: encoderWeightsURL) // encoder.safetensors
|
| 42 |
+
let codes = encoder.encode(audio: audioArray) // [16, T]
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
## Source
|
| 46 |
+
|
| 47 |
+
- **Original model:** https://huggingface.co/kyutai/mimi
|
| 48 |
+
- **Swift consumer:** https://github.com/xocialize/mimi-encoder-mlx-swift
|
| 49 |
+
|
| 50 |
+
## License
|
| 51 |
+
|
| 52 |
+
CC-BY-4.0 (Kyutai) — permissive, attribution required. This is a derivative (encoder-only,
|
| 53 |
+
format-converted) of `kyutai/mimi`; attribution to Kyutai is retained.
|
encoder.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c901ea2a23d911e5b23efa40bb003e3896aecf858f4473a43022f69cb98605d2
|
| 3 |
+
size 191138104
|