File size: 4,571 Bytes
64825a3 5cc731b 64825a3 5cc731b 64825a3 5cc731b 64825a3 5cc731b 64825a3 5cc731b 64825a3 5cc731b 64825a3 5cc731b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
license: mit
language:
- ar
- da
- de
- el
- en
- es
- fi
- fr
- he
- hi
- it
- ja
- ko
- ms
- nl
- 'no'
- pl
- pt
- ru
- sv
- sw
- tr
- zh
tags:
- gguf
- tts
- chatterbox
- chatterbox-multilingual
- text-to-speech
- codec-lm
base_model: ResembleAI/chatterbox
---
# Chatterbox-Multilingual T3 GGUF
End-to-end GGUF conversion of [ResembleAI's Chatterbox multilingual T3](https://huggingface.co/ResembleAI/chatterbox) (`t3_mtl23ls_v3.safetensors` — 23-language coverage).
This release adopts the **native codec_lm split** used by CSM / Qwen3-TTS / MOSS-TTSD:
- **Backbone (`chatterbox-mtl-t3-<quant>.gguf`)** — stock `llama` arch GGUF of Chatterbox T3's Llama-520M `tfmr.*` weights (30 layers, hidden 1024, 16 heads, head_dim=64, MLP 4096, llama3 RoPE scaling, rope_theta=500000, vocab placeholder = `tokenizer.ggml.model = "none"`). Runs in stock **llama.cpp** with `embeddings=true`.
- **Codec + codec_lm (`chatterbox-mtl-codec-<quant>.gguf`)** — Chatterbox S3G (flow-matching decoder + HiFi-GAN vocoder) bundled with the T3 LM-adaptor side (audio embed table, speech head, text embed/head, learned positional embeddings, cond encoder weights). Runs in [codec.cpp](https://github.com/mybigday/codec.cpp) as a `parallel_heads_delay` codec_lm with `n_cb=1`.
- **S3T tokenizer (`chatterbox-mtl-s3t.gguf`)** — speech tokenizer needed when registering a custom voice from a reference WAV.
Compared to the earlier release in this repo, the old `t3-*.gguf` (custom shape) and `t3-extras.gguf` are gone — everything the host runtime needs to drive the LM adaptor is now bundled into `chatterbox-mtl-codec-*.gguf` and exposed through codec.cpp's `codec_lm` API.
## Files
### Backbone (`chatterbox-mtl-t3-<quant>.gguf`)
| File | Size |
| ----------------------------------- | ------- |
| `chatterbox-mtl-t3-f32.gguf` | 1.9 GB |
| `chatterbox-mtl-t3-f16.gguf` | 961 MB |
| `chatterbox-mtl-t3-bf16.gguf` | 961 MB |
| `chatterbox-mtl-t3-q8_0.gguf` | 511 MB |
| `chatterbox-mtl-t3-q6_k.gguf` | 395 MB |
| `chatterbox-mtl-t3-q5_1.gguf` | 361 MB |
| `chatterbox-mtl-t3-q5_k_m.gguf` | 340 MB |
| `chatterbox-mtl-t3-q5_k_s.gguf` | 331 MB |
| `chatterbox-mtl-t3-q5_0.gguf` | 331 MB |
| `chatterbox-mtl-t3-q4_1.gguf` | 301 MB |
| `chatterbox-mtl-t3-q4_k_m.gguf` | 289 MB |
| `chatterbox-mtl-t3-q4_k_s.gguf` | 273 MB |
| `chatterbox-mtl-t3-q4_0.gguf` | 271 MB |
| `chatterbox-mtl-t3-q3_k_l.gguf` | 254 MB |
| `chatterbox-mtl-t3-q3_k_m.gguf` | 232 MB |
| `chatterbox-mtl-t3-q3_k_s.gguf` | 207 MB |
| `chatterbox-mtl-t3-q2_k.gguf` | 177 MB |
### Codec + codec_lm (`chatterbox-mtl-codec-<quant>.gguf`)
| File | Size |
| ----------------------------------- | ------- |
| `chatterbox-mtl-codec-f32.gguf` | 572 MB |
| `chatterbox-mtl-codec-f16.gguf` | 317 MB |
| `chatterbox-mtl-codec-q8_0.gguf` | 226 MB |
| `chatterbox-mtl-codec-q5_k_m.gguf` | 190 MB |
| `chatterbox-mtl-codec-q4_k_m.gguf` | 178 MB |
### S3T speech tokenizer
`chatterbox-mtl-s3t.gguf` (F16, 237 MB) — needed for **voice-clone**, encodes a reference WAV into the speech token IDs that the codec_lm consumes when registering a custom speaker. Same weights as English Chatterbox.
## Inference shape
Per-frame AR loop, single-codebook `parallel_heads_delay`:
```
backbone (Llama-520M, embeddings=true) hidden h
→ codec_lm_step_begin(state, h)
→ codec_lm_step_logits(0) → sample speech-token → codec_lm_step_push_code
→ codec_lm_step_finish → codes[1]
→ codec_lm_compose_audio_embd(codes) + speech_pos_emb[step] → next-step embedding
→ feed via b.embd; loop until stop_speech_token (6562)
```
Prompt prefix assembly (text + cond) lives inside codec.cpp's chatterbox path; the host application doesn't directly touch `lm.chatterbox.*` tensors.
## Sources
- Upstream model: [`ResembleAI/chatterbox`](https://huggingface.co/ResembleAI/chatterbox) (`t3_mtl23ls_v3.safetensors`)
- Conversion tooling: [`mybigday/codec.cpp`](https://github.com/mybigday/codec.cpp) (`prep_chatterbox_t3` + `lm_adaptor/chatterbox.py` + `ChatterboxS3GConverter` with `lm_source`)
- Inference runtime: [`mybigday/llama.rn`](https://github.com/mybigday/llama.rn)
## Supported languages
Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese (23 total).
|