---
library_name: coremltools
base_model:
- ResembleAI/chatterbox-flash
language:
- en
license: mit
tags:
- coreml
- text-to-speech
- voice-cloning
- zero-shot-tts
- block-diffusion
- speech-synthesis
- apple-silicon
- ios
- macos
pipeline_tag: text-to-speech
---

# Chatterbox Flash CoreML

Compiled Core ML export of [ResembleAI/chatterbox-flash](https://huggingface.co/ResembleAI/chatterbox-flash) for Apple runtimes.

This bundle contains the Chatterbox Flash T3 token generator and the S3Gen audio back-half as static-shape `.mlmodelc` graphs. It is intended for an application runtime that owns text tokenization, T3 denoising/sampling, reference conditioning, and graph orchestration.

## Links

- [speech-swift](https://github.com/soniqo/speech-swift) — Apple SDK
- [Speech Studio](https://soniqo.audio/speech-studio) — local speech generation and voice cloning app
- [Docs](https://soniqo.audio/getting-started) — install and CLI docs
- [soniqo.audio](https://soniqo.audio) — website
- [blog](https://soniqo.audio/blog) — blog

## Model

| Component | Parameters | Format | Precision | Static shape |
|---|---:|---|---|---|
| T3 block-diffusion token generator | 532.4M | Core ML `.mlmodelc` | fp16 | text_len 256, block_size 16, max_seq 1024 |
| S3Gen audio back-half | 266.0M | Core ML `.mlmodelc` | fp16 | token_len 192, mel_len 384 |
| Total | 798.4M | Core ML `.mlmodelc` | fp16 | 24 kHz waveform output |

## Files

| Path | Size | Description |
|---|---:|---|
| `config.json` | 4 KB | Root metadata for download tracking and runtime discovery |
| `t3/ConditioningEncoder.mlmodelc` | 25 MB | Speaker/prompt/emotion conditioning to T3 conditioning embedding |
| `t3/TextPrefill.mlmodelc` | 963 MB | Causal `[cond, text, start_speech]` prefix prefill and flat KV cache |
| `t3/BlockDecoder.mlmodelc` | 1.0 GB | Full-visible Flash speech-block logits with explicit KV cache |
| `t3/uncond_block_prior.npy` | 36 KB | Unconditional PMI prior for Flash scoring |
| `t3/tokenizer.json` | 28 KB | Chatterbox Flash text tokenizer |
| `t3/config.json` | 4 KB | T3 export metadata |
| `audio/FlowSpeakerProjector.mlmodelc` | 44 KB | S3Gen reference embedding to projected speaker conditioning |
| `audio/FlowEncoder.mlmodelc` | 79 MB | `prompt_token ++ speech_tokens` to flow `mu` and mask |
| `audio/FlowEstimator.mlmodelc` | 141 MB | One meanflow Euler derivative step |
| `audio/HiFTVocoder.mlmodelc` | 41 MB | Mel frames to 24 kHz waveform |
| `audio/audio_config.json` | 4 KB | S3Gen audio export metadata |

## Runtime Boundary

The exported graphs cover:

- T3 conditioning, text prefill, and block decoding.
- S3Gen speaker projection, flow encoder, meanflow estimator step, and HiFT vocoder.

The host runtime must still provide:

- text normalization and tokenization
- T3 denoising loop, PMI/CFG scoring, sampling, and EOS trimming
- reference waveform encoders:
  - VoiceEncoder: `ref.wav -> speaker_emb`
  - prompt speech tokenizer: `ref.wav -> prompt_speech_tokens`
  - S3Gen reference encoder: `ref.wav -> prompt_token`, `prompt_feat`, `embedding`
- padding/cropping and the two-step meanflow loop over `(t,r) = (0,0.5), (0.5,1.0)`

This means the bundle supports voice-cloning TTS when the runtime supplies reference conditioning tensors, but it is not yet a fully Core ML `ref.wav -> cloned wav` pipeline.

## Validation

| Test | Result |
|---|---|
| T3 graph roundtrip vs PyTorch wrappers | Pass at 2% relative tolerance |
| Audio graph roundtrip vs PyTorch wrappers | Pass for token_len 192, mel_len 384 |
| Stitched S3Gen meanflow-to-audio roundtrip | Pass |
| Prompted synthesis smoke test | Pass |
| Whisper tiny transcript of generated wav | `Core ML speech test.` |
| Smoke-test WER | 0.000 |

Core ML warnings from local export:

- `CPU_ONLY` prediction crashed for the T3 package in local coremltools 8.3 testing. Use `ALL`, `CPU_AND_NE`, or a compiled-device runtime.
- The uploaded artifact ships compiled `.mlmodelc` folders. The numerical parity tests were run against the source `.mlpackage` exports before compilation.

## Usage Sketch

The runtime loads graphs from `t3/` and `audio/`, then:

1. Prepare T3 reference conditioning tensors and S3Gen `ref_dict` tensors from a prompt wav.
2. Tokenize text with `t3/tokenizer.json`.
3. Run T3 prefill and block denoising/sampling to produce S3 speech tokens.
4. Concatenate S3Gen `prompt_token` and generated speech tokens.
5. Run `audio/FlowEncoder.mlmodelc`.
6. Build `cond` by copying `prompt_feat.T` into the mel prefix.
7. Run `audio/FlowEstimator.mlmodelc` twice for `(0,0.5)` and `(0.5,1.0)`.
8. Crop generated mel frames after the prompt prefix.
9. Run `audio/HiFTVocoder.mlmodelc` and crop padded samples.

## Source

Converted from [ResembleAI/chatterbox-flash](https://huggingface.co/ResembleAI/chatterbox-flash), revision `4385507288b8197e6dab8b4e6b1603328d549d9d`.