Chatterbox-Flash

Made with ❤️ by

Chatterbox-Flash is a block-diffusion zero-shot TTS model that extends the Chatterbox-TTS pipeline with a parallel masked decoder while preserving streaming generation. It unmasks multiple speech tokens in parallel within each block while keeping native block-by-block streaming, delivering autoregressive-class quality at a fraction of the latency.

Streaming & Speed Highlights

• Native block-by-block streaming — emits audio as each block is committed, no full-sequence wait.
• ~9× real-time synthesis at the default config (D = 16, α = 0.5), up to ~13× real-time at D = 32, α = 0.75.
• Time-to-first-packet from 103 ms, on par with streaming AR systems.
• RTF as low as 0.076 — substantially faster than autoregressive streaming TTS at the same scale.
• Early-decoding schedule adaptively ends denoising early, cutting average steps per block by ~20% at negligible quality cost.
• Built on a FlashInfer paged KV cache + CUDA-graph inference engine for low per-step overhead.

The released weights cover the four checkpoints needed at inference time:

File	What it is
t3_flash.safetensors	Block-diffusion T3 decoder (Llama-520M + 1 extra [MASK] token).
s3gen.safetensors	Flow-matching S3Gen vocoder.
ve.safetensors	GE2E voice encoder (taken verbatim from ResembleAI/chatterbox).
tokenizer.json	English BPE tokenizer (taken verbatim from ResembleAI/chatterbox).

Quick start

pip install chatterbox-flash

import torchaudio as ta
from chatterbox_flash import ChatterboxFlashTTS
tts = ChatterboxFlashTTS.from_pretrained("ResembleAI/chatterbox-flash", device="cuda")
wav = tts.generate(
    "Hello, world.",
    audio_prompt_path="reference.wav",
)
ta.save("out.wav", wav.unsqueeze(0).cpu(), tts.sr)

Inference defaults (paper configuration)

• Block size D = 16
• Maximum K = 10 denoising steps per block
• Sampling temperature 0.2
• shift outlier schedule with tau = 0.5
• CFG with w = 1.0, pmi_cfg combination
• FlashInfer paged KV cache + CUDA graph capture

Streaming Efficiency

Latency and throughput at concurrency 1, measured over 50 utterances. TTFP is the wall-clock time from request to the first emitted audio packet; RTF (real-time factor) is generation time divided by synthesized audio duration — lower is faster, and RTF < 1 means faster than real time.

Config (25 Hz, 0.5B)	TTFP (ms) ↓	RTF ↓
D = 16, α = 0.5 (default)	118	0.107
D = 16, α = 0.75	106	0.091
D = 24, α = 0.5	119	0.100
D = 24, α = 0.75	105	0.084
D = 32, α = 0.5	115	0.090
D = 32, α = 0.75	103	0.076

Even on a single concurrent request, Chatterbox-Flash sustains roughly 9× real-time synthesis at the default setting and ~13× real-time at D = 32, α = 0.75, while keeping time-to-first-packet low enough for interactive streaming.

Apple Silicon (MLX)

Chatterbox-Flash also runs locally on Apple Silicon via MLX. The numbers below were measured on a Mac M4 at the default configuration; both stay comfortably under real time (RTF < 1), and 4-bit quantization gives a further speedup.

Backend	RTF ↓
MLX	0.778
MLX (4-bit quantized)	0.665

Quality (Seed-TTS test-en)

Zero-shot TTS quality on the Seed-TTS English benchmark, under the canonical configuration (D = 16). SIM-o is speaker similarity to the reference (higher is better), WER is word error rate from ASR transcription (lower is better), and UTMOS is a predicted naturalness score (higher is better). Results are shown for our two main decoding settings against the Chatterbox backbone and ground-truth audio for reference.

System	SIM-o ↑	WER ↓	UTMOS ↑
Ground-truth	0.734	2.14	3.52
Chatterbox (AR backbone)	0.685	2.20	4.10
Chatterbox-Flash (α = 0)	0.704	1.96	4.09
Chatterbox-Flash (α = 0.5, early decoding)	0.704	2.04	4.08

Converting the autoregressive backbone into a block-diffusion decoder improves both speaker similarity (0.685 → 0.704) and intelligibility (2.20 → 1.96 WER) while keeping naturalness essentially unchanged — all while unlocking parallel, streaming-friendly decoding.

License

MIT — see LICENSE in the source repository.