Instructions to use Zyphra/ZONOS3-Mini-transformers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Zyphra/ZONOS3-Mini-transformers with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="Zyphra/ZONOS3-Mini-transformers")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("Zyphra/ZONOS3-Mini-transformers", device_map="auto") - Zonos
How to use Zyphra/ZONOS3-Mini-transformers with Zonos:
# pip install git+https://github.com/Zyphra/Zonos.git import torchaudio from zonos.model import Zonos from zonos.conditioning import make_cond_dict model = Zonos.from_pretrained("Zyphra/ZONOS3-Mini-transformers", device="cuda") wav, sr = torchaudio.load("speaker.wav") # 5-10s reference clip speaker = model.make_speaker_embedding(wav, sr) cond = make_cond_dict(text="Hello, world!", speaker=speaker, language="en-us") codes = model.generate(model.prepare_conditioning(cond)) audio = model.autoencoder.decode(codes)[0].cpu() torchaudio.save("sample.wav", audio, model.autoencoder.sampling_rate) - Notebooks
- Google Colab
- Kaggle
ZONOS3-Mini (native 🤗 Transformers)
ZONOS3-Mini is Zyphra's compact multilingual text-to-speech model (43
languages, about 100 million active parameters per token). This repo holds the checkpoint for the native
transformers port (zonos3_mini, no trust_remote_code): the language model, the MECT-B2 speaker encoder, the
SuperBPE tokenizer, the processor and the 27 release voices in voices/. The processor loads the
ZAC-1 codec as its audio tokenizer.
Model details, samples and evaluations: the blog post and technical report.
Usage
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from transformers import AutoModelForTextToWaveform, AutoProcessor
model_id = "Zyphra/ZONOS3-Mini-transformers"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForTextToWaveform.from_pretrained(model_id, device_map="auto")
voice = load_file(hf_hub_download(model_id, "voices/Savannah.safetensors"))
conversation = [{"role": "user", "content": "Hello! This is ZONOS3-Mini running in transformers."}]
inputs = processor.apply_chat_template(
conversation,
tokenize=True,
return_dict=True,
processor_kwargs={"speaker_reference_codes": [voice["codes"]]},
).to(model.device)
audio = processor.decode(model.generate(**inputs))
processor.save_audio(audio, "zonos3_mini.wav")
To clone a voice from a clip, put the clip (44.1 kHz) in the user turn:
conversation = [
{
"role": "user",
"content": [
{"type": "text", "text": "This voice was cloned from a short clip."},
{"type": "audio", "path": "my_voice.wav"},
],
}
]
inputs = processor.apply_chat_template(
conversation, tokenize=True, return_dict=True, processor_kwargs={"sampling_rate": 44100}
).to(model.device)
audio = processor.decode(model.generate(**inputs))
The clip is encoded into reference codes by ZAC-1 by default; speaker_mode="embedding" conditions on its speaker
embedding instead, and speaker_guidance=True with guidance_scale > 1 in generate strengthens the voice. The
documentation covers conditions (loudness, fmax, speaking rate), batching, torch.compile, the pipeline and training.
Install zyphra-textnorm to read numbers, dates, currencies and units aloud as the model was trained; without it, the
text is used as written.
Voices
voices/<Name>.safetensors holds each release voice's reference codes (codes, [frames, 10]) and speaker embedding
(embedding, [192]); voices/voices.json lists their language, style and gender.
Architecture
| Component | ZONOS3-Mini |
|---|---|
| Backbone | 11 layers, dimension 1,024; latent MoE with 64 experts, top-2 routing from layer 1 |
| Sequence mixer | 9 Gated DeltaNet layers + 2 global attention layers, Full AttnRes; context 32,768 positions |
| Text | SuperBPE, 8,192 entries + 4 special tokens |
| Audio tokens | 15 Hz; 10 codebooks of 4,096 entries (1 semantic + 9 acoustic); hop 2,940 samples at 44.1 kHz |
| Speaker prompt | codec reference prefix (up to 60 s) or a 192-d MECT-B2 embedding |
| Codec | ZAC-1 keep2_plain decoder |
The language model is stored in bfloat16 (the release dtype); the speaker encoder stays in float32.
License
Apache-2.0.
Citation
@misc{zyphra2026zonos3mini,
title = {{ZONOS3-Mini Technical Report}},
author = {Gabriel Clark and Sofian Mejjoute and George Close and Beren Millidge},
year = {2026},
eprint = {2611.16525},
archivePrefix = {arXiv},
primaryClass = {cs.SD},
url = {https://arxiv.org/abs/2611.16525},
}
- Downloads last month
- 75
Model tree for Zyphra/ZONOS3-Mini-transformers
Base model
Zyphra/ZONOS3-Mini