ZONOS3-Mini-GGUF

ZONOS3-Mini is Zyphra's open-source, multilingual text-to-speech model. It supports 43 languages and offers voice cloning from a reference clip or a single speaker embedding. It has about 800M total parameters, with roughly 100M active per token. The backbone is an 11-layer hybrid of 9 Gated DeltaNet layers and 2 global attention layers, with a latent mixture-of-experts (64 experts, top-2 routing). It generates 15 Hz, 10-codebook audio tokens, and a causal iSTFT head renders them to 44.1 kHz audio as they are produced. Users can design a voice by shaping age, accent and vocal texture, and can control loudness, bandwidth and speaking rate. Text is normalized with NeMo for nine languages and tokenized with an 8,192-entry SuperBPE tokenizer. A single generation can run up to 10 minutes. The card reports a median time to first audio of 80 ms with 8 concurrent requests on an H100, and peak throughput of 457 seconds of audio per second. The model runs in real time on phones and laptops, including in the browser via WebGPU. It can also be served with SGLang-Omni through an OpenAI-compatible /v1/audio/speech endpoint, or run with native runtimes (CUDA, MLX for Apple Silicon, and C++/ggml for CPU, CUDA, Metal or Vulkan) that load the provided GGUF files. The weights are released under Apache 2.0.

Model Files

File Name Quant Type File Size File Link Description
ZONOS3-Mini.BF16.gguf BF16 1.61 GB Link Full BF16 weights. Highest quality, largest file size.
ZONOS3-Mini.Q2_K.gguf Q2_K 300 MB Link Lowest quality, smallest file size, only for extreme resource constraints.
ZONOS3-Mini.Q3_K_L.gguf Q3_K_L 420 MB Link Lower quality but usable, good for very low RAM availability.
ZONOS3-Mini.Q3_K_M.gguf Q3_K_M 387 MB Link Low quality.
ZONOS3-Mini.Q4_0.gguf Q4_0 456 MB Link Standard 4-bit quantization.
ZONOS3-Mini.Q4_K_M.gguf Q4_K_M 484 MB Link Good quality, default size for most use cases, recommended.
ZONOS3-Mini.Q4_K_S.gguf Q4_K_S 457 MB Link Slightly lower quality with more space savings, recommended.
ZONOS3-Mini.Q5_0.gguf Q5_0 555 MB Link Standard 5-bit quantization.
ZONOS3-Mini.Q5_K_M.gguf Q5_K_M 570 MB Link High quality, recommended.
ZONOS3-Mini.Q5_K_S.gguf Q5_K_S 555 MB Link High quality with slight space savings, recommended.
ZONOS3-Mini.Q6_K.gguf Q6_K 661 MB Link Very high quality, near perfect, recommended.
ZONOS3-Mini.Q8_0.gguf Q8_0 855 MB Link Extremely high quality, generally unneeded but max available standard quant.
ZONOS3-Mini.mmproj-bf16.gguf mmproj-bf16 689 MB Link Multimodal projection file in BF16 format. Used for audio/voice capabilities.

Run with llama.cpp

Install dependencies and clone llama.cpp

!pip install transformers huggingface-hub

!git clone https://github.com/ggml-org/llama.cpp.git
%cd /data/llama.cpp

Download GGUF model files

Download the BF16 model and BF16 multimodal projector:

from huggingface_hub import snapshot_download

# Target local directory
local_dir = "./model/ZONOS3-Mini-GGUF"

# Download only the specified files
snapshot_download(
    repo_id="prithivMLmods/ZONOS3-Mini-GGUF",
    local_dir=local_dir,
    allow_patterns=[
        "ZONOS3-Mini.BF16.gguf",
        "ZONOS3-Mini.mmproj-bf16.gguf",
    ],
    resume_download=True,
)

Build llama-tts

Build the TTS CLI with the CUDA backend enabled:

!cmake --build . --config Release -j$(nproc) --target llama-tts

Text-to-speech generation (GPU offloading)

Run inference using llama-tts with GPU offloading enabled:

!/data/llama.cpp/build/bin/llama-tts \
    -m /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.BF16.gguf \
    -mm /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.mmproj-bf16.gguf \
    -p "ZONOS3-Mini is Zyphra’s compact, open-source text-to-speech model, delivering natural reliable speech and high-fidelity voice cloning with just 100 million active parameters. It runs in real time on phones and laptops, including directly in the browser." \
    -ngl 99 \
    -n 1000 \
    --seed 42 \
    --output /data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav

Output

The generated audio file will be saved at:

/data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav

You can play the generated WAV file using an audio player or a notebook audio widget.

llama.cpp

LLM inference in C/C++ β€” https://github.com/ggml-org/llama.cpp

Quantized using llama.cpp by prithivMLmods.

Downloads last month
339
GGUF
Model size
0.8B params
Architecture
zonos3
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for prithivMLmods/ZONOS3-Mini-GGUF

Quantized
(3)
this model

Collections including prithivMLmods/ZONOS3-Mini-GGUF