ZONOS3-Mini B70 Turbo
One Intel Arc Pro B70 serves 20 live voices with no stalls with Zyphra's ZONOS3-Mini, where the stock runtime serves 8. Same model files, same voice quality.
This repo is two patches for Zyphra's zonos3.cpp runtime (Vulkan backend). The model weights are Zyphra's own GGUFs, downloaded from their repo. All credit for the model goes to Zyphra.
How to run
You need git, CMake, a C++ compiler, a Rust toolchain (cargo), and the Vulkan development files plus glslc (the LunarG
Vulkan SDK, or your distro's packages such as libvulkan-dev and glslc).
git clone https://github.com/Zyphra/ZONOS3-Mini.git && cd ZONOS3-Mini
git checkout a50e14117d99181ba7f28047b451ba03ba1a09a9
git submodule update --init servers/ggml/ggml third_party/zyphra-textnorm
hf download Frosty40/ZONOS3-Mini-B70-Turbo --include "patches/*" --local-dir b70-turbo
cp b70-turbo/patches/vulkan-b70-turbo.patch servers/ggml/patches/ # applied by CMake at configure time
git apply b70-turbo/patches/zonos3-b70-turbo.patch
cd servers/ggml
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
hf download Zyphra/ZONOS3-Mini gguf/backbone-ptq.gguf gguf/zac-keep2.gguf gguf/spk-encoder.gguf gguf/steer.gguf --local-dir zonos3
./build/zonos3-server zonos3/gguf/backbone-ptq.gguf --gpu --batch 32 --port 1919
curl -fsS http://127.0.0.1:1919/v2/generate \
-d '{"text":"Hello from one Intel Arc B70.","stream":false,"format":"wav","seed":1}' -o hello.wav
The server speaks Zyphra's HTTP API, including streaming and the OpenAI-compatible /v1/audio/speech. The first
launch on a machine compiles GPU shaders (about 40 seconds); later launches start in about two seconds.
Results
One Arc Pro B70 (Mesa 26.2.6, Vulkan SDK 1.4.363.1, Linux 7.0), Ryzen 9 5950X. Zyphra's server with
--batch 32, streaming, zac-keep2 decoder, a speaker embedding on every request. Stock and turbo use the same
zonos3.cpp commit (a50e141), the same model files and the same prompts, and ran alternately in one session (stock,
turbo, stock, turbo), two rounds each. All voices start at once.
A stall is a gap in playback for a listener whose player keeps 200 ms of audio buffered.
| Voices at once | Stock: longest stall (4 rounds) | Turbo: longest stall (4 rounds) |
|---|---|---|
| 8 | none | none |
| 12 | 321 ms | none |
| 16 | 489 ms | none |
| 20 | 745 ms | none |
| 24 | 873 ms | 58 ms |
| Stock | Turbo | |
|---|---|---|
| Live voices with no stalls in every round | 8 | 20 |
| Audio served at 16 voices, seconds per second (mean of 4 rounds) | 18.3 | 39.4 |
| Time to first audio, one voice (median of 10) | 43 ms | 28 ms |
| Word error rate, 48 takes at 16 voices | 0.0% | 0.0% |
Speech is checked by transcribing every take with Whisper large-v3 (faster-whisper, int8) and comparing it with the
prompt text after Whisper's English normalizer. The 12 prompts (prompts_heldout_v1.jsonl) were never used to tune
the patches. Raw results and the scripts that produced them are in results/ and scripts/.
What the patches change
vulkan-b70-turbo.patch(ggml Vulkan backend):- Batches of 16, 20, 24 and 32 voices stay on the fast matrix-vector kernels.
- Each mixture-of-experts layer runs as one GPU dispatch for the whole batch.
- Attention for many voices runs in one dispatch and sums in the same order as for a single voice.
zonos3-b70-turbo.patch(runtime):- The audio decoder reuses its GPU graphs from chunk to chunk instead of rebuilding them, and uploads each chunk's inputs in one transfer.
- The 44.1 kHz resampler skips the filter taps that are zero and runs vectorized on Linux.
- The language model reads every voice's output in one transfer.
- Top-k sampling finds its candidates faster and draws the same tokens from the same scores.
Licence
The patches are MIT, the licence of the zonos3.cpp runtime and ggml that they modify. The model is Apache 2.0 from Zyphra and is downloaded from their repo, not redistributed here. Voice cloning should only be used with the voice owner's consent.
Built by OmniSpace Technologies.