license: other
license_name: tontaube-community-model-license-1.0
license_link: https://huggingface.co/TontaubeAI/TontaubeV1/blob/main/LICENSE
pipeline_tag: text-to-speech
base_model:
- Qwen/Qwen3-1.7B
- Qwen/Qwen3-0.6B
language:
- en
- de
- es
- fr
- it
- nl
- pt
tags:
- text-to-speech
- voice-cloning
- multilingual
- audio
- vllm
inference: false
TontaubeV1
TontaubeV1 is a multilingual text-to-speech model designed for expressive voice cloning, long-form generation, and low-latency streaming. English and German are its primary languages; Spanish, French, Italian, Dutch, and Portuguese are also supported.
The release contains four causal codebook predictors. CB0 generates semantic
audio and duration from text, while progressively smaller CB1--CB3 models add
acoustic detail. Spoken text uses character-level tokenization, with compact
BPE-encoded language and speaking-style controls. The public styles are
audiobook, conversational, and agentic.
Technical report · Try TontaubeV1 online · Inference repository
Quick start
These checkpoints require the Tontaube vLLM model adapter and input layout;
they are not drop-in AutoModelForCausalLM checkpoints. Use the matching
release of the inference repository:
git clone --branch v1.0.0 --depth 1 \
https://github.com/craitech/tontaube.git
cd tontaube
uv run --python 3.12 --frozen tontaube serve
Once the API is ready, generate a WAV with the bundled English voice from a second terminal:
curl --silent --show-error --fail http://127.0.0.1:8080/predict \
-H 'content-type: application/json' \
-d '{"text":"Welcome to Tontaube.","language":"english","tag":"conversational","format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> welcome.wav
Clone a voice by sending about 5--60 seconds of clean, single-speaker speech with the request:
VOICE_FILE=/path/to/reference.wav
python3 -c 'import base64,json,sys; print(json.dumps({"text":"This voice was cloned from a local reference.","language":"english","tag":"conversational","voice_audio_b64":base64.b64encode(open(sys.argv[1],"rb").read()).decode(),"format":"wav"}))' "$VOICE_FILE" \
| curl --silent --show-error --fail http://127.0.0.1:8080/predict \
-H 'content-type: application/json' \
--data-binary @- \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> cloned.wav
The server downloads this model and its pinned runtime dependencies into local
caches on first use. Follow the inference repository's installation
instructions for system requirements, the browser interface, and Docker. The
optional English verbalizer is available separately from
TontaubeAI/TontaubeV1-Verbalizer and is downloaded only when enabled.
The default semantic sampling temperature is 0.8. A value around 0.6
generally favors text correctness, while values approaching 1.0 can produce
more varied and expressive delivery.
Capabilities
- Bundled synthetic voices and zero-shot cloning from up to 60 seconds of reference audio
audiobook,conversational, andagenticspeaking styles- Bounded-context long-form generation
- WAV generation and low-latency MP3 or Opus streaming
- Optional English text verbalizer, distributed separately
Examples
These MP3 samples were normalized to a consistent speech listening level.
Audiobook
English 1
English 2
English 3
German 1
Agentic
English 1
English 2
English 3
Model architecture
| Predictor | Role | Transformer blocks | Width | Stored parameters |
|---|---|---|---|---|
| CB0 | Semantic audio and duration | 28 | 2,048 | 1,829,116,930 |
| CB1 | Acoustic refinement | 16 | 1,024 | 448,960,512 |
| CB2 | Acoustic refinement | 8 | 1,024 | 327,307,264 |
| CB3 | Acoustic refinement | 4 | 1,024 | 268,577,792 |
| Total | 56 | 2,873,962,498 |
CB0 uses a Qwen3-1.7B-derived transformer trunk. CB1--CB3 use progressively shallower Qwen3-0.6B-derived trunks. Each predictor replaces the inherited language-model output layer with a two-layer audio-token head. The four output streams are decoded with DualCodec; the release inference path then uses the acoustic encoder and decoder from VibeVoice for continuous reconstruction and streaming.
Training
All four predictors were trained exclusively with supervised fine-tuning on approximately 200,000 hours of paired speech and text across seven languages, predominantly from public-domain audiobook recordings and openly released speech corpora.
Repository contents
TontaubeV1/
├── cb0/
├── cb1/
├── cb2/
└── cb3/
Each directory contains its safetensors weights, runtime configuration, and tokenizer files. The repository does not redistribute DualCodec, W2V-BERT, or VibeVoice weights. The inference setup downloads those components from their upstream repositories at pinned revisions.
Serving performance
With weights resident on one NVIDIA GeForce RTX 5090 and the process warmed, the streaming path reaches approximately 200 ms to first encoded audio. In separate non-streaming measurements, end-to-end real-time factor (RTF) is 0.08 for one input and aggregate RTF is 0.02 across eight concurrent inputs. Startup, model loading, and network latency are excluded.
Evaluation
The LLM-as-a-judge pairwise audiobook-reading benchmark contains 400 fixed English passages of 250--500 characters. For each passage, Gemini 3.1 Pro Preview judges the same output pair twice, once in each presentation order, on prosody and word-by-word correctness. A TontaubeV1 preference, tie, or comparator preference scores 1, 0.5, or 0; the figure reports the mean over all 800 order-balanced judgments. A score of 50% denotes parity. Whiskers show 95% passage-cluster bootstrap intervals; labels beneath the points give tie rates across individual judge calls.
Exact preference scores
| Comparator | Prosody preference | Correctness preference |
|---|---|---|
| ElevenLabs Flash v2.5 | 50.1% | 48.9% |
| Fish Audio S2 Pro | 82.1% | 49.6% |
| Gradium API, April 2026 | 86.2% | 54.6% |
| Cartesia Sonic 3 | 82.3% | 60.8% |
TontaubeV1 uses semantic sampling temperature 0.55 and acoustic temperature
zero in these comparisons. Each waveform is independently normalized to an
average level of -20 dBFS before judging. The technical report gives the
complete judge instructions and further methodological limitations.
Fish Audio uses the same frozen cloning reference as TontaubeV1. ElevenLabs, Gradium, and Cartesia instead use fixed provider voices while TontaubeV1 uses the cloning reference. Although voice identity and timbre are excluded from the rubric, prosody is not fully separable from the reference; this asymmetry may favor TontaubeV1 in those three comparisons.
On the 1,088 English zero-shot examples of the Seed-TTS evaluation set,
TontaubeV1 obtains 1.66% mean utterance-level word error rate using Whisper
large-v3 transcription at semantic sampling temperature 0.6.
These automated evaluations measure English reading prosody and text correctness. They do not establish voice similarity, overall sound quality, multilingual quality, long-form continuity, or streaming quality.
Limitations and responsible use
Autoregressive generation can omit, repeat, or alter text and can terminate too early or too late. Reference conditioning may reproduce incidental recording properties or imperfectly preserve identity. Quality varies by language, speaker, style, and text domain. The optional verbalizer can normalize incorrectly or alter wording. Training emphasized audiobook speech, so audiobook generation may be more reliable than conversational or agentic generation.
English and German are the primary languages. German phoneme realization can be imperfect, and Spanish, French, Italian, Dutch, and Portuguese have not been evaluated by native speakers; no comparative quality claims are made for those languages.
Voice cloning can enable impersonation, fraud, non-consensual synthesis, and misleading media. Obtain permission for reference voices and apply appropriate authentication, rate limiting, disclosure, logging, and abuse-response measures in deployed systems. Generated speech must not be treated as verified identity or factual evidence.
License and attribution
TontaubeV1 is distributed under the Tontaube Community Model License 1.0, which is not an open-source license. The license applies to the Tontaube model materials identified by the release manifest; third-party material remains subject to its respective terms. See Third-Party Notices.
TontaubeV1 builds on Qwen3 and uses the DualCodec speech representation. The release inference stack also uses components from VibeVoice, W2V-BERT, PyTorch, Transformers, and vLLM. Their respective licenses and notices remain applicable.
The generated audio examples in assets/showcase/ are provided as
demonstration material and are not part of the model materials covered by the
Tontaube Community Model License 1.0.