| --- |
| language: |
| - af |
| - am |
| - ar |
| - as |
| - ast |
| - az |
| - be |
| - bg |
| - bn |
| - bs |
| - ca |
| - ceb |
| - ckb |
| - cs |
| - cy |
| - da |
| - de |
| - el |
| - en |
| - es |
| - et |
| - fa |
| - ff |
| - fi |
| - fil |
| - fr |
| - ga |
| - gl |
| - gu |
| - ha |
| - he |
| - hi |
| - hr |
| - hu |
| - hy |
| - id |
| - ig |
| - is |
| - it |
| - ja |
| - jv |
| - ka |
| - kam |
| - kea |
| - kk |
| - km |
| - kn |
| - ko |
| - ky |
| - lb |
| - lg |
| - ln |
| - lo |
| - lt |
| - luo |
| - lv |
| - mi |
| - mk |
| - ml |
| - mn |
| - mr |
| - ms |
| - mt |
| - my |
| - nb |
| - ne |
| - nl |
| - nso |
| - ny |
| - oc |
| - om |
| - or |
| - pa |
| - pl |
| - ps |
| - pt |
| - ro |
| - ru |
| - sd |
| - sk |
| - sl |
| - sn |
| - so |
| - sr |
| - su |
| - sv |
| - sw |
| - ta |
| - te |
| - tg |
| - th |
| - tr |
| - tw |
| - uk |
| - umb |
| - ur |
| - uz |
| - vi |
| - wo |
| - xh |
| - yo |
| - yue |
| - zh |
| - zu |
| pipeline_tag: text-to-speech |
| tags: |
| - zero-shot |
| - multilingual |
| - voice-cloning |
| - voice-design |
| - omnivoice |
| - qwen3 |
| - dac |
| - trained-from-scratch |
| library_name: omnivoice |
| license: other |
| license_name: genpio-voices-model-license |
| datasets: |
| - google/fleurs |
| - facebook/multilingual_librispeech |
| - AISHELL/AISHELL-1 |
| extra_gated_heading: Request access to Genpio Voices |
| extra_gated_prompt: >- |
| The weights are gated and reviewed manually. Access is granted to identifiable |
| organisations and individuals whose Hub profile shows who they are and what |
| they intend to build. Requests without an attributable identity or a stated |
| use case are declined. Approval is at Genpio's discretion and may be revoked. |
| Commercial use is not granted by approval alone — contact hi@genpio.com for terms. |
| extra_gated_fields: |
| Full name: text |
| Organisation or affiliation: text |
| Country: country |
| Intended use: text |
| I will not use this model for unauthorized voice cloning, impersonation or fraud: checkbox |
| I agree that approval does not grant commercial rights: checkbox |
| extra_gated_button_content: Request access |
| --- |
| |
| # Genpio Voices 🌍 |
|
|
| Genpio Voices is a massively multilingual zero-shot text-to-speech model **trained from scratch, in-house, on the OmniVoice architecture**. It is not a fine-tune or a continuation of any pre-existing speech checkpoint: the architecture and the language-backbone shape are open source (Apache-2.0), the weights are Genpio's own, produced by a single balanced training run over publicly documented corpora. |
|
|
| - **Website / technology page:** <https://genpio.com/technology> |
| - **Architecture reference:** [OmniVoice](https://github.com/k2-fsa/OmniVoice) — audio-codebook TTS over a Qwen3-0.6B-shaped LM |
| - **Contact:** <hi@genpio.com> |
|
|
| ## Access |
|
|
| The weights are **gated and reviewed by hand**. The card, the licensing story and the corpus provenance are public; the model files are not. Downloading `model.safetensors`, `tokenizer.json` or the configs requires an approved request from an authenticated account — anonymous `hf download` and `from_pretrained` will fail until access is granted. |
|
|
| Requests are approved case by case. Genpio grants access to organisations and individuals who are identifiable: a Hub profile that shows who you are, an organisation or affiliation, and a stated intended use. Requests from empty or anonymous profiles, or without a use case, are declined. Access is at Genpio's discretion and may be revoked. |
|
|
| Approval grants **read access to the weights only**. It is not a commercial licence and it does not sublicense any of the rights described in [Licensing and attribution](#licensing-and-attribution) — for commercial terms, contact <hi@genpio.com> before deploying. |
|
|
| ## What "trained from zero" means here |
|
|
| | Component | Origin | What Genpio did | |
| | --- | --- | --- | |
| | OmniVoice architecture | `k2-fsa/OmniVoice`, Apache-2.0 | Adopted as the model definition (audio codebooks + LM backbone) | |
| | LM backbone topology | `Qwen/Qwen3-0.6B`, Apache-2.0 | Adopted as the *shape* — 28 layers, hidden size 1024, 16 heads / 8 KV heads | |
| | Audio codec | Descript Audio Codec (DAC), MIT | Adopted as the speech tokenizer — see [Audio codec](#audio-codec) below | |
| | **Weights in this repository** | **Genpio** | **Randomly initialised and trained end-to-end on the corpus mix below** | |
|
|
| Because no upstream weights were inherited, there is no licence to renegotiate downstream: Genpio chose the corpora, built the mix, ran the training, and owns the resulting weights outright with full commercial rights. Every dependency in the stack is Apache-2.0 or MIT — permissive, permanent and irrevocable, with no user-count ceiling, no field-of-use restriction and no acceptable-use policy incorporated by reference. |
|
|
| The training is **single-run and simultaneous** — every language is present in the same optimisation, never bolted on in sequential passes. Sequential fine-tuning of one language after another causes catastrophic forgetting and degrades the languages trained earlier; a joint run avoids it. Per-language sampling caps keep high-resource languages (English, Mandarin) from overshadowing under-represented ones. |
|
|
| ## Training data |
|
|
| Seven public corpora, all with published provenance so the mix can be checked against OpenSLR and Google directly rather than taken on assertion. |
|
|
| | Corpus | ID | Licence | Scale | Role in the mix | |
| | --- | --- | --- | --- | --- | |
| | Multilingual LibriSpeech | SLR94 | CC-BY-4.0 | ~50,000 h | Speaker diversity across 8 European languages: en, de, nl, fr, it, es, pt, pl | |
| | Google multi-speaker TTS data | SLR37, SLR41–44 | CC-BY-SA-4.0 | ~100 h @ 48 kHz | Bengali (bn-BD, bn-IN), Javanese, Khmer, Nepali, Sundanese | |
| | Google crowdsourced multi-speaker data | SLR63–66, SLR78–80 | CC-BY-SA-4.0 | ~350 h @ 48 kHz | Malayalam, Marathi, Tamil, Telugu, Gujarati, Kannada, Burmese | |
| | BibleTTS | SLR129 | CC-BY-SA-4.0 | ~380 h @ 48 kHz | Twi (Asante/Akuapem), Hausa, Lingala, Yoruba — highest impact per hour | |
| | AISHELL-1 | SLR33 | Apache-2.0 | 178 h, 400 speakers | Mandarin Chinese voice-quality refinement | |
| | Yoruba multi-speaker speech data | SLR86 | CC-BY-SA-4.0 | ~40 h, studio-clean | Yoruba | |
| | Google FLEURS | — | CC-BY-4.0 | ~10 h per language | Breadth layer across 102 languages | |
|
|
| Corpus identifiers and licences are as published on [OpenSLR](https://www.openslr.org/resources.php). Full attribution for every component is in [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md), and the same data is available machine-readable in [training_datasets.json](training_datasets.json). |
|
|
| The `language:` list in this card is the union of those corpora: the 102 FLEURS locales, plus Twi from BibleTTS and Sundanese from SLR44, neither of which FLEURS covers. |
|
|
| ## Coverage |
|
|
| - **60+ languages** carry baseline fluency directly from the training mix. |
| - **85+ languages** are exposed as output languages in the Genpio product catalogue. |
| - **700+ localized neural voices** plus multilingual cloned voices. Public first-party voices are returned with a `genpio:` prefix in API responses. |
|
|
| Training coverage and product catalogue are deliberately reported as separate numbers — the corpora above establish the former; the latter includes voices built on top of it. |
|
|
| ## Key features |
|
|
| - **Zero-shot voice cloning** from a short reference clip. |
| - **Voice design** — control by speaker attributes (gender, age, pitch, dialect/accent, whisper). |
| - **Fine-grained control** — non-verbal symbols (e.g. `[laughter]`) and pronunciation correction via pinyin or phonemes. |
| - **Fast inference** — RTF as low as 0.025 (~40× faster than real time). |
| - **Self-hosted serving** — inference runs on Genpio's own GPUs; nothing in the serving path is a third-party API. |
|
|
| ## Audio codec |
|
|
| Speech is tokenised with the **Descript Audio Codec (DAC)** — [`descriptinc/descript-audio-codec`](https://github.com/descriptinc/descript-audio-codec), **MIT licensed**. The LM predicts 8 residual-vector-quantised codebooks of 1024 entries each (plus one mask id), which the DAC decoder reconstructs to 24 kHz waveform audio. |
|
|
| DAC was chosen over community-licensed alternatives for licensing reasons as much as quality ones. **Genpio Voices does not use the Boson Higgs Audio 2 tokenizer, and the Boson Higgs Audio 2 Community License does not apply to this model.** That means none of the following bind Genpio or its downstream users: |
|
|
| - the 100,000-annual-active-user ceiling above which an expanded licence must be requested, |
| - the Llama 3 Acceptable Use Policy incorporated by reference, |
| - the restriction on using model outputs to improve other large language models, |
| - the mandatory Boson attribution notice. |
|
|
| MIT imposes only attribution, which is given here. |
|
|
| ## Model configuration |
|
|
| Taken from [config.json](config.json): |
|
|
| | Field | Value | |
| | --- | --- | |
| | `model_type` | `omnivoice` | |
| | `num_audio_codebook` | 8 | |
| | `audio_codebook_weights` | `[8, 8, 6, 6, 4, 4, 2, 2]` | |
| | `audio_vocab_size` | 1025 (mask id 1024) | |
| | Codec | Descript Audio Codec (DAC), MIT | |
| | Backbone | Qwen3, 28 layers, hidden 1024, FFN 3072 | |
| | Attention | 16 heads / 8 KV heads, head dim 128, full attention | |
| | Context | 40,960 positions, RoPE θ = 1e6 | |
| | Tokenizer limit | `model_max_length` 40,960 — matched to the position budget | |
| | Text vocab | 151,676 (151,643 BPE + 33 special tokens) | |
| | Load dtype | `bfloat16` — weights are stored as float32 in `model.safetensors` and cast at load | |
| | Sample rate | 24 kHz output | |
|
|
| There is no long-context mechanism behind the 40,960-position budget — `sliding_window` is null and `use_sliding_window` is false — so the tokenizer's `model_max_length` in [tokenizer_config.json](tokenizer_config.json) is set to the same figure rather than a larger nominal one. Requests longer than that must be chunked upstream. |
|
|
| ### Prompt format |
|
|
| This is not a chat model, and no `chat_template.jinja` is shipped: `tokenizer.apply_chat_template()` cannot express how the model is conditioned, so it is better that the call fails than that it silently returns a ChatML prompt the model was never trained on. Conditioning is built from two text segments, in this order: |
|
|
| ```text |
| style: [<|denoise|>] <|lang_start|>{language}<|lang_end|><|instruct_start|>{instruction}<|instruct_end|> |
| text: <|text_start|>{reference_transcript} {target_text}<|text_end|> |
| ``` |
|
|
| - `<|denoise|>` is prepended only when reference audio is supplied and denoising is requested. |
| - `{language}` and `{instruction}` fall back to the literal string `None` when unset — the field is always present. |
| - The reference transcript and the target text are concatenated into a single string; there is no separator token between them. |
|
|
| Each segment is tokenised and then repeated across all `num_audio_codebook` (8) channels, and the full input is the concatenation `[style, text, reference audio tokens, target slots]` along the time axis, where the target slots are filled with `audio_mask_id` (1024) for iterative unmasked decoding. The model input is therefore a `(8, T)` tensor, not a flat token sequence — which is the structural reason a Jinja string template cannot represent it. |
|
|
| | Token | ID | |
| | --- | --- | |
| | `<|denoise|>` | 151669 | |
| | `<|lang_start|>` / `<|lang_end|>` | 151670 / 151671 | |
| | `<|instruct_start|>` / `<|instruct_end|>` | 151672 / 151673 | |
| | `<|text_start|>` / `<|text_end|>` | 151674 / 151675 | |
|
|
| The ChatML and agent tokens inherited from the Qwen3 tokenizer — `<|im_start|>`, `<|im_end|>`, `<think>`, `<tool_call>`, the FIM and vision ids — are unused by this model, though `<|im_end|>` remains the nominal `eos_token`. |
|
|
| ### File integrity |
|
|
| [CHECKSUMS.sha256](CHECKSUMS.sha256) pins a SHA-256 for every distributed file, including `model.safetensors` and `tokenizer.json`, which are served from the model repository rather than the git tree. Verify a download with `sha256sum -c CHECKSUMS.sha256`. Hashes are over LF bytes, so clone with `core.autocrlf=false` if you intend to check the text files on Windows. |
|
|
| ## Licensing and attribution |
|
|
| - **Weights in this repository:** © Genpio. Trained from scratch by Genpio; no upstream weights inherited. Commercial use is Genpio's own right and is not sublicensed by this card — contact <hi@genpio.com> for terms. |
| - **Architecture and runtime code:** OmniVoice and Qwen3 are Apache-2.0, used under a permanent, irrevocable grant with no field-of-use or revenue restrictions. |
| - **Audio codec:** Descript Audio Codec, MIT. No community or research-only licence sits anywhere in the stack; in particular the Boson Higgs Audio 2 Community License does not apply — see [Audio codec](#audio-codec). |
| - **Training corpora:** attribution is required by the corpus licences and is given in full in [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md). MLS and FLEURS are CC-BY-4.0; SLR37/41–44, SLR63–66/78–80, SLR86 and SLR129 are CC-BY-SA-4.0; AISHELL-1 is Apache-2.0. Downstream users redistributing derivatives of the *data* must honour those terms; Genpio's position — not settled law — is that they do not restrict the trained weights. |
| - **Notices:** [NOTICE](NOTICE) and [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md) must be distributed with the model to satisfy the Apache-2.0, MIT and Creative Commons attribution terms. |
|
|
| No CC-BY-NC corpus (e.g. Emilia) is in the mix, so the non-commercial constraint that applies to some open TTS checkpoints does not apply here. |
|
|
| ## Citation |
|
|
| The architecture this model is trained on: |
|
|
| ```bibtex |
| @article{zhu2026omnivoice, |
| title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models}, |
| author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel}, |
| journal={arXiv preprint arXiv:2604.00688}, |
| year={2026} |
| } |
| ``` |
|
|
| ## Disclaimer |
|
|
| Voice cloning is consent-based: no likeness or voice may be cloned without the permission of the person it belongs to. Users are prohibited from using this model for unauthorized voice cloning, impersonation, fraud, scams, or any other illegal or unethical activity, and must comply with applicable local laws, regulations and ethical standards. Genpio assumes no liability for misuse and advocates responsible AI development and use. |
|
|
