arktts-multi-realistic-unicode
ArkTTS voice for phoonnx, converted to ONNX from Edge0/Audio8-TTS-Realistic-Air.
Model details
| Field | Value |
|---|---|
| Repository | Phoonnx/arktts-multi-realistic-unicode |
| Voice id | arktts/realistic-air-lj/en |
| Languages | en |
| Architecture | ArkTTS |
| Text input | The text, split into subwords by the BPE tokenizer in tokenizer.json |
| Punctuation | Not recorded in the config. phoonnx removes punctuation before synthesis. |
| Sample rate | 44100 Hz |
| Speakers | 1 |
| Training code | Upstream training code; see Edge0/Audio8-TTS-Realistic-Air |
| Phonemizer | None: a BPE tokenizer (tokenizer.json) reads the text |
| Training data | Not recorded upstream |
| Base model | Edge0/Audio8-TTS-Preview-0.6b |
| Training checkpoint | None in this repository |
| Upstream | Edge0/Audio8-TTS-Realistic-Air |
| Trained by | Edge0 (upstream) |
| Licence | Apache 2.0 |
QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The voice voices/lj.json holds the codec codes of LJ Speech LJ001-0009 (public domain) and its transcription. QA covers English only.
| Check | fp32 | int8 |
|---|---|---|
| Result | pass | pass |
| WER | 0.000 | 0.000 |
| Characters per second | 11.4 to 14.1 | - |
| Length, int8 / fp32 | - | 0.81 to 0.97 |
| Graph size | 3486.6 MB | 959.9 MB |
| Peak RSS | 3766.3 MB | 2005.4 MB |
| RTF (median of 3, 4 threads) | 8.098 | 3.365 |
Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.
Each <graph>_int8.onnx file is the int8 copy of <graph>_fp32.onnx, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.
How to use
Install phoonnx:
pip install --pre phoonnx
Then run:
import wave
from phoonnx.model_manager import TTSModelManager
manager = TTSModelManager()
manager.load()
manager.merge_default_voices()
voice = manager.voices["arktts/realistic-air-lj/en"].load()
with wave.open("output.wav", "wb") as wav_file:
voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file)
In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select this voice in
mycroft.conf:
{
"tts": {
"module": "ovos-tts-plugin-phoonnx",
"ovos-tts-plugin-phoonnx": {
"voice": "arktts/realistic-air-lj/en"
}
}
}
Training data
The upstream card does not name the training data. See Edge0/Audio8-TTS-Realistic-Air.
Related links
- Upstream model: Edge0/Audio8-TTS-Realistic-Air
- phoonnx: https://github.com/TigreGotico/phoonnx
- TigreGotico: https://tigregotico.pt
Ownership and licensing
Edge0 trained this model and publishes it at Edge0/Audio8-TTS-Realistic-Air under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit Edge0 when you use this voice.
- Downloads last month
- 49
Model tree for Phoonnx/arktts-multi-realistic-unicode
Base model
Edge0/Audio8-TTS-Preview-0.6b