arktts-multi-realistic-unicode

ArkTTS voice for phoonnx, converted to ONNX from Edge0/Audio8-TTS-Realistic-Air.

Model details

Field Value
Repository Phoonnx/arktts-multi-realistic-unicode
Voice id arktts/realistic-air-lj/en
Languages en
Architecture ArkTTS
Text input The text, split into subwords by the BPE tokenizer in tokenizer.json
Punctuation Not recorded in the config. phoonnx removes punctuation before synthesis.
Sample rate 44100 Hz
Speakers 1
Training code Upstream training code; see Edge0/Audio8-TTS-Realistic-Air
Phonemizer None: a BPE tokenizer (tokenizer.json) reads the text
Training data Not recorded upstream
Base model Edge0/Audio8-TTS-Preview-0.6b
Training checkpoint None in this repository
Upstream Edge0/Audio8-TTS-Realistic-Air
Trained by Edge0 (upstream)
Licence Apache 2.0

QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The voice voices/lj.json holds the codec codes of LJ Speech LJ001-0009 (public domain) and its transcription. QA covers English only.

Check fp32 int8
Result pass pass
WER 0.000 0.000
Characters per second 11.4 to 14.1 -
Length, int8 / fp32 - 0.81 to 0.97
Graph size 3486.6 MB 959.9 MB
Peak RSS 3766.3 MB 2005.4 MB
RTF (median of 3, 4 threads) 8.098 3.365

Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.

Each <graph>_int8.onnx file is the int8 copy of <graph>_fp32.onnx, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.

How to use

Install phoonnx:

pip install --pre phoonnx

Then run:

import wave
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["arktts/realistic-air-lj/en"].load()

with wave.open("output.wav", "wb") as wav_file:
    voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file)

In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select this voice in mycroft.conf:

{
  "tts": {
    "module": "ovos-tts-plugin-phoonnx",
    "ovos-tts-plugin-phoonnx": {
      "voice": "arktts/realistic-air-lj/en"
    }
  }
}

Training data

The upstream card does not name the training data. See Edge0/Audio8-TTS-Realistic-Air.

Related links

Ownership and licensing

Edge0 trained this model and publishes it at Edge0/Audio8-TTS-Realistic-Air under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit Edge0 when you use this voice.

Downloads last month
49
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Phoonnx/arktts-multi-realistic-unicode

Quantized
(13)
this model

Collection including Phoonnx/arktts-multi-realistic-unicode