Text-to-Speech
ONNX
F5-TTS
phoonnx
Jarbas's picture
README.md: card follows the f5tts_pinyin text route of the pinned config, adds the text preparation section
d6ab1c8 verified
|
Raw History Blame Contribute Delete
5.83 kB
---
license: apache-2.0
language:
- as
- bn
- bo
- gu
- hi
- kn
- ks
- kok
- mai
- ml
- mni
- mr
- ne
- or
- pa
- sa
- sat
- sd
- ta
- te
- ur
- doi
- raj
- en
base_model:
- SPRINGLab/SPRING_F5
base_model_relation: quantized
library_name: phoonnx
pipeline_tag: text-to-speech
tags:
- phoonnx
- onnx
- f5-tts
- text-to-speech
datasets:
- ai4bharat/IndicVoices
- ai4bharat/Rasa
---
# f5tts-multi-spring-unicode
F5-TTS voice for [phoonnx](https://github.com/TigreGotico/phoonnx), converted to ONNX from [SPRINGLab/SPRING_F5](https://huggingface.co/SPRINGLab/SPRING_F5).
## Model details
| Field | Value |
| --- | --- |
| Repository | `Phoonnx/f5tts-multi-spring-unicode` |
| Voice id | `f5tts/spring` |
| Languages | `mul` |
| Architecture | F5-TTS |
| Text input | The characters of the text, capitals and marks included (`f5tts_pinyin`) |
| Punctuation | Trained with the clause punctuation , . ! ? : ;. phoonnx 1.96.1a1 and later keep it at synthesis. |
| Sample rate | 24000 Hz |
| Speakers | 1 |
| Training code | Upstream training code; see [SPRINGLab/SPRING_F5](https://huggingface.co/SPRINGLab/SPRING_F5) |
| Phonemizer | scriptconv `f5tts_pinyin` (0.0.4a110 and later), a port of the F5-TTS function `convert_char_to_pinyin`. It reads a Chinese character as its pinyin syllable |
| Training data | [ai4bharat/IndicVoices](https://huggingface.co/datasets/ai4bharat/IndicVoices), [ai4bharat/Rasa](https://huggingface.co/datasets/ai4bharat/Rasa) |
| Base model | [SWivid/F5-TTS](https://huggingface.co/SWivid/F5-TTS) |
| Training checkpoint | None in this repository |
| Upstream | [SPRINGLab/SPRING_F5](https://huggingface.co/SPRINGLab/SPRING_F5) |
| Trained by | SPRINGLab (upstream) |
| Licence | Apache 2.0 |
QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The reference clip is LJ Speech LJ001-0009 (public domain). QA covers English only. The QA ran with the `unicode` text input. On its three sentences, that input and `f5tts_pinyin` give the same ids.
Text ids: phoonnx gives the model the characters of the text, marks included, as the F5-TTS text function does. On FLORES-200 rows of the card languages, the ids equal the F5-TTS ids on 176 of 185 rows. On 3 more rows, the ids of each sentence are equal: phoonnx runs the model once for each sentence. On the other 6 rows, the only difference is a space or a no-break space at the start or the end of the row. F5-TTS inference also removes it.
The voice config holds no pinyin syllable, so phoonnx gives a Chinese character id 0, where F5-TTS gives the id of its syllable. The card languages do not include Chinese.
| Check | fp32 | int8 |
| --- | --- | --- |
| Result | pass | pass |
| WER | 0.000 | 0.000 |
| Characters per second | 13.8 to 13.8 | - |
| Length, int8 / fp32 | - | 1.00 to 1.00 |
| Graph size | 1455.0 MB | 463.1 MB |
| Peak RSS | 3201.0 MB | 1193.8 MB |
| RTF (median of 3, 4 threads) | 31.127 | 19.886 |
Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.
Each `<graph>_int8.onnx` file is the int8 copy of `<graph>.onnx`, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.
## How to use
Install [phoonnx](https://github.com/TigreGotico/phoonnx):
```bash
pip install --pre phoonnx
```
Then run:
```python
import wave
from phoonnx.config import SynthesisConfig
from phoonnx.model_manager import TTSModelManager
manager = TTSModelManager()
manager.load()
manager.merge_default_voices()
voice = manager.voices["f5tts/spring"].load()
with wave.open("output.wav", "wb") as wav_file:
# The voice clones the speaker of a short clip. Give the clip and its transcription.
syn_config = SynthesisConfig(speaker_reference="reference.wav",
speaker_reference_text="What the speaker says in the clip.")
voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file, syn_config=syn_config)
```
In OVOS, phoonnx installs the `ovos-tts-plugin-phoonnx` plugin. Select this voice in
`mycroft.conf`:
```json
{
"tts": {
"module": "ovos-tts-plugin-phoonnx",
"ovos-tts-plugin-phoonnx": {
"voice": "f5tts/spring"
}
}
}
```
This voice is not for commercial use. See "Ownership and licensing".
### Prepare the text
Run these lines on the text before it goes to `voice.synthesize_wav`:
```python
# uv pip install --prerelease=allow text2spoken
from text2spoken import normalize
text = normalize(text, "en") # the tag of the language of the text
```
[text2spoken](https://github.com/TigreGotico/text2spoken) writes numbers, dates, times, money, units and abbreviations as the words that a speaker says. This voice does not name one language, so give text2spoken the tag of the language of the text.
## Training data
Trained on [ai4bharat/IndicVoices](https://huggingface.co/datasets/ai4bharat/IndicVoices), [ai4bharat/Rasa](https://huggingface.co/datasets/ai4bharat/Rasa).
## Related links
- Upstream model: [SPRINGLab/SPRING_F5](https://huggingface.co/SPRINGLab/SPRING_F5)
- phoonnx: https://github.com/TigreGotico/phoonnx
- TigreGotico: https://tigregotico.pt
## Ownership and licensing
SPRINGLab trained this model and publishes it at [SPRINGLab/SPRING_F5](https://huggingface.co/SPRINGLab/SPRING_F5) under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit SPRINGLab when you use this voice. SPRINGLab fine-tuned this model from SWivid/F5-TTS, which is CC BY-NC 4.0. Commercial use of this voice is therefore not allowed.