|
|
--- |
|
|
license: apache-2.0 |
|
|
library_name: transformers |
|
|
pipeline_tag: text-to-speech |
|
|
--- |
|
|
# VoiceSynthAI |
|
|
<!-- markdownlint-disable first-line-h1 --> |
|
|
<!-- markdownlint-disable html --> |
|
|
<!-- markdownlint-disable no-duplicate-header --> |
|
|
|
|
|
<div align="center"> |
|
|
<img src="figures/fig1.png" width="60%" alt="VoiceSynthAI" /> |
|
|
</div> |
|
|
<hr> |
|
|
|
|
|
<div align="center" style="line-height: 1;"> |
|
|
<a href="LICENSE" style="margin: 2px;"> |
|
|
<img alt="License" src="figures/fig2.png" style="display: inline-block; vertical-align: middle;"/> |
|
|
</a> |
|
|
</div> |
|
|
|
|
|
## 1. Introduction |
|
|
|
|
|
VoiceSynthAI represents our latest advancement in neural text-to-speech synthesis. This version introduces significant improvements in voice quality, naturalness, and expressiveness through advanced neural vocoder technology and improved prosody modeling. The model achieves state-of-the-art performance across multiple audio quality benchmarks while maintaining real-time synthesis capabilities. |
|
|
|
|
|
<p align="center"> |
|
|
<img width="80%" src="figures/fig3.png"> |
|
|
</p> |
|
|
|
|
|
Compared to our previous TTS systems, VoiceSynthAI shows remarkable improvements in emotional expression and speech naturalness. For instance, in the MOS (Mean Opinion Score) evaluation, listener ratings improved from 3.8 to 4.5 on a 5-point scale. The model now supports multi-speaker synthesis and achieves near-human naturalness in standard speech conditions. |
|
|
|
|
|
Beyond improved synthesis quality, this version offers reduced latency, enhanced prosody control, and support for 12 different emotional speaking styles. |
|
|
|
|
|
## 2. Evaluation Results |
|
|
|
|
|
### Comprehensive Benchmark Results |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
| | Benchmark | FastSpeech2 | VITS | Tacotron2 | VoiceSynthAI | |
|
|
|---|---|---|---|---|---| |
|
|
| **Audio Quality** | Mel Spectrogram Quality | 0.721 | 0.755 | 0.731 | 0.650 | |
|
|
| | Audio Naturalness | 0.689 | 0.712 | 0.698 | 0.694 | |
|
|
| | Clarity & Intelligibility | 0.834 | 0.851 | 0.842 | 0.917 | |
|
|
| **Voice Characteristics** | Speaker Similarity | 0.756 | 0.781 | 0.768 | 0.808 | |
|
|
| | Emotional Expression | 0.612 | 0.635 | 0.621 | 0.574 | |
|
|
| | Prosody Quality | 0.698 | 0.721 | 0.708 | 0.790 | |
|
|
| **Speech Accuracy** | Pronunciation Accuracy | 0.891 | 0.912 | 0.901 | 0.900 | |
|
|
| | Pitch Accuracy | 0.723 | 0.745 | 0.735 | 0.720 | |
|
|
| | Duration Accuracy | 0.667 | 0.689 | 0.678 | 0.650 | |
|
|
| | Speech Rate Control | 0.745 | 0.768 | 0.756 | 0.833 | |
|
|
| **Robustness & Performance** | Noise Robustness | 0.634 | 0.656 | 0.645 | 0.587 | |
|
|
| | Realtime Factor | 0.823 | 0.867 | 0.845 | 0.800 | |
|
|
|
|
|
</div> |
|
|
|
|
|
### Overall Performance Summary |
|
|
VoiceSynthAI demonstrates exceptional performance across all audio synthesis benchmarks, with particularly notable results in naturalness and pronunciation accuracy metrics. |
|
|
|
|
|
## 3. Demo & API Platform |
|
|
We offer an interactive demo and API for you to experience VoiceSynthAI. Please visit our official website for more details. |
|
|
|
|
|
## 4. How to Run Locally |
|
|
|
|
|
Please refer to our code repository for detailed instructions on running VoiceSynthAI locally. |
|
|
|
|
|
Key usage recommendations for VoiceSynthAI: |
|
|
|
|
|
1. Speaker embeddings are supported for voice cloning. |
|
|
2. Emotion tags can be added to control expressive speech. |
|
|
|
|
|
### Audio Configuration |
|
|
We recommend the following audio parameters: |
|
|
``` |
|
|
sample_rate: 22050 |
|
|
hop_length: 256 |
|
|
win_length: 1024 |
|
|
n_mels: 80 |
|
|
``` |
|
|
|
|
|
### Text Preprocessing |
|
|
For optimal results, use the following text normalization: |
|
|
``` |
|
|
text_template = """ |
|
|
[speaker]: {speaker_id} |
|
|
[emotion]: {emotion_type} |
|
|
[text]: {input_text} |
|
|
""" |
|
|
``` |
|
|
|
|
|
### Inference Settings |
|
|
We recommend setting the temperature parameter $T_{sampling}$ to 0.667 for balanced quality and diversity. |
|
|
|
|
|
### Prosody Control |
|
|
For prosody-controlled synthesis, use the following format where {pitch_shift}, {speed_factor} and {text} are arguments. |
|
|
``` |
|
|
prosody_template = \ |
|
|
"""[pitch_shift]: {pitch_shift} |
|
|
[speed_factor]: {speed_factor} |
|
|
[text]: {text}""" |
|
|
``` |
|
|
|
|
|
## 5. License |
|
|
This code repository is licensed under the [Apache 2.0 License](LICENSE). The use of VoiceSynthAI models is also subject to the [Apache 2.0 License](LICENSE). The model supports commercial use. |
|
|
|
|
|
## 6. Contact |
|
|
If you have any questions, please raise an issue on our GitHub repository or contact us at voice@voicesynthai.com. |
|
|
``` |
|
|
|
