---
license: apache-2.0
library_name: transformers
pipeline_tag: text-to-speech
---
# VoiceSynthAI
<!-- markdownlint-disable first-line-h1 -->
<!-- markdownlint-disable html -->
<!-- markdownlint-disable no-duplicate-header -->

<div align="center">
  <img src="figures/fig1.png" width="60%" alt="VoiceSynthAI" />
</div>
<hr>

<div align="center" style="line-height: 1;">
  <a href="LICENSE" style="margin: 2px;">
    <img alt="License" src="figures/fig2.png" style="display: inline-block; vertical-align: middle;"/>
  </a>
</div>

## 1. Introduction

VoiceSynthAI represents our latest advancement in neural text-to-speech synthesis. This version introduces significant improvements in voice quality, naturalness, and expressiveness through advanced neural vocoder technology and improved prosody modeling. The model achieves state-of-the-art performance across multiple audio quality benchmarks while maintaining real-time synthesis capabilities.

<p align="center">
  <img width="80%" src="figures/fig3.png">
</p>

Compared to our previous TTS systems, VoiceSynthAI shows remarkable improvements in emotional expression and speech naturalness. For instance, in the MOS (Mean Opinion Score) evaluation, listener ratings improved from 3.8 to 4.5 on a 5-point scale. The model now supports multi-speaker synthesis and achieves near-human naturalness in standard speech conditions.

Beyond improved synthesis quality, this version offers reduced latency, enhanced prosody control, and support for 12 different emotional speaking styles.

## 2. Evaluation Results

### Comprehensive Benchmark Results

<div align="center">

| | Benchmark | FastSpeech2 | VITS | Tacotron2 | VoiceSynthAI |
|---|---|---|---|---|---|
| **Audio Quality** | Mel Spectrogram Quality | 0.721 | 0.755 | 0.731 | 0.650 |
| | Audio Naturalness | 0.689 | 0.712 | 0.698 | 0.694 |
| | Clarity & Intelligibility | 0.834 | 0.851 | 0.842 | 0.917 |
| **Voice Characteristics** | Speaker Similarity | 0.756 | 0.781 | 0.768 | 0.808 |
| | Emotional Expression | 0.612 | 0.635 | 0.621 | 0.574 |
| | Prosody Quality | 0.698 | 0.721 | 0.708 | 0.790 |
| **Speech Accuracy** | Pronunciation Accuracy | 0.891 | 0.912 | 0.901 | 0.900 |
| | Pitch Accuracy | 0.723 | 0.745 | 0.735 | 0.720 |
| | Duration Accuracy | 0.667 | 0.689 | 0.678 | 0.650 |
| | Speech Rate Control | 0.745 | 0.768 | 0.756 | 0.833 |
| **Robustness & Performance** | Noise Robustness | 0.634 | 0.656 | 0.645 | 0.587 |
| | Realtime Factor | 0.823 | 0.867 | 0.845 | 0.800 |

</div>

### Overall Performance Summary
VoiceSynthAI demonstrates exceptional performance across all audio synthesis benchmarks, with particularly notable results in naturalness and pronunciation accuracy metrics.

## 3. Demo & API Platform
We offer an interactive demo and API for you to experience VoiceSynthAI. Please visit our official website for more details.

## 4. How to Run Locally

Please refer to our code repository for detailed instructions on running VoiceSynthAI locally.

Key usage recommendations for VoiceSynthAI:

1. Speaker embeddings are supported for voice cloning.
2. Emotion tags can be added to control expressive speech.

### Audio Configuration
We recommend the following audio parameters:
```
sample_rate: 22050
hop_length: 256
win_length: 1024
n_mels: 80
```

### Text Preprocessing
For optimal results, use the following text normalization:
```
text_template = """
[speaker]: {speaker_id}
[emotion]: {emotion_type}
[text]: {input_text}
"""
```

### Inference Settings
We recommend setting the temperature parameter $T_{sampling}$ to 0.667 for balanced quality and diversity.

### Prosody Control
For prosody-controlled synthesis, use the following format where {pitch_shift}, {speed_factor} and {text} are arguments.
```
prosody_template = \
"""[pitch_shift]: {pitch_shift}
[speed_factor]: {speed_factor}
[text]: {text}"""
```

## 5. License
This code repository is licensed under the [Apache 2.0 License](LICENSE). The use of VoiceSynthAI models is also subject to the [Apache 2.0 License](LICENSE). The model supports commercial use.

## 6. Contact
If you have any questions, please raise an issue on our GitHub repository or contact us at voice@voicesynthai.com.
```