BSC-LT/styletts2-catalan-multispeaker

StyleTTS2 Catalan

This model is a StyleTTS2 version trained from scratch in Catalan. The text aligner and the prosodic text encoder have also been specifically trained for Catalan.

Model Details

Model Description

StyleTTS2 is an end-to-end expressive text-to-speech (TTS) model that generates the most suitable style for the text without requiring reference speech. The model leverages style diffusion and adversarial training with large Speech Language Models (SLMs) to achieve human-level TTS synthesis. It uses an end-to-end (E2E) training process that jointly optimizes all components, along with direct waveform synthesis and adversarial training with large SLMs enabled by differentiable duration modeling. The speech style is modeled as a latent variable sampled through diffusion models, allowing diverse speech generation without reference audio.

• Developed by: AI Institute, BSC
• Language(s): Catalan
• License: gpl-3.0

Model Sources

• You can try our model in this HuggingFace space demo
• Original StyleTTS2 GitHub Repository
• Original StyleTTS2 Paper

Intended Uses and Limitations

This model is intended to serve as multispeaker text-to-speech system for the Catalan language. It has been trained using a Catalan phonemizer, therefore if the model is used for other languages it will not produce intelligible samples. The quality of the samples can vary depending on the quality and length of the audio prompt.

Training Details

Training Data

Festcat and Openslr69 open source datasets have been used as training and validation data. The data has been filtered to include only audios up to 10 seconds, which corresponds to a maximum length of 800 frames. The total duration of the dataset is 14.8 hours for training and 0.2 hours for validation. All samples were resampled to 24kHz.

Training Procedure

The model in this repository (styletts2-catalan-multispeaker) was trained from scratch. Other four models were fine-tuned for comparison and evaluation. The original pitch extractor was used for the five models. A catalan AuxiliaryASR was trained as text aligner and used for some models. All fine-tuned models used yl4579/StyleTTS2-LibriTTS as pre-trained model, except multispeaker_FT_kokoro that uses hexgrad/Kokoro-82M.

styletts2-catalan-multispeaker was trained from scratch. The first stage was executed for 83 epochs. The original text aligner was used and papercup-ai/multilingual-pl-bert was used as prosodic text Encoder. The second stage was executed for 70 epochs. The catalan text aligner was used and BSC-LT/PL-BERT-wp-ca was used as prosodic text Encoder. microsoft/wavlm-base-plus was used as SLM.

FT-libri-tts_asr-en_multilingual-pl-bert_wavlm-base-plus was fine-tuned for 34 epochs. The original text aligner was used and papercup-ai/multilingual-pl-bert was used as prosodic text Encoder. microsoft/wavlm-base-plus was used as SLM.

FT-libri-tts_asr-ca_pl-bert-subword-ca_wavlm-base-plus was fine-tuned for 34 epochs. The catalan text aligner was used and BSC-LT/PL-BERT-wp-ca was used as prosodic text Encoder. microsoft/wavlm-base-plus was used as SLM.

FT-libri-tts_asr-ca_pl-bert-subword-ca_hubert-base-ca was fine-tuned for 34 epochs. The catalan text aligner was used and BSC-LT/PL-BERT-wp-ca was used as prosodic text Encoder. BSC-LT/hubert-base-ca-2k was used as SLM.

FT-kokoro_asr-ca_pl-bert-subword-ca_wavlm-base-plus was fine-tuned for 47 epochs. The catalan text aligner was used and BSC-LT/PL-BERT-wp-ca was used as prosodic text Encoder. microsoft/wavlm-base-plus was used as SLM.

Training Hyperparameters

styletts2-catalan-multispeaker first stage hyperparameters can be found here and second stage hyperparameters here.

FT-libri-tts_asr-en_multilingual-pl-bert_wavlm-base-plus fine-tuning hyperparameters can be found here.

FT-libri-tts_asr-ca_pl-bert-subword-ca_wavlm-base-plus fine-tuning hyperparameters can be found here.

FT-libri-tts_asr-ca_pl-bert-subword-ca_hubert-base-ca fine-tuning hyperparameters can be found here.

FT-kokoro_asr-ca_pl-bert-subword-ca_wavlm-base-plus fine-tuning hyperparameters can be found here.

Evaluation

Testing Data

The test corpus is comprised of 350 text sentences from La creació d’Eva i altres contes by Josep Carner. The text was sourced from Project Gutenberg’s website. We used 4 speakers (2 males and 2 females) to generate the audio samples, arriving at a total evaluation corpus of 1400 audio samples.

Metrics

A mix of automatic and subjective metrics has been used to evaluate this model. For the automatic metrics we used VERSA library. UTMOS and DNSMOS Pro BVCCC where used to assess the audio quality and WER was used to assess the intelligibility of the models.

Results

A total of 5 models has been evaluated with 3 automatic metrics (UTMOS, DNSMOS and WER). The model in this repository (styletts2-catalan-multispeaker) has been compared to different finetunings of the original model with Catalan data:

Model	UTMOS	DNSMOS Pro BVCC	WER
styletts2-catalan-multispeaker	3.839	3.030	0.090
FT-libri-tts_asr-en_multilingual-pl-bert_wavlm-base-plus	3.858	3.059	0.116
FT-libri-tts_asr-ca_pl-bert-subword-ca_wavlm-base-plus	3.727	3.034	0.114
FT-libri-tts_asr-ca_pl-bert-subword-ca_hubert-base-ca	3.909	3.117	0.108
FT-kokoro_asr-ca_pl-bert-subword-ca_wavlm-base-plus	3.488	2.885	0.167

Additional Information

Citation

If this model contributes to your research, please cite the work:

@misc{giraldo2026styletts2-ca-multispeaker,
      title={Catalan Multispeaker pre-trained StyleTTS2.}, 
      author={Giraldo, Jose Omar; Zevallos, Rodolfo; Armentano Oller, Carme; Peiro Lilja, Alexandre; Costa, Federico, España-Bonet, Cristina},
      organization={AI Institute, Barcelona Supercomputing Center},
      url={https://huggingface.co/langtech-veu/styletts2-catalan-multispeaker},
      year={2026}
}

Author

The Speech team of the Barcelona Supercomputing Center.

Contact

For further information, please send an email to bsc-lt@bsc.es.

Copyright

Copyright(c) 2026 by AI Institute, Barcelona Supercomputing Center.

License

GPL-3.0

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA. The training of the model was possible thanks to the computing time provided by Barcelona Supercomputing Center through MareNostrum 5. We acknowledge EuroHPC Joint Undertaking for awarding us access to MareNostrum5 as BSC, Spain.

Disclaimer

Click to expand

The model published in this repository is intended for a generalist purpose and is available to third parties under a permissive Apache License, Version 2.0.

Be aware that the model may have biases and/or any other undesirable distortions.

When third parties deploy or provide systems and/or services to other parties using this model (or any system based on it) or become users of the model, they should note that it is their responsibility to mitigate the risks arising from its use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.

In no event shall the owner and creator of the model (Barcelona Supercomputing Center) be liable for any results arising from the use made by third parties.