PsarAI-Speech-Flash

The small, fast text-to-speech part of PsarAI-Speech: Khmer, English and French, mixed Khmer + English / Khmer + French sentences included, in the Jarvis voice or the voice type of a reference clip. No ASR, no clone v3 (that is PsarAI-Speech); 281 MB instead of 3.2 GB.

numbers -> words -> characters -> speaker-conditioned T2U (CTC over 1,000 mHuBERT-147 units + unit 4-gram beam search) -> durations and pitch -> unit HiFi-GAN with speaker + pitch input, 24 kHz. T2U and vocoder weights are stored in float16 (computed in float32).

from psarai_speech import Speech
speech = Speech()                                               # this folder
wav = speech.tts("αžŸαž½αžŸαŸ’αžαžΈ αž›αŸ„αž€αŸ” Bonsoir, monsieur.")             # numpy float32, 24 kHz, Jarvis
wav = speech.tts("Good evening, sir.", voice="reference.wav")   # voice type of the reference (CAM++)
python psarai_speech.py tts --text "Good evening, sir." --out out.wav [--voice ref.wav]

Requirements: torch, safetensors, kenlm, numpy, soundfile, soxr (torchaudio for --voice).

Cloning here copies voice type, pitch and rhythm (male / female, high / low), not the person; for zero-shot voice cloning use PsarAI-Speech (clone v3). Same model as the fast engine of PsarAI-Speech and as PsarAI-Speech-Flash-GGUF for audio.cpp.

Check against the float32 package (12 sentences km / en / fr / mixed / numbers, unit 4-gram, no listen-back): ASR CER float32 6.5%, float16 6.2%; 7 / 12 sentences identical (correlation > 0.999).

Listen

Generated by the audio.cpp server from the GGUF in this family (the Python package gives the same output). Heard is what the PsarAI-Speech ASR transcribes from the audio, automatically (its own spelling slips included).

Jarvis voice (default)

Text Listen Heard
Khmer αžŸαž½αžŸαŸ’αžαžΈ! αžαŸ’αž„αŸƒαž“αŸαŸ‡αž’αžΆαž€αžΆαžŸαž’αžΆαžαž»αž›αŸ’αž’αžŽαžΆαžŸαŸ‹ αž αžΎαž™αžαŸ’αž‰αž»αŸ†αžšαžΈαž€αžšαžΆαž™αžŠαŸ‚αž›αž”αžΆαž“αž‡αž½αž”αž’αŸ’αž“αž€αž˜αŸ’αžαž„αž‘αŸ€αžαŸ”
Hello! The weather is lovely today, and I am happy to see you again.
αžŸαž½αžŸαŸ’αžαžΈ αžαŸ’αž„αŸƒαž“αŸαŸ‡αž’αžΆαž€αžΆαžŸαž’αžΆαžαž»αž›αŸ’αž’αžŽαžΆαžŸαŸ‹ αž αžΎαž™αžαŸ’αž‰αž»αŸ†αžšαžΈαž€αžšαžΆαž™αžŠαŸ‚αž›αž”αžΆαž“αž‡αž½αž”αž’αŸ’αž“αž€αž˜αŸ’αžαž„αž‘αŸ€αžαŸ”
English Hello! This voice was cloned from a ten-second sample. The weather is lovely today. hello this voice was closed from a ten second sample the weather is lovely today
French Bonjour ! Cette voix a été clonée à partir d'un court extrait. Il fait très beau aujourd'hui.
Hello! This voice was cloned from a short sample. The weather is lovely today.
bonjour cette voie était clonnée à partir d'un cours extrait il fait très beau aujourd'hui.
Khmer + English αžαŸ’αž„αŸƒαž“αŸαŸ‡ αžαŸ’αž‰αž»αŸ†αž˜αžΆαž“ meeting αž‡αžΆαž˜αž½αž™ team αž“αŸ…αž˜αŸ‰αŸ„αž„αžŠαž”αŸ‹ αž αžΎαž™αž”αž“αŸ’αž‘αžΆαž”αŸ‹αž˜αž€αž‘αŸ…αž‰αŸ‰αžΆαŸ†αž”αžΆαž™αŸ”
Today I have a meeting with the team at ten, and then I'll go eat.
αžαŸ’αž„αŸƒαž“αŸαŸ‡αž…αž»αž„αž˜αžΆαž“αž˜αžΈαž‘αžΆαŸ†αž„αž‡αžΆαž˜αž½αž™αž‘αžΈαž˜ αž“αž·αž„αž˜αŸ‰αŸ„αž„αžŠαž”αŸ‹ αž αžΎαž™αž”αž“αŸ’αž‘αžΆαž”αŸ‹αž”αž»αž‚αž‘αž·αž‰αž‰αŸ‰αžΆαŸ†αž”αžΆαž™αŸ”

Voice type from a reference clip

Flash copies the voice type (gender, pitch, rhythm), not the person; for real cloning use PsarAI-Speech-GGUF.

Reference voice Reference clip Khmer sentence in that voice similarity
elderly woman, warm 0.37
middle-aged woman, deep 0.61
young man, deep and smooth 0.56

Text: αžŸαž½αžŸαŸ’αžαžΈ! αžαŸ’αž„αŸƒαž“αŸαŸ‡αž’αžΆαž€αžΆαžŸαž’αžΆαžαž»αž›αŸ’αž’αžŽαžΆαžŸαŸ‹ αž αžΎαž™αžαŸ’αž‰αž»αŸ†αžšαžΈαž€αžšαžΆαž™αžŠαŸ‚αž›αž”αžΆαž“αž‡αž½αž”αž’αŸ’αž“αž€αž˜αŸ’αžαž„αž‘αŸ€αžαŸ”
Hello! The weather is lovely today, and I am happy to see you again.

License

CC-BY-NC-4.0 (non-commercial). Speech units from utter-project/mHuBERT-147 (codebook only, no mHuBERT weights); speaker encoder CAM++ (FunASR / 3D-Speaker, weights from ResembleAI/chatterbox, MIT).

Downloads last month
55
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support