PsarAI-Speech-Flash
The small, fast text-to-speech part of PsarAI-Speech: Khmer, English and French, mixed Khmer + English / Khmer + French sentences included, in the Jarvis voice or the voice type of a reference clip. No ASR, no clone v3 (that is PsarAI-Speech); 281 MB instead of 3.2 GB.
numbers -> words -> characters -> speaker-conditioned T2U (CTC over 1,000 mHuBERT-147 units + unit 4-gram beam search) -> durations and pitch -> unit HiFi-GAN with speaker + pitch input, 24 kHz. T2U and vocoder weights are stored in float16 (computed in float32).
from psarai_speech import Speech
speech = Speech() # this folder
wav = speech.tts("αα½ααααΈ αααα Bonsoir, monsieur.") # numpy float32, 24 kHz, Jarvis
wav = speech.tts("Good evening, sir.", voice="reference.wav") # voice type of the reference (CAM++)
python psarai_speech.py tts --text "Good evening, sir." --out out.wav [--voice ref.wav]
Requirements: torch, safetensors, kenlm, numpy, soundfile, soxr (torchaudio for --voice).
Cloning here copies voice type, pitch and rhythm (male / female, high / low), not the person; for zero-shot voice cloning use PsarAI-Speech (clone v3). Same model as the fast engine of PsarAI-Speech and as PsarAI-Speech-Flash-GGUF for audio.cpp.
Check against the float32 package (12 sentences km / en / fr / mixed / numbers, unit 4-gram, no listen-back): ASR CER float32 6.5%, float16 6.2%; 7 / 12 sentences identical (correlation > 0.999).
Listen
Generated by the audio.cpp server from the GGUF in this family (the Python package gives the same output). Heard is what the PsarAI-Speech ASR transcribes from the audio, automatically (its own spelling slips included).
Jarvis voice (default)
| Text | Listen | Heard | |
|---|---|---|---|
| Khmer | αα½ααααΈ! αααααααα’αΆααΆαααΆαα»ααα’ααΆαα α αΎααααα»αααΈαααΆααααααΆααα½αα’ααααααααααα Hello! The weather is lovely today, and I am happy to see you again. |
αα½ααααΈ αααααααα’αΆααΆαααΆαα»ααα’ααΆαα α αΎααααα»αααΈαααΆααααααΆααα½αα’ααααααααααα | |
| English | Hello! This voice was cloned from a ten-second sample. The weather is lovely today. | hello this voice was closed from a ten second sample the weather is lovely today | |
| French | Bonjour ! Cette voix a été clonée à partir d'un court extrait. Il fait très beau aujourd'hui. Hello! This voice was cloned from a short sample. The weather is lovely today. |
bonjour cette voie était clonnée à partir d'un cours extrait il fait très beau aujourd'hui. | |
| Khmer + English | ααααααα αααα»αααΆα meeting ααΆαα½α team αα
ααααααα α αΎααααααΆαααααα
αααΆαααΆαα Today I have a meeting with the team at ten, and then I'll go eat. |
αααααααα α»αααΆαααΈααΆααααΆαα½αααΈα αα·αααααααα α αΎααααααΆαααα»ααα·ααααΆαααΆαα |
Voice type from a reference clip
Flash copies the voice type (gender, pitch, rhythm), not the person; for real cloning use PsarAI-Speech-GGUF.
| Reference voice | Reference clip | Khmer sentence in that voice | similarity |
|---|---|---|---|
| elderly woman, warm | 0.37 | ||
| middle-aged woman, deep | 0.61 | ||
| young man, deep and smooth | 0.56 |
Text: αα½ααααΈ! αααααααα’αΆααΆαααΆαα»ααα’ααΆαα α αΎααααα»αααΈαααΆααααααΆααα½αα’ααααααααααα
Hello! The weather is lovely today, and I am happy to see you again.
License
CC-BY-NC-4.0 (non-commercial). Speech units from utter-project/mHuBERT-147 (codebook only, no mHuBERT weights); speaker encoder CAM++ (FunASR / 3D-Speaker, weights from ResembleAI/chatterbox, MIT).
- Downloads last month
- 55