| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - text-to-speech |
| - tts |
| - voice-cloning |
| - emotion-control |
| - speech-synthesis |
| - packed-model |
| - pytorch |
| --- |
| |
| # PackedTTS |
|
|
| PackedTTS is a self-contained text-to-speech runtime bundle that packages the full synthesis stack into a single `tts.pt` file. |
|
|
| The bundle is designed to be loaded directly by the runtime script and used without rebuilding the model stack. It stores the model weights, tokenizer data, packed voices, packed emotions, resolution indexes, and runtime defaults in one artifact. |
|
|
| The example bundle in this repo is intended to be used as-is and currently includes a voice set and emotion set. |
|
|
| ## What is included |
|
|
| `tts.pt` contains: |
|
|
| - T3 weights |
| - S3Gen weights |
| - VoiceEncoder weights |
| - tokenizer JSON |
| - packed voices |
| - packed emotions |
| - lookup indexes |
| - default resolution settings |
|
|
| This is not a training checkpoint meant to be unpacked and rebuilt from ingredients. It is the runtime artifact. |
|
|
| ## Repository contents |
|
|
| - `tts.pt` — packed TTS bundle |
| - `PackedTTS.py` — runtime loader, resolver, and inference script |
| - `requirements.txt` — Python dependencies for the runtime |
| - `README.md` — usage and overview |
|
|
| ## Features |
|
|
| - Single-file bundle loading |
| - Voice selection by name |
| - Emotion selection by name |
| - Fuzzy matching for names |
| - Default voice and emotion fallback |
| - Optional reference-audio overrides |
| - Packed voices and emotions inside one artifact |
| - CLI usage for quick testing |
| - Python API usage |
|
|
| ## Requirements |
|
|
| You will need: |
|
|
| - Python 3.10+ |
| - A working PyTorch environment |
| - the dependencies listed in `requirements.txt` |
|
|
| A GPU is recommended, but CPU mode is supported if your environment can handle the runtime cost. |
|
|
| ## Quick start |
|
|
| ### 1) Install dependencies |
|
|
| ```bash |
| pip install -r requirements.txt |
| ```` |
|
|
| ### 2) Download or place `tts.pt` |
|
|
| If you are using the Hugging Face repo, download the bundle and place it next to `PackedTTS.py`, or pass the path with `--bundle`. |
|
|
| ### 3) List available voices and emotions |
|
|
| ```bash |
| python PackedTTS.py --bundle tts.pt --list |
| ``` |
|
|
| ### 4) Generate speech with an explicit voice and emotion |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "Hello world, this is a test." \ |
| --voice "Sarah" \ |
| --emotion "Angry" \ |
| --output output.wav |
| ``` |
|
|
| --- |
|
|
| ## Command-line examples |
|
|
| ### List mode |
|
|
| Print all packed voices and emotions without generating audio: |
|
|
| ```bash |
| python PackedTTS.py --bundle tts.pt --list |
| ``` |
|
|
| ### Basic generation |
|
|
| Generate speech with explicit voice and emotion: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This is a normal synthesis test." \ |
| --voice "Sarah" \ |
| --emotion "Disgust" \ |
| --output output.wav |
| ``` |
|
|
| ### Voice only |
|
|
| Let the bundle choose the emotion from the packed voice defaults or fallback rules: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This is voice-only generation." \ |
| --voice "Sarah" \ |
| --output output.wav |
| ``` |
|
|
| ### Emotion only |
|
|
| Let the bundle choose a default or fallback voice while forcing an emotion: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This is emotion-only generation." \ |
| --emotion "Happy" \ |
| --output output.wav |
| ``` |
|
|
| ### No voice and no emotion |
|
|
| Use the bundle defaults or random fallback selection: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This uses the bundle defaults." \ |
| --output output.wav |
| ``` |
|
|
| ### Custom sampling parameters |
|
|
| Adjust guidance, temperature, and style strength: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "More expressive speech with custom sampling." \ |
| --voice "Sarah" \ |
| --emotion "Angry" \ |
| --cfg-weight 0.7 \ |
| --temperature 0.9 \ |
| --exaggeration 0.6 \ |
| --seed 123 \ |
| --output output.wav |
| ``` |
|
|
| ### Different output path |
|
|
| Write the result anywhere you want: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "Saving to a custom file." \ |
| --voice "Sarah" \ |
| --emotion "Calm" \ |
| --output results/custom_output.wav |
| ``` |
|
|
| ### Fuzzy matching for names |
|
|
| If the voice or emotion name is close but not exact, PackedTTS will try normalized matching and then fuzzy matching. |
|
|
| Example: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This uses fuzzy matching." \ |
| --voice "sara" \ |
| --emotion "angr" \ |
| --output output.wav |
| ``` |
|
|
| ### Reference voice override |
|
|
| Use a reference voice file instead of a packed voice: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This voice comes from reference audio." \ |
| --voice-ref path/to/voice_reference.wav \ |
| --output output.wav |
| ``` |
|
|
| ### Reference emotion override |
|
|
| Use a reference emotion file instead of a packed emotion: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "This emotion comes from reference audio." \ |
| --emo-ref path/to/emotion_reference.wav \ |
| --output output.wav |
| ``` |
|
|
| ### Both reference overrides |
|
|
| Override both the voice and emotion with reference audio: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "Both voice and emotion are driven by reference audio." \ |
| --voice-ref path/to/voice_reference.wav \ |
| --emo-ref path/to/emotion_reference.wav \ |
| --output output.wav |
| ``` |
|
|
| ### Seeded generation |
|
|
| Use a fixed seed to make results more repeatable: |
|
|
| ```bash |
| python PackedTTS.py \ |
| --bundle tts.pt \ |
| --text "Seeded generation example." \ |
| --voice "Sarah" \ |
| --emotion "Disgust" \ |
| --seed 42 \ |
| --output output.wav |
| ``` |
|
|
| --- |
|
|
| ## Python usage |
|
|
| ### Basic API usage |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="Hi, this is Sarah speaking with a disgust emotion.", |
| voice="Sarah", |
| emotion="Disgust", |
| cfg_weight=0.5, |
| temperature=0.8, |
| exaggeration=0.5, |
| seed=42, |
| ) |
| |
| sf.write("output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with defaults |
|
|
| Let the model choose the default voice and emotion: |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="This uses the bundle defaults.", |
| seed=7, |
| ) |
| |
| sf.write("default_output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with voice only |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="Voice selected, emotion resolved by the bundle.", |
| voice="Sarah", |
| seed=12, |
| ) |
| |
| sf.write("voice_only.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with emotion only |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="Emotion selected, voice resolved by the bundle.", |
| emotion="Happy", |
| seed=12, |
| ) |
| |
| sf.write("emotion_only.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with reference voice override |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="This uses voice reference audio.", |
| voice_ref="path/to/voice_reference.wav", |
| emotion="Calm", |
| seed=42, |
| ) |
| |
| sf.write("voice_ref_output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with reference emotion override |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="This uses emotion reference audio.", |
| voice="Sarah", |
| emo_ref="path/to/emotion_reference.wav", |
| seed=42, |
| ) |
| |
| sf.write("emo_ref_output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage with both reference overrides |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.generate( |
| text="This uses both reference audio inputs.", |
| voice_ref="path/to/voice_reference.wav", |
| emo_ref="path/to/emotion_reference.wav", |
| seed=42, |
| ) |
| |
| sf.write("both_refs_output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| ### Python usage through the `forward` alias |
|
|
| `forward` is an alias for `generate`, so the model can be used like a callable runtime component: |
|
|
| ```python |
| from pathlib import Path |
| import soundfile as sf |
| |
| from PackedTTS import PackedTTS |
| |
| tts = PackedTTS.load(Path("tts.pt")) |
| |
| sr, audio, meta = tts.forward( |
| text="This uses the forward alias.", |
| voice="Sarah", |
| emotion="Disgust", |
| cfg_weight=0.5, |
| temperature=0.8, |
| exaggeration=0.5, |
| seed=42, |
| ) |
| |
| sf.write("forward_output.wav", audio, sr) |
| print(meta) |
| ``` |
|
|
| --- |
|
|
| ## How it works |
|
|
| PackedTTS restores the full runtime bundle and then runs synthesis in three stages: |
|
|
| 1. **Resolve voice and emotion** |
|
|
| * The bundle stores named voices and emotions. |
| * A voice can be selected by name or replaced with reference audio. |
| * An emotion can be selected by name or replaced with reference audio. |
| * If exact matching fails, the runtime tries normalized matching and then fuzzy matching. |
|
|
| 2. **Build conditionals** |
|
|
| * The runtime loads the packed speaker embedding. |
| * It loads the packed prompt tokens if available. |
| * It loads the emotion conditioning vector. |
| * It uses any packed generation reference state stored in the voice entry. |
|
|
| 3. **Generate audio** |
|
|
| * `T3` generates speech tokens from text. |
| * `S3Gen` converts those tokens into waveform audio. |
|
|
| The result is a single packed synthesis workflow that does not require rebuilding the voice/emotion registry at runtime. |
|
|
| --- |
|
|
| ## Expected file behavior |
|
|
| The script expects the bundle to contain: |
|
|
| * `models.t3_state` |
| * `models.s3gen_state` |
| * `models.ve_state` |
| * `models.tokenizer_json` |
| * `voices` |
| * `emotions` |
| * `defaults` |
| * `indexes` |
|
|
| If a voice or emotion is not found by exact name, PackedTTS will try normalized matching and then fuzzy matching. |
|
|
| --- |
|
|
| ## Example command-line options |
|
|
| * `--bundle` — path to `tts.pt` |
| * `--text` — text to synthesize |
| * `--voice` — packed voice name |
| * `--emotion` — packed emotion name |
| * `--voice-ref` — override voice with reference audio |
| * `--emo-ref` — override emotion with reference audio |
| * `--cfg-weight` — classifier-free guidance weight |
| * `--temperature` — sampling temperature |
| * `--exaggeration` — emotion strength / style strength |
| * `--seed` — random seed |
| * `--output` — output WAV path |
| * `--list` — print packed voices and emotions |
|
|
| --- |
|
|
| ## Notes |
|
|
| * This repo is meant for inference and testing. |
| * The bundle is treated as a trusted artifact. |
| * If the underlying model architecture, tokenizer, or conditioning schema changes, rebuild `tts.pt`. |
| * Voice and emotion names depend on the bundle version. |
|
|
| --- |
|
|
| ## Credits |
|
|
| Built on top of: |
|
|
| * T3 |
| * S3Gen |
| * VoiceEncoder |
|
|
