File size: 10,777 Bytes

---
license: apache-2.0
language:
  - en
tags:
  - text-to-speech
  - tts
  - voice-cloning
  - emotion-control
  - speech-synthesis
  - packed-model
  - pytorch
---

# PackedTTS

PackedTTS is a self-contained text-to-speech runtime bundle that packages the full synthesis stack into a single `tts.pt` file.

The bundle is designed to be loaded directly by the runtime script and used without rebuilding the model stack. It stores the model weights, tokenizer data, packed voices, packed emotions, resolution indexes, and runtime defaults in one artifact.

The example bundle in this repo is intended to be used as-is and currently includes a voice set and emotion set.

## What is included

`tts.pt` contains:

- T3 weights
- S3Gen weights
- VoiceEncoder weights
- tokenizer JSON
- packed voices
- packed emotions
- lookup indexes
- default resolution settings

This is not a training checkpoint meant to be unpacked and rebuilt from ingredients. It is the runtime artifact.

## Repository contents

- `tts.pt` — packed TTS bundle
- `PackedTTS.py` — runtime loader, resolver, and inference script
- `requirements.txt` — Python dependencies for the runtime
- `README.md` — usage and overview

## Features

- Single-file bundle loading
- Voice selection by name
- Emotion selection by name
- Fuzzy matching for names
- Default voice and emotion fallback
- Optional reference-audio overrides
- Packed voices and emotions inside one artifact
- CLI usage for quick testing
- Python API usage

## Requirements

You will need:

- Python 3.10+
- A working PyTorch environment
- the dependencies listed in `requirements.txt`

A GPU is recommended, but CPU mode is supported if your environment can handle the runtime cost.

## Quick start

### 1) Install dependencies

```bash
pip install -r requirements.txt
````

### 2) Download or place `tts.pt`

If you are using the Hugging Face repo, download the bundle and place it next to `PackedTTS.py`, or pass the path with `--bundle`.

### 3) List available voices and emotions

```bash
python PackedTTS.py --bundle tts.pt --list
```

### 4) Generate speech with an explicit voice and emotion

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "Hello world, this is a test." \
  --voice "Sarah" \
  --emotion "Angry" \
  --output output.wav
```

---

## Command-line examples

### List mode

Print all packed voices and emotions without generating audio:

```bash
python PackedTTS.py --bundle tts.pt --list
```

### Basic generation

Generate speech with explicit voice and emotion:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This is a normal synthesis test." \
  --voice "Sarah" \
  --emotion "Disgust" \
  --output output.wav
```

### Voice only

Let the bundle choose the emotion from the packed voice defaults or fallback rules:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This is voice-only generation." \
  --voice "Sarah" \
  --output output.wav
```

### Emotion only

Let the bundle choose a default or fallback voice while forcing an emotion:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This is emotion-only generation." \
  --emotion "Happy" \
  --output output.wav
```

### No voice and no emotion

Use the bundle defaults or random fallback selection:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This uses the bundle defaults." \
  --output output.wav
```

### Custom sampling parameters

Adjust guidance, temperature, and style strength:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "More expressive speech with custom sampling." \
  --voice "Sarah" \
  --emotion "Angry" \
  --cfg-weight 0.7 \
  --temperature 0.9 \
  --exaggeration 0.6 \
  --seed 123 \
  --output output.wav
```

### Different output path

Write the result anywhere you want:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "Saving to a custom file." \
  --voice "Sarah" \
  --emotion "Calm" \
  --output results/custom_output.wav
```

### Fuzzy matching for names

If the voice or emotion name is close but not exact, PackedTTS will try normalized matching and then fuzzy matching.

Example:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This uses fuzzy matching." \
  --voice "sara" \
  --emotion "angr" \
  --output output.wav
```

### Reference voice override

Use a reference voice file instead of a packed voice:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This voice comes from reference audio." \
  --voice-ref path/to/voice_reference.wav \
  --output output.wav
```

### Reference emotion override

Use a reference emotion file instead of a packed emotion:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "This emotion comes from reference audio." \
  --emo-ref path/to/emotion_reference.wav \
  --output output.wav
```

### Both reference overrides

Override both the voice and emotion with reference audio:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "Both voice and emotion are driven by reference audio." \
  --voice-ref path/to/voice_reference.wav \
  --emo-ref path/to/emotion_reference.wav \
  --output output.wav
```

### Seeded generation

Use a fixed seed to make results more repeatable:

```bash
python PackedTTS.py \
  --bundle tts.pt \
  --text "Seeded generation example." \
  --voice "Sarah" \
  --emotion "Disgust" \
  --seed 42 \
  --output output.wav
```

---

## Python usage

### Basic API usage

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="Hi, this is Sarah speaking with a disgust emotion.",
    voice="Sarah",
    emotion="Disgust",
    cfg_weight=0.5,
    temperature=0.8,
    exaggeration=0.5,
    seed=42,
)

sf.write("output.wav", audio, sr)
print(meta)
```

### Python usage with defaults

Let the model choose the default voice and emotion:

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="This uses the bundle defaults.",
    seed=7,
)

sf.write("default_output.wav", audio, sr)
print(meta)
```

### Python usage with voice only

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="Voice selected, emotion resolved by the bundle.",
    voice="Sarah",
    seed=12,
)

sf.write("voice_only.wav", audio, sr)
print(meta)
```

### Python usage with emotion only

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="Emotion selected, voice resolved by the bundle.",
    emotion="Happy",
    seed=12,
)

sf.write("emotion_only.wav", audio, sr)
print(meta)
```

### Python usage with reference voice override

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="This uses voice reference audio.",
    voice_ref="path/to/voice_reference.wav",
    emotion="Calm",
    seed=42,
)

sf.write("voice_ref_output.wav", audio, sr)
print(meta)
```

### Python usage with reference emotion override

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="This uses emotion reference audio.",
    voice="Sarah",
    emo_ref="path/to/emotion_reference.wav",
    seed=42,
)

sf.write("emo_ref_output.wav", audio, sr)
print(meta)
```

### Python usage with both reference overrides

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.generate(
    text="This uses both reference audio inputs.",
    voice_ref="path/to/voice_reference.wav",
    emo_ref="path/to/emotion_reference.wav",
    seed=42,
)

sf.write("both_refs_output.wav", audio, sr)
print(meta)
```

### Python usage through the `forward` alias

`forward` is an alias for `generate`, so the model can be used like a callable runtime component:

```python
from pathlib import Path
import soundfile as sf

from PackedTTS import PackedTTS

tts = PackedTTS.load(Path("tts.pt"))

sr, audio, meta = tts.forward(
    text="This uses the forward alias.",
    voice="Sarah",
    emotion="Disgust",
    cfg_weight=0.5,
    temperature=0.8,
    exaggeration=0.5,
    seed=42,
)

sf.write("forward_output.wav", audio, sr)
print(meta)
```

---

## How it works

PackedTTS restores the full runtime bundle and then runs synthesis in three stages:

1. **Resolve voice and emotion**

   * The bundle stores named voices and emotions.
   * A voice can be selected by name or replaced with reference audio.
   * An emotion can be selected by name or replaced with reference audio.
   * If exact matching fails, the runtime tries normalized matching and then fuzzy matching.

2. **Build conditionals**

   * The runtime loads the packed speaker embedding.
   * It loads the packed prompt tokens if available.
   * It loads the emotion conditioning vector.
   * It uses any packed generation reference state stored in the voice entry.

3. **Generate audio**

   * `T3` generates speech tokens from text.
   * `S3Gen` converts those tokens into waveform audio.

The result is a single packed synthesis workflow that does not require rebuilding the voice/emotion registry at runtime.

---

## Expected file behavior

The script expects the bundle to contain:

* `models.t3_state`
* `models.s3gen_state`
* `models.ve_state`
* `models.tokenizer_json`
* `voices`
* `emotions`
* `defaults`
* `indexes`

If a voice or emotion is not found by exact name, PackedTTS will try normalized matching and then fuzzy matching.

---

## Example command-line options

* `--bundle` — path to `tts.pt`
* `--text` — text to synthesize
* `--voice` — packed voice name
* `--emotion` — packed emotion name
* `--voice-ref` — override voice with reference audio
* `--emo-ref` — override emotion with reference audio
* `--cfg-weight` — classifier-free guidance weight
* `--temperature` — sampling temperature
* `--exaggeration` — emotion strength / style strength
* `--seed` — random seed
* `--output` — output WAV path
* `--list` — print packed voices and emotions

---

## Notes

* This repo is meant for inference and testing.
* The bundle is treated as a trusted artifact.
* If the underlying model architecture, tokenizer, or conditioning schema changes, rebuild `tts.pt`.
* Voice and emotion names depend on the bundle version.

---

## Credits

Built on top of:

* T3
* S3Gen
* VoiceEncoder