README.md

---
license: gpl-3.0
language:
- he
- en
pipeline_tag: text-to-speech
base_model:
- herimor/voxtream2
base_model_relation: finetune
tags:
- text-to-speech
- tts
- streaming
- hebrew
- voice-cloning
- zero-shot
- voxtream
library_name: voxtream2-he
---

# voxtream2-he

**Streaming Hebrew text-to-speech.** A Hebrew (and mixed Hebrew/English) finetune of
[`herimor/voxtream2`](https://huggingface.co/herimor/voxtream2), packaged as a clean,
installable Python inference library.

- **Base model:** [`herimor/voxtream2`](https://huggingface.co/herimor/voxtream2)
- **Code / GitHub repo:** [`flowtyone/voxtream2-he`](https://github.com/flowtyone/voxtream2-he)

This repository hosts the **model weights**. The code that loads them lives in the GitHub
repo above; weights are fetched automatically the first time you construct `SpeechGenerator()`.

> Because the model is small and fully streaming, outputs are not always stable. Commercial
> use is allowed but not recommended.

## What it does

Voxtream2 is a **zero-shot, full-stream** text-to-speech model: you give it a few seconds of
any voice plus some text, and it speaks that text in that voice — no per-voice training needed.

- **Built for streaming.** It produces audio in small chunks (one ~80 ms frame at a time)
  *while* it reads the text. The first chunk comes back in tens of milliseconds, so it suits
  live, interactive, real-time use (voice agents, dubbing, assistants).
- **Faster than real-time.** On a consumer GPU it generates audio several times faster than
  it takes to play.
- **Voice cloning from a short sample.** Point it at a 3–10 second clip of a target voice and
  it mimics that speaker — accent, tone, and timbre — without any fine-tuning.
- **Adjustable speaking rate.** You can ask for faster or slower speech, even mid-sentence.

This is the **Hebrew** finetune. It speaks Hebrew (with English mixed in) and keeps all of the
streaming behavior above. The generator consumes **phonemes** (a Hebrew G2P front-end with
`espeak-ng` as the English fallback) and emits 24 kHz mono audio frames.

## Usage

Install the library (see the [GitHub repo](https://github.com/flowtyone/voxtream2-he) for full
instructions, including the `espeak-ng` system dependency):

```bash
uv pip install "git+https://github.com/flowtyone/voxtream2-he.git"
```

Weights download from this repo automatically on first use:

```python
import numpy as np
import soundfile as sf
from voxtream2_he import SpeechGenerator

gen = SpeechGenerator()                          # weights auto-download from this repo
phonemizer = gen.make_phonemizer()

prompt = gen.prepare_prompt_from_path("voices/shaul.wav")
phonemes = phonemizer.phonemize("שלום, מה שלומך היום?", language="he")

frames = [
    frame
    for frame, _ in gen.generate_stream(
        text=phonemes,
        prepared_prompt=prompt,
        language="he",
        normalize_text=False,
        input_is_phonemes=True,
    )
]
sf.write("out.wav", np.concatenate(frames), gen.config.mimi_sr)  # 24 kHz mono
```

Pre-download ahead of time (e.g. to bake into an image or warm the cache):

```bash
hf download flowty1/voxtream2-he
```

## Files in this repo

| File                       | What it is                                                |
| -------------------------- | --------------------------------------------------------- |
| `model.safetensors`        | Full model weights (phone/temp/dep formers + heads)       |
| `config.json`              | Model architecture configuration                          |
| `phoneme_to_token.json`    | Phoneme → token vocabulary used by the generator          |
| `renikud/model.onnx`       | Hebrew grapheme-to-phoneme (G2P) front-end (ReNikud, ONNX) |

The Mimi codec (`kyutai/moshiko-pytorch-bf16`), the ReDimNet speaker encoder
(`IDRnD/ReDimNet`, via `torch.hub`), and the optional Sidon prompt enhancer
(`sarulab-speech/sidon-v0.1`) are pulled from their own upstream repos, not from here.

## Acknowledgements

- **Voxtream2** — the upstream full-stream TTS model. [herimor/voxtream](https://github.com/herimor/voxtream)
- **ReNikud** — audio-supervised Hebrew grapheme-to-phoneme (G2P) model used as the Hebrew front-end. [renikud/renikud](https://github.com/renikud/renikud)
- **Mimi** audio codec, from Kyutai's [Moshi](https://github.com/kyutai-labs/moshi).
- **ReDimNet** speaker encoder ([IDRnD/ReDimNet](https://github.com/IDRnD/ReDimNet)).
- **Sidon** speech enhancement ([sarulab-speech/Sidon](https://github.com/sarulab-speech/Sidon)), optional prompt enhancement.

## License

GPL-3.0. Bundled third-party components (Moshi/Mimi, ReDimNet, Sidon) retain their respective
upstream licenses.