|
Download README.md from flockspeech/goldfinch: direct link, hf CLI and curl.
- Browser
- Download file 12 kB
-
https://huggingface.co/flockspeech/goldfinch/resolve/main/README.md
- Command line
-
hf download hf://flockspeech/goldfinch/README.md
-
curl -L -o README.md https://huggingface.co/flockspeech/goldfinch/resolve/main/README.md
12 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| library_name: onnx | |
| pipeline_tag: text-to-speech | |
| tags: | |
| - text-to-speech | |
| - tts | |
| - speech-synthesis | |
| - neural-tts | |
| - offline-tts | |
| - on-device | |
| - edge-ai | |
| - edge | |
| - offline | |
| - cpu | |
| - onnx | |
| - onnxruntime | |
| - lightweight | |
| - small-model | |
| - vocos | |
| - english | |
| - raspberry-pi | |
| - flockspeech | |
| - conversational | |
| - conversational-ai | |
| - expressive-tts | |
| - nonverbal | |
| - laughter | |
| - disfluency | |
| - voice-assistant | |
| - game-audio | |
| datasets: | |
| - amaai-lab/DisfluencySpeech | |
| # Goldfinch v0.1: tiny conversational text-to-speech (TTS) that laughs on cue | |
| **A small, expressive English text-to-speech model that laughs, breathes and says "uh", offline on any CPU. 44 MB, no | |
| GPU, no cloud.** | |
| Goldfinch is an 11.4M-parameter text-to-speech model from [Flock Speech](https://huggingface.co/flockspeech). Where | |
| its sibling [Junco](https://huggingface.co/flockspeech/junco) reads aloud, Goldfinch talks: it keeps fillers such as | |
| "uh" and "you know", restarts like "I, I think", and non-speech sounds you place with tags such as `[laugh]`. It runs | |
| faster than real time on an ordinary CPU, with no GPU, network or PyTorch dependency. | |
| Goldfinch is the **conversational voice** of the Flock Speech family, for chat companions, game characters, | |
| conversational AI and voice assistants that should sound casual and human, on-device and offline. Install it with | |
| `pip install flockspeech`. | |
| | Flock Speech voices | Style | Model card | | |
| |---|---|---| | |
| | **Goldfinch** | conversational voice that laughs and breathes on cue with `[laugh]` tags | this page | | |
| | **Junco** | clear, neutral reading voice | [flockspeech/junco](https://huggingface.co/flockspeech/junco) | | |
| | | | | |
| |---|---| | |
| | **Developed by** | Flock Speech | | |
| | **Model type** | Neural text-to-speech: compact acoustic model + Vocos vocoder | | |
| | **Language** | English (US) | | |
| | **Voices** | 1 (conversational style) | | |
| | **Sound tags** | `[laugh]`, `[breath]`, `[sigh]`, `[lipsmack]`, `[throat]`, `[noise]` | | |
| | **Output** | 24 kHz mono audio | | |
| | **Parameters** | 11.4M acoustic model + 13.5M vocoder | | |
| | **Download** | 44 MB (ONNX; int8 decoder, fp16 encoder and vocoder). A 100 MB full-precision build is also included | | |
| | **Runtime** | ONNX Runtime on CPU (Linux, macOS, Windows; x86-64 and ARM64) | | |
| | **License** | Apache-2.0 (vocoder weights MIT) | | |
| | **Version** | v0.1, November 2026 | | |
| ## Samples | |
| All samples were made with the files in this repository at default settings (8 steps, seed 0), with no editing. | |
| | Use | Text | Audio | | |
| |---|---|---| | |
| | Laugh | *Oh my gosh, [laugh] I can't believe you actually did that.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/laugh.wav"></audio> | | |
| | Chat | *Yeah, you know, I used to work there, but, um, not anymore.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/chat.wav"></audio> | | |
| | Planning | *[breath] Okay, so here's the plan for tomorrow. We meet at nine, and, uh, we'll figure out the rest.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/plan.wav"></audio> | | |
| | Reaction | *That's hilarious. [laugh] No, really, it is.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/funny.wav"></audio> | | |
| | Plain | *The weather today is sunny, with a light breeze from the west.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/calm.wav"></audio> | | |
| ## Quick start | |
| The [`flockspeech`](https://pypi.org/project/flockspeech/) Python package (0.2.0 or later) bundles these weights, so | |
| nothing is downloaded at run time. | |
| ```sh | |
| pip install flockspeech | |
| flockspeech "Oh my gosh, [laugh] I can't believe you actually did that." --voice goldfinch -o laugh.wav --play | |
| ``` | |
| ```python | |
| from flockspeech import Goldfinch | |
| tts = Goldfinch() # loads in under a second | |
| audio = tts("Yeah, um, we usually eat dinner around six. [laugh] Late, I know.") # float32, 24 kHz mono | |
| tts.save("Well, you know, it depends.", "reply.wav") | |
| for chunk in tts.stream(long_text): # sentence by sentence, for low latency | |
| play(chunk) | |
| ``` | |
| To use the files in this repository directly: | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| from flockspeech import Goldfinch | |
| path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx/*"]) | |
| tts = Goldfinch(model_dir=f"{path}/onnx") | |
| # or the full-precision build (100 MB, about 40% slower on a CPU) | |
| path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx-fp32/*"]) | |
| tts = Goldfinch(model_dir=f"{path}/onnx-fp32") | |
| ``` | |
| ### Sound tags | |
| Write a tag in square brackets where the sound should go. Each tag is one symbol in the model's input, so the model | |
| decides how the sound is shaped and how long it lasts, in the context of the words around it. | |
| | Tag | Also accepted | Training examples | Reliability | | |
| |---|---|---:|---| | |
| | `[laugh]` | `[laughs]`, `[giggle]`, `[chuckle]` | 701 | good | | |
| | `[breath]` | `[inhale]`, `[exhale]` | 98 | good | | |
| | `[throat]` | `[clears throat]`, `[cough]` | 57 | fair | | |
| | `[lipsmack]` | `[lip smack]` | 56 | fair | | |
| | `[noise]` | | 19 | weak | | |
| | `[sigh]` | | 16 | weak | | |
| By default the engine gives each tag 1.4× its predicted length and doubles the short pauses on either side. In | |
| our listening comparison this made laughs sound smoother than the raw prediction. These settings are `tag_len` and `tag_pad` | |
| in `onnx/goldfinch.json`. | |
| Fillers need no tags: write them as words ("uh", "um", "you know", "I mean"). | |
| ### Options | |
| | Parameter | Default | Effect | | |
| |---|---|---| | |
| | `speed` | 1.0 | Speaking rate; 1.2 is 20% faster | | |
| | `steps` | 8 | Generation steps; fewer is faster | | |
| | `seed` | 0 | Fixes the random variation in delivery, so output is reproducible | | |
| | `threads` | up to 4 | CPU threads for ONNX Runtime | | |
| ## Intended use | |
| **Suitable for:** | |
| - Voice output that should sound casual and human: chat companions, game characters, conversational assistants, | |
| podcasts and prototypes of spoken dialogue. | |
| - Devices and apps that must work offline or keep text private. | |
| - Research on small expressive TTS models and nonverbal sound generation. | |
| **Out of scope:** | |
| - Languages other than English, or speaker voices other than the one provided. Goldfinch has no voice cloning. | |
| - Formal reading, such as news, audiobooks or announcements. Use Junco for those. | |
| - Emotion control beyond the tags listed above. | |
| - Impersonating a real person, or producing audio meant to deceive listeners about who is speaking or whether the | |
| speech is synthetic. | |
| ## Limitations | |
| - **Weak tags.** `[sigh]` and `[noise]` had fewer than 20 training examples each, and can come out faint or wrong. | |
| - **Variable delivery.** Goldfinch was trained on spontaneous-sounding speech, so pacing and emphasis vary more than | |
| in a read voice. Try another `seed` if one take sounds off. | |
| - **No text normalisation.** Write numbers, currency, dates and abbreviations in words. | |
| - **Pronunciation comes from espeak-ng.** Rare names and some homographs can be mispronounced. | |
| - **One voice, one accent.** The model learned from a single US English speaker and about 8 hours of audio. | |
| - **Sentence by sentence.** Prosody does not carry across sentences. | |
| ## Risks and responsible use | |
| Synthetic speech that laughs, breathes and hesitates sounds more human, which makes it easier to mistake for a real | |
| person. Goldfinch produces a single fixed voice and cannot clone speakers. We ask that you: | |
| - tell listeners when speech is synthetic; | |
| - not use the model for fraud, harassment, scams or deceptive impersonation; | |
| - review output before you use it in high-stakes settings. | |
| ## Model details | |
| ### How it works | |
| Text is converted to phonemes with espeak-ng, a compact neural acoustic model turns the phonemes into a mel | |
| spectrogram, and the [Vocos](https://huggingface.co/charactr/vocos-mel-24khz) vocoder (MIT) turns that into 24 kHz | |
| audio. All three run in ONNX Runtime on the CPU. | |
| ### Files | |
| | File | Contents | Size | | |
| |---|---|---:| | |
| | `onnx/encoder.onnx` | Acoustic model, part 1 (fp16 weights) | 7.2 MB | | |
| | `onnx/decoder-int8.onnx` | Acoustic model, part 2 (int8) | 9.9 MB | | |
| | `onnx/vocoder.onnx` | Vocos vocoder (fp16 weights) | 27.1 MB | | |
| | `onnx/goldfinch.json` | Mel statistics, tag ids and default inference settings | <1 kB | | |
| | `onnx-fp32/encoder.onnx` | Acoustic model, part 1, fp32 | 14.2 MB | | |
| | `onnx-fp32/decoder.onnx` | Acoustic model, part 2, fp32 | 31.5 MB | | |
| | `onnx-fp32/vocoder.onnx` | Vocos, fp32 | 54.2 MB | | |
| | `onnx-fp32/goldfinch.json` | As above, pointing at the fp32 files | <1 kB | | |
| | `samples/` | The audio samples above | | | |
| ### Training data | |
| [DisfluencySpeech](https://huggingface.co/datasets/amaai-lab/DisfluencySpeech) (AMAAI Lab, Apache-2.0): one studio | |
| speaker re-performing conversations from the Switchboard corpus transcripts, with fillers, restarts and non-speech | |
| events kept. | |
| We trained on 8.3 hours of it. | |
| ### Training | |
| | | | | |
| |---|---| | |
| | **Compute** | about 3.1 hours on one NVIDIA T4 GPU, from scratch | | |
| **Environmental impact.** About 3.1 GPU-hours on one NVIDIA T4 (70 W TDP), or about 0.2 kWh. | |
| ## FAQ | |
| **How do I make it laugh?** | |
| Write `[laugh]` where the laugh should go: `Goldfinch()("That's hilarious. [laugh] No, really.")`. The same works for | |
| `[breath]`, `[throat]`, `[lipsmack]`, `[sigh]` and `[noise]`; see [Sound tags](#sound-tags). | |
| **Do I need to tag "uh" and "um"?** | |
| No. Write fillers as ordinary words ("uh", "um", "you know", "I mean") and Goldfinch says them naturally. | |
| **Does it need a GPU or an internet connection?** | |
| No. It runs on an ordinary CPU with ONNX Runtime. Once installed, speech is generated fully offline and your text | |
| never leaves the machine. | |
| **How big is it?** | |
| 44.2 MB in total: a 17.1 MB acoustic model plus the 27.1 MB Vocos vocoder, shared with Junco. Both voices together | |
| take 61.3 MB. A full-precision (fp32) build is also in this repository. | |
| **Can it run on a Raspberry Pi or other ARM boards?** | |
| It runs on 64-bit ARM Linux wherever `onnxruntime` and `espeakng-loader` publish wheels, including 64-bit Raspberry | |
| Pi OS. | |
| **Does it support voice cloning or emotions?** | |
| No voice cloning: Goldfinch is one English (US) voice. Expressiveness comes from fillers and the sound tags above, | |
| not from emotion labels. | |
| **Can I use it commercially?** | |
| The weights are Apache-2.0 and the vocoder is MIT. The phonemiser (espeak-ng, phonemizer) is GPL-3.0 and is installed | |
| separately, and the training texts come from Switchboard transcripts; see the License section below. | |
| ## License | |
| - **Goldfinch weights and configuration:** [Apache License 2.0](LICENSE). | |
| - **Vocos vocoder weights** (`onnx/vocoder.onnx`): MIT, © 2023 Charactr Inc. See `onnx/LICENSE-vocos`. | |
| - **Training data:** DisfluencySpeech, released by its authors under Apache-2.0. Its texts come from Switchboard | |
| transcripts, and the Switchboard corpus itself is distributed by the Linguistic Data Consortium (LDC) under its own | |
| licence. The recordings and the released model are new works, but if your use is sensitive to data provenance, | |
| review these terms yourself. | |
| **Run-time dependency.** Text-to-phoneme conversion uses espeak-ng and phonemizer. Both are GPL-3.0 and are **not** | |
| included in this repository. See [NOTICE](NOTICE). | |
| ## Citation | |
| ```bibtex | |
| @misc{flockspeech2026goldfinch, | |
| title = {Goldfinch: a small conversational text-to-speech model with sound tags}, | |
| author = {{Flock Speech}}, | |
| year = {2026}, | |
| howpublished = {\url{https://huggingface.co/flockspeech/goldfinch}}, | |
| note = {Version 0.1} | |
| } | |
| ``` | |
| ## Links | |
| - Python package: [pypi.org/project/flockspeech](https://pypi.org/project/flockspeech/) | |
| - Source code: [github.com/flockspeech/flockspeech](https://github.com/flockspeech/flockspeech) | |
| - Reading voice: [flockspeech/junco](https://huggingface.co/flockspeech/junco) | |
| ## Acknowledgements | |
| Goldfinch is trained on DisfluencySpeech from the AMAAI Lab. We thank its authors and the authors of the Vocos vocoder. | |