File size: 12,010 Bytes
61c0446
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
---
license: apache-2.0
language:
  - en
library_name: onnx
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - tts
  - speech-synthesis
  - neural-tts
  - offline-tts
  - on-device
  - edge-ai
  - edge
  - offline
  - cpu
  - onnx
  - onnxruntime
  - lightweight
  - small-model
  - vocos
  - english
  - raspberry-pi
  - flockspeech
  - conversational
  - conversational-ai
  - expressive-tts
  - nonverbal
  - laughter
  - disfluency
  - voice-assistant
  - game-audio
datasets:
  - amaai-lab/DisfluencySpeech
---

# Goldfinch v0.1: tiny conversational text-to-speech (TTS) that laughs on cue

**A small, expressive English text-to-speech model that laughs, breathes and says "uh", offline on any CPU. 44 MB, no
GPU, no cloud.**

Goldfinch is an 11.4M-parameter text-to-speech model from [Flock Speech](https://huggingface.co/flockspeech). Where
its sibling [Junco](https://huggingface.co/flockspeech/junco) reads aloud, Goldfinch talks: it keeps fillers such as
"uh" and "you know", restarts like "I, I think", and non-speech sounds you place with tags such as `[laugh]`. It runs
faster than real time on an ordinary CPU, with no GPU, network or PyTorch dependency.

Goldfinch is the **conversational voice** of the Flock Speech family, for chat companions, game characters,
conversational AI and voice assistants that should sound casual and human, on-device and offline. Install it with
`pip install flockspeech`.

| Flock Speech voices | Style | Model card |
|---|---|---|
| **Goldfinch** | conversational voice that laughs and breathes on cue with `[laugh]` tags | this page |
| **Junco** | clear, neutral reading voice | [flockspeech/junco](https://huggingface.co/flockspeech/junco) |

| | |
|---|---|
| **Developed by** | Flock Speech |
| **Model type** | Neural text-to-speech: compact acoustic model + Vocos vocoder |
| **Language** | English (US) |
| **Voices** | 1 (conversational style) |
| **Sound tags** | `[laugh]`, `[breath]`, `[sigh]`, `[lipsmack]`, `[throat]`, `[noise]` |
| **Output** | 24 kHz mono audio |
| **Parameters** | 11.4M acoustic model + 13.5M vocoder |
| **Download** | 44 MB (ONNX; int8 decoder, fp16 encoder and vocoder). A 100 MB full-precision build is also included |
| **Runtime** | ONNX Runtime on CPU (Linux, macOS, Windows; x86-64 and ARM64) |
| **License** | Apache-2.0 (vocoder weights MIT) |
| **Version** | v0.1, November 2026 |

## Samples

All samples were made with the files in this repository at default settings (8 steps, seed 0), with no editing.

| Use | Text | Audio |
|---|---|---|
| Laugh | *Oh my gosh, [laugh] I can't believe you actually did that.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/laugh.wav"></audio> |
| Chat | *Yeah, you know, I used to work there, but, um, not anymore.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/chat.wav"></audio> |
| Planning | *[breath] Okay, so here's the plan for tomorrow. We meet at nine, and, uh, we'll figure out the rest.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/plan.wav"></audio> |
| Reaction | *That's hilarious. [laugh] No, really, it is.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/funny.wav"></audio> |
| Plain | *The weather today is sunny, with a light breeze from the west.* | <audio controls src="https://huggingface.co/flockspeech/goldfinch/resolve/main/samples/calm.wav"></audio> |

## Quick start

The [`flockspeech`](https://pypi.org/project/flockspeech/) Python package (0.2.0 or later) bundles these weights, so
nothing is downloaded at run time.

```sh
pip install flockspeech
flockspeech "Oh my gosh, [laugh] I can't believe you actually did that." --voice goldfinch -o laugh.wav --play
```

```python
from flockspeech import Goldfinch

tts = Goldfinch()                    # loads in under a second
audio = tts("Yeah, um, we usually eat dinner around six. [laugh] Late, I know.")   # float32, 24 kHz mono
tts.save("Well, you know, it depends.", "reply.wav")

for chunk in tts.stream(long_text):  # sentence by sentence, for low latency
    play(chunk)
```

To use the files in this repository directly:

```python
from huggingface_hub import snapshot_download
from flockspeech import Goldfinch

path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx/*"])
tts = Goldfinch(model_dir=f"{path}/onnx")

# or the full-precision build (100 MB, about 40% slower on a CPU)
path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx-fp32/*"])
tts = Goldfinch(model_dir=f"{path}/onnx-fp32")
```

### Sound tags

Write a tag in square brackets where the sound should go. Each tag is one symbol in the model's input, so the model
decides how the sound is shaped and how long it lasts, in the context of the words around it.

| Tag | Also accepted | Training examples | Reliability |
|---|---|---:|---|
| `[laugh]` | `[laughs]`, `[giggle]`, `[chuckle]` | 701 | good |
| `[breath]` | `[inhale]`, `[exhale]` | 98 | good |
| `[throat]` | `[clears throat]`, `[cough]` | 57 | fair |
| `[lipsmack]` | `[lip smack]` | 56 | fair |
| `[noise]` | | 19 | weak |
| `[sigh]` | | 16 | weak |

By default the engine gives each tag 1.4× its predicted length and doubles the short pauses on either side. In
our listening comparison this made laughs sound smoother than the raw prediction. These settings are `tag_len` and `tag_pad`
in `onnx/goldfinch.json`.

Fillers need no tags: write them as words ("uh", "um", "you know", "I mean").

### Options

| Parameter | Default | Effect |
|---|---|---|
| `speed` | 1.0 | Speaking rate; 1.2 is 20% faster |
| `steps` | 8 | Generation steps; fewer is faster |
| `seed` | 0 | Fixes the random variation in delivery, so output is reproducible |
| `threads` | up to 4 | CPU threads for ONNX Runtime |

## Intended use

**Suitable for:**
- Voice output that should sound casual and human: chat companions, game characters, conversational assistants,
  podcasts and prototypes of spoken dialogue.
- Devices and apps that must work offline or keep text private.
- Research on small expressive TTS models and nonverbal sound generation.

**Out of scope:**
- Languages other than English, or speaker voices other than the one provided. Goldfinch has no voice cloning.
- Formal reading, such as news, audiobooks or announcements. Use Junco for those.
- Emotion control beyond the tags listed above.
- Impersonating a real person, or producing audio meant to deceive listeners about who is speaking or whether the
  speech is synthetic.

## Limitations

- **Weak tags.** `[sigh]` and `[noise]` had fewer than 20 training examples each, and can come out faint or wrong.
- **Variable delivery.** Goldfinch was trained on spontaneous-sounding speech, so pacing and emphasis vary more than
  in a read voice. Try another `seed` if one take sounds off.
- **No text normalisation.** Write numbers, currency, dates and abbreviations in words.
- **Pronunciation comes from espeak-ng.** Rare names and some homographs can be mispronounced.
- **One voice, one accent.** The model learned from a single US English speaker and about 8 hours of audio.
- **Sentence by sentence.** Prosody does not carry across sentences.

## Risks and responsible use

Synthetic speech that laughs, breathes and hesitates sounds more human, which makes it easier to mistake for a real
person. Goldfinch produces a single fixed voice and cannot clone speakers. We ask that you:
- tell listeners when speech is synthetic;
- not use the model for fraud, harassment, scams or deceptive impersonation;
- review output before you use it in high-stakes settings.

## Model details

### How it works

Text is converted to phonemes with espeak-ng, a compact neural acoustic model turns the phonemes into a mel
spectrogram, and the [Vocos](https://huggingface.co/charactr/vocos-mel-24khz) vocoder (MIT) turns that into 24 kHz
audio. All three run in ONNX Runtime on the CPU.

### Files

| File | Contents | Size |
|---|---|---:|
| `onnx/encoder.onnx` | Acoustic model, part 1 (fp16 weights) | 7.2 MB |
| `onnx/decoder-int8.onnx` | Acoustic model, part 2 (int8) | 9.9 MB |
| `onnx/vocoder.onnx` | Vocos vocoder (fp16 weights) | 27.1 MB |
| `onnx/goldfinch.json` | Mel statistics, tag ids and default inference settings | <1 kB |
| `onnx-fp32/encoder.onnx` | Acoustic model, part 1, fp32 | 14.2 MB |
| `onnx-fp32/decoder.onnx` | Acoustic model, part 2, fp32 | 31.5 MB |
| `onnx-fp32/vocoder.onnx` | Vocos, fp32 | 54.2 MB |
| `onnx-fp32/goldfinch.json` | As above, pointing at the fp32 files | <1 kB |
| `samples/` | The audio samples above | |

### Training data

[DisfluencySpeech](https://huggingface.co/datasets/amaai-lab/DisfluencySpeech) (AMAAI Lab, Apache-2.0): one studio
speaker re-performing conversations from the Switchboard corpus transcripts, with fillers, restarts and non-speech
events kept.

We trained on 8.3 hours of it.

### Training

| | |
|---|---|
| **Compute** | about 3.1 hours on one NVIDIA T4 GPU, from scratch |

**Environmental impact.** About 3.1 GPU-hours on one NVIDIA T4 (70 W TDP), or about 0.2 kWh.

## FAQ

**How do I make it laugh?**
Write `[laugh]` where the laugh should go: `Goldfinch()("That's hilarious. [laugh] No, really.")`. The same works for
`[breath]`, `[throat]`, `[lipsmack]`, `[sigh]` and `[noise]`; see [Sound tags](#sound-tags).

**Do I need to tag "uh" and "um"?**
No. Write fillers as ordinary words ("uh", "um", "you know", "I mean") and Goldfinch says them naturally.

**Does it need a GPU or an internet connection?**
No. It runs on an ordinary CPU with ONNX Runtime. Once installed, speech is generated fully offline and your text
never leaves the machine.

**How big is it?**
44.2 MB in total: a 17.1 MB acoustic model plus the 27.1 MB Vocos vocoder, shared with Junco. Both voices together
take 61.3 MB. A full-precision (fp32) build is also in this repository.

**Can it run on a Raspberry Pi or other ARM boards?**
It runs on 64-bit ARM Linux wherever `onnxruntime` and `espeakng-loader` publish wheels, including 64-bit Raspberry
Pi OS.

**Does it support voice cloning or emotions?**
No voice cloning: Goldfinch is one English (US) voice. Expressiveness comes from fillers and the sound tags above,
not from emotion labels.

**Can I use it commercially?**
The weights are Apache-2.0 and the vocoder is MIT. The phonemiser (espeak-ng, phonemizer) is GPL-3.0 and is installed
separately, and the training texts come from Switchboard transcripts; see the License section below.

## License

- **Goldfinch weights and configuration:** [Apache License 2.0](LICENSE).
- **Vocos vocoder weights** (`onnx/vocoder.onnx`): MIT, © 2023 Charactr Inc. See `onnx/LICENSE-vocos`.
- **Training data:** DisfluencySpeech, released by its authors under Apache-2.0. Its texts come from Switchboard
  transcripts, and the Switchboard corpus itself is distributed by the Linguistic Data Consortium (LDC) under its own
  licence. The recordings and the released model are new works, but if your use is sensitive to data provenance,
  review these terms yourself.

**Run-time dependency.** Text-to-phoneme conversion uses espeak-ng and phonemizer. Both are GPL-3.0 and are **not**
included in this repository. See [NOTICE](NOTICE).

## Citation

```bibtex
@misc{flockspeech2026goldfinch,
  title        = {Goldfinch: a small conversational text-to-speech model with sound tags},
  author       = {{Flock Speech}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/flockspeech/goldfinch}},
  note         = {Version 0.1}
}
```

## Links

- Python package: [pypi.org/project/flockspeech](https://pypi.org/project/flockspeech/)
- Source code: [github.com/flockspeech/flockspeech](https://github.com/flockspeech/flockspeech)
- Reading voice: [flockspeech/junco](https://huggingface.co/flockspeech/junco)

## Acknowledgements

Goldfinch is trained on DisfluencySpeech from the AMAAI Lab. We thank its authors and the authors of the Vocos vocoder.