Sayso
Sayso guesses how an English word is pronounced (grapheme-to-phoneme). It is made for the words a pronunciation dictionary doesn't know, such as names, tools and jargon, and it answers in the phoneme format of misaki, so its output can go straight into Kokoro-82M. It speaks both American and British English: one token at the start chooses the accent.
Sayso is the pronunciation model of Papegai, an app that reads text aloud with highlighting. There it replaces espeak-ng as the fallback for unknown words.
Files
| File | What it is |
|---|---|
g2p-encoder.onnx |
encoder: input src (int64, [1, S]), output mem ([1, S, 256]) |
g2p-decoder.onnx |
decoder: inputs tgt (int64, [1, T]) and mem, output logits (the next phoneme) |
g2p-vocab.json |
src: the input tokens (<pad>, <us>, <gb>, a-z, '), tgt: the phonemes, max_src 32, max_tgt 48 |
How to use
Lowercase the word, put <us> or <gb> in front, and decode greedily until
<eos>:
import json
import numpy as np
import onnxruntime as ort
v = json.load(open("g2p-vocab.json", encoding="utf-8"))
src_id = {c: i for i, c in enumerate(v["src"])}
tgt = v["tgt"]
BOS, EOS = tgt.index("<bos>"), tgt.index("<eos>")
enc = ort.InferenceSession("g2p-encoder.onnx")
dec = ort.InferenceSession("g2p-decoder.onnx")
def phonemes(word, accent="us"): # accent: "us" or "gb"
ids = [src_id[f"<{accent}>"]] + [src_id[c] for c in word.lower() if c in src_id]
mem = enc.run(["mem"], {"src": np.array([ids], dtype=np.int64)})[0]
out = [BOS]
for _ in range(v["max_tgt"] - 1):
logits = dec.run(["logits"], {"tgt": np.array([out], dtype=np.int64), "mem": mem})[0]
nxt = int(logits.reshape(-1, len(tgt))[-1].argmax())
if nxt in (0, BOS, EOS):
break
out.append(nxt)
return "".join(tgt[i] for i in out[1:])
print(phonemes("casanovo")) # kˌɑsənˈOvO
print(phonemes("casanovo", "gb")) # kˌasənˈQvQ
Words can be up to 31 letters. Sayso reads single words only: split compounds, numbers and abbreviations before asking it.
Training
- Data: the American and British dictionaries of misaki 0.9.4 (gold and silver, Apache-2.0), about 356,000 word and pronunciation pairs. 2% of the words were held out as the test set.
- Model: a transformer encoder and decoder, 256 wide, 3 + 3 layers, 4 heads, 5.59 M parameters.
- Training: 120 epochs, batch 1024, learning rate 1e-3 (cosine, 2,000 warm-up steps), label smoothing 0.1; 34 minutes on one RTX 5090.
Results
On the held-out test words (3,483 American, 3,776 British), compared with misaki's own pronunciation:
| exact | ignoring stress | sound errors | |
|---|---|---|---|
| Sayso, American | 78.0% | 82.4% | 4.6% |
| Sayso, British | 80.2% | 84.5% | 4.5% |
| espeak-ng, American | 28.1% | 30.2% | 17.4% |
| espeak-ng, British | 72.1% | 77.5% | 6.7% |
"Sound errors" is the number of phoneme characters that differ (edit distance) divided by the length of the right answer.
Limitations
Sayso learned from English dictionary words. Names from other languages are pronounced as if they were English, and abbreviations that should be spelled out letter by letter can come out as one word. It only knows the letters a to z and the apostrophe.
License
Apache-2.0, like the misaki dictionaries it was trained on.