Sayso

Sayso guesses how an English word is pronounced (grapheme-to-phoneme). It is made for the words a pronunciation dictionary doesn't know, such as names, tools and jargon, and it answers in the phoneme format of misaki, so its output can go straight into Kokoro-82M. It speaks both American and British English: one token at the start chooses the accent.

Sayso is the pronunciation model of Papegai, an app that reads text aloud with highlighting. There it replaces espeak-ng as the fallback for unknown words.

Files

File What it is
g2p-encoder.onnx encoder: input src (int64, [1, S]), output mem ([1, S, 256])
g2p-decoder.onnx decoder: inputs tgt (int64, [1, T]) and mem, output logits (the next phoneme)
g2p-vocab.json src: the input tokens (<pad>, <us>, <gb>, a-z, '), tgt: the phonemes, max_src 32, max_tgt 48

How to use

Lowercase the word, put <us> or <gb> in front, and decode greedily until <eos>:

import json
import numpy as np
import onnxruntime as ort

v = json.load(open("g2p-vocab.json", encoding="utf-8"))
src_id = {c: i for i, c in enumerate(v["src"])}
tgt = v["tgt"]
BOS, EOS = tgt.index("<bos>"), tgt.index("<eos>")
enc = ort.InferenceSession("g2p-encoder.onnx")
dec = ort.InferenceSession("g2p-decoder.onnx")

def phonemes(word, accent="us"):  # accent: "us" or "gb"
    ids = [src_id[f"<{accent}>"]] + [src_id[c] for c in word.lower() if c in src_id]
    mem = enc.run(["mem"], {"src": np.array([ids], dtype=np.int64)})[0]
    out = [BOS]
    for _ in range(v["max_tgt"] - 1):
        logits = dec.run(["logits"], {"tgt": np.array([out], dtype=np.int64), "mem": mem})[0]
        nxt = int(logits.reshape(-1, len(tgt))[-1].argmax())
        if nxt in (0, BOS, EOS):
            break
        out.append(nxt)
    return "".join(tgt[i] for i in out[1:])

print(phonemes("casanovo"))        # kˌɑsənˈOvO
print(phonemes("casanovo", "gb"))  # kˌasənˈQvQ

Words can be up to 31 letters. Sayso reads single words only: split compounds, numbers and abbreviations before asking it.

Training

  • Data: the American and British dictionaries of misaki 0.9.4 (gold and silver, Apache-2.0), about 356,000 word and pronunciation pairs. 2% of the words were held out as the test set.
  • Model: a transformer encoder and decoder, 256 wide, 3 + 3 layers, 4 heads, 5.59 M parameters.
  • Training: 120 epochs, batch 1024, learning rate 1e-3 (cosine, 2,000 warm-up steps), label smoothing 0.1; 34 minutes on one RTX 5090.

Results

On the held-out test words (3,483 American, 3,776 British), compared with misaki's own pronunciation:

exact ignoring stress sound errors
Sayso, American 78.0% 82.4% 4.6%
Sayso, British 80.2% 84.5% 4.5%
espeak-ng, American 28.1% 30.2% 17.4%
espeak-ng, British 72.1% 77.5% 6.7%

"Sound errors" is the number of phoneme characters that differ (edit distance) divided by the length of the right answer.

Limitations

Sayso learned from English dictionary words. Names from other languages are pronounced as if they were English, and abbreviations that should be spelled out letter by letter can come out as one word. It only knows the letters a to z and the apostrophe.

License

Apache-2.0, like the misaki dictionaries it was trained on.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support