KemeTone v1.0 — Egyptian Arabic TTS

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/logo-card.png filter=lfs diff=lfs merge=lfs -text
+assets/logo.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,186 @@
+---
+license: apache-2.0
+language:
+- ar
+pipeline_tag: text-to-speech
+library_name: kokoro
+tags:
+- text-to-speech
+- tts
+- arabic
+- egyptian-arabic
+- masri
+- dialect
+- speech-synthesis
+- kokoro
+---
+
+<div align="center">
+  <img src="assets/logo-card.png" alt="KemeTone" width="460">
+
+  ### Egyptian Arabic that sounds Egyptian.
+
+  **82M parameters · 24 kHz · female voice · runs on a CPU**
+</div>
+
+---
+
+## The problem with Arabic TTS
+
+Almost every Arabic voice you can buy or download speaks **Modern Standard
+Arabic** — the language of news broadcasts and textbooks. Roughly no one speaks
+it at home.
+
+Point one of those models at Egyptian text and it does not fail loudly. It
+fails *politely*: it reads your words with textbook consonants and textbook
+vowels, and the result lands somewhere between a newsreader and a foreigner.
+`جَمِيل` comes out **/ʤamiːl/** when a hundred million people say **/ɡamiːl/**.
+`دِلْوَقْتِي` comes out **/dilwaqti/** when the street says **/dilwaʔti/**.
+
+KemeTone is built for the dialect, not adapted to it.
+
+| | Standard Arabic TTS | **KemeTone** |
+|---|---|---|
+| ج | /ʤ/ — *gemeel* | **/ɡ/ — *gameel*** |
+| ق | /q/ — *dilwaqti* | **/ʔ/ — *dilwa'ti*** |
+| ث | /θ/ — *thalatha* | **/t/ — *talata*** |
+| ذ | /ð/ — *dhahab* | **/z/ — *zahab*** |
+| ظ | /ðˤ/ | **/zˤ/** |
+| Register | broadcast | **conversational** |
+
+And it does this without becoming naive about it. Egyptians do not flatten
+*every* ق to a glottal stop — `الْقُرْآن` keeps its /q/, `ثَقَافَة` keeps its
+/s/, and a model that applies the rule everywhere sounds like a caricature.
+KemeTone ships curated exception lexicons for exactly the learned and Qur'anic
+vocabulary that resists the shift, so the rules fire where a Cairene speaker
+fires them and stay quiet where they don't.
+
+## Listen for
+
+- **A natural female voice**, warm and conversational — not a formal announcer.
+- **Long-form in a single pass.** 15–20 seconds of continuous speech with
+  coherent phrasing, no chunk-and-stitch seams.
+- **Correct dialect consonants**, with the learned-vocabulary exceptions intact.
+- **ع and ح that are actually pronounced** — the two pharyngeals that
+  general-purpose phonemisers silently drop.
+
+## Quick start
+
+```bash
+pip install -r requirements.txt
+```
+
+You also need **espeak-ng** available to the phonemiser
+(`apt install espeak-ng`, `brew install espeak-ng`, or set
+`KEMETONE_ESPEAK_LIB` to a build of `libespeak-ng.so`).
+
+```python
+import torch, soundfile as sf
+from kokoro import KModel
+from kemetone import EgyptianG2P
+
+model = KModel(repo_id="Rabe3/kemetone",
+               config="config.json", model="kemetone.pth").eval()
+voice = torch.load("voices/kemetone.pt")
+
+ipa = EgyptianG2P()("النَّهَارْدَه الْجَوّ حِلْو أَوِي")
+audio = model(ipa, voice[len(ipa) - 1])
+
+sf.write("out.wav", audio.numpy(), 24000)
+```
+
+Or straight from the command line:
+
+```bash
+python example.py "النَّهَارْدَه الْجَوّ حِلْو أَوِي" out.wav
+```
+
+### Give it diacritics
+
+Arabic script does not write short vowels. When they are missing, the
+phonemiser has to guess them — and it guesses the **Standard Arabic** way,
+which is precisely the accent KemeTone exists to avoid. The consonants will
+still be Egyptian; the vowels will not.
+
+```
+حلو            →  vowels guessed, MSA-flavoured
+حِلْو           →  vowels read from the text  ✓
+```
+
+Undiacritized input works and is often acceptable. Diacritized input is
+noticeably better. If your text is bare, run it through an Arabic diacritizer
+first — the quality gain is larger than anything else you can do at inference
+time.
+
+## Specifications
+
+| | |
+|---|---|
+| Architecture | StyleTTS2 / Kokoro (ISTFTNet decoder) |
+| Parameters | 81.8 M |
+| Sample rate | 24 kHz mono |
+| Voice | single female speaker |
+| Language | Egyptian (Cairene) Arabic |
+| Input | diacritized Arabic text |
+| Weights | `kemetone.pth` (327 MB, fp32) |
+| Voice embedding | `voices/kemetone.pt` |
+| Hardware | CPU or any CUDA GPU; ~300 MB of VRAM |
+| Licence | Apache 2.0 |
+
+### What ships here
+
+```
+kemetone.pth          model weights
+config.json           architecture + 178-symbol vocabulary
+voices/kemetone.pt    the voice embedding
+kemetone/             Egyptian G2P front-end and its exception lexicons
+example.py            end-to-end synthesis
+```
+
+The front-end is part of the model, not a convenience wrapper. The weights were
+trained on the phoneme strings this G2P produces; feeding them phonemes from a
+generic Arabic phonemiser will degrade output in ways that are subtle enough to
+miss and consistent enough to matter.
+
+## Limitations
+
+- **One voice.** No multi-speaker support, no voice cloning, no style prompts.
+- **Cairene.** Not Sa'idi, not Alexandrian, and not any other Arabic dialect.
+- **Diacritics carry the vowels.** See above.
+- **Latin script, digits, and code-switched English** are not handled — convert
+  numbers to Arabic words before synthesis.
+- **Very long inputs** must be split at sentence boundaries; the context window
+  is 510 tokens.
+- Emotional range is conversational-neutral. There is no shouting, whispering,
+  or singing in this model.
+
+## Responsible use
+
+This voice is modelled on a real person. Please treat it accordingly:
+
+- **Do not** use it to impersonate anyone, to put words in a real person's
+  mouth, or to produce audio that a listener could mistake for a genuine
+  recording of someone.
+- **Do not** use it for fraud, harassment, political disinformation, or to
+  defeat voice-based authentication.
+- **Do** disclose synthetic speech when the audience might otherwise assume it
+  is human — accessibility tools, narration, learning material, and assistants
+  are all better with a label on them.
+
+Downstream users are responsible for complying with the laws on synthetic media
+and personality rights in their own jurisdiction.
+
+## Licence and credits
+
+Released under **Apache 2.0**, inheriting the licence of the Kokoro
+architecture it builds on.
+
+Built on [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) and
+[Nabra-82M](https://huggingface.co/oddadmix/Nabra-82M-v0.1), with
+phonemisation by [espeak-ng](https://github.com/espeak-ng/espeak-ng).
+
+---
+
+<div align="center">
+  <sub><b>KemeTone</b> — from <i>Kemet</i>, the old name for Egypt: <i>the black land</i>.</sub>
+</div>

config.json

@@ -0,0 +1,237 @@
+{
+  "istftnet": {
+    "upsample_kernel_sizes": [
+      20,
+      12
+    ],
+    "upsample_rates": [
+      10,
+      6
+    ],
+    "gen_istft_hop_size": 5,
+    "gen_istft_n_fft": 20,
+    "resblock_dilation_sizes": [
+      [
+        1,
+        3,
+        5
+      ],
+      [
+        1,
+        3,
+        5
+      ],
+      [
+        1,
+        3,
+        5
+      ]
+    ],
+    "resblock_kernel_sizes": [
+      3,
+      7,
+      11
+    ],
+    "upsample_initial_channel": 512
+  },
+  "dim_in": 64,
+  "dropout": 0.2,
+  "hidden_dim": 512,
+  "max_conv_dim": 512,
+  "max_dur": 50,
+  "multispeaker": true,
+  "n_layer": 3,
+  "n_mels": 80,
+  "n_token": 178,
+  "style_dim": 128,
+  "text_encoder_kernel_size": 5,
+  "plbert": {
+    "hidden_size": 768,
+    "num_attention_heads": 12,
+    "intermediate_size": 2048,
+    "max_position_embeddings": 512,
+    "num_hidden_layers": 12,
+    "dropout": 0.1
+  },
+  "sample_rate": 24000,
+  "vocab": {
+    "$": 0,
+    ";": 1,
+    ":": 2,
+    ",": 3,
+    ".": 4,
+    "!": 5,
+    "?": 6,
+    "ʕ": 7,
+    "ħ": 8,
+    "—": 9,
+    "…": 10,
+    "\"": 11,
+    "(": 12,
+    ")": 13,
+    "“": 14,
+    "”": 15,
+    " ": 16,
+    "̃": 17,
+    "ʣ": 18,
+    "ʥ": 19,
+    "ʦ": 20,
+    "ʨ": 21,
+    "ᵝ": 22,
+    "ꭧ": 23,
+    "A": 24,
+    "I": 25,
+    "": 26,
+    "": 27,
+    "": 28,
+    "": 29,
+    "": 30,
+    "O": 31,
+    "": 32,
+    "Q": 33,
+    "": 34,
+    "S": 35,
+    "T": 36,
+    "": 37,
+    "": 38,
+    "W": 39,
+    "": 40,
+    "Y": 41,
+    "ᵊ": 42,
+    "a": 43,
+    "b": 44,
+    "c": 45,
+    "d": 46,
+    "e": 47,
+    "f": 48,
+    "": 49,
+    "h": 50,
+    "i": 51,
+    "j": 52,
+    "k": 53,
+    "l": 54,
+    "m": 55,
+    "n": 56,
+    "o": 57,
+    "p": 58,
+    "q": 59,
+    "r": 60,
+    "s": 61,
+    "t": 62,
+    "u": 63,
+    "v": 64,
+    "w": 65,
+    "x": 66,
+    "y": 67,
+    "z": 68,
+    "ɑ": 69,
+    "ɐ": 70,
+    "ɒ": 71,
+    "æ": 72,
+    "": 73,
+    "": 74,
+    "β": 75,
+    "ɔ": 76,
+    "ɕ": 77,
+    "ç": 78,
+    "": 79,
+    "ɖ": 80,
+    "ð": 81,
+    "ʤ": 82,
+    "ə": 83,
+    "": 84,
+    "ɚ": 85,
+    "ɛ": 86,
+    "ɜ": 87,
+    "": 88,
+    "": 89,
+    "ɟ": 90,
+    "": 91,
+    "ɡ": 92,
+    "": 93,
+    "": 94,
+    "": 95,
+    "": 96,
+    "": 97,
+    "": 98,
+    "ɥ": 99,
+    "": 100,
+    "ɨ": 101,
+    "ɪ": 102,
+    "ʝ": 103,
+    "": 104,
+    "": 105,
+    "": 106,
+    "": 107,
+    "": 108,
+    "": 109,
+    "ɯ": 110,
+    "ɰ": 111,
+    "ŋ": 112,
+    "ɳ": 113,
+    "ɲ": 114,
+    "ɴ": 115,
+    "ø": 116,
+    "": 117,
+    "ɸ": 118,
+    "θ": 119,
+    "œ": 120,
+    "": 121,
+    "": 122,
+    "ɹ": 123,
+    "": 124,
+    "ɾ": 125,
+    "ɻ": 126,
+    "": 127,
+    "ʁ": 128,
+    "ɽ": 129,
+    "ʂ": 130,
+    "ʃ": 131,
+    "ʈ": 132,
+    "ʧ": 133,
+    "": 134,
+    "ʊ": 135,
+    "ʋ": 136,
+    "": 137,
+    "ʌ": 138,
+    "ɣ": 139,
+    "ɤ": 140,
+    "": 141,
+    "χ": 142,
+    "ʎ": 143,
+    "": 144,
+    "": 145,
+    "": 146,
+    "ʒ": 147,
+    "ʔ": 148,
+    "": 149,
+    "": 150,
+    "": 151,
+    "": 152,
+    "": 153,
+    "": 154,
+    "": 155,
+    "ˈ": 156,
+    "ˌ": 157,
+    "ː": 158,
+    "": 159,
+    "": 160,
+    "": 161,
+    "ʰ": 162,
+    "": 163,
+    "ʲ": 164,
+    "": 165,
+    "": 166,
+    "": 167,
+    "": 168,
+    "↓": 169,
+    "": 170,
+    "→": 171,
+    "↗": 172,
+    "↘": 173,
+    "": 174,
+    "": 175,
+    "": 176,
+    "ᵻ": 177
+  }
+}

example.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+"""Synthesise Egyptian Arabic with KemeTone.
+
+    python example.py "النَّهَارْدَه الْجَوّ حِلْو أَوِي" out.wav
+
+Input should be diacritized. Undiacritized Arabic still synthesises, but short
+vowels are then guessed by the phonemiser rather than read from the text, and
+the guesses follow Modern Standard patterns — which is audibly wrong in
+Egyptian. See the model card.
+
+If espeak-ng is not on the default library path, point KemeTone at it:
+
+    export KEMETONE_ESPEAK_LIB=/path/to/libespeak-ng.so
+    export KEMETONE_ESPEAK_DATA=/path/to/espeak-ng-data
+"""
+import sys
+import torch
+import soundfile as sf
+from kokoro import KModel
+from kemetone import EgyptianG2P
+
+SR = 24000
+REPO = "Rabe3/kemetone"
+
+
+def main() -> int:
+    text = sys.argv[1] if len(sys.argv) > 1 else "النَّهَارْدَه الْجَوّ حِلْو أَوِي"
+    out = sys.argv[2] if len(sys.argv) > 2 else "out.wav"
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+
+    model = KModel(repo_id=REPO, config="config.json",
+                   model="kemetone.pth").to(device).eval()
+    voice = torch.load("voices/kemetone.pt", map_location=device)
+
+    ipa = EgyptianG2P()(text)
+    print("phonemes:", ipa)
+
+    with torch.no_grad():
+        audio = model(ipa, voice[len(ipa) - 1])
+
+    sf.write(out, audio.cpu().numpy(), SR)
+    print(f"wrote {out}  {len(audio) / SR:.1f}s")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

kemetone.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7880f9db23c5b9adb46d26ea166181ccef208db812322256ff004ab83420c5f0
+size 327220427

kemetone/__init__.py

@@ -0,0 +1,3 @@
+"""KemeTone — Egyptian Arabic text-to-speech."""
+from .g2p import EgyptianG2P  # noqa: F401
+__version__ = "1.0.0"

kemetone/arabic.py

@@ -0,0 +1,42 @@
+"""Arabic orthography helpers used by the front-end."""
+from __future__ import annotations
+import unicodedata
+
+SHADDA = "ّ"
+FINAL_MARKS = frozenset(["ً", "ٌ", "ٍ", "َ", "ُ", "ِ", "ْ", "ٰ"])
+ALL_HARAKAT = FINAL_MARKS | {SHADDA}
+KEPT_FINAL = frozenset([SHADDA, "ْ"])
+ALEF_VARIANTS = str.maketrans({"أ": "ا", "إ": "ا", "آ": "ا", "ٱ": "ا", "ة": "ه", "ى": "ي"})
+ARABIC_LETTERS = frozenset("ابتثجحخدذرزسشصضطظعغفقكلمنهويءأإآئؤةى")
+
+
+def is_mark(ch: str) -> bool:
+    return unicodedata.category(ch) == "Mn"
+
+
+def letters(word: str) -> str:
+    return "".join(c for c in word if not is_mark(c))
+
+
+def skeleton(word: str) -> str:
+    """Lookup key: Arabic letters only, orthography normalised."""
+    return "".join(
+        c for c in letters(word).translate(ALEF_VARIANTS) if c in ARABIC_LETTERS
+    )
+
+
+def strip_irab(text: str) -> str:
+    """Drop word-final case endings, keeping final sukun and shadda.
+
+    Egyptian does not pronounce i'rab, and leaving it in leads the phonemiser to
+    voice a final vowel that is not there. Applied to every input.
+
+        رَجُلٌ -> رَجُل      حَقٌّ -> حَقّ      مِنْ -> مِنْ
+    """
+    out = []
+    for word in text.split(" "):
+        i = len(word)
+        while i > 0 and word[i - 1] in ALL_HARAKAT:
+            i -= 1
+        out.append(word[:i] + "".join(c for c in word[i:] if c in KEPT_FINAL))
+    return " ".join(out)

kemetone/g2p.py

@@ -0,0 +1,229 @@
+"""Egyptian (Cairene) Arabic grapheme-to-phoneme front-end.
+
+Turns diacritized Egyptian text into the IPA symbol set the model was trained
+on. Modern Standard Arabic phonemisation will not do: five consonants are
+realised differently in Cairene, and a model fed MSA phonemes produces MSA
+pronunciation regardless of how the text was written.
+
+    ج  ->  /ɡ/     unconditional
+    ق  ->  /ʔ/     except in Qur'anic and learned vocabulary, which keeps /q/
+    ث  ->  /t/     except in learned vocabulary, which takes /s/
+    ذ  ->  /z/     except in a small inherited set, which takes /d/
+    ظ  ->  /zˤ/    except in a small inherited set, which takes /dˤ/
+
+The exceptions are lexical, not rule-governed, so each lives in a word list
+under `lexicons/`. Lookup strips fused proclitics (و ف ب ك ل ال …) because a
+proclitic never changes how the stem's consonants are realised.
+
+Two implementation notes that the code depends on:
+
+* espeak-ng emits **ð for both ذ and ظ** — it produces no ðˤ — so those two
+  rules cannot be told apart on the phoneme string and are decided from the
+  word's letters instead. A word carrying both letters is left alone.
+* Rewrites run **before** the phoneme fixups, which strip the pharyngealisation
+  marker and the dental bridge. Running them after would collapse distinctions
+  the rules still need.
+
+Usage:
+    from kemetone import EgyptianG2P
+    g2p = EgyptianG2P()
+    g2p("قَالَ لِي جَمِيل")        # -> 'ʔˈaːla liː ɡˈamiːl'
+"""
+from __future__ import annotations
+
+import re
+from collections import Counter
+from pathlib import Path
+
+from .arabic import skeleton, strip_irab
+from .normalize_tashkeel import normalize as normalize_marks
+
+LEXDIR = Path(__file__).resolve().parent / "lexicons"
+
+# Symbols outside the model's inventory, mapped or dropped.
+PHONEME_FIXUPS = {
+    "̪": "",   # combining bridge below (dental)
+    "ˤ": "",   # pharyngealisation marker
+    "[": "", "]": "", "{": "", "}": "",
+}
+
+_SYLLABLE_DOT = re.compile(r"(?<=\S)\.(?=\S)")
+_LATIN_RUN = re.compile(r"[A-Za-z]+")
+_CITATION = re.compile(r"\[[^\]]*\]")
+_WS = re.compile(r"\s+")
+
+# What espeak-ng's Arabic voice emits for each letter we rewrite. ج is a single
+# codepoint (U+02A4), not a d+ʒ tie bar, so a plain replace is safe.
+PH_JEEM, PH_QAF, PH_THEH, PH_DHAL = "ʤ", "q", "θ", "ð"
+
+# A word that makes the NEXT word a Qur'anic proper noun, which keeps /q/.
+# Sura names cannot go in the flat lexicon: most double as ordinary words said
+# with /ʔ/ — البقرة is "the cow", القصص is "stories".
+Q_NEXT_TRIGGERS = {"سوره", "سور"}
+
+_PROCLITICS = ("وال", "فال", "بال", "كال", "لل", "ال", "و", "ف", "ب", "ك", "ل")
+
+
+def load_lexicon(letter: str, path: str | Path | None = None) -> set[str]:
+    p = Path(path) if path else LEXDIR / f"{letter}.tsv"
+    if not p.exists():
+        return set()
+    out = set()
+    for line in p.read_text(encoding="utf-8").splitlines():
+        line = line.split("#", 1)[0].strip()
+        if line:
+            out.add(line.split("\t", 1)[0])
+    return out
+
+
+def lexicon_lookup(skel: str, lex: set[str]) -> bool:
+    """Membership test that tolerates fused proclitics."""
+    if skel in lex:
+        return True
+    seen, frontier = {skel}, [skel]
+    for _ in range(2):
+        nxt = []
+        for w in frontier:
+            for p in _PROCLITICS:
+                if w.startswith(p) and len(w) - len(p) >= 3:
+                    s = w[len(p):]
+                    if s in lex:
+                        return True
+                    if s not in seen:
+                        seen.add(s)
+                        nxt.append(s)
+        frontier = nxt
+    return False
+
+
+def normalize_text(text: str) -> tuple[str, int]:
+    text = _CITATION.sub(" ", text)
+    # Tatweel has no phonetic value, but espeak reads it BY NAME when bare —
+    # "الـ" comes out as the spoken word تطويل.
+    text = text.replace("ـ", "")
+    latin = _LATIN_RUN.findall(text)
+    if latin:
+        text = _LATIN_RUN.sub(" ", text)
+    return _WS.sub(" ", text).strip(), len(latin)
+
+
+def clean_phonemes(ph: str) -> str:
+    ph = _SYLLABLE_DOT.sub("", ph)
+    for old, new in PHONEME_FIXUPS.items():
+        ph = ph.replace(old, new)
+    return ph
+
+
+class EgyptianG2P:
+    """Diacritized Egyptian text -> IPA.
+
+    Args:
+        dialect: apply the Egyptian rewrites. False gives plain MSA output.
+        pausal:  drop word-final case endings before phonemising. On by default,
+                 and applied here so that every input is treated identically.
+    """
+
+    def __init__(self, dialect: bool = True, lexicon_dir: str | Path | None = None,
+                 pausal: bool = True):
+        from misaki import espeak
+
+        from .runtime import init_espeak
+        init_espeak()
+
+        self._g2p = espeak.EspeakG2P(language="ar")
+        self.dialect = dialect
+        self.pausal = pausal
+        d = Path(lexicon_dir) if lexicon_dir else LEXDIR
+        self.lex = {ch: load_lexicon(ch, d / f"{ch}.tsv") for ch in ("ق", "ث", "ذ", "ظ")}
+
+        self.n_words = 0
+        self.n_rule = Counter()
+        self.n_exception = Counter()
+        self.n_ambiguous = 0
+        self.oov = Counter()
+
+    def _rewrite_word(self, ph: str, word: str, prev: str | None = None) -> str:
+        skel = skeleton(word)
+
+        if PH_JEEM in ph:
+            self.n_rule["ج>ɡ"] += 1
+            ph = ph.replace(PH_JEEM, "ɡ")
+
+        if PH_QAF in ph:
+            keep_q = lexicon_lookup(skel, self.lex["ق"]) or (
+                prev is not None and skeleton(prev) in Q_NEXT_TRIGGERS
+            )
+            if keep_q:
+                self.n_exception["ق>q"] += 1
+            else:
+                self.n_rule["ق>ʔ"] += 1
+                ph = ph.replace(PH_QAF, "ʔ")
+
+        if PH_THEH in ph:
+            if lexicon_lookup(skel, self.lex["ث"]):
+                self.n_exception["ث>s"] += 1
+                ph = ph.replace(PH_THEH, "s")
+            else:
+                self.n_rule["ث>t"] += 1
+                ph = ph.replace(PH_THEH, "t")
+
+        # ð is both ذ and ظ — decide from the spelling.
+        if PH_DHAL in ph:
+            has_dhal, has_zah = "ذ" in skel, "ظ" in skel
+            if has_dhal and has_zah:
+                self.n_ambiguous += 1
+            elif has_zah:
+                if lexicon_lookup(skel, self.lex["ظ"]):
+                    self.n_exception["ظ>d"] += 1
+                    ph = ph.replace(PH_DHAL, "dˤ")
+                else:
+                    self.n_rule["ظ>z"] += 1
+                    ph = ph.replace(PH_DHAL, "zˤ")
+            elif has_dhal:
+                if lexicon_lookup(skel, self.lex["ذ"]):
+                    self.n_exception["ذ>d"] += 1
+                    ph = ph.replace(PH_DHAL, "d")
+                else:
+                    self.n_rule["ذ>z"] += 1
+                    ph = ph.replace(PH_DHAL, "z")
+        return ph
+
+    def phonemize(self, text: str) -> str:
+        text, _ = normalize_text(text)
+        if not text:
+            return ""
+        # A vowel written before a shadda makes espeak drop it silently, and
+        # both orders occur in real text — canonicalise on the way in.
+        text = normalize_marks(text)
+        if self.pausal:
+            text = strip_irab(text)
+
+        raw, _ = self._g2p(text)
+        if not self.dialect:
+            return clean_phonemes(raw)
+
+        words, parts = text.split(" "), raw.split(" ")
+        self.n_words += len(words)
+        if len(parts) == len(words):
+            out = " ".join(
+                self._rewrite_word(p, w, words[i - 1] if i else None)
+                for i, (p, w) in enumerate(zip(parts, words))
+            )
+        else:
+            # espeak merged or split something; phonemise word by word so the
+            # rules stay word-scoped.
+            out = " ".join(
+                self._rewrite_word(self._g2p(w)[0], w, words[i - 1] if i else None)
+                for i, w in enumerate(words)
+            )
+        return clean_phonemes(out)
+
+    __call__ = phonemize
+
+    def stats(self) -> dict:
+        return {
+            "words": self.n_words,
+            "rules_fired": dict(self.n_rule),
+            "lexicon_exceptions": dict(self.n_exception),
+            "ambiguous_words": self.n_ambiguous,
+        }

kemetone/lexicons/ث.tsv

@@ -0,0 +1,105 @@
+# Words taking the exception reading for ث.
+ابحاث
+اثار
+اثبات
+اثر
+اثري
+اثرياء
+اثناء
+احاديث
+اضغاث
+الاثار
+الاثر
+الاثناء
+الاثير
+الاضغاث
+الباحث
+البث
+البحث
+البراغيث
+التاثير
+التلوث
+الثابت
+الثانويه
+الثروه
+الثقافه
+الثقالبه
+الثقه
+الثوره
+الجثه
+الحادث
+الحادثه
+الحديث
+الحديثه
+الحوادث
+الكارثه
+المثال
+المثقفين
+المثلث
+المستثمرين
+الميثين
+الميراث
+الوثائق
+الوثيقه
+الوراثه
+امثله
+باحث
+باحثين
+بحث
+بحوث
+تؤثر
+تاثير
+تثير
+تحديث
+تلوث
+تندثر
+ثابت
+ثانوي
+ثانويه
+ثبات
+ثروه
+ثري
+ثقافه
+ثقافي
+ثقافيه
+ثقه
+ثنائي
+ثنائيه
+ثورات
+ثوره
+ثوري
+ثوريه
+جثث
+جثه
+حادث
+حديث
+حديثه
+حوادث
+حيث
+فثم
+كارثه
+لوث
+مؤثر
+متاثر
+مثال
+مثبت
+مثقال
+مثقف
+مثل
+مثلا
+مثلث
+مثير
+مثيره
+ملوث
+ميثان
+ميراث
+نموذج
+واثق
+وثائق
+وثيقه
+وراثه
+وراثي
+يؤثر
+يثبت
+يثير
+يلبثوا

kemetone/lexicons/ذ.tsv

@@ -0,0 +1,9 @@
+# Words taking the exception reading for ذ.
+الذهب
+بالذهب
+تاخذ
+ذراعه
+ذكر
+ذهب
+ياخذ
+يذوب

kemetone/lexicons/ظ.tsv

@@ -0,0 +1,6 @@
+# Words taking the exception reading for ظ.
+الظهر
+بالظبط
+نظاره
+نظيف
+نظيفه

kemetone/lexicons/ق.tsv

@@ -0,0 +1,59 @@
+# Words taking the exception reading for ق.
+اقتصاد
+اقتصادي
+اقتصاديه
+الاقتصاد
+الانشقاق
+التقوي
+الثقافه
+العلق
+الفرقان
+الفقه
+الفقهاء
+الفلق
+القارعه
+القانون
+القبله
+القدر
+القدس
+القران
+القوانين
+القوميه
+القيامه
+المقرئ
+المنافقون
+المنطق
+الموسيقي
+الوثائق
+الوثيقه
+بالقران
+تقدير
+تقوي
+ثقافه
+ثقافي
+ثقافيه
+فقه
+فقهاء
+فقهي
+قانون
+قانوني
+قانونيه
+قبله
+قدر
+قران
+قرانا
+قراني
+قرانيه
+قوانين
+قوميه
+للقران
+مقرئ
+منطق
+منطقي
+منطقيه
+موسيقي
+موسيقيه
+موسيقيين
+والقران
+وثائق
+وثيقه

kemetone/normalize_tashkeel.py

@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+"""
+Canonical ordering for Arabic combining marks.
+
+Unicode allows a vowel mark either side of a shadda, and both orderings occur
+in real text. espeak-ng only honours the canonical one:
+
+    consonant + shadda + harakat   ->  الشَّمْس  ʔaʃʃˈams   (correct)
+    consonant + harakat + shadda   ->  الشَّمْس  ʔaʃʃms     (vowel SILENTLY dropped)
+
+The wrong order does not error, it just deletes a vowel from the phoneme
+string. Applied across a corpus that is a systematic pronunciation defect that
+no symbol-map assertion would catch, so normalize before phonemizing.
+"""
+
+import re
+
+SHADDA = "ّ"
+HARAKAT = "ً-ْٰ"
+
+_SWAP = re.compile(f"([{HARAKAT}])({SHADDA})")
+_DUP = re.compile(f"([{HARAKAT}])\\1+")
+
+
+def normalize(text: str) -> str:
+    """Reorder harakat+shadda -> shadda+harakat and collapse duplicated marks."""
+    prev = None
+    while prev != text:  # a mark may need to migrate past several neighbours
+        prev = text
+        text = _SWAP.sub(r"\2\1", text)
+    return _DUP.sub(r"\1", text)
+
+
+def count_misordered(text: str) -> int:
+    return len(_SWAP.findall(text))
+
+
+if __name__ == "__main__":
+    import sys
+
+    for path in sys.argv[1:]:
+        with open(path, encoding="utf-8") as fh:
+            src = fh.read()
+        n = count_misordered(src)
+        out = normalize(src)
+        with open(path, "w", encoding="utf-8") as fh:
+            fh.write(out)
+        print(f"{path}: reordered {n} harakat+shadda pairs")

kemetone/runtime.py

@@ -0,0 +1,45 @@
+"""espeak-ng wiring.
+
+The front-end phonemises through espeak-ng's Arabic voice. Some environments
+ship a `libespeak-ng.so` that will not load (an older glibc than the wheel was
+built against is the usual cause). Point `KEMETONE_ESPEAK_LIB` at a working
+build to override:
+
+    export KEMETONE_ESPEAK_LIB=/path/to/libespeak-ng.so
+
+The override must be applied AFTER `misaki.espeak` is imported, because that
+module sets the library path at import time and phonemizer reads its value
+before consulting the environment. `init_espeak()` handles the ordering.
+"""
+from __future__ import annotations
+import os
+
+_done = False
+
+
+def init_espeak(library: str | None = None) -> str | None:
+    """Prepare espeak-ng. Safe to call repeatedly."""
+    global _done
+    if _done:
+        return os.environ.get("PHONEMIZER_ESPEAK_LIBRARY")
+
+    import misaki.espeak  # noqa: F401  — sets its own path at import time
+
+    lib = library or os.environ.get("KEMETONE_ESPEAK_LIB")
+    if lib:
+        from phonemizer.backend.espeak.wrapper import EspeakWrapper
+
+        EspeakWrapper.set_library(lib)          # after misaki, so ours wins
+        os.environ["PHONEMIZER_ESPEAK_LIBRARY"] = lib
+        data = os.environ.get("KEMETONE_ESPEAK_DATA")
+        if data:
+            EspeakWrapper.set_data_path(data)
+    _done = True
+    return os.environ.get("PHONEMIZER_ESPEAK_LIBRARY")
+
+
+def espeak_version() -> tuple:
+    init_espeak()
+    from phonemizer.backend import EspeakBackend
+
+    return EspeakBackend.version()

requirements.txt

@@ -0,0 +1,6 @@
+# KemeTone needs the Kokoro runtime for the model class, and espeak-ng for the
+# Arabic phonemiser. `kokoro` pulls in misaki and phonemizer-fork itself.
+kokoro>=0.9.4
+torch>=2.1
+soundfile
+numpy

voices/kemetone.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f8c27cf931b2b7852cd4dc90f07c9f1189a18edf32d3c9973a33f14f8575d38
+size 523824