---
language: ja
license: apache-2.0
pipeline_tag: text-to-speech
library_name: eeh
tags:
- text-to-speech
- japanese
- rust
- webassembly
- tiny
---
# eeh
**A tiny Japanese voice that fits in a browser tab.** eeh is a Japanese
text-to-speech model with two voices and 4.5 million parameters, waveform
generator included. It reads kanji, numbers and Tokyo pitch accent, and runs on
a CPU or fully inside a browser, with nothing sent anywhere.
There is no neural vocoder. A transformer reads OpenJTalk's phonemes with their
pitch-accent tones, predicts how long each one lasts, then predicts pitch,
loudness and the settings of a small source-filter synthesizer, which makes the
waveform from a pulse train and noise. That is why it is small and fast. It was
trained from scratch on 18.6 hours of speech by
[Irodori-TTS v4.1 Small Yomi](https://huggingface.co/takuma104/Irodori-TTS-v4.1-Small-Yomi).
| | |
|---|---|
| Parameters | 4,534,219 |
| Voices | F1 (female, late 20s, calm and clear), M1 (male, early 30s, warm), Tokyo Japanese |
| Output | 24 kHz mono |
| Input | Japanese text (kanji, kana, numbers), read with OpenJTalk |
| Speed | 120x real time on one CPU thread, 280x on four (Apple M4 Max); about 50x in a browser tab. First sound after 9 ms (median) on the CPU, 30–50 ms in a browser |
| Pretrained weights in the model | None |
| License | Apache-2.0 |
## Samples
| Text | F1 | M1 |
|---|---|---|
| こんにちは!これは、ノートパソコンでも動く小さな音声合成モデルです。 | | |
| 会議は三月十四日の午後三時半に変わったので、あと四十五分ほどあります。 | | |
| 本当に、あなたの誕生日を忘れると思ったの? | | |
| 彼女は海を見たことがなかった。しばらくの間、何も言わずに波打ち際に立っていた。 | | |
| 恐れ入りますが、お名前とご連絡先をこちらにご記入いただけますか。 | | |
## Files
| Path | Contents |
|---|---|
| `eeh.safetensors` | Weights, float32 (18.2 MB) |
| `eeh-dict.bin` | The Japanese dictionary the front end reads (36 MB) |
| `python/` | Python wheels for macOS (arm64) and Linux (aarch64, x86_64) |
| `bin//eeh` | Command-line program |
| `lib//`, `include/eeh.h` | C library and header, for any language with a C FFI |
| `demo/` | The browser demo: the whole engine compiled to WebAssembly, the weights and the dictionary |
| `samples/` | The audio on this page |
| `eval/report.json`, `eval/metrics.jsonl` | Evaluation report and the training log |
| `LICENSE` | Apache-2.0 |
## Usage
eeh runs on its own engine, written in Rust: a Japanese front end that reads
text exactly as OpenJTalk does (identical on more than 500,000 test texts) and
the model. The same engine is behind the Python package, the command-line
program, the C library and the browser demo.
```bash
hf download ctrltokyo/eeh-1 --local-dir eeh
cd eeh
```
### Python
```bash
pip install python/eeh-0.1.0-py3-none-.whl
```
```python
import eeh, soundfile
synth = eeh.Synthesizer("eeh.safetensors", "eeh-dict.bin")
audio = synth.speak("こんにちは。今日はいい天気ですね。", voice="F1") # float32 array, 24 kHz
soundfile.write("hello.wav", audio, synth.sample_rate)
```
`speak` takes `voice` (`F1` or `M1`), `speed` (1.0), `pitch_shift` (0; 0.058
is about one semitone), `pitch_range` (1.0) and `seed`. `speak_each(text,
callback)` hands over each sentence as soon as it is made, for streaming. The
package has no dependencies; it returns a NumPy array when NumPy is installed.
The synthesizer can be shared between threads.
### Command line
```bash
chmod +x bin/macos-arm64/eeh # the Hub does not keep the executable bit
bin/macos-arm64/eeh speak --weights eeh.safetensors --dictionary eeh-dict.bin \
--voice M1 --text "こんにちは。今日はいい天気ですね。" --out hello.wav
```
### C and other languages
`lib//libeeh_ffi` with `include/eeh.h`: `eeh_synthesizer_new` takes
the weights and dictionary as bytes, `eeh_speak` returns the audio and
`eeh_speak_each` streams it sentence by sentence. Errors come back as status
codes, never as crashes. Go, Node, Java, Ruby and Swift can call it through
their C FFI (all five are tested).
Text of any length works. It is split into sentences at 。!? and read one
sentence at a time; a quote followed by と or って stays with its sentence, and
a sentence too long for one call is cut after 、. A text without a final mark
gets 。. The audio never peaks above 0.99.
### In a browser
```bash
python -m http.server --directory demo 8000
```
Then open `http://localhost:8000`. The engine runs in a Web Worker and plays
long text sentence by sentence as it is made. The first visit downloads about
26 MB (gzip): 17 MB of weights, 9 MB of dictionary and 0.4 MB of program, all
kept by the browser afterwards.
## How it works
**Front end.** eeh's own Rust front end reads the text the way OpenJTalk does
with the NAIST Japanese Dictionary: it splits it into words, picks each kanji's
reading, expands numbers, joins words into accent phrases and assigns their
accent types. It turns the result into tokens:
- one token per phoneme (`k`, `sh`, `a`, `N` for ん, `cl` for っ);
a long vowel is two tokens, one per mora;
- devoiced vowels (the u of です, the i of した) are their own tokens `A I U E O`;
- `` marks a boundary between accent phrases, and punctuation is kept;
- each token carries the Tokyo pitch-accent tone of its mora, low or high,
computed from the phrase's accent type and the mora's position.
It is fast: the 8,650 training sentences, 11.8 hours of speech, are read in
0.29 s on one CPU thread (0.03 ms per sentence, about 150,000x real time), 5.5x
faster than pyopenjtalk. Its dictionary loads in 3 ms.
**Model.** A transformer encoder (4 blocks) reads the tokens and tones, a
predictor gives each token a length in frames, and two frame-level transformers
(2 and 5 blocks) predict pitch, voicing and loudness and then the settings of
the synthesizer: a band-limited pulse train at the predicted pitch plus noise,
each shaped by a filter predicted every frame (256 samples, 10.7 ms). All blocks
use 192 channels, 4 heads and rotary position embeddings.
| Component | Parameters |
|---|---:|
| Embeddings (token, tone, voice) | 13,632 |
| Text encoder | 1,588,864 |
| Duration and pause predictor | 123,906 |
| Frame transformer A, pitch/voicing/energy head | 795,971 |
| Frame transformer B | 1,985,984 |
| Source-filter waveform head | 25,862 |
| Total | 4,534,219 |
## Training data
The speech is synthetic. Two voices were designed once from Japanese
descriptions with Irodori-TTS v4.1 Small (`落ち着いた20代後半の女性。明瞭で自然な東京の話し言葉。`
and `30代前半の男性。温かく親しみやすい、自然な東京の話し言葉。`), then every
sentence was read by [Irodori-TTS v4.1 Small Yomi](https://huggingface.co/takuma104/Irodori-TTS-v4.1-Small-Yomi)
(MIT) cloning those two references. No recording of a real speaker is involved.
On the 300 test sentences in both voices, the teacher's speech has 1.7% kana
reading error, 0.70 pitch-accent agreement and a UTMOS of 3.38.
The 8,650 sentences (8,000 train, 350 validation, 300 test, split by sentence)
were chosen to cover as many morae and mora pairs as possible; 25% are
questions and 10% exclamations.
Every clip was screened before training:
- **Reading.** [kana-whisper](https://huggingface.co/sbintuitions/kana-whisper)
transcribed it to katakana, compared with OpenJTalk's reading; more than 10%
kana error was rejected.
- **Alignment.** Phoneme durations come from forced alignment with
`facebook/wav2vec2-lv-60-espeak-cv-ft`, used only to label the data. Its
labels were recalibrated on the teacher's audio, and clips it aligned with
low confidence were rejected.
- **Speaking rate and length.**
| | Clips | Hours |
|---|---:|---:|
| Generated (sentences x voices) | 17,300 | |
| Passed screening | 15,684 | |
| Used for training (F1 7,301, M1 6,783) | 14,084 | 18.6 |
| Validation / test | 614 / 527 | 1.3 / 1.0 |
## Training
| | |
|---|---|
| Steps | 104,000 |
| Batches | about 9,000 frames (96 s of audio) per step |
| Losses | Multi-resolution spectral losses; adversarial loss with multi-period and multi-resolution discriminators after step 8,000 |
| Hardware | One NVIDIA GB10 (Lenovo ThinkStation PGX) for training; an Apple M4 Max MacBook generated and scored half of the corpus |
| Time | 11.5 hours of training; 5.7 hours to generate the corpus on both machines |
Two changes to the training step made the adversarial phase twice as fast
without changing the gradients: the discriminators' gradient at the waveform is
computed once instead of twice, and their convolutions run in bfloat16 with
autotuned kernels, on batches padded to a few fixed shapes.
## Evaluation
Speech from the final checkpoint was scored the same way as the teacher's
recordings of the same sentences: kana-whisper transcribes it to katakana
(reading error against OpenJTalk's reading), the pitch tracker checks each
expected rise and fall of the pitch accent, and UTMOS rates naturalness.
| Sentences | Clips | Reading error, eeh | Reading error, teacher | Pitch-accent agreement, eeh | Pitch-accent agreement, teacher | UTMOS, eeh | UTMOS, teacher |
|---|---:|---:|---:|---:|---:|---:|---:|
| Test | 300 | 0.8% | 1.2% | 0.70 | 0.69 | 2.94 | 3.38 |
| Validation | 300 | 1.0% | 1.7% | 0.71 | 0.68 | 2.94 | 3.34 |
| New sentences, not in the dataset | 16 | 0.0% | | 0.69 | | 2.88 | |
eeh reads and accents as well as the teacher it learned from; it sounds less
natural (UTMOS about 0.4 lower), which is the cost of the small synthesizer. No
listening test has been run.
## Limitations
- Japanese only, two fixed voices, no voice cloning.
- Readings come from OpenJTalk's dictionary. Rare kanji readings, names and
new words can be read wrong, and the accent of compound phrases is predicted
by rules. Whatever OpenJTalk gets wrong, eeh says wrong.
- Latin letters are spelled out one by one (`eeh` is read イー・イー・エイチ).
- The same text is always read the same way.
- The training data is synthetic, so eeh cannot sound more natural than the
teacher; the small source-filter synthesizer sounds more buzzy than it.
## License
Apache-2.0 for the weights and code.
## Citation
```bibtex
@software{nicholson2026eeh,
author = {Nicholson, Alexander},
title = {eeh: a 4.5M-parameter two-voice Japanese text-to-speech model},
year = {2026},
url = {https://huggingface.co/ctrltokyo/eeh-1}
}
```
## Acknowledgements
Inspired by [tiny-voca](https://github.com/n4ze3m/tiny-voca), trained on speech from [Irodori-TTS](https://github.com/Aratako/Irodori-TTS), and reading Japanese with [OpenJTalk](https://open-jtalk.sourceforge.net/).