eg2-ko-reader / README.md
topabaem's picture
Card: ZeroGPU demo speed (source 52d2c586e8b2)
2cbe124 verified
|
Raw History Blame Contribute Delete
6.09 kB
---
language: ko
license: other
license_name: eg2-ko-reader-license-1.0
license_link: LICENSE
base_model: google/embeddinggemma-2
pipeline_tag: automatic-speech-recognition
tags: [speech-recognition, ctc, korean, embeddinggemma, core-ai]
---
![EG2-KO Reader](https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/assets/banner.png)
# EG2-KO Reader
Korean speech-to-text on top of the EmbeddingGemma 2 (EG2) audio encoder. The encoder stays exactly as it is; a small reader (19.7M parameters, 79 MB) reads its output, 25 vectors per second, and picks a Hangul syllable or a blank for each one. An app that already ships EG2 gets Korean transcription without a second speech model.
This repository holds the reader and an unmodified copy of the encoder, so one download is enough.
**Try it in the browser:** [demo Space](https://huggingface.co/spaces/topabaem/eg2-ko-reader-demo) · **How it works, animated:** [project page](https://topabaem05.github.io/eg2-ko-reader-demo/)
## Use
```bash
pip install -U huggingface_hub
hf download Haverbex/eg2-ko-reader --local-dir eg2-ko-reader
pip install -r eg2-ko-reader/requirements.txt
python eg2-ko-reader/eg2_ko_reader.py audio.wav
```
From Python:
```python
import sys
sys.path.insert(0, "eg2-ko-reader")
from eg2_ko_reader import load
stt = load() # load(device="cuda") on a GPU, load(device="mps") on a Mac
print(stt.transcribe("audio.wav")) # a file path, or stt.transcribe(array, sample_rate=44100)
```
* Any length. `transcribe` cuts the audio into 30 s windows that overlap by 3.84 s and joins them.
* For long recordings, `stt.chunks("lecture.wav")` cuts the audio at pauses into pieces of at most 30 s and yields `{"start", "end", "text"}` for each, so you can show text as it comes. On a 55-minute lecture both ways gave the same accuracy.
* Any file libsndfile reads (wav, flac, ogg, mp3). It is converted to 16 kHz mono.
* The download is about 1.6 GB, of which the encoder is 1.49 GB.
## How it works
1. Audio (16 kHz) → log-mel → **EG2 audio encoder**, frozen → one 512-dimensional vector every 40 ms
2. Vectors → **reader** (6 bidirectional Transformer layers) → one of 1,035 syllables, or a blank, at every position
3. Repeated symbols merge and blanks drop out (greedy CTC). There is no decoder that writes one token at a time, so one pass gives the whole text.
## Benchmarks
![Accuracy](https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/assets/accuracy.png)
![Speed](https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/assets/speed.png)
![Weights an app adds](https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/assets/size.png)
| Test set | Utterances | Character error rate |
| -- | -- | -- |
| Zeroth-Korean test (10 speakers and sentences not seen in training) | 457 | **7.26%** |
| FLEURS Korean test (125 utterances that cannot be normalized to Hangul excluded) | 257 | 5.59% |
| Zeroth-Korean dev (speakers overlap with training, sentences do not) | 965 | 6.23% |
| FLEURS Korean dev | 157 | 4.68% |
* Greedy CTC with no language model, spaces not counted. These numbers come from the files in this repository.
* No confidence interval for the test sets. One training seed.
* Speed: 60 minutes of audio in 5.50 s with Core AI on an Apple M5 Pro (654× real time, peak memory 3.4 GB), measured with earlier weights of the same architecture. On the demo Space (ZeroGPU), 56 minutes of audio took 17.8 s on the GPU (about 190× real time, measured once on 2026-11-18).
* The gauges and the banner are drawn with [dither-kit-rs](https://github.com/DeepanshuMishraa/dither-kit-rs).
## Files
| Path | Contents | License |
| -- | -- | -- |
| `reader/model.safetensors`, `reader/vocab_ko.txt` | Reader weights (fp32) and the syllable vocabulary | EG2-KO Reader License 1.0 |
| `encoder/` | Unmodified files of `google/embeddinggemma-2` (revision `914f7f89`) | Apache-2.0 |
| `coreai/reader_w30_fp16.aimodel/` | Core AI reader bundle for macOS and iOS (fp16, 40 MB). Pair it with the encoder bundle `macos/eg2_audio_fp16_T3000.aimodel` of `mlboydaisuke/EmbeddingGemma-2-Transcribe-CoreAI`, which is not included | EG2-KO Reader License 1.0 |
| `eg2_ko_reader.py`, `requirements.txt` | Inference in one file | EG2-KO Reader License 1.0 |
| `eval/v1.3/*.json` | The evaluation summaries above | — |
| `assets/*.png` | Banner and benchmark images | — |
## Limits
* **Evaluated on read speech only.** Conversation, noise and overlapping speakers were not measured.
* Output is Hangul syllables and spaces only: no punctuation, digits or Latin letters. Numbers and English come out as Hangul pronunciation.
* Syllables outside the vocabulary (0.33% of Zeroth test syllables) cannot be produced and come out as `<unk>`.
* No streaming and no speaker labels.
* The whole encoder is included (1.49 GB), although only its audio part is used.
## Training data
The data sets and their licenses are listed in [`NOTICE`](NOTICE). No audio and no transcript is included in this repository.
## License
* **Reader, inference code and Core AI bundle:** [EG2-KO Reader License 1.0](LICENSE)
* Individuals (people using it for themselves) may use, modify and redistribute it for any purpose.
* Organizations (companies, institutions, associations) may use it for non-commercial purposes such as research, evaluation and teaching. Use by employees for their work is use by the organization.
* **Organizations must ask for written permission before any commercial use** (see below).
* Because of this condition it is source-available, not an OSI open-source license.
* **`encoder/`:** Apache License 2.0 (`encoder/LICENSE`, `encoder/NOTICE`). The EmbeddingGemma 2 model card asks users to follow the [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy); please follow it when you use this model too.
## Commercial use
Organizations that want to use the model in a product, a service or internal operations: email **guritopabaem@gmail.com** and start the subject line with `[Commercial]`.