|
Download README.md from Haverbex/eg2-ko-reader: direct link, hf CLI and curl.
- Browser
- Download file 6.09 kB
-
https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/README.md
- Command line
-
hf download hf://Haverbex/eg2-ko-reader/README.md
-
curl -L -o README.md https://huggingface.co/Haverbex/eg2-ko-reader/resolve/main/README.md
6.09 kB
| language: ko | |
| license: other | |
| license_name: eg2-ko-reader-license-1.0 | |
| license_link: LICENSE | |
| base_model: google/embeddinggemma-2 | |
| pipeline_tag: automatic-speech-recognition | |
| tags: [speech-recognition, ctc, korean, embeddinggemma, core-ai] | |
|  | |
| # EG2-KO Reader | |
| Korean speech-to-text on top of the EmbeddingGemma 2 (EG2) audio encoder. The encoder stays exactly as it is; a small reader (19.7M parameters, 79 MB) reads its output, 25 vectors per second, and picks a Hangul syllable or a blank for each one. An app that already ships EG2 gets Korean transcription without a second speech model. | |
| This repository holds the reader and an unmodified copy of the encoder, so one download is enough. | |
| **Try it in the browser:** [demo Space](https://huggingface.co/spaces/topabaem/eg2-ko-reader-demo) · **How it works, animated:** [project page](https://topabaem05.github.io/eg2-ko-reader-demo/) | |
| ## Use | |
| ```bash | |
| pip install -U huggingface_hub | |
| hf download Haverbex/eg2-ko-reader --local-dir eg2-ko-reader | |
| pip install -r eg2-ko-reader/requirements.txt | |
| python eg2-ko-reader/eg2_ko_reader.py audio.wav | |
| ``` | |
| From Python: | |
| ```python | |
| import sys | |
| sys.path.insert(0, "eg2-ko-reader") | |
| from eg2_ko_reader import load | |
| stt = load() # load(device="cuda") on a GPU, load(device="mps") on a Mac | |
| print(stt.transcribe("audio.wav")) # a file path, or stt.transcribe(array, sample_rate=44100) | |
| ``` | |
| * Any length. `transcribe` cuts the audio into 30 s windows that overlap by 3.84 s and joins them. | |
| * For long recordings, `stt.chunks("lecture.wav")` cuts the audio at pauses into pieces of at most 30 s and yields `{"start", "end", "text"}` for each, so you can show text as it comes. On a 55-minute lecture both ways gave the same accuracy. | |
| * Any file libsndfile reads (wav, flac, ogg, mp3). It is converted to 16 kHz mono. | |
| * The download is about 1.6 GB, of which the encoder is 1.49 GB. | |
| ## How it works | |
| 1. Audio (16 kHz) → log-mel → **EG2 audio encoder**, frozen → one 512-dimensional vector every 40 ms | |
| 2. Vectors → **reader** (6 bidirectional Transformer layers) → one of 1,035 syllables, or a blank, at every position | |
| 3. Repeated symbols merge and blanks drop out (greedy CTC). There is no decoder that writes one token at a time, so one pass gives the whole text. | |
| ## Benchmarks | |
|  | |
|  | |
|  | |
| | Test set | Utterances | Character error rate | | |
| | -- | -- | -- | | |
| | Zeroth-Korean test (10 speakers and sentences not seen in training) | 457 | **7.26%** | | |
| | FLEURS Korean test (125 utterances that cannot be normalized to Hangul excluded) | 257 | 5.59% | | |
| | Zeroth-Korean dev (speakers overlap with training, sentences do not) | 965 | 6.23% | | |
| | FLEURS Korean dev | 157 | 4.68% | | |
| * Greedy CTC with no language model, spaces not counted. These numbers come from the files in this repository. | |
| * No confidence interval for the test sets. One training seed. | |
| * Speed: 60 minutes of audio in 5.50 s with Core AI on an Apple M5 Pro (654× real time, peak memory 3.4 GB), measured with earlier weights of the same architecture. On the demo Space (ZeroGPU), 56 minutes of audio took 17.8 s on the GPU (about 190× real time, measured once on 2026-11-18). | |
| * The gauges and the banner are drawn with [dither-kit-rs](https://github.com/DeepanshuMishraa/dither-kit-rs). | |
| ## Files | |
| | Path | Contents | License | | |
| | -- | -- | -- | | |
| | `reader/model.safetensors`, `reader/vocab_ko.txt` | Reader weights (fp32) and the syllable vocabulary | EG2-KO Reader License 1.0 | | |
| | `encoder/` | Unmodified files of `google/embeddinggemma-2` (revision `914f7f89`) | Apache-2.0 | | |
| | `coreai/reader_w30_fp16.aimodel/` | Core AI reader bundle for macOS and iOS (fp16, 40 MB). Pair it with the encoder bundle `macos/eg2_audio_fp16_T3000.aimodel` of `mlboydaisuke/EmbeddingGemma-2-Transcribe-CoreAI`, which is not included | EG2-KO Reader License 1.0 | | |
| | `eg2_ko_reader.py`, `requirements.txt` | Inference in one file | EG2-KO Reader License 1.0 | | |
| | `eval/v1.3/*.json` | The evaluation summaries above | — | | |
| | `assets/*.png` | Banner and benchmark images | — | | |
| ## Limits | |
| * **Evaluated on read speech only.** Conversation, noise and overlapping speakers were not measured. | |
| * Output is Hangul syllables and spaces only: no punctuation, digits or Latin letters. Numbers and English come out as Hangul pronunciation. | |
| * Syllables outside the vocabulary (0.33% of Zeroth test syllables) cannot be produced and come out as `<unk>`. | |
| * No streaming and no speaker labels. | |
| * The whole encoder is included (1.49 GB), although only its audio part is used. | |
| ## Training data | |
| The data sets and their licenses are listed in [`NOTICE`](NOTICE). No audio and no transcript is included in this repository. | |
| ## License | |
| * **Reader, inference code and Core AI bundle:** [EG2-KO Reader License 1.0](LICENSE) | |
| * Individuals (people using it for themselves) may use, modify and redistribute it for any purpose. | |
| * Organizations (companies, institutions, associations) may use it for non-commercial purposes such as research, evaluation and teaching. Use by employees for their work is use by the organization. | |
| * **Organizations must ask for written permission before any commercial use** (see below). | |
| * Because of this condition it is source-available, not an OSI open-source license. | |
| * **`encoder/`:** Apache License 2.0 (`encoder/LICENSE`, `encoder/NOTICE`). The EmbeddingGemma 2 model card asks users to follow the [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy); please follow it when you use this model too. | |
| ## Commercial use | |
| Organizations that want to use the model in a product, a service or internal operations: email **guritopabaem@gmail.com** and start the subject line with `[Commercial]`. | |