eg2-ko-reader / README.md
topabaem's picture
Card: ZeroGPU demo speed (source 52d2c586e8b2)
2cbe124 verified
|
Raw History Blame Contribute Delete
6.09 kB
metadata
language: ko
license: other
license_name: eg2-ko-reader-license-1.0
license_link: LICENSE
base_model: google/embeddinggemma-2
pipeline_tag: automatic-speech-recognition
tags:
  - speech-recognition
  - ctc
  - korean
  - embeddinggemma
  - core-ai

EG2-KO Reader

EG2-KO Reader

Korean speech-to-text on top of the EmbeddingGemma 2 (EG2) audio encoder. The encoder stays exactly as it is; a small reader (19.7M parameters, 79 MB) reads its output, 25 vectors per second, and picks a Hangul syllable or a blank for each one. An app that already ships EG2 gets Korean transcription without a second speech model.

This repository holds the reader and an unmodified copy of the encoder, so one download is enough.

Try it in the browser: demo Space · How it works, animated: project page

Use

pip install -U huggingface_hub
hf download Haverbex/eg2-ko-reader --local-dir eg2-ko-reader
pip install -r eg2-ko-reader/requirements.txt
python eg2-ko-reader/eg2_ko_reader.py audio.wav

From Python:

import sys

sys.path.insert(0, "eg2-ko-reader")
from eg2_ko_reader import load

stt = load()  # load(device="cuda") on a GPU, load(device="mps") on a Mac
print(stt.transcribe("audio.wav"))  # a file path, or stt.transcribe(array, sample_rate=44100)
  • Any length. transcribe cuts the audio into 30 s windows that overlap by 3.84 s and joins them.
  • For long recordings, stt.chunks("lecture.wav") cuts the audio at pauses into pieces of at most 30 s and yields {"start", "end", "text"} for each, so you can show text as it comes. On a 55-minute lecture both ways gave the same accuracy.
  • Any file libsndfile reads (wav, flac, ogg, mp3). It is converted to 16 kHz mono.
  • The download is about 1.6 GB, of which the encoder is 1.49 GB.

How it works

  1. Audio (16 kHz) → log-mel → EG2 audio encoder, frozen → one 512-dimensional vector every 40 ms
  2. Vectors → reader (6 bidirectional Transformer layers) → one of 1,035 syllables, or a blank, at every position
  3. Repeated symbols merge and blanks drop out (greedy CTC). There is no decoder that writes one token at a time, so one pass gives the whole text.

Benchmarks

Accuracy

Speed

Weights an app adds

Test set Utterances Character error rate
Zeroth-Korean test (10 speakers and sentences not seen in training) 457 7.26%
FLEURS Korean test (125 utterances that cannot be normalized to Hangul excluded) 257 5.59%
Zeroth-Korean dev (speakers overlap with training, sentences do not) 965 6.23%
FLEURS Korean dev 157 4.68%
  • Greedy CTC with no language model, spaces not counted. These numbers come from the files in this repository.
  • No confidence interval for the test sets. One training seed.
  • Speed: 60 minutes of audio in 5.50 s with Core AI on an Apple M5 Pro (654× real time, peak memory 3.4 GB), measured with earlier weights of the same architecture. On the demo Space (ZeroGPU), 56 minutes of audio took 17.8 s on the GPU (about 190× real time, measured once on 2026-11-18).
  • The gauges and the banner are drawn with dither-kit-rs.

Files

Path Contents License
reader/model.safetensors, reader/vocab_ko.txt Reader weights (fp32) and the syllable vocabulary EG2-KO Reader License 1.0
encoder/ Unmodified files of google/embeddinggemma-2 (revision 914f7f89) Apache-2.0
coreai/reader_w30_fp16.aimodel/ Core AI reader bundle for macOS and iOS (fp16, 40 MB). Pair it with the encoder bundle macos/eg2_audio_fp16_T3000.aimodel of mlboydaisuke/EmbeddingGemma-2-Transcribe-CoreAI, which is not included EG2-KO Reader License 1.0
eg2_ko_reader.py, requirements.txt Inference in one file EG2-KO Reader License 1.0
eval/v1.3/*.json The evaluation summaries above —
assets/*.png Banner and benchmark images —

Limits

  • Evaluated on read speech only. Conversation, noise and overlapping speakers were not measured.
  • Output is Hangul syllables and spaces only: no punctuation, digits or Latin letters. Numbers and English come out as Hangul pronunciation.
  • Syllables outside the vocabulary (0.33% of Zeroth test syllables) cannot be produced and come out as <unk>.
  • No streaming and no speaker labels.
  • The whole encoder is included (1.49 GB), although only its audio part is used.

Training data

The data sets and their licenses are listed in NOTICE. No audio and no transcript is included in this repository.

License

  • Reader, inference code and Core AI bundle: EG2-KO Reader License 1.0
    • Individuals (people using it for themselves) may use, modify and redistribute it for any purpose.
    • Organizations (companies, institutions, associations) may use it for non-commercial purposes such as research, evaluation and teaching. Use by employees for their work is use by the organization.
    • Organizations must ask for written permission before any commercial use (see below).
    • Because of this condition it is source-available, not an OSI open-source license.
  • encoder/: Apache License 2.0 (encoder/LICENSE, encoder/NOTICE). The EmbeddingGemma 2 model card asks users to follow the Gemma Prohibited Use Policy; please follow it when you use this model too.

Commercial use

Organizations that want to use the model in a product, a service or internal operations: email guritopabaem@gmail.com and start the subject line with [Commercial].