gramota-gemma4-e2b-ru

Core ML build of google/gemma-4-E2B-it (Apache-2.0, ungated), fine-tuned with a LoRA adapter for Russian proofreading and converted for the Apple Neural Engine. It is the on-device model for the mangled-word lane of Грамота, a Russian writing assistant: the app's tagger fixes what it can describe, and this model is asked only about words that are mangled past recognition («заче», «шесотдвацадсемь»). Its output is filtered so that only edits landing on those flagged words are kept.

Layout

The repo IS the on-device bundle. The app downloads these paths verbatim into Application Support/GemmaBundle/ and reads model_config.json to find the rest.

path	what
decode_dec{1..4}_s2048_v2_pal8.mlpackage	4-chunk stateful decode chain, 8-bit palettized, ctx 2048
pruned_vocab/lmhead/	lm head over the pruned vocabulary (int4)
pruned_vocab/tokenizer/	tokenizer speaking the pruned ids natively
pruned_vocab/sidecars/	token + per-layer embedding tables (int8 + fp16 scales)
sidecars/*.npy	RoPE tables (vocabulary-independent)

The token and per-layer embedding tables are kept OUT of the Core ML graphs and looked up on the CPU per token; that is what lets the chain sit on the ANE.

Vocabulary prune

The vocabulary is reduced from 262,144 to 49,344 tokens (Cyrillic, Latin, digits, punctuation, all 256 byte-fallbacks and specials), which shrinks the embedding tables from ~2.6 GB to ~0.5 GB. All byte-fallback tokens are kept, so the tokenizer cannot emit a dropped id by construction. Ids in this repo are the NEW (pruned) ids; old2new.npy / new2old.npy sit beside the artifacts.

Prefill packages included

This bundle also ships the T=512 stateful prefill packages (prefill_pre{1..4}_*.mlpackage, see model_config.json's prefill block for the exact files and call pattern). They cut long-input latency by stepping 512 tokens at a time instead of one, and hand off to the decode chain via a state-buffer memcpy (identical state shapes by construction). Short one-sentence inputs still go through the decode-stepping path; the choice is the app's, not this repo's.

Licence

Apache-2.0, inherited from google/gemma-4-E2B-it. See LICENSE and NOTICE.