mlboydaisuke/Tashkeel-350M-v2-LiteRT

Tashkeel-350M-v2 — LiteRT-LM (on-device Arabic diacritization)

On-device conversion of Etherll/Tashkeel-350M-v2 — an Arabic diacritization (tashkeel) fine-tune of ibm-granite/granite-4.0-h-350m trained on Misraj/Sadeed_Tashkeela — to .litertlm bundles for the LiteRT-LM runtime. All credit for the model itself goes to its author; this repo only packages it for phones and desktops.

Requires litert-lm ≥ 0.15 to run (the Mamba2 hybrid state binds through the ExecutorMetadata section). CPU backend — this architecture's selective-scan ops exceed the mobile GPU delegate's tensor-rank limit, like its base model.

file	quant	size	task-gate parity vs HF fp32
Tashkeel-350M-v2_fp16.litertlm	fp16 weights (convs/SSM fp32)	769 MB	10/10 byte-identical
Tashkeel-350M-v2_int8.litertlm	dynamic int8 (linears + embedding)	481 MB	8/10 (see note)

Which file: fp16 is the exact-parity build — greedy output byte-identical to the HF fp32 reference on all 10 diacritization probes. int8 is 38% of the size; its two misses are single-diacritic greedy flips (both probes are byte-exact on the float parent, i.e. quantization cost, not conversion error — the same 350M-scale int8 sensitivity the base model's card documents). On phones the CPU runtime unpacks fp16 to fp32 in RAM, so int8 is the practical mobile choice; use fp16 where RAM allows.

Conversion & verification

Converted with hf-to-litertlm (family recipe granite_work/convert_granite4h.py, 2026-08-25):

• The checkpoint's chat template (byte-equal to the granite base's, 6418/6418) is embedded and applied at runtime.
• The metadata start token is dropped — granite's template has no leading BOS, and at 350M scale a prepended <|end_of_text|> flips correct diacritization into garbage (measured on this checkpoint).
• Task gate: the model card's worked example plus nine undiacritized MSA probes, bundle vs HF fp32 greedy on identical rendered strings (granite_work/gate_tashkeel.py).

Performance (Apple M4 Max, litert-lm 0.16.0, -p 256 -d 256 --runs 3 --cache no, CPU)

file	prefill tok/s	decode tok/s	TTFT
fp16	661.6	58.2	0.40 s
int8	761.3	97.2	0.35 s

Usage

The model diacritizes the text you send as a user message in its trained prompt form:

pip install litert-lm
litert-lm run Tashkeel-350M-v2_fp16.litertlm --backend cpu \
  --prompt 'قم بتشكيل هذا النص :
السلام عليكم'

Output: السَّلَامُ عَلَيْكُمْ

License

apache-2.0, inherited from the source model and its granite base.