Hugging Face's logo

mlboydaisuke
/
Mordant-3B-Think-LiteRT

Text Generation
LiteRT
LiteRT-LM
English
on-device
granite
reasoning
image-prompt
Model card Files
xet
 Community
Mordant-3B-Think-LiteRT / README.md
mlboydaisuke's picture
mlboydaisuke
Upload README.md with huggingface_hub
b6c51bc verified
preview code
|
Raw
History Blame Contribute Delete
2.61 kB
metadata 
license: apache-2.0
base_model: Kezmark/Mordant-3B-Think
base_model_relation: quantized
tags:
  - litert
  - litert-lm
  - on-device
  - granite
  - reasoning
  - image-prompt
language:
  - en
pipeline_tag: text-generation

Mordant-3B-Think — LiteRT-LM (on-device)

On-device conversion of Kezmark/Mordant-3B-Think — a full fine-tune of ibm-granite/granite-4.1-3b for AI image-generation prompt composition with chain-of-thought reasoning — to a .litertlm bundle for the LiteRT-LM runtime. All credit for the model itself goes to its author; this repo only packages it for phones and desktops.

Requires litert-lm ≥ 0.16 to run.

file quant size
Mordant-3B-Think_int8.litertlm dynamic int8 (linears + embedding) 3.76 GB

Conversion & verification

Converted with one command by hf-to-litertlm (python scripts/convert.py Kezmark/Mordant-3B-Think, 2026-08-25):

  • The finetune's own chat template — a thinking-form template that opens the assistant turn with <think> — is embedded verbatim (byte-equal to the checkpoint's chat_template.jinja, 1474/1474).
  • The spurious metadata start token is dropped: this family declares bos == eos == <|end_of_text|> and its template never renders a leading BOS, so an engine-prepended start token reads as "this document already ended" — measured on this checkpoint, it flips HF bf16 greedy output into a code-fence loop.
  • Reduced 7-signature prefill ladder + externalized embedder (the ≥3B ship shape; the full 11-signature ladder is killed by iOS at Metal init on this family).
  • Quality gate: 8/8 on the 8-question sanity gate (think-aware budget), non-degenerate.

Performance (Apple M4 Max, litert-lm 0.16.0, -p 256 -d 256 --runs 3 --cache no)

backend prefill tok/s decode tok/s TTFT
CPU 97.3 20.2 2.68 s
GPU 1129 71.9 0.24 s

Usage 

pip install litert-lm
litert-lm run Mordant-3B-Think_int8.litertlm \
  --prompt "A cat sitting on a windowsill at sunset" --max-num-tokens 4096

The model answers with a <think>…</think> block followed by the composed image prompt — budget generation length accordingly. On Android, load the bundle in an app embedding the LiteRT-LM engine (e.g. Google AI Edge Gallery-style hosts).

License

apache-2.0, inherited from the source model and its granite-4.1 base.