Ornith-1.5-9B — LiteRT-LM

ornith-ai/Ornith-1.5-9B converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime: an int8 file and a Mixed INT4 file. The model is Ornith's post-training of Qwen/Qwen3.5-9B (its text decoder's configuration and weight names equal Qwen3.5-9B's), so these files come off the same conversion path as the Ornith-1.0-9B, Qwen3.5-4B, 2B and 0.8B conversions. Converted with litert-torch (commit 115a1360 with the Qwen3.5 hybrid patch) and tested on litert-lm 0.18.0 on a Mac Studio M4 Max, where the Mixed INT4 file decodes at 45.6 tok/s on the GPU (measured 2026-10-15); the phone status is under Performance. On litert-lm 0.17.1 both files answered a plain run ("Paris", Mac CPU, one run each).

Ornith describes Ornith-1.5 as "a major step toward building foundation models through end-to-end self-improvement", extending Ornith-1.0 (post-trained on Qwen3.5 and Gemma4) by having the model generate its own training tasks, discover strategies for solving them and improve the policy through reinforcement learning; the 9B is "the most lightweight member of the Ornith-1.5 family — a 9B dense model designed for efficient single-GPU deployment", about 19 GB in bf16. Ornith's table reports Terminal-Bench 2.1 (Terminus-2) 46.2 against Ornith-1.0-9B's 43.1 and Qwen3.5-9B's 21.3, and SWE-bench Verified 70.6 against 69.4 and 53.2, averaged over five runs (vendor-reported; not run here). Ornith recommends temperature 1.0, top-p 0.95, top-k 20 and a presence penalty of 1.5 for general tasks, and temperature 0.6 with no presence penalty for precise coding; the context is 262,144 tokens (4096 in these files). The README also names a quantized "Ornith-1.5-9B-Mobile" variant; no public repository of that name exists at the time of writing, and these files are not it.

The architecture is Qwen3.5's hybrid: 32 layers, 24 of them GatedDeltaNet linear attention and 8 full attention (one in every four), hidden size 4096, 16 attention heads with 4 KV heads of 256, linear attention with 16 key and 32 value heads of 128, a 248,320-token vocabulary with untied embeddings. The upstream checkpoint is multimodal and carries a multi-token-prediction head; these files hold the text decoder only (8,953,803,264 of the release's 9,653,104,368 parameters; the vision tower and the MTP head are not in them, the same text-only load as upstream's Qwen3_5ForCausalLM). Only the 8 attention layers keep a growing KV cache (a 4096-token budget here); the 24 linear-attention layers carry a constant-size conv and recurrent state.

Ornith-1.5-9B is a reasoning model whose assistant turn opens with a <think> block by default. These files run it with thinking off: the bundle's chat template writes the empty <think>\n\n</think> block that Ornith's own template writes when enable_thinking is false, in every assistant turn and in the generation prompt, and the replies are the answers alone. Why no thinking switch: Ornith's template writes every past assistant turn with its think block, and the switch template of the Qwen3.5-4B conversion drops it, so a multi-turn prompt would differ from the form Ornith trained on; the shipped template's renders equal Ornith's enable_thinking=False renders byte for byte in every conversation shape without tools (Usage).

File	Recipe	Size	sha256
Ornith-1.5-9B_int8.litertlm	int8 dynamic on every linear layer, int8 embedding; the convs, the delta rule and the 48 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) float; fp32 GPU activations declared	9.20 GB (9,200,832,896 bytes)	f21058268a2b66a351007a1c3268dba79387943ea27cb595e6203c3b36d05586
Ornith-1.5-9B_mixed_int4.litertlm	Mixed INT4: int4 blockwise-32 (min-max) on the linear layers, int8 on the lm_head and the embedding, the same 48 projections float; fp32 GPU activations declared	6.06 GB (6,060,357,536 bytes)	e4cad4a90869a2f6589a0f47268016e5ce5ac438ff8b22d68ff9b82b2e0183b4

On GSM8K (thinking off, 2,048 output tokens, table in Accuracy) the int8 file scored 95 and the Mixed INT4 file 92 on the Mac GPU against 94 for the bf16 model in transformers. The Mixed INT4 file is 34 % smaller. Time for a 1024-token prefill plus 256 decoded tokens at context 2048, from the rates in Performance:

Seconds	Mac M4 Max	Galaxy S26	Pixel 8a
Mixed INT4 GPU	7.28	GPU engine does not build (Performance)	not run (Performance)
int8 GPU	8.16	not run (Performance)	not run (Performance)
int8 CPU	21.47	not run (Performance)	not run (Performance)
Mixed INT4 CPU	32.37	not run (Performance)	not run (Performance)

On the Mac the Mixed INT4 file is the faster file on the GPU (10.7 % less time, decode 45.6 vs 39.6 tok/s at about the same prefill rate) and the int8 file on the CPU (the int4 file's blockwise prefill runs at 68 tok/s against 234). Not measured on iOS: each file is one TFLite section (8.57 GiB int8, 5.64 GiB Mixed INT4, the embedding table inside it), over the ~2 GiB single-section limit the Ministral-3-3B-Instruct-2512 card describes.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 8/8 on CPU and GPU, Mixed INT4 8/8 on CPU and GPU, degenerate 0; the bf16 model in transformers 8/8, and the int8 file's eight GPU answers equal its eight word for word. The int8 weights with the runtime's default fp16 GPU activations answered 0 of 8 (every reply a run of !), which is why both files declare fp32 activations.

All 7 signatures of both files are fully delegated on the Mac GPU (34,373 / 25,157 / 22,469 / 22,589 / 22,589 / 2,866 / 2,920 nodes, one partition each, no validation error). No start token: the engine adds nothing before the template's text, and at every swept prompt length its prefill count equals the transformers token count of the render. A three-turn conversation ("Give me one word that means 'very big'." / "Use it in a short sentence." / "Now make that sentence a question.") on both files and both backends chose "Huge" and kept it across the turns with no chat marker in the text, as the bf16 model did (its later sentences differ from the files' in wording). A prompt-length check (17 prompt lengths between 21 and 60 tokens, a fresh engine each): the expected word every time on the int8 file's GPU and on the Mixed INT4 file's CPU and GPU, 51 of 51; on the int8 file's CPU 15 of 17: at 23 and 24 tokens it declined politely ("I appreciate the playful request, but I can't comply …"), the same reply in three fresh processes each, where the bf16 model answered at all 17 lengths with that refusal a close second at those two (a logit margin of 1.6 and 1.1). This is consistent with the CPU backend's run-time int8 quantization of each int8 layer's input (the path the 2026-10-13 note on the Qwen3.5-4B card describes); the CPU logits themselves were not read, so the cause is not measured.

Multi-step arithmetic on the Mac CPU (six multiplication questions, three fresh processes each, thinking off, 400-token cap): the right product in 18 of 18 runs per file, every run ending by itself with no repeated line; on the Mac GPU 6 of 6 per file. With the thinking block opened by a raw <think> prefill (3 runs, CPU) both files closed the thought and answered right in 3 of 3, none at the cap. Long-output check (four prompts, greedy and the runtime's default sampling, up to 512 tokens, Mac CPU): no reply of either file looped; no int8 reply reached the cap, 3 of 8 Mixed INT4 replies did (answers longer than 512 tokens, cut mid-sentence, none repeating). Greedy CPU runs can still differ between processes (the same question gave a 192-character answer in two processes and a 291-character one in a third, all right).

Accuracy

GSM8K: test questions 0–99 with a step-by-step instruction and a final #### <number> line, the bundle's template (thinking off), greedy, at most 2,048 output tokens, a 4096-token KV budget, a fresh process per question (litert-lm 0.18.0 Python API, Mac GPU). The bf16 reference is transformers 5.14.1 on MPS with Ornith's template at enable_thinking=False (byte-identical to the bundle's render), the same prompts, greedy, 2,048 new tokens. Every reply of every run had its final line and ended by itself. IFEval was not run; Ornith's own coding and agentic benchmarks were not run here.

Configuration	GSM8K	wrong questions
bf16 reference (transformers, MPS)	94 / 100	12, 37, 41, 43, 87, 89
int8, litert-lm Mac GPU	95 / 100	3, 12, 26, 62, 89
Mixed INT4, litert-lm Mac GPU	92 / 100	3, 7, 12, 40, 62, 85, 87, 89

Question by question, the int8 file agrees with bf16 on 93 of 100 (right in both 91, right only in bf16 3, right only in the int8 file 4); the Mixed INT4 file agrees on 92 (right in both 89, right only in bf16 5, right only in the int4 file 3). The files declare fp32 activations for the GPU: with the runtime's fp16 default the int8 file answered none of the 8 check questions, for 4.5 % less time.

Usage

litert-lm run ./Ornith-1.5-9B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./Ornith-1.5-9B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm run ./Ornith-1.5-9B_mixed_int4.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./Ornith-1.5-9B_mixed_int4.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./Ornith-1.5-9B_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

All four run commands listed the two dates ("The dates in the sentence are: - May 3 - May 17"), with no think block and no control token in the output, in 37–63 s including engine init. Without --backend, run uses the CPU; without --cache, it writes a compiled-weights cache next to the model on its first CPU run (7.99 GB for int8, 4.96 GB for Mixed INT4). The bundle sets no sampler, so Ornith's recommended sampling is applied only when you pass it (run passes a sampler only when you give --top-k, --top-p, --temperature or --seed). For Android, see android-gpu.md.

The chat template (701 bytes) writes ChatML turns, reads message content as a string or as a list of text parts, and writes the empty think block into every assistant turn and the generation prompt; it has no tools branch, no thinking switch and no default system message. The runtime's template engine renders it byte for byte as transformers renders Ornith's chat_template.jinja (the file transformers loads; the repo's tokenizer config embeds a second variant that differs in where it writes past think blocks) with enable_thinking=False in six conversation shapes (a single question, with a system message, two and three turns, with history); the two tool shapes differ: this template renders no tool list, and a tool message becomes a plain tool turn. One stop token, <|im_end|> (id 248046): the repo has no generation config, and its model config lists that one end token (the Qwen3.5-9B and Ornith-1.0-9B conversions also stop on <|endoftext|>, id 248044). The tokenizer section is the repo's tokenizer.json byte for byte (Conversion notes).

Tool calling: a tool passed through the API changes nothing in the prompt (this template renders no tool list), and the model answers in text without a call. With Ornith's own # Tools block (its template's text) as the system message, the model wrote the call in Qwen3.5's XML form as text, <tool_call><function=get_weather><parameter=city>Paris …, which the runtime does not parse into a tool call and does not run (the bundle's generic model type has no parser for this form; LiteRT-LM issue 3927). A tool result sent back as a tool message was not used: the reply repeated the turn before word for word (Ornith's template wraps a tool result in a user turn as <tool_response>, which this template does not write). An app that needs Ornith's tool calls parses that text itself and feeds the result back in Ornith's form.

• One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's recurrent state: it reuses the prompt tokens the two conversations share and does not reset the state (the open report). On these two files the probe (three questions in a pirate-persona conversation, then in a conversation told to answer in one word, on the same Engine and on a fresh Engine; CPU, greedy, 3 runs) gave the same answers on both Engines, where the Qwen3.5-4B conversion's Mixed INT4 file showed the carry-over; the state is there all the same, so give each conversation its own Engine.

Performance

Mac Studio M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations, two processes per file and backend (each in its own held measurement window, the CPU ones alone on the machine), the median of the two; GPU is WebGPU on Metal. Init is the CLI's printed init time of the first iteration; in litert-lm 0.18.0 it sums overlapping init phases and reads about twice the engine build measured from the log's timestamps (on two other files of this series: a 9B Mixed INT4 GPU run printed 89.2 s for a 44.5 s build, a granite-4.1-8b int4 GPU run 7.38 s for 3.66 s; the Python API built the 9B GPU engines in 43–45 s).

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init (s)
int8 GPU	608.97	39.57	1.707	89.74
Mixed INT4 GPU	615.54	45.58	1.686	89.28
int8 CPU	234.14	14.98	4.443	91.70
Mixed INT4 CPU	68.06	14.78	15.122	50.87

Memory during the benchmark on the CPU (--cache no): int8 50.5 GB, Mixed INT4 30.1 GB; during a GPU run: 9.7 GB and 8.4 GB. A three-signature variant of the Mixed INT4 file (prefill 1024 / 256 / 1, the same weights, 50,244,960 bytes smaller) also answers the 8 questions 8/8 on the Mac GPU with the same replies as the shipped file; on a Galaxy S26 neither it nor the shipped file builds a GPU engine (below).

Phones (Android 16, the prebuilt litert_lm_advanced_main of 2026-09-18 from LiteRT-LM's public binary bucket):

• Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno 840, MemTotal 11,389,756 kB): the GPU engine of the Mixed INT4 file does not build on a phone of this memory class. On a platform device-lab Galaxy S26 (SM-S942U1, MemTotal 11,389,764 kB; 2026-10-16) the Ornith-1.0-9B files of the same architecture and recipe (that Mixed INT4 file is 294,912 bytes larger than this one) failed four sessions, the six-signature file and a three-signature variant, each in a benchmark run and in a prompt run, during the engine build: while the OpenCL delegate was taking the signatures the low-memory killer removed other apps 376 to 473 times, the kernel logged thrashing, and each job failed 3 to 6 minutes after the engine creation started, with no output. The benchmark session of this file on that unit (session-eed400ba) ended the same way: the delegate took prefill_1024, prefill_256 and prefill_64, the runtime log ended with an allocator failure (Can not populate more pages for size class 96), and the low-memory killer had removed other apps 348 times. Our own phone was not tried with these files (a 7.9 GB LiteRT-LM process has already restarted its framework once). The int8 file was not started on the phone's GPU: after a sibling 8B-class int8 file restarted this phone during its GPU engine warm-up on 2026-10-14, no 8B-class int8 engine is started on it. The CPU was not run: by the earlier granite-4.0-micro measurements on the same phone (a CPU peak of 1.5 × the int8 file and 2.0 × the int4 file) these files would need 13.8 GB and 12.1 GB against the phone's 11.66 GB.
• Pixel 8a (Tensor G3, Mali-G715, MemTotal 7,754,700 kB): not run, and GPU execution on a Mali phone is not verified for these files. No Qwen3.5 file on the Hub, the 0.8B included, built a GPU engine on this 7.75 GB Mali phone under our memory guard (a run ends at the first MemAvailable reading under 400,000 kB; 0 of 5 engines on 2026-10-12, each file accepted whole by the OpenCL delegate while the engine build exhausts memory), and these files are the same architecture at 2–9 × the size; on the CPU the same multipliers give 13.8 GB and 12.1 GB against 7.94 GB.
• Not run on an NPU: these dynamic-range int8 and blockwise int4 graphs have no HTP path (the NPU path takes static-range or float graphs; the GatedDeltaNet layers' ops come on top).

Conversion notes

• Source and stack: ornith-ai/Ornith-1.5-9B at revision 489cb979 (every file checked against the Hub's record); litert-torch at commit 115a1360 with the Qwen3.5 hybrid patch of hf-to-litertlm (the 2026-08-21 version: the GatedDeltaNet cache layers, the rank-4 chunk kernel and head interleave of the 4B's notes), transformers 5.14.1, torch 2.12.1, ai-edge-quantizer 0.8.0; packed and checked with litert-lm 0.18.0. qwen35_work/convert_qwen35_9b.sh in hf-to-litertlm is the one command (the same script bytes as the Qwen3.5-9B and Ornith-1.0-9B runs, with this repo's id and revision), followed by the tokenizer step below; the re-run of both rebuilt all four sections of both files byte for byte (only the packer's uuid and timestamp differ, so compare sections), and the script as published rebuilt the Ornith-1.0-9B files from their pinned checkpoint on 2026-10-16 (one command, the environment of its header, 18 minutes on a 128 GB Mac, all four sections byte for byte).
• Tokenizer section: the export bundles the tokenizer as transformers writes it for the Qwen2Tokenizer class, whose pre-tokenizer has no \p{M} class, while this repo's tokenizer.json has one; the two give different ids on texts with combining marks (Hindi 49 vs 36 tokens, Thai 24 vs 12, Tamil 29 vs 16, Arabic with harakat 30 vs 29) and the same ids on English, code, CJK and every prompt of the checks here. After the script, the tokenizer section of each file was replaced with the repo's tokenizer.json (tools/tokenizer_parity/swap_tokenizer_section.py in hf-to-litertlm; the other three sections byte-identical), and the runtime's tokenization equals the tokenizers library's on 174 of 174 probes per file on litert-lm 0.18.0 and 0.17.1.
• Export: the text decoder through litert-torch's lightweight conversion (constants of 1e8 bytes and more stay lazy), 6 prefill signatures (1024, 256, 64, 16, 4, 1) plus decode over a 4096-position KV budget, 64 state buffers (8 KV pairs and 48 linear-attention states); 17 minutes and a 49.4 GB peak on a 128 GiB Mac, each quantize step 35–37 GB. Run one export at a time.
• Quantization after the export: int8 = dynamic int8 on every FULLY_CONNECTED plus an int8 embedding, with the 48 decay/gate projections of the 24 linear-attention layers kept float (the CPU backend quantizes each int8 layer's input at run time, and those heads do not take that noise: the 2026-10-13 note on the Qwen3.5-4B card); Mixed INT4 = int4 blockwise-32 min-max on the linear layers, the lm_head and the embedding int8, the same 48 float. The int4 file's 216,006,656 block scales are all normal floats (none zero or subnormal), so no scale floor was needed.
• Metadata: the thinking-off template above (the only field the metadata step changed, every other section byte-identical); no start token; stop id 248046; generic_model; fp32 GPU activations declared; max_num_tokens 4096.

License and changes

Ornith-1.5-9B is released under the MIT License (the repo's metadata; the repo carries no LICENSE file, its metadata's license link points to a page that does not exist, and its README has no license sentence). The base model, Qwen/Qwen3.5-9B, is Apache-2.0. Changes from the original work: the text decoder of the multimodal checkpoint exported (the vision tower and the MTP head not loaded), weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above with the 48 decay/gate projections float; the chat template replaced by the thinking-off template (byte-identical renders to Ornith's enable_thinking=False form in the conversation shapes without tools); the repo's tokenizer.json as the tokenizer section; stop id 248046, fp32 GPU activations and a 4096-token KV budget declared. This repository is a community conversion and is not affiliated with Ornith (the ornith-ai organization) or the Ornith team.