Ornith-1.0-9B — LiteRT-LM

ornith-ai/Ornith-1.0-9B converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime: an int8 file and a Mixed INT4 file. The model is Ornith's post-training of Qwen/Qwen3.5-9B (its text decoder's configuration equals Qwen3.5-9B's key for key), so these files come off the same conversion path as the Qwen3.5-4B, 2B and 0.8B conversions. Converted with litert-torch (commit 115a1360 with the Qwen3.5 hybrid patch) and tested on litert-lm 0.18.0 on a Mac Studio M4 Max, where the Mixed INT4 file decodes at 44.9 tok/s on the GPU (measured 2026-10-15); the phone status is under Performance. On litert-lm 0.17.1 both files answered a plain run ("Paris", Mac CPU, one run each).

Ornith describes Ornith-1.0 as "a self-improving family of open-source models for agentic coding", and the 9B as "the most lightweight member of the Ornith family, designed for efficient single-GPU deployment": a dense model of about 19 GB in bf16, post-trained on Qwen 3.5 with a training framework that uses RL to learn the scaffold that drives its solution rollouts as well as the solutions. Ornith's table reports Terminal-Bench 2.1 (Terminus-2) 43.1 against Qwen3.5-9B's 21.3 and SWE-bench Verified 69.4 against 53.2 (vendor-reported; not run here). Ornith recommends temperature 0.6, top-p 0.95 and top-k 20 (temperature 1.0 to reproduce its benchmark setup) and a 262,144-token context (4096 in these files).

The architecture is Qwen3.5's hybrid: 32 layers, 24 of them GatedDeltaNet linear attention and 8 full attention (one in every four), hidden size 4096, 16 attention heads with 4 KV heads of 256, linear attention with 16 key and 32 value heads of 128, a 248,320-token vocabulary with untied embeddings. The upstream checkpoint is multimodal; these files hold the text decoder only (8,953,803,264 of the release's 9,409,813,744 parameters; the vision tower is not in them, the same text-only load as upstream's Qwen3_5ForCausalLM). Only the 8 attention layers keep a growing KV cache (a 4096-token budget here); the 24 linear-attention layers carry a constant-size conv and recurrent state.

Ornith-1.0-9B is a reasoning model whose assistant turn opens with a <think> block by default. These files run it with thinking off: the bundle's chat template writes the empty <think>\n\n</think> block that Ornith's own template writes when enable_thinking is false, in every assistant turn and in the generation prompt, and the replies are the answers alone. Why no thinking switch: Ornith's template writes every past assistant turn with its think block, and the switch template of the Qwen3.5-4B conversion drops it, so a multi-turn prompt would differ from the form Ornith trained on; the shipped template's renders equal Ornith's enable_thinking=False renders byte for byte in every conversation shape without tools (Usage).

File	Recipe	Size	sha256
Ornith-1.0-9B_int8.litertlm	int8 dynamic on every linear layer, int8 embedding; the convs, the delta rule and the 48 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) float; fp32 GPU activations declared	9.20 GB (9,201,127,808 bytes)	b42d5c1e7393b26b2006e79d2f339b2051c6bae0a7e49f48981f7bcbe37e122f
Ornith-1.0-9B_mixed_int4.litertlm	Mixed INT4: int4 blockwise-32 (min-max) on the linear layers, int8 on the lm_head and the embedding, the same 48 projections float; fp32 GPU activations declared	6.06 GB (6,060,652,448 bytes)	bf01eff637705a82c1a7dab65c922b9bab5a8fdb445ffeb6ac6aa67c860ee80c

On GSM8K (thinking off, 2,048 output tokens, table in Accuracy) the int8 file scored 97 and the Mixed INT4 file 95 on the Mac GPU against 95 for the bf16 model in transformers. The Mixed INT4 file is 34 % smaller. Time for a 1024-token prefill plus 256 decoded tokens at context 2048, from the rates in Performance:

Seconds	Mac M4 Max	Galaxy S26	Pixel 8a
Mixed INT4 GPU	7.45	GPU engine does not build (Performance)	not run (Performance)
int8 GPU	8.17	not run (Performance)	not run (Performance)
int8 CPU	21.38	not run (Performance)	not run (Performance)
Mixed INT4 CPU	31.49	not run (Performance)	not run (Performance)

On the Mac the Mixed INT4 file is the faster file on the GPU (8.8 % less time, decode 44.9 vs 40.0 tok/s at about the same prefill rate) and the int8 file on the CPU (the int4 file's blockwise prefill runs at 68 tok/s against 227). Not measured on iOS: each file is one TFLite section (8.57 GiB int8, 5.64 GiB Mixed INT4, the embedding table inside it), over the ~2 GiB single-section limit the Ministral-3-3B-Instruct-2512 card describes.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 8/8 on CPU and GPU, Mixed INT4 8/8 on CPU and GPU, degenerate 0; the bf16 model in transformers 8/8. The int8 weights with the runtime's default fp16 GPU activations answered 0 of 8 (every reply a run of !), which is why both files declare fp32 activations.

All 7 signatures of both files are fully delegated on the Mac GPU (34,373 / 25,157 / 22,469 / 22,589 / 22,589 / 2,866 / 2,920 nodes, one partition each, no validation error). No start token: the engine adds nothing before the template's text, and at every swept prompt length its prefill count equals the transformers token count of the render. A three-turn conversation ("Give me one word that means 'very big'." / "Use it in a short sentence." / "Now make that sentence a question.") on both files and both backends kept the word across the turns with no chat marker in the text; three of the four runs gave the bf16 model's three replies word for word ("Colossal" / "The colossal statue dominated the entire city skyline." / "Did the colossal statue dominate the entire city skyline?"), the Mixed INT4 file on the GPU chose "Enormous". A prompt-length check (17 prompt lengths between 21 and 60 tokens, a fresh engine each, both files, CPU and GPU): the expected word every time, 68 of 68.

Multi-step arithmetic on the Mac CPU (six multiplication questions, three fresh processes each, thinking off, 400-token cap): the right product in 18 of 18 runs per file, every run ending by itself with no repeated line; on the Mac GPU 6 of 6 per file. With the thinking block opened by a raw <think> prefill (3 runs, CPU) both files closed the thought and answered right in 3 of 3 (the Mixed INT4 runs reached the 400-token cap after the answer). Long-output check (four prompts, greedy and the runtime's default sampling, up to 512 tokens, Mac CPU): no reply of either file looped; 3 of 8 int8 replies and 4 of 8 Mixed INT4 replies reached the cap, none repeating a sentence. Greedy CPU runs can still differ between processes ("Paris." in one process, "Paris" in the next on the same prompt).

Accuracy

GSM8K: test questions 0–99 with a step-by-step instruction and a final #### <number> line, the bundle's template (thinking off), greedy, at most 2,048 output tokens, a 4096-token KV budget, a fresh process per question (litert-lm 0.18.0 Python API, Mac GPU). The bf16 reference is transformers 5.14.1 on MPS with Ornith's template at enable_thinking=False (byte-identical to the bundle's render), the same prompts, greedy, 2,048 new tokens. Every reply of every run had its final line. IFEval was not run; Ornith's own coding-agent benchmarks were not run here.

Configuration	GSM8K	wrong questions
bf16 reference (transformers, MPS)	95 / 100	12, 37, 73, 87, 89
int8, litert-lm Mac GPU	97 / 100	12, 37, 73
Mixed INT4, litert-lm Mac GPU	95 / 100	7, 62, 75, 89, 93

Question by question, the int8 file agrees with bf16 on 98 of 100 (right in both 95, right only in the int8 file 2); the Mixed INT4 file agrees on 92 (right in both 91, right only in bf16 4, right only in the int4 file 4). The files declare fp32 activations for the GPU: with the runtime's fp16 default the int8 file printed WebGPU validation errors and answered none of the 8 check questions, for 2.8 % less time.

Usage

litert-lm run ./Ornith-1.0-9B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./Ornith-1.0-9B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm run ./Ornith-1.0-9B_mixed_int4.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./Ornith-1.0-9B_mixed_int4.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./Ornith-1.0-9B_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

All four run commands listed the two dates ("The dates extracted from the sentence are: … May 3 … May 17"), with no think block and no control token in the output, in 37–57 s including engine init. Without --backend, run uses the CPU; without --cache, it writes a compiled-weights cache next to the model on its first CPU run (7.99 GB for int8, 4.96 GB for Mixed INT4). The bundle sets no sampler; run passes one only when you give --top-k, --top-p, --temperature or --seed. For Android, see android-gpu.md.

The chat template (701 bytes) writes ChatML turns, reads message content as a string or as a list of text parts, and writes the empty think block into every assistant turn and the generation prompt; it has no tools branch, no thinking switch and no default system message. The runtime's template engine renders it byte for byte as transformers renders Ornith's template with enable_thinking=False in six conversation shapes (a single question, with a system message, two and three turns, with history); the two tool shapes differ: this template renders no tool list, and a tool message becomes a plain tool turn. Stop tokens <|endoftext|> (id 248044) and <|im_end|> (id 248046), from Ornith's generation config. The tokenizer section is the repo's tokenizer.json byte for byte (a byte-level BPE with 33 added tokens, no BOS).

Tool calling: a tool passed through the API changes nothing in the prompt (this template renders no tool list), and the model answers in text without a call. With Ornith's own # Tools block (its template's text) as the system message, the model wrote the call in Qwen3.5's XML form as text, <tool_call><function=get_weather><parameter=city>Paris …, which the runtime does not parse into a tool call and does not run (the bundle's generic model type has no parser for this form; LiteRT-LM issue 3927); a tool result sent back as a tool message was used in the answer. An app that needs Ornith's tool calls parses that text itself.

• One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's recurrent state: it reuses the prompt tokens the two conversations share and does not reset the state (the open report). On these two files the probe (three questions in a pirate-persona conversation, then in a conversation told to answer in one word, on the same Engine and on a fresh Engine; CPU, greedy, 3 runs) gave the same answers on both Engines, where the Qwen3.5-4B conversion's Mixed INT4 file showed the carry-over; the state is there all the same, so give each conversation its own Engine.

Performance

Mac Studio M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations, two processes per file and backend (each in its own held measurement window, the CPU ones alone on the machine), the median of the two; GPU is WebGPU on Metal. Init is the CLI's printed init time of the first iteration; in litert-lm 0.18.0 it sums overlapping init phases and reads about twice the engine build measured from the log's timestamps (on two other files of this series: a 9B Mixed INT4 GPU run printed 89.2 s for a 44.5 s build, a granite-4.1-8b int4 GPU run 7.38 s for 3.66 s; the Python API built the 9B GPU engines in 43–45 s).

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init (s)
int8 GPU	581.39	39.99	1.789	87.27
Mixed INT4 GPU	585.36	44.93	1.774	93.33
int8 CPU	227.37	15.17	4.586	90.95
Mixed INT4 CPU	68.44	15.77	15.049	58.73

Memory during the benchmark on the CPU (--cache no): int8 50.5 GB, Mixed INT4 30.1 GB; during a GPU run: 9.3 GB and 8.4 GB. A three-signature variant of the Mixed INT4 file (prefill 1024 / 256 / 1, the same weights, 50,244,960 bytes smaller) also answers the 8 questions 8/8 on the Mac GPU; on a Galaxy S26 neither it nor the shipped file builds a GPU engine (below).

Phones (Android 16, the prebuilt litert_lm_advanced_main of 2026-09-18 from LiteRT-LM's public binary bucket):

• Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno 840, MemTotal 11,389,756 kB): the GPU engine of the Mixed INT4 file does not build on a phone of this memory class. On a platform device-lab Galaxy S26 of the same model line (SM-S942U1, MemTotal 11,389,764 kB; 2026-10-16) four sessions, the shipped six-signature file and the three-signature variant, each in a benchmark run and in a prompt run, ended during the engine build: while the OpenCL delegate was taking the signatures the low-memory killer removed other apps 376 to 473 times, the kernel logged thrashing, and each job failed 3 to 6 minutes after the engine creation started, with no output (the build had reached the second prefill signature at most; the three-signature prompt run had delegated all four of its signatures and ended two seconds later). Our own phone was not tried with these files after that (a 7.9 GB LiteRT-LM process has already restarted its framework once). The int8 file was not started on the phone's GPU: after a sibling 8B-class int8 file restarted this phone during its GPU engine warm-up on 2026-10-14, no 8B-class int8 engine is started on it. The CPU was not run: by the earlier granite-4.0-micro measurements on the same phone (a CPU peak of 1.5 × the int8 file and 2.0 × the int4 file) these files would need 13.8 GB and 12.1 GB against the phone's 11.66 GB.
• Pixel 8a (Tensor G3, Mali-G715, MemTotal 7,754,700 kB): not run, and GPU execution on a Mali phone is not verified for these files. No Qwen3.5 file on the Hub, the 0.8B included, built a GPU engine on this 7.75 GB Mali phone under our memory guard (a run ends at the first MemAvailable reading under 400,000 kB; 0 of 5 engines on 2026-10-12, each file accepted whole by the OpenCL delegate while the engine build exhausts memory), and these files are the same architecture at 2–9 × the size; on the CPU the same multipliers give 13.8 GB and 12.1 GB against 7.94 GB.
• Not run on an NPU: these dynamic-range int8 and blockwise int4 graphs have no HTP path (the NPU path takes static-range or float graphs; the GatedDeltaNet layers' ops come on top).

Conversion notes

• Source and stack: ornith-ai/Ornith-1.0-9B at revision 83dc1f5e (every file checked against the Hub's record); litert-torch at commit 115a1360 with the Qwen3.5 hybrid patch of hf-to-litertlm (the 2026-08-21 version: the GatedDeltaNet cache layers, the rank-4 chunk kernel and head interleave of the 4B's notes), transformers 5.14.1, torch 2.12.1, ai-edge-quantizer 0.8.0; packed and checked with litert-lm 0.18.0. qwen35_work/convert_qwen35_9b.sh in hf-to-litertlm is the one command (the Qwen3.5-9B recipe run with this repo's id and revision); run as published on 2026-10-16 (one command, the environment of its header, 18 minutes on a 128 GB Mac), it rebuilt all four sections of both files byte for byte from the pinned checkpoint, as the conversion run of 2026-10-15 behind the shipped files had (only the packer's uuid and timestamp differ, so compare sections).
• Export: the text decoder through litert-torch's lightweight conversion (constants of 1e8 bytes and more stay lazy), 6 prefill signatures (1024, 256, 64, 16, 4, 1) plus decode over a 4096-position KV budget, 64 state buffers (8 KV pairs and 48 linear-attention states); 20 minutes and a 49.4 GB peak on a 128 GiB Mac, each quantize step 36.5 GB. Run one export at a time.
• Quantization after the export: int8 = dynamic int8 on every FULLY_CONNECTED plus an int8 embedding, with the 48 decay/gate projections of the 24 linear-attention layers kept float (the CPU backend quantizes each int8 layer's input at run time, and those heads do not take that noise: the 2026-10-13 note on the Qwen3.5-4B card); Mixed INT4 = int4 blockwise-32 min-max on the linear layers, the lm_head and the embedding int8, the same 48 float. The int4 file's 216,006,656 block scales are all normal floats (none zero or subnormal), so no scale floor was needed.
• Metadata: the thinking-off template above (the only field the metadata step changed, every other section byte-identical); no start token; stop ids 248044 and 248046; generic_model; fp32 GPU activations declared; max_num_tokens 4096.

License and changes

Ornith-1.0-9B is released under the MIT License (the repo's metadata and README: "MIT licensed, globally accessible, and free from regional limitations"; the repo carries no LICENSE file, and its metadata's license link points to the old deepreinforce-ai path, which redirects to a page that does not exist). The base model, Qwen/Qwen3.5-9B, is Apache-2.0. Changes from the original work: the text decoder of the multimodal checkpoint exported (the vision tower not loaded), weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above with the 48 decay/gate projections float; the chat template replaced by the thinking-off template (byte-identical renders to Ornith's enable_thinking=False form in the conversation shapes without tools); stop ids 248044 and 248046, fp32 GPU activations and a 4096-token KV budget declared. This repository is a community conversion and is not affiliated with Ornith (the ornith-ai organization) or the DeepReinforce team.