Ornith-1.5-35B-A3B — hipfire

hipfire quantization of ornith-ai/Ornith-1.5-35B-A3B: a 35B-A3B hybrid MoE (30 linear-attention + 10 full-attention layers, 256 experts, top-8) with a vision tower and a multi-token-prediction draft head.

File	Size	What
ornith-1.5-35b-a3b.mq4	17.7 GiB	Text trunk, MQ4G256V2 (qt44)
ornith-1.5-35b-a3b.mq4r	17.4 GiB	Text trunk, uniform MQ4G256V2 (qt44), zero qt13
ornith-1.5-35b-a3b.mtp	462 MiB	MTP draft head — replacement, see below
ornith-1.5-35b-a3b.vl	865 MiB	Vision tower, F16

Needs ~22 GB of VRAM with the default Q8 KV cache.

Runtime requirement for .mq4r: use hipfire PR #664 (commit 0b368cf17) or a later release containing it. Earlier runtimes can select an invalid MQ4G256V2 MoE gate route on gfx1100 and emit corrupted output. The PR is open for review and is not merged as of this card revision.

Renamed 2026-08-25. The artifacts were published briefly as ornith1.5-* and are now ornith-1.5-* — repo id included — matching the hyphenated family spelling used everywhere else. Only the filenames changed — the weights are byte-identical (same LFS hashes, nothing re-uploaded). If you downloaded before the rename, nothing breaks: ornith1.5:35b-a3b stays a registry alias for the canonical ornith-1.5:35b-a3b, and hipfire run ornith1.5:35b-a3b still finds the file already on your disk. Renaming your local copy to ornith-1.5-35b-a3b.mq4 also works and avoids a re-download.

The MTP head is NOT the one from the base model

This repository ships a third-party trained MTP head, not Ornith 1.5's. That is a deliberate substitution and you should know about it.

The head in the official Ornith 1.5 release drafts poorly. Its mtp.* tensors carry a standard deviation of exactly 0.0200 — the config's initializer_range — with clean Gaussian statistics, and it is discussed upstream in ornith-ai/Ornith-1.5-35B-A3B discussion #10, which reports ~13% acceptance. We measured the same behaviour independently in two engines: hipfire and llama.cpp both accept the first drafted token at 0.82–0.93 and then collapse (hipfire 0.29 at depth 2, llama.cpp 0.15). With that head, speculative decoding is slower than plain autoregressive decode.

So the shipped .mtp is built from shisa-ai/Ornith-1.5-35B-A3B-MTP-ONLY, a head warm-started from Qwen3.6's MTP weights and KL-distilled against Ornith 1.5 hidden states.

Per-position draft acceptance, gfx1151, greedy, code prompt:

Head	depth-1	depth-2	depth-3
Ornith 1.5 native head	0.823	0.292	0.179
This head	0.877	0.620	0.528

Speculative decoding is lossless with respect to the trunk regardless of head quality — the trunk verifies every drafted token — so a weak head costs throughput, never correctness.

If you want the vendor's original head instead, extract it from the base checkpoint yourself; it is not redistributed here.

Throughput

Measured on a Strix Halo (gfx1151, Radeon 8060S), 256 generated tokens, greedy, reasoning off, Q8 KV, code-editing prompt. Medians of repeated runs on an otherwise idle machine.

Configuration	tok/s	τ	accept
Autoregressive (no speculation)	~70	—	—
MTP, K=3	77.4	2.5	0.61
MTP + ngram-modulated drafting	~150	5.43	0.937

The ngram-modulated arm is the large win — roughly 2.1× over autoregressive decode. It engages only when reasoning is disabled (max_think_tokens = 1).

Numbers are hardware- and prompt-specific. Prose prompts accept far fewer drafted tokens than code; treat these as an upper band, not a guarantee.

MQ4R autoregressive validation

The .mq4r artifact was exercised through hipfire's native daemon path on an RX 7900 XTX (gfx1100) and Radeon AI PRO R9700 (gfx1201), with Q8 VMM KV, speculation and reasoning off. The hard pagoda/HTML prompt produced coherent HTML on both architectures with no token attractor. Redline's stable prefill/decode checks and AQL shadow parity passed on gfx1100; PM4 shadow parity passed on gfx1201. Gfx1100 PM4 still falls back to HIP for the scratch-using MQ4G256V2 residual GEMV.

For a byte-identical merge-sort prompt (MD5 ec4f982ad6a442261af3e329b9b947bd), three measured runs after three warmups, 128 generated tokens:

HIPFIRE_VERIFY_GRAPH=0 was fixed for both arms. Artifact MD5s were .mq4r=33e4f6bc2e48a527fbfbf8a101ef44b0 and .mq4=f6cb95300d29c0c4c10d017180c12ab8; benchmark binary MD5s were hipfire=436ae74c1a8c78d484e2fb5a307ed61b and daemon=71b36e753ae134947c263595177ead09.

GPU	Artifact	Decode median	Samples (tok/s)
RX 7900 XTX (gfx1100)	.mq4r	169.8	169.9, 169.8, 169.8
RX 7900 XTX (gfx1100)	.mq4	155.4	155.4, 155.3, 155.4
Radeon AI PRO R9700 (gfx1201)	.mq4r	179.0	176.7, 179.0, 180.6
Radeon AI PRO R9700 (gfx1201)	.mq4	58.2	58.7, 58.2, 58.2

These are two different quantization recipes, not a one-to-one kernel comparison: .mq4 retains its graded MQ6/Q8 tiers, while .mq4r is the new uniform MQ4G256V2 trunk. Treat the numbers as an artifact throughput gap under this fixture, not a performance regression or a general quality claim.

Quantization

The original .mq4 trunk is a graded MQ4G256V2 (qt44) artifact: 4-bit weights in 256-element groups with an FWHT rotation and two fp16 scale/zero pairs per group. Attention tensors in the first and last two layers are promoted to MQ6G256V2; lm_head, embeddings, MoE routers and shared-expert gates retain their fixed tiers.

The .mq4r trunk was converted from ornith-ai/Ornith-1.5-35B-A3B@10fbf86fed7ecee4a061f8b499a618f46001cac1 with:

hipfire-quantize --input Ornith-1.5-35B-A3B \
  --output ornith-1.5-35b-a3b.mq4r \
  --format mq4 --no-q8-router --uniform

Its 21,093-tensor census is qt44=20,871, qt3=31, qt1=191, qt13=0, qt15=0. The artifact is 18,700,570,368 bytes; SHA-256 84103fcc8ade42aa2ac8ec01176df7a4ead5e94810597c9fae2f6763152a3ac6.

The draft head is qt44 with a Q8 lm_head_draft compressed to a 16,384-token draft vocabulary (the trunk's own lm_head still verifies against the full 248,320-token vocabulary, so the compression cannot change what is committed).

Usage

hipfire run ornith-1.5:35b-a3b
hipfire run ornith-1.5:35b-a3b-mq4r

HIPFIRE_MTP_K=3 is the measured sweet spot on gfx1151 (K=4 75.5, K=5 72.3, K=6 73.5 tok/s). Recommended sampling follows the base model: temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5.

Licensing

This repository is mixed-licence. See NOTICE for the full chain.

• Trunk and vision tower derive from ornith-ai/Ornith-1.5-35B-A3B, which declares MIT in its repository metadata. Upstream ships no LICENSE file and its own license_link returned 404 when this artifact was built; the canonical MIT terms are reproduced in LICENSE-MIT for reference, and copyright remains with the upstream authors.
• MTP draft head derives from shisa-ai/Ornith-1.5-35B-A3B-MTP-ONLY (Apache-2.0, LICENSE-APACHE-2.0), itself a modified derivative of the MTP head from Qwen/Qwen3.6-35B-A3B (Apache-2.0).

hipfire's contribution is quantization only: conversion to qt44 with FWHT rotation, the original trunk's selective MQ6/Q8 tiers, the uniform MQ4R recipe, and the compressed draft vocabulary. No warranty; quantization is lossy.