README.md

---
license: apache-2.0
base_model:
  - deepreinforce-ai/Ornith-1.0-35B
  - Qwen/Qwen3.6-35B-A3B
base_model_relation: merge
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - rocmfp4
  - strix-halo
  - gfx1151
  - mtp
  - speculative-decoding
  - qwen35moe
  - code
  - agentic
---

# Ornith-1.0-35B-MTP — ROCmFP4 (Strix Halo)

> ⚠️ **Requires the [ROCmFPX fork](https://github.com/charlie12345/ROCmFPX) of llama.cpp — this will NOT load on mainline llama.cpp.**
> ROCmFP4 is an experimental AMD FP4 quant format (ggml tensor types 100–107) that only exists in that fork.
> Targets **AMD Strix Halo / Radeon 8060S (gfx1151)**. For standard GGUF that runs anywhere, use the base repo:
> **→ [singulared/Ornith-1.0-35B-MTP-GGUF](https://huggingface.co/singulared/Ornith-1.0-35B-MTP-GGUF)** (Q8_0 / Q4_K_M, mainline llama.cpp).

[Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B) (DeepReinforce, a `qwen35moe`
agentic coder) with an embedded **MTP (`nextn`) head grafted in**, quantized to **ROCmFP4-COHERENT**
for fast self-speculative decoding on Strix Halo.

## What it is / provenance

- **Body:** `Q4_0_ROCMFP4_COHERENT` (4.70 bpw — ROCmFP4 experts + `Q6_K` token embeddings),
  quantized **directly from the BF16 original** (`deepreinforce-ai/Ornith-1.0-35B`) — a single clean
  quantization pass, no intermediate Q8 (avoids double-quant).
- **MTP head:** `blk.40.nextn.*` grafted **raw** from [`Qwen/Qwen3.6-35B-A3B`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)'s
  native head (same `qwen35moe` arch), after quantization — so `eh_proj` stays at its original **Q8_0**.
  The head's `eh_proj` precision is the throughput lever, and grafting it raw keeps it high. Ornith is a
  barely-shifted fine-tune of Qwen3.6-35B-A3B, so the head transfers cleanly (~88% draft acceptance).

## Performance (Radeon 8060S / **gfx1151**, ROCmFPX fork · 19.2 GiB)

**Recommended: Vulkan (`-dev Vulkan0`)** — best decode, which dominates a coder's token cost:

- **decode: 86.7 t/s** (MTP `n4`, `-ub 2048`) · 72 t/s without MTP · draft acceptance **0.88**
- **prefill: ~1050 t/s** (pp4096, `-ub 2048`)

### Backend / ROCm-version comparison (`-ub 2048`)

| backend | prefill (pp4096) | decode (MTP `n4`) |
|---|---:|---:|
| **Vulkan** | 1051 | **86.7** |
| ROCm/HIP @ ROCm 7.2 | 876 | 70.5 |
| ROCm/HIP @ ROCm 7.15.0a20260721 nightly *(compiled)* | **1155** | 68.1 |

- **Vulkan wins decode decisively** (86.7 vs 70.5) → **the serving default** (decode dominates a coder's token cost).
- **Compiling the fork against ROCm 7.15 gives +32% ROCm prefill** (876→1155, beating Vulkan's) — but it's
  the *slowest* decode, so it only helps prefill-bound work. The gain is **compile-time** (running a
  7.2-built binary on 7.15 libs does nothing). **⚠ ROCm 7.15 is the current [TheRock](https://github.com/ROCm/TheRock)
  nightly, not a stable release** — specifically `therock-dist-linux-gfx1151-7.15.0a20260721`
  (build 2026-07-21), built against the nightly tarball, not a packaged ROCm. Newer nightlies may differ.
- Numbers are **gfx1151-specific**; other Strix Halo parts may differ.

## Usage (ROCmFPX fork)

```bash
llama-server -m ornith-1.0-35b-MTP-ROCmFP4-COHERENT.gguf \
  -dev Vulkan0 -fa on -ngl 99 -c 131072 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.6 --alias ornith
```

The MTP head is **embedded** — no separate draft model. `--spec-draft-n-max 4` is the MoE sweet spot
(deeper drafts get rejected). Build the fork per its Strix Halo quickstart.

## Licensing & attribution

Derivative of two permissively-licensed models; both credited, their licenses apply to their parts:
- **Ornith-1.0-35B** — © DeepReinforce — **MIT** — the base weights.
- **Qwen3.6-35B-A3B** — © Alibaba Cloud / Qwen — **Apache-2.0** — the grafted MTP head.

Quantization: ROCmFP4-COHERENT (ROCmFPX fork). Not affiliated with or endorsed by DeepReinforce, Qwen, or the ROCmFPX author.