---
license: mit
language:
  - en
base_model: deepreinforce-ai/Ornith-1.0-9B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - rocmfp4
  - rocmfpx
  - quantized
  - imatrix
  - amd
  - rocm
  - strix-halo
  - agent
  - hal0
---

# Ornith-1.0-9B — ROCmFPX quants for AMD Strix Halo

ROCmFPX-family GGUF quantizations of [deepreinforce-ai/Ornith-1.0-9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B), produced for AMD Strix Halo (Ryzen AI Max, gfx1151) and the [hal0](https://hal0.dev) home inference platform.

> ⚠️ **These files require the [Hal0ai/Hal0_ROCmFPX](https://github.com/Hal0ai/Hal0_ROCmFPX) llama.cpp fork** (or the `ghcr.io/hal0ai/hal0-rocmfpx` container image that hal0 uses). Stock llama.cpp will reject the tensor types (`invalid ggml type 101`).

## Files

| File | Quant | BPW | Size | Notes |
|---|---|---|---|---|
| `Ornith-1.0-9B-Q4_0_ROCMFP4_STRIX_LEAN.gguf` | Q4_0_ROCMFP4_STRIX_LEAN | 4.42 | 4.96 GB | Size-biased Strix recipe, Q5_K token embeddings — best size/speed |
| `Ornith-1.0-9B-Q4_0_ROCMFP4_STRIX.gguf` | Q4_0_ROCMFP4_STRIX | 4.54 | 5.09 GB | Quality-biased attention K/V recipe |
| `Ornith-1.0-9B-Q6_0_ROCMFPX_STRIX_QUALITY.gguf` | Q6_0_ROCMFPX_STRIX_QUALITY | 7.50 | 8.41 GB | FP6 bulk + Q8 protected tensors — near-lossless daily driver |
| `Ornith-1.0-9B-Q8_0_ROCMFPX_AGENT.gguf` | Q8_0_ROCMFPX_AGENT | 8.41 | 9.42 GB | Agent profile: protects embeddings, attn Q/K/V/O and select FFN tensors for tool-calling / JSON fidelity |
| `mmproj-BF16.gguf` | BF16 | — | 0.92 GB | Vision projector (Ornith is multimodal) — load alongside any quant |
| `imatrix_unsloth.gguf_file` | — | — | 5 MB | Importance matrix used for calibration (from unsloth, included for reproducibility) |

## Measured performance

On AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X, ROCm backend, `hal0-rocmfpx` image):

- **Q4_0_ROCMFP4_STRIX_LEAN: 42 tok/s decode, 317 tok/s prompt processing** (32K ctx slot) — ≈200 GB/s effective bandwidth, near the platform's memory-bandwidth ceiling.

## How they were made

BF16 GGUF source and imatrix from [unsloth/Ornith-1.0-9B-GGUF](https://huggingface.co/unsloth/Ornith-1.0-9B-GGUF), quantized with the Hal0_ROCmFPX fork's `llama-quantize`:

```bash
llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Ornith-1.0-9B-BF16.gguf Ornith-1.0-9B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN
```

(same invocation per type for the other three)

## Serving on hal0

```bash
hal0 model pull gsrunion/Ornith-1.0-9B-ROCmFPX-GGUF   # or download + add-from-path
hal0 slot create ornith --type llm --hardware rocm --model <id> --ctx-size 32768
curl -sS --max-time 300 -X POST http://127.0.0.1:8080/api/slots/ornith/load
```

## Credits

- Base model: [deepreinforce-ai](https://huggingface.co/deepreinforce-ai) — Ornith-1.0-9B (MIT)
- BF16 GGUF conversion + imatrix: [unsloth](https://huggingface.co/unsloth)
- ROCmFPX quant formats + fork: [Hal0ai](https://github.com/Hal0ai/Hal0_ROCmFPX)