Ornith 1.5 9B 路 bf16 路 for radiance
ornith-ai/Ornith-1.5-9B in one 16.6 GiB file for the radiance inference engine on AMD RDNA4. A dense 9B reasoning model, tuned for coding and agent work. It runs on one card.
| File | ornith-1.5-9b-bf16.rad, 16.60 GiB |
| Needs | radiance 2.0.0 or newer; 1 or 2 cards with 32 GB (tested on one and two Radeon AI PRO R9700) |
| Format | weights bf16 as in the original; lm_head int8, one bf16 scale per 128 weights; activations bf16 |
| Speculation | none: this file carries no draft head |
| Inputs | text 路 image |
| Context | 262,144 tokens |
| Version | 2.0 (2026-11-15) |
Speed
A short check on one card, radiance 2.0.0, i8 KV, greedy, two prompts: 33.6 and 33.7 tokens per second (no speculation).
Quality
Distance to the bf16 original on the 80 MT-Bench questions, both turns. The text is the GPT-4 answers from lmsys/mt_bench_human_judgments, with the model's chat template and thinking off. The reference is the bf16 original in HF transformers, with fp32 arithmetic and the full vocabulary. radiance scores this file with --kld-ref, on the answer tokens only (42,305 tokens). radiance 2.0.0 (release image), i8 KV, one card.
| mean KL | 99th percentile KL | top token the same | perplexity ratio | |
|---|---|---|---|---|
| this file | 0.00017 | 0.0017 | 99.6% | 脳1.000 |
| keptn/ornith-1.5-9b-iq8r | 0.00067 | 0.0068 | 99.2% | 脳1.000 |
Quick start
Start the model on one card. Save this as radiance.yaml:
server:
host: 0.0.0.0
port: 8000
models:
ornith-1.5-9b-bf16:
model: /models/ornith-1.5-9b-bf16.rad
tp: 1
kv-cache-dtype: i8
max-model-len: 262144
max-num-seqs: 8
max-num-batched-tokens: 2048
temp: 1.0
top-p: 0.95
top-k: 20
presence-penalty: 1.5
The entry name ornith-1.5-9b-bf16 is the model id in requests. Then run:
hf download keptn/ornith-1.5-9b-bf16 ornith-1.5-9b-bf16.rad config.json --local-dir ~/models
docker pull stilldeadcode/radiance:2.0.0
docker run -d --init --name ornith --network host --group-add video --group-add render \
--device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
-v ~/models:/models:ro -v $PWD/radiance.yaml:/etc/radiance/radiance.yaml:ro \
-e ROCR_VISIBLE_DEVICES=0 \
stilldeadcode/radiance:2.0.0 --config /etc/radiance/radiance.yaml
| change | |
|---|---|
| two cards | ROCR_VISIBLE_DEVICES=0,1, tp: 2, add p2p: on and -e HSA_ENABLE_IPC_MODE_LEGACY=1 |
| four cards | ROCR_VISIBLE_DEVICES=0,1,2,3, tp: 4, add p2p: on and -e HSA_ENABLE_IPC_MODE_LEGACY=1 |
| coding | temp: 0.6, presence-penalty: 0 (Ornith's setting for precise coding) |
| thinking | on by default; per request "chat_template_kwargs": {"enable_thinking": false} turns it off |
The sampling values are Ornith's: temperature 1.0, top_p 0.95, top_k 20, presence penalty 1.5.
Memory and context
KV cache at max-model-len: 262144 and 8 sequences, i8 KV, radiance 2.0.0, from radiance's startup report:
| KV cache | tokens of context in total | full-length sessions | |
|---|---|---|---|
| 1 card | 16.22 GiB | 1,030,096 | 3.9 |
| 2 cards | 16.32 GiB a card | 2,097,200 | 8.0 |
radiance starts and serves when the cache holds less than 8 full-length sessions. It reports the number of sessions at startup.
How it was made
- Started from the bf16 release
ornith-ai/Ornith-1.5-9B(revision489cb979). - Kept every weight as it is. Rounded only the lm_head to int8, one bf16 scale per 128 weights (round to nearest).
rad-convert Ornith-1.5-9B --tokenizer Ornith-1.5-9B/tokenizer.json \
--recipe ornith-1.5-9b-bf16.recipe -o ornith-1.5-9b-bf16.rad
Calibration: no calibration data.
Limits
- radiance runs on AMD RDNA4 cards.
- The file needs radiance 2.0.0 or newer.
- This file has no speculation. For draft tokens, use keptn/ornith-1.5-9b-iq8r.
- Speed is checked with two short prompts only. BetterBench is not run on radiance 2.0 yet.
- Four cards are not tested on radiance 2.0.
- Images are not compared with HF transformers for this version.
- With thinking on, hard tasks can need more than 32k output tokens.
Versions
| version | date | change |
|---|---|---|
| 2.0 | 2026-11-15 | lm_head int8 (radiance 2.0 does not serve the FP8 blocks of 1.0) |
| 1.0 | 2026-11-03 | lm_head FP8 in 128脳128 blocks; radiance 1.15.1 to 1.17.0 |
Version 1.0: hf download keptn/ornith-1.5-9b-bf16 ornith-1.5-9b-bf16.rad --revision v1.0 --local-dir ~/models. Its card at that revision has the command-line flags.
Credits
- Model: the Ornith team (github.com/ornith-ai/Ornith-1), built on Qwen3.5 (Apache 2.0).
- radiance: StillDeadcode (codeberg.org/StillDeadcode/radiance).
- Test set: MT-Bench.
License
MIT, the license of the base model. See LICENSE (from github.com/ornith-ai/Ornith-1).
- Downloads last month
- 53
Model tree for keptn/ornith-1.5-9b-bf16
Base model
ornith-ai/Ornith-1.5-9B