Ornith 1.5 9B 路 bf16 路 for radiance

ornith-ai/Ornith-1.5-9B in one 16.6 GiB file for the radiance inference engine on AMD RDNA4. A dense 9B reasoning model, tuned for coding and agent work. It runs on one card.

File ornith-1.5-9b-bf16.rad, 16.60 GiB
Needs radiance 2.0.0 or newer; 1 or 2 cards with 32 GB (tested on one and two Radeon AI PRO R9700)
Format weights bf16 as in the original; lm_head int8, one bf16 scale per 128 weights; activations bf16
Speculation none: this file carries no draft head
Inputs text 路 image
Context 262,144 tokens
Version 2.0 (2026-11-15)

Speed

A short check on one card, radiance 2.0.0, i8 KV, greedy, two prompts: 33.6 and 33.7 tokens per second (no speculation).

Quality

Distance to the bf16 original on the 80 MT-Bench questions, both turns. The text is the GPT-4 answers from lmsys/mt_bench_human_judgments, with the model's chat template and thinking off. The reference is the bf16 original in HF transformers, with fp32 arithmetic and the full vocabulary. radiance scores this file with --kld-ref, on the answer tokens only (42,305 tokens). radiance 2.0.0 (release image), i8 KV, one card.

mean KL 99th percentile KL top token the same perplexity ratio
this file 0.00017 0.0017 99.6% 脳1.000
keptn/ornith-1.5-9b-iq8r 0.00067 0.0068 99.2% 脳1.000

Quick start

Start the model on one card. Save this as radiance.yaml:

server:
  host: 0.0.0.0
  port: 8000
models:
  ornith-1.5-9b-bf16:
    model: /models/ornith-1.5-9b-bf16.rad
    tp: 1
    kv-cache-dtype: i8
    max-model-len: 262144
    max-num-seqs: 8
    max-num-batched-tokens: 2048
    temp: 1.0
    top-p: 0.95
    top-k: 20
    presence-penalty: 1.5

The entry name ornith-1.5-9b-bf16 is the model id in requests. Then run:

hf download keptn/ornith-1.5-9b-bf16 ornith-1.5-9b-bf16.rad config.json --local-dir ~/models
docker pull stilldeadcode/radiance:2.0.0
docker run -d --init --name ornith --network host --group-add video --group-add render \
    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
    -v ~/models:/models:ro -v $PWD/radiance.yaml:/etc/radiance/radiance.yaml:ro \
    -e ROCR_VISIBLE_DEVICES=0 \
    stilldeadcode/radiance:2.0.0 --config /etc/radiance/radiance.yaml
change
two cards ROCR_VISIBLE_DEVICES=0,1, tp: 2, add p2p: on and -e HSA_ENABLE_IPC_MODE_LEGACY=1
four cards ROCR_VISIBLE_DEVICES=0,1,2,3, tp: 4, add p2p: on and -e HSA_ENABLE_IPC_MODE_LEGACY=1
coding temp: 0.6, presence-penalty: 0 (Ornith's setting for precise coding)
thinking on by default; per request "chat_template_kwargs": {"enable_thinking": false} turns it off

The sampling values are Ornith's: temperature 1.0, top_p 0.95, top_k 20, presence penalty 1.5.

Memory and context

KV cache at max-model-len: 262144 and 8 sequences, i8 KV, radiance 2.0.0, from radiance's startup report:

KV cache tokens of context in total full-length sessions
1 card 16.22 GiB 1,030,096 3.9
2 cards 16.32 GiB a card 2,097,200 8.0

radiance starts and serves when the cache holds less than 8 full-length sessions. It reports the number of sessions at startup.

How it was made

  1. Started from the bf16 release ornith-ai/Ornith-1.5-9B (revision 489cb979).
  2. Kept every weight as it is. Rounded only the lm_head to int8, one bf16 scale per 128 weights (round to nearest).
rad-convert Ornith-1.5-9B --tokenizer Ornith-1.5-9B/tokenizer.json \
    --recipe ornith-1.5-9b-bf16.recipe -o ornith-1.5-9b-bf16.rad

Calibration: no calibration data.

Limits

  • radiance runs on AMD RDNA4 cards.
  • The file needs radiance 2.0.0 or newer.
  • This file has no speculation. For draft tokens, use keptn/ornith-1.5-9b-iq8r.
  • Speed is checked with two short prompts only. BetterBench is not run on radiance 2.0 yet.
  • Four cards are not tested on radiance 2.0.
  • Images are not compared with HF transformers for this version.
  • With thinking on, hard tasks can need more than 32k output tokens.

Versions

version date change
2.0 2026-11-15 lm_head int8 (radiance 2.0 does not serve the FP8 blocks of 1.0)
1.0 2026-11-03 lm_head FP8 in 128脳128 blocks; radiance 1.15.1 to 1.17.0

Version 1.0: hf download keptn/ornith-1.5-9b-bf16 ornith-1.5-9b-bf16.rad --revision v1.0 --local-dir ~/models. Its card at that revision has the command-line flags.

Credits

License

MIT, the license of the base model. See LICENSE (from github.com/ornith-ai/Ornith-1).

Downloads last month
53
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for keptn/ornith-1.5-9b-bf16

Finetuned
(49)
this model