D3-Ultra-NPU2 for FastFlowLM

3945726 verified 3 days ago

15.2 kB

---
license: apache-2.0
base_model: vllm-sr/d3-ultra
pipeline_tag: zero-shot-classification
tags:
- decision-model
- system-one
- fastflowlm
- amd-npu
- q4nx
---
# D3-Ultra-NPU2

[vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra) (`52ba9e7a`; Decision 3.0, the 27B multimodal decision
model of vLLM Semantic Router, Qwen3.5-27B text tower, a newer fine-tune of the d3-turbo architecture) for AMD Ryzen AI NPUs,
served by [FastFlowLM](https://github.com/ROCm/FastFlowLM) 1.0.11+.
It answers *System One* requests -- choice, yes / no and score questions about a state, optionally with images and videos,
with a probability for every option; `box` questions (where is it in an image or a video) and `track` questions (a box per
moment over a whole video) as well -- over the same `POST /v1/systemone` API as the upstream package. Up to 32,768 input
tokens per question. Like upstream d3-ultra it reads the state once per request and answers every question on it
("ask once, answer many"), so many questions about one input cost far less than one request per question.

```shell
flm pull d3:ultra                     # 16.6 GB download
flm serve --decision d3:ultra         # optionally next to a chat model: flm serve qwen3.5:9b --decision d3:ultra
curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" \
  -d '{"state": "The order arrived damaged.", "questions": {"route": {"type": "choice",
       "criteria": {"returns": "Refunds and replacements", "billing": "Payments", "general": null}}}}'
```

Images go in an `images` list of base64 data URLs (PNG, JPEG or WebP; each at most 8 MB and 16 MP, resized to at most
1.6 MP), shared by every question of the request. `d3:lite` (0.8B), `d3:nano` (2B) and `d3:mini` (4B) are the smaller
siblings; they answer a single short question in well under a second, d3:ultra needs a few seconds (see Speed).
`d3:turbo` is the previous 27B model on the same engine.

## What this package supports on the NPU

| Question / input | Status |
|---|---|
| `choice`, `noul` (yes / no), `score` on text | supported |
| the same with images in the state (several images per request) | supported |
| Maximum length | 32,768 tokens per question (a longer prompt is answered with `max_length_exceeded`; nothing is truncated). Upstream's `validated_length` is 131,072; this build keeps the 32,768-token limit of d3:turbo |
| `long_context=True` (upstream: up to 131,072 tokens) | **not supported** (the request is refused with HTTP 422) |

| Question / input | Status |
|---|---|
| `box` (where is it in an image; also at a moment of a video with `at`) | supported: one box per question with a calibrated confidence |
| `assets` (images inside the state, a question's instructions or its options) | supported |
| videos in the state (2 frames per second, 4 to 32 frames, at most 0.2 MP per frame) | supported; a video is at most 60 s and 16,384 vision tokens per request; decoding mp4 / webm / mkv / mov / gif needs the FFmpeg shared libraries on the machine (without them the request is refused with HTTP 422; frame lists `{"frames": [...]}` always work) |
| `track` (a box per moment over a whole video, 1 or 2 updates per second) | supported; at most 256 moments per request (`D3_MOMENT_ROWS`) |

`GET /v1/systemone/models` lists exactly what the running build accepts.

**What is new against d3:turbo** (both are inside this package's `decision_config.json` and are applied by the server as the
upstream runtime does):

- `prompt_layout` `header-v1`: for a text request, the prompt of every question starts with the block
  `Questions in this request:` (all questions of the request, numbered, with their instructions and option texts), then the
  state and the question. The header is part of the shared state block, so it is computed once per request; it adds tokens
  to every request (13 to 3,875 tokens, median 219 on the test set).
  Requests with images or videos, and `assets` / `box` / `track` requests, use the plain layout, as upstream.
- `temperature_by_questions`: the softmax temperature is 1.046 for a request with exactly one `choice` / `noul` / `score`
  question and 1.0 for two or more; `box` and `track` answers keep the base temperature.

## Requirements

- An AMD Ryzen AI processor with an XDNA 2 NPU (this package was tested on a Strix Halo machine), Linux with the XRT
  stack, FastFlowLM as above.
- Memory: the package is 16.6 GB on disk and about 15.9 GiB resident once loaded. Measured peak resident memory of the
  server process: 17.2 GiB after the load, 18.0 GiB for prompts up to 4k tokens and **27.0 GiB at 32,768 tokens** (one request
  on a fresh server); the FLM catalog reserves a footprint of 28 GB (plus 8.6 GB of locked scratch memory). All numbers are from
  a 128 GB machine; smaller machines were not tested, plan for a 32 GB or larger system and do not run it next to other
  large models.
- Load time: about 13 s for the engine with the files in the page cache (`flm serve` also checks the 16.6 GB of files against
  their hashes the first time it sees them or after an upgrade).

## What was changed

- **Weights:** the Qwen3.5-27B text tower of d3-ultra quantized to FastFlowLM's Q4NX format: every projection Q4_K with
  GPTQ-calibrated rounding (our own sequential GPTQ over all 64 layers, 400 tensors; calibration: 300 synthetic System
  One questions with the header-v1 layout and the model's `prefix_bidir` mask, 147,008 tokens, none from the Decision Index),
  token embedding Q8_0, norms and the Gated DeltaNet `alpha` / `beta` parameters kept in BF16.
- **Vision tower:** `vision_weight.q4nx` is made from d3-ultra's own vision encoder (27 layers, 1152 wide, 5120 out; it differs
  from stock Qwen3.5-27B's in every tensor and slightly from d3-turbo's), then quantized to Q4_K. It runs on FastFlowLM's
  existing vision kernels.
- **Attention numerics (`prefix_bidir`):** as upstream, the state block attends bidirectionally to itself, each question
  block attends to the state and to itself, never the other way, and the Gated DeltaNet layers stay causal. The engine
  runs the state once and the question blocks on its cached keys / values and recurrent state; a question's answer does
  not depend on the other questions of the request (the header, which lists them, is part of the state). `usage.input_tokens`
  counts the state once plus every question block, as upstream.
- **Engine:** none; d3:ultra runs on the same libraries and the same four kernels (`mm`, `attn`, `vision_mm`, `vision_attn`)
  as d3:turbo.
- **Unchanged:** `readout.safetensors`, `decision_config.json`, `tokenizer.json`, `tokenizer_config.json`,
  `chat_template.jinja`, `LICENSE` are the upstream files byte for byte (`MANIFEST.sha256`); `NOTICE` adds the
  modification statement. Prompts, token ids and answer objects follow the upstream runtime, with one known difference:
  FastFlowLM tokenizes with the model's own `tokenizer.json`, while upstream's `transformers` tokenizer drops the
  combining-mark class from its pre-tokenizer, so text with combining marks (for example an emoji variation selector)
  can have one token more upstream (3 of 212 text test requests, one token each).

## Accuracy (against the upstream BF16 model)

All comparisons use the same requests: the upstream runtime in BF16 as the reference (run on an AMD iGPU, the vision
tower of the image references on the CPU or in chunked BF16 on the GPU) and this package on the NPU.

| | this package on the NPU | upstream BF16 |
|---|---|---|
| Final-norm hidden state vs BF16, cosine (736 text questions, median / minimum) | 0.9985 / 0.919 | -- |
| Probability error vs BF16 (736 text questions, median / 99th percentile / maximum) | 0.006 / 0.22 / 0.95 | -- |
| Same top answer as BF16 (736 text questions) | 95.1% | -- |
| Same, 5 long prompts of 6k to 24k tokens (cosine median) | 5 / 5 (0.9988) | -- |
| Same, 2 prompts of 32,767 and 32,768 tokens (cosine) | 2 / 2 (0.9988) | -- |
| Same, 60 questions with images (cosine median) | 91.7% (0.9972) | -- |

The plain Q4_K quantization of the same model without GPTQ agreed on 91.5% of the text questions (cosine 0.9955), so
GPTQ is what makes this package usable. Rounding noise of 4-bit weights accumulates over 64 layers, so this 27B model
sits at the level of d3:turbo (95.7%) and d3:nano (95.7%) on this measure. The text set is deliberately hard (many near-ties);
on the Decision Index rows below the agreement is 99%. The weakest items of the set are Japanese-language prompts
(37 of 42 questions of the 20 multilingual requests agree, with the plain Q4_K package as well): non-English input is the
part of the model that quantization hurts most. A request with one question is answered at temperature 1.046, as upstream.

| Decision Index public-suite subset (100 rows, same file and order as the d3:lite / nano / mini / turbo samples) | this package on the NPU | upstream BF16 (same requests) |
|---|---|---|
| top-1 agreement with BF16 on the same requests (716 choice answers) | 99.0% (709 / 716; 95% CI 98.0 to 99.5) | -- |
| accuracy against the expected labels (593 labelled choice answers) | 95.1% | 94.8% |

Accuracy, BF16 minus this package: -0.3 pt (95% CI -1.3 to +0.5 over the 100 rows). The mean absolute probability
difference is 0.0060. The subset is the first 100 rows of the 4,002-row seeded sample (round-robin over 43 benchmarks; about
2 rows per benchmark, so per-benchmark numbers are not meaningful; one row of 32 questions and a 31k-token state does not fit
the 80 GiB iGPU in BF16 and is left out). It is a small check, not a replacement for the full suite.

| Decision Index public vision suite subset (600 questions, 10 benchmarks x 60) | this package on the NPU | upstream BF16 |
|---|---|---|
| accuracy | 77.8% | 80.7% |
| top-1 agreement with BF16 on the same questions | 85.7% (514 / 600) | -- |

Paired accuracy, BF16 minus this package: +2.8 pt (95% CI +0.0 to +5.7; d3:turbo +2.0 pt, d3:mini +2.5 pt on the same
rows). The largest gaps are in BLINK (75.0% vs 83.3%) and Mind2Web (83.3% vs 88.3%); Hateful Memes moderation is near chance for both models and
flips between them (61.7% vs 66.7%, 42% agreement). The image rows are limited by the vision tower: on FastFlowLM's current
vision kernels the features of the d3:turbo tower differ from the BF16 tower's by a relative L2 error of about 0.3 (measured there; d3-ultra's tower is a slight further fine-tune of it).
These 600 rows are the sample used for d3:turbo, so the numbers are comparable.

Box and track questions against the BF16 answers of the same requests: `box` on images (29 requests, 99 boxes) median edge
difference 0.015 of the image size and median IoU 0.86 with BF16's box; `assets` (15 requests) top-1 100%, median IoU 0.95;
`box` at a video moment (11 requests, 22 boxes) edge 0.024 and IoU 0.77; `track` (13 requests, 120 moments) edge 0.017, IoU 0.83,
`visible` decision agrees on all; plain questions about videos (45 answers) top-1 97.8%. A few boxes of the image set miss the
reference object entirely (the 5th percentile of the IoU is about 0), a confident box on the wrong object.

## Speed (AMD Ryzen AI MAX+ PRO 395 (Strix Halo, XDNA 2 NPU), 128 GB, Ubuntu 24.04, XRT, FastFlowLM)

Measured over HTTP (`flm serve --decision d3:ultra`, this GPTQ package, FastFlowLM pull request build with the CI-built
libraries and xclbins, nothing else running on the NPU): median of 5 requests after a warm-up request (one request for 32k);
"tokens" is the `usage.input_tokens` of the request, the header included.

| request | tokens | latency |
|---|---:|---|
| 1 question, short state | 241 | 3.10 s (2.96 to 3.10) |
| 1 question | 441 | 3.43 s (3.42 to 3.44) |
| 1 question | 1,131 | 9.50 s (9.43 to 9.51) |
| 1 question | 4,201 | 30.6 s (30.5 to 30.6) |
| 1 question | 32,750 | 314 s |
| 4 questions about one 330-token state | 717 | 6.33 s (6.30 to 6.36) |
| 16 questions about one 330-token state | 1,688 | 12.8 s (12.7 to 12.9) |

The floor is about 3 s per request (the 15.5 GB of weights are streamed from memory once per pass, so a short question is
memory-bound; d3:mini needs 0.5 s for the same question), then roughly 5 to 10 ms per token. The header makes the prompts a little
longer than d3:turbo's for the same request (a request lists its questions once), so the same text costs somewhat more than
the d3:turbo numbers. State once gives bit-identical answers to the one-pass-per-question path (checked on 55 requests / 389
answers). Over HTTP the 736 text questions of the accuracy set (212 requests) took 3.3 s per request at the median.

Requests on the same machine, one pass over the accuracy request sets (GPTQ package, medians):

| request set | median latency (range) |
|---|---|
| plain question with 1 to 2 images (31 requests) | 4.1 s (max 15.7 s) |
| `box` questions about one image (1 to 8 boxes, 29 requests) | 11.4 s (max 19.6 s) |
| `assets` (images in options or instructions, 15 requests) | 27.5 s (max 113 s) |
| a plain question about a video in the state (11 requests) | 18.3 s (max 26.6 s) |
| `box` at one moment of a video (11 requests) | 10.4 s (max 41 s) |
| `track` (13 requests: clips of 4 to 11 s) | 11.1 s (max 103 s) |

Soak: 100 mixed requests (text, image and long-prompt requests, several questions per request, empty and malformed
bodies) first one after the other, then the same schedule from 4 concurrent clients next to a streaming chat model: 0 status and
0 answer mismatches between the two runs (91 answered, 9 refused with 422 as expected), no crash; resident memory of the server
18.1 GB after the load, a peak of 26.1 GB, 19.6 GB afterwards.

## Comparison

- **Against d3:turbo (same engine):** upstream reports (its own private tests) SR-Text 87.2 for d3-ultra against 83.1 for
  d3-turbo and SR-Box 58.2% against 60.0%. The NPU port costs the same kind of quantization noise as d3:turbo: 95.1% top-1
  agreement on the hard text set (turbo 95.7%) and 99% on Decision Index rows; the vision gap to BF16 is +2.8 pt (turbo +2.0).
  Speed and memory are those of d3:turbo; the header adds tokens to a request.
- **Against the smaller NPU siblings:** d3:mini answers a 330-token question in 0.5 s and d3:nano in 0.25 s; d3:ultra
  takes about 3.4 s, but reads a state once for any number of questions (the NPU siblings in FastFlowLM read the prompt again
  for every question).
- **Against the BF16 model on the same requests:** see Accuracy. The quantization costs a few points of top-1
  agreement on hard items, more than for d3:mini, and the image path costs somewhat more than text.

## License

Apache-2.0, as the upstream model; see `LICENSE` and `NOTICE`. Built on Qwen/Qwen3.5-27B (Apache-2.0). Quantized and
repackaged for FastFlowLM by the FastFlowLM team; the readout, decision config, tokenizer and chat template are unmodified.

## Citation

Please cite the upstream model ([vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra)):

```bibtex
@misc{d3ultra_2026,
  title        = {{d3-ultra}: A Multimodal Decision Model},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-ultra}}
}
```

	---
	license: apache-2.0
	base_model: vllm-sr/d3-ultra
	pipeline_tag: zero-shot-classification
	tags:
	- decision-model
	- system-one
	- fastflowlm
	- amd-npu
	- q4nx
	---

	# D3-Ultra-NPU2

	[vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra) (`52ba9e7a`; Decision 3.0, the 27B multimodal decision
	model of vLLM Semantic Router, Qwen3.5-27B text tower, a newer fine-tune of the d3-turbo architecture) for AMD Ryzen AI NPUs,
	served by [FastFlowLM](https://github.com/ROCm/FastFlowLM) 1.0.11+.
	It answers System One requests -- choice, yes / no and score questions about a state, optionally with images and videos,
	with a probability for every option; `box` questions (where is it in an image or a video) and `track` questions (a box per
	moment over a whole video) as well -- over the same `POST /v1/systemone` API as the upstream package. Up to 32,768 input
	tokens per question. Like upstream d3-ultra it reads the state once per request and answers every question on it
	("ask once, answer many"), so many questions about one input cost far less than one request per question.

	```shell
	flm pull d3:ultra # 16.6 GB download
	flm serve --decision d3:ultra # optionally next to a chat model: flm serve qwen3.5:9b --decision d3:ultra
	curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" \
	-d '{"state": "The order arrived damaged.", "questions": {"route": {"type": "choice",
	"criteria": {"returns": "Refunds and replacements", "billing": "Payments", "general": null}}}}'
	```

	Images go in an `images` list of base64 data URLs (PNG, JPEG or WebP; each at most 8 MB and 16 MP, resized to at most
	1.6 MP), shared by every question of the request. `d3:lite` (0.8B), `d3:nano` (2B) and `d3:mini` (4B) are the smaller
	siblings; they answer a single short question in well under a second, d3:ultra needs a few seconds (see Speed).
	`d3:turbo` is the previous 27B model on the same engine.

	## What this package supports on the NPU

	\| Question / input \| Status \|
	\|---\|---\|
	\| `choice`, `noul` (yes / no), `score` on text \| supported \|
	\| the same with images in the state (several images per request) \| supported \|
	\| Maximum length \| 32,768 tokens per question (a longer prompt is answered with `max_length_exceeded`; nothing is truncated). Upstream's `validated_length` is 131,072; this build keeps the 32,768-token limit of d3:turbo \|
	\| `long_context=True` (upstream: up to 131,072 tokens) \| not supported (the request is refused with HTTP 422) \|

	\| Question / input \| Status \|
	\|---\|---\|
	\| `box` (where is it in an image; also at a moment of a video with `at`) \| supported: one box per question with a calibrated confidence \|
	\| `assets` (images inside the state, a question's instructions or its options) \| supported \|
	\| videos in the state (2 frames per second, 4 to 32 frames, at most 0.2 MP per frame) \| supported; a video is at most 60 s and 16,384 vision tokens per request; decoding mp4 / webm / mkv / mov / gif needs the FFmpeg shared libraries on the machine (without them the request is refused with HTTP 422; frame lists `{"frames": [...]}` always work) \|
	\| `track` (a box per moment over a whole video, 1 or 2 updates per second) \| supported; at most 256 moments per request (`D3_MOMENT_ROWS`) \|

	`GET /v1/systemone/models` lists exactly what the running build accepts.

	What is new against d3:turbo (both are inside this package's `decision_config.json` and are applied by the server as the
	upstream runtime does):

	- `prompt_layout` `header-v1`: for a text request, the prompt of every question starts with the block
	`Questions in this request:` (all questions of the request, numbered, with their instructions and option texts), then the
	state and the question. The header is part of the shared state block, so it is computed once per request; it adds tokens
	to every request (13 to 3,875 tokens, median 219 on the test set).
	Requests with images or videos, and `assets` / `box` / `track` requests, use the plain layout, as upstream.
	- `temperature_by_questions`: the softmax temperature is 1.046 for a request with exactly one `choice` / `noul` / `score`
	question and 1.0 for two or more; `box` and `track` answers keep the base temperature.

	## Requirements

	- An AMD Ryzen AI processor with an XDNA 2 NPU (this package was tested on a Strix Halo machine), Linux with the XRT
	stack, FastFlowLM as above.
	- Memory: the package is 16.6 GB on disk and about 15.9 GiB resident once loaded. Measured peak resident memory of the
	server process: 17.2 GiB after the load, 18.0 GiB for prompts up to 4k tokens and 27.0 GiB at 32,768 tokens (one request
	on a fresh server); the FLM catalog reserves a footprint of 28 GB (plus 8.6 GB of locked scratch memory). All numbers are from
	a 128 GB machine; smaller machines were not tested, plan for a 32 GB or larger system and do not run it next to other
	large models.
	- Load time: about 13 s for the engine with the files in the page cache (`flm serve` also checks the 16.6 GB of files against
	their hashes the first time it sees them or after an upgrade).

	## What was changed

	- Weights: the Qwen3.5-27B text tower of d3-ultra quantized to FastFlowLM's Q4NX format: every projection Q4_K with
	GPTQ-calibrated rounding (our own sequential GPTQ over all 64 layers, 400 tensors; calibration: 300 synthetic System
	One questions with the header-v1 layout and the model's `prefix_bidir` mask, 147,008 tokens, none from the Decision Index),
	token embedding Q8_0, norms and the Gated DeltaNet `alpha` / `beta` parameters kept in BF16.
	- Vision tower: `vision_weight.q4nx` is made from d3-ultra's own vision encoder (27 layers, 1152 wide, 5120 out; it differs
	from stock Qwen3.5-27B's in every tensor and slightly from d3-turbo's), then quantized to Q4_K. It runs on FastFlowLM's
	existing vision kernels.
	- Attention numerics (`prefix_bidir`): as upstream, the state block attends bidirectionally to itself, each question
	block attends to the state and to itself, never the other way, and the Gated DeltaNet layers stay causal. The engine
	runs the state once and the question blocks on its cached keys / values and recurrent state; a question's answer does
	not depend on the other questions of the request (the header, which lists them, is part of the state). `usage.input_tokens`
	counts the state once plus every question block, as upstream.
	- Engine: none; d3:ultra runs on the same libraries and the same four kernels (`mm`, `attn`, `vision_mm`, `vision_attn`)
	as d3:turbo.
	- Unchanged: `readout.safetensors`, `decision_config.json`, `tokenizer.json`, `tokenizer_config.json`,
	`chat_template.jinja`, `LICENSE` are the upstream files byte for byte (`MANIFEST.sha256`); `NOTICE` adds the
	modification statement. Prompts, token ids and answer objects follow the upstream runtime, with one known difference:
	FastFlowLM tokenizes with the model's own `tokenizer.json`, while upstream's `transformers` tokenizer drops the
	combining-mark class from its pre-tokenizer, so text with combining marks (for example an emoji variation selector)
	can have one token more upstream (3 of 212 text test requests, one token each).

	## Accuracy (against the upstream BF16 model)

	All comparisons use the same requests: the upstream runtime in BF16 as the reference (run on an AMD iGPU, the vision
	tower of the image references on the CPU or in chunked BF16 on the GPU) and this package on the NPU.

	\| \| this package on the NPU \| upstream BF16 \|
	\|---\|---\|---\|
	\| Final-norm hidden state vs BF16, cosine (736 text questions, median / minimum) \| 0.9985 / 0.919 \| -- \|
	\| Probability error vs BF16 (736 text questions, median / 99th percentile / maximum) \| 0.006 / 0.22 / 0.95 \| -- \|
	\| Same top answer as BF16 (736 text questions) \| 95.1% \| -- \|
	\| Same, 5 long prompts of 6k to 24k tokens (cosine median) \| 5 / 5 (0.9988) \| -- \|
	\| Same, 2 prompts of 32,767 and 32,768 tokens (cosine) \| 2 / 2 (0.9988) \| -- \|
	\| Same, 60 questions with images (cosine median) \| 91.7% (0.9972) \| -- \|

	The plain Q4_K quantization of the same model without GPTQ agreed on 91.5% of the text questions (cosine 0.9955), so
	GPTQ is what makes this package usable. Rounding noise of 4-bit weights accumulates over 64 layers, so this 27B model
	sits at the level of d3:turbo (95.7%) and d3:nano (95.7%) on this measure. The text set is deliberately hard (many near-ties);
	on the Decision Index rows below the agreement is 99%. The weakest items of the set are Japanese-language prompts
	(37 of 42 questions of the 20 multilingual requests agree, with the plain Q4_K package as well): non-English input is the
	part of the model that quantization hurts most. A request with one question is answered at temperature 1.046, as upstream.

	\| Decision Index public-suite subset (100 rows, same file and order as the d3:lite / nano / mini / turbo samples) \| this package on the NPU \| upstream BF16 (same requests) \|
	\|---\|---\|---\|
	\| top-1 agreement with BF16 on the same requests (716 choice answers) \| 99.0% (709 / 716; 95% CI 98.0 to 99.5) \| -- \|
	\| accuracy against the expected labels (593 labelled choice answers) \| 95.1% \| 94.8% \|

	Accuracy, BF16 minus this package: -0.3 pt (95% CI -1.3 to +0.5 over the 100 rows). The mean absolute probability
	difference is 0.0060. The subset is the first 100 rows of the 4,002-row seeded sample (round-robin over 43 benchmarks; about
	2 rows per benchmark, so per-benchmark numbers are not meaningful; one row of 32 questions and a 31k-token state does not fit
	the 80 GiB iGPU in BF16 and is left out). It is a small check, not a replacement for the full suite.

	\| Decision Index public vision suite subset (600 questions, 10 benchmarks x 60) \| this package on the NPU \| upstream BF16 \|
	\|---\|---\|---\|
	\| accuracy \| 77.8% \| 80.7% \|
	\| top-1 agreement with BF16 on the same questions \| 85.7% (514 / 600) \| -- \|

	Paired accuracy, BF16 minus this package: +2.8 pt (95% CI +0.0 to +5.7; d3:turbo +2.0 pt, d3:mini +2.5 pt on the same
	rows). The largest gaps are in BLINK (75.0% vs 83.3%) and Mind2Web (83.3% vs 88.3%); Hateful Memes moderation is near chance for both models and
	flips between them (61.7% vs 66.7%, 42% agreement). The image rows are limited by the vision tower: on FastFlowLM's current
	vision kernels the features of the d3:turbo tower differ from the BF16 tower's by a relative L2 error of about 0.3 (measured there; d3-ultra's tower is a slight further fine-tune of it).
	These 600 rows are the sample used for d3:turbo, so the numbers are comparable.

	Box and track questions against the BF16 answers of the same requests: `box` on images (29 requests, 99 boxes) median edge
	difference 0.015 of the image size and median IoU 0.86 with BF16's box; `assets` (15 requests) top-1 100%, median IoU 0.95;
	`box` at a video moment (11 requests, 22 boxes) edge 0.024 and IoU 0.77; `track` (13 requests, 120 moments) edge 0.017, IoU 0.83,
	`visible` decision agrees on all; plain questions about videos (45 answers) top-1 97.8%. A few boxes of the image set miss the
	reference object entirely (the 5th percentile of the IoU is about 0), a confident box on the wrong object.

	## Speed (AMD Ryzen AI MAX+ PRO 395 (Strix Halo, XDNA 2 NPU), 128 GB, Ubuntu 24.04, XRT, FastFlowLM)

	Measured over HTTP (`flm serve --decision d3:ultra`, this GPTQ package, FastFlowLM pull request build with the CI-built
	libraries and xclbins, nothing else running on the NPU): median of 5 requests after a warm-up request (one request for 32k);
	"tokens" is the `usage.input_tokens` of the request, the header included.

	\| request \| tokens \| latency \|
	\|---\|---:\|---\|
	\| 1 question, short state \| 241 \| 3.10 s (2.96 to 3.10) \|
	\| 1 question \| 441 \| 3.43 s (3.42 to 3.44) \|
	\| 1 question \| 1,131 \| 9.50 s (9.43 to 9.51) \|
	\| 1 question \| 4,201 \| 30.6 s (30.5 to 30.6) \|
	\| 1 question \| 32,750 \| 314 s \|
	\| 4 questions about one 330-token state \| 717 \| 6.33 s (6.30 to 6.36) \|
	\| 16 questions about one 330-token state \| 1,688 \| 12.8 s (12.7 to 12.9) \|

	The floor is about 3 s per request (the 15.5 GB of weights are streamed from memory once per pass, so a short question is
	memory-bound; d3:mini needs 0.5 s for the same question), then roughly 5 to 10 ms per token. The header makes the prompts a little
	longer than d3:turbo's for the same request (a request lists its questions once), so the same text costs somewhat more than
	the d3:turbo numbers. State once gives bit-identical answers to the one-pass-per-question path (checked on 55 requests / 389
	answers). Over HTTP the 736 text questions of the accuracy set (212 requests) took 3.3 s per request at the median.

	Requests on the same machine, one pass over the accuracy request sets (GPTQ package, medians):

	\| request set \| median latency (range) \|
	\|---\|---\|
	\| plain question with 1 to 2 images (31 requests) \| 4.1 s (max 15.7 s) \|
	\| `box` questions about one image (1 to 8 boxes, 29 requests) \| 11.4 s (max 19.6 s) \|
	\| `assets` (images in options or instructions, 15 requests) \| 27.5 s (max 113 s) \|
	\| a plain question about a video in the state (11 requests) \| 18.3 s (max 26.6 s) \|
	\| `box` at one moment of a video (11 requests) \| 10.4 s (max 41 s) \|
	\| `track` (13 requests: clips of 4 to 11 s) \| 11.1 s (max 103 s) \|

	Soak: 100 mixed requests (text, image and long-prompt requests, several questions per request, empty and malformed
	bodies) first one after the other, then the same schedule from 4 concurrent clients next to a streaming chat model: 0 status and
	0 answer mismatches between the two runs (91 answered, 9 refused with 422 as expected), no crash; resident memory of the server
	18.1 GB after the load, a peak of 26.1 GB, 19.6 GB afterwards.

	## Comparison

	- Against d3:turbo (same engine): upstream reports (its own private tests) SR-Text 87.2 for d3-ultra against 83.1 for
	d3-turbo and SR-Box 58.2% against 60.0%. The NPU port costs the same kind of quantization noise as d3:turbo: 95.1% top-1
	agreement on the hard text set (turbo 95.7%) and 99% on Decision Index rows; the vision gap to BF16 is +2.8 pt (turbo +2.0).
	Speed and memory are those of d3:turbo; the header adds tokens to a request.
	- Against the smaller NPU siblings: d3:mini answers a 330-token question in 0.5 s and d3:nano in 0.25 s; d3:ultra
	takes about 3.4 s, but reads a state once for any number of questions (the NPU siblings in FastFlowLM read the prompt again
	for every question).
	- Against the BF16 model on the same requests: see Accuracy. The quantization costs a few points of top-1
	agreement on hard items, more than for d3:mini, and the image path costs somewhat more than text.

	## License

	Apache-2.0, as the upstream model; see `LICENSE` and `NOTICE`. Built on Qwen/Qwen3.5-27B (Apache-2.0). Quantized and
	repackaged for FastFlowLM by the FastFlowLM team; the readout, decision config, tokenizer and chat template are unmodified.

	## Citation

	Please cite the upstream model ([vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra)):

	```bibtex
	@misc{d3ultra_2026,
	title = {{d3-ultra}: A Multimodal Decision Model},
	author = {{vLLM Semantic Router Team}},
	year = {2026},
	howpublished = {\url{https://huggingface.co/vllm-sr/d3-ultra}}
	}
	```