d3-turbo-FP8

d3-turbo-FP8 is d3-turbo, the upgraded 27B multimodal decision model of Decision 3.0, with its decoder's projections in FP8. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, rate on a scale, or point to where something is in an image or a video. It reads the input once per request, answers every question on it and returns a probability for every answer, as d3-turbo does, in 31.9 GB of weights instead of 52.2 GB.

Parameters	26.09B, including the 0.46B vision encoder
Base model	vllm-sr/d3-turbo v1.1 at revision 17a01e22
Quantization	FP8 W8A8 (compressed-tensors float-quantized, FP8_DYNAMIC): E4M3 weights with one BF16 scale per output channel, activations quantized to FP8 per token at run time. 304 projections; the Gated DeltaNet input projections, the vision tower, the embeddings and the readout stay as published
Calibration data	None: weight scales come from the weights, activation scales from each request
Weights on disk	31.9 GB (d3-turbo: 52.2 GB)
Needs	FP8 matrix units: AMD Instinct MI300 / MI350 series, or NVIDIA compute capability 8.9 or later. Measured on one AMD Instinct MI300X
Inputs	Text or JSON, plus images and videos (several per request); images can also sit inside questions and options
Decision types	Choice · Yes / No · Score · Box · Track, all measured against d3-turbo
License	Apache-2.0

Highlights

• 71.54 on the Jev Decision Index 0.3 public suite against d3-turbo's 71.62, every area within 0.6 points, with d3-turbo's top answer on 99.2% of 272,229 questions.
• 76.81 on the public vision suite against 76.64 for d3-turbo on the same GPU, with d3-turbo's top answer on 97.7% of 12,210 questions.
• 0.92× to 1.35× d3-turbo's speed per request on one AMD Instinct MI300X, depending on the kind of request (text 100 ms against 135 ms at the median), and 1.36× its text throughput under load.
• 31.9 GB of weights instead of 52.2 GB; 59.6 GiB of GPU memory at most in the speed benchmark against 78.3 GiB.
• Box and tracking hold: Box accuracy at IoU 0.5 is 0.895 against 0.896, tracking success AUC 0.792 / 0.804 against 0.796 / 0.807, and images in questions and options 0.957 against 0.957.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.17" torch torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention  # optional: fast GPU kernels for the linear-attention layers

import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-turbo-FP8")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")


def asset(name):
    return hf_hub_download("vllm-sr/d3-turbo", f"assets/{name}")


# Text: several questions, one call
print(model.system_one(
    state="Ticket: the invoice total is wrong after the plan change.",
    questions={
        "route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "tech": "Technical support"}},
        "urgent": {"type": "noul", "instructions": "Is the customer asking for an urgent fix?"},
    },
))

# Box: where is it in the image? Four questions, one read of the photo
print(model.system_one(
    state="A photo from a dog festival.",
    images=[asset("example-frisbee.jpg")],  # local paths, http(s) URLs, PIL images or base64-encoded images
    questions={
        "frisbee": {"type": "box", "instructions": "The frisbee"},
        "tail": {"type": "box", "instructions": "The dog's tail"},
        "cone": {"type": "box", "instructions": "The orange cone"},
        "camera": {"type": "box", "instructions": "The camera on the tripod"},
    },
))

# Images in options: each option is an image that only this question sees
print(model.system_one(
    state="This animal: <animal>",
    questions={"pick": {"type": "choice", "instructions": "Which picture shows the same kind of animal?",
                        "criteria": {"a": "<a>", "b": "<b>", "c": "<c>"}}},
    assets={"<animal>": asset("example-panda.jpg"), "<a>": asset("example-option-a.jpg"),
            "<b>": asset("example-option-b.jpg"), "<c>": asset("example-option-c.jpg")},
))

# Video: a track over the whole clip, two boxes per second
print(model.system_one(
    state="A bird at a feeder.",
    videos=[asset("example-hummingbird.mp4")],  # local paths, http(s) URLs, base64-encoded videos or frame arrays
    questions={"target": {"type": "track", "instructions": "The hummingbird", "rate": 2}},
))

load_d3 is d3-turbo's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its weights in FP8, and each quantized projection becomes a layer that quantizes its input to FP8 per row with one Triton kernel and multiplies in torch._scaled_mm. Prompting, reading the input once, the FP32 readout, Box and Track answers, images and videos are d3-turbo's own; the answers are formatted as on d3-turbo's card. AutoModel.from_pretrained(..., trust_remote_code=True) alone does not run FP8 weights.

Checked on one AMD Instinct MI300X with transformers 5.17.0 and compressed-tensors 0.17.0: the four requests above are answered as d3-turbo answers them through the same path (each edge of the four boxes within 0.011 of d3-turbo's, picture b at 0.999 against 1.000, a 22-point track for both). With compressed-tensors 0.17.1, on 300 of the tasksource test requests below, picked from those where the model runtime kept d3-turbo's answer (200) or changed it (100), this path gives the kept answer on 198 of 200, and on the other 100 the runtime's answer on 60 and d3-turbo's on 40.

Run with vLLM Semantic Router

The vLLM Semantic Router model runtime loads d3-turbo-FP8 as it loads d3-turbo, verifies the package against its MODEL_MANIFEST.json, and keeps the weights in FP8 on the GPU (#5066):

pip install -e "src/model-runtime[multimodal]" flash-linear-attention   # in a checkout of vllm-project/semantic-router
vllm-srun serve vllm-sr/d3-turbo-FP8 --revision f1f84f60cd52d61c3f8d76f9ccf1450724c97080 --port 8899 --max-request-bytes 67108864

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-turbo-FP8",
  "state": "Write a Python function that reverses a string.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

vllm-srun takes a Hub package that is not built in only at a full 40-hex --revision; requests with videos need --max-request-bytes above the default 8 MiB. The Box, images-in-options and tracking results below come from vllm-srun serve run on a local copy of this repository (the same weights and loaded files). Serving it from the Hub at the revision above was checked with the request above and a Box request; engine mode (vllm-sr serve vllm-sr/d3-turbo-FP8 --engine) has not been tried with this repository yet.

Serving performance

GPU	One request at a time	d3-turbo (BF16), p50 / p90	d3-turbo-FP8, p50 / p90	Speed-up (p50)
AMD Instinct MI300X	Text (3,500 requests)	135 / 768 ms	100 / 568 ms	1.35×
	Images (700)	228 / 362 ms	211 / 304 ms	1.08×
	Box (300)	158 / 217 ms	171 / 204 ms	0.92×
	Ask once (300)	174 / 1,384 ms	187 / 1,030 ms	0.93×
	Video tracking (200)	2.18 / 8.15 s	1.72 / 6.33 s	1.27×
	Text throughput, 4 to 64 at a time	4.27 requests / s	5.81 requests / s	1.36×
	GPU memory, most during the benchmark	78.3 GiB	59.6 GiB	0.76×
	Weights on disk	52.2 GB	31.9 GB	0.61×

By input length, by number of questions, by area and family

Text by input length (tokens)	< 128	128-256	256-512	512-1k	1k-2k	2k-4k	≥ 4k
d3-turbo p50	42 ms	53 ms	76 ms	126 ms	179 ms	341 ms	773 ms
d3-turbo-FP8 p50	35 ms	42 ms	58 ms	95 ms	133 ms	251 ms	570 ms
Speed-up	1.23×	1.26×	1.31×	1.33×	1.35×	1.36×	1.35×

Ask once: questions about one photo	1	4	16	64	128	255
d3-turbo p50	152 ms	131 ms	151 ms	386 ms	713 ms	1.39 s
d3-turbo-FP8 p50	167 ms	144 ms	145 ms	289 ms	525 ms	1.03 s
Speed-up	0.91×	0.91×	1.04×	1.34×	1.36×	1.34×

Text under load	d3-turbo: requests / s, questions / s, p50	d3-turbo-FP8: requests / s, questions / s, p50	Ratio
4 at a time	4.25, 25.2, 412 ms	5.77, 34.1, 294 ms	1.36×
16 at a time	4.29, 25.4, 1.67 s	5.85, 34.6, 1.15 s	1.36×
64 at a time	4.27, 25.3, 6.59 s	5.81, 34.4, 4.58 s	1.36×

By area the text speed-up is 1.28× to 1.35× (arts 1.28×, knowledge 1.35×, language 1.33×, retrieval 1.28×, tools 1.34×); by image benchmark 0.93× (CV-Bench) to 1.23× (Winoground); by Box family 0.91× to 0.98×; video tracking 1.22× at 1 update per second and 1.28× at 2.

Earlier, separate timings on the same GPU: one question per request on the tasksource test split below, one request at a time, median 34.3 ms against 42.3 ms, p90 75.0 against 96.3 ms, p95 99.0 against 133.8 ms (2026-10-20).

• Throughput under load does not grow past 4 requests at a time for any of the three packages with these requests, and latency grows with the queue.
• Box and ask once with up to 4 questions are slower with FP8 (0.91× to 0.93× at the median); the speed-up appears from 16 questions about one photo and grows with text length.
• GPU memory is the device's used memory (rocm-smi), PyTorch's cached blocks included, with requests of up to about 15,000 tokens.
• The benchmark is the fixed set of 5,000 requests in the dataset vllm-sr/d3-perf-bench (private to the vLLM Semantic Router team; text across seven input lengths and the Decision Index's five areas, images, Box, ask once and video), which times any d3-turbo package on the same requests.

_{Measured on 2026-10-25 on one AMD Instinct MI300X (amdgpu 6.19.14), ROCm 7.2 image with PyTorch 2.12, Triton 3.7.0 and FLA 0.5.2, with the vLLM Semantic Router model runtime from main as of 2026-10-20 and its default settings (--max-request-bytes 67108864). Each package was served alone on the GPU, one after the other: every request one at a time, then the text requests 4, 16 and 64 at a time.}

Evaluation

Every comparison is against d3-turbo (BF16) on the same questions. The text index compares with d3-turbo v1.1's own run of the public suite; images, Box, images in options and tracking compare with d3-turbo run on the same MI300X through the same engine. Through the same server, requests and scoring, d3-turbo gives the same hit or miss as its own evaluation rows on 99.5% of the Box items and the same right or wrong answer on 99.7% of the images-in-options items, so the differences below are the quantization's.

Text: Jev Decision Index 0.3, public suite

	d3-turbo (BF16)	d3-turbo-FP8
Decision Index (public)	71.62	71.54
Raw index	78.47	78.39
Knowledge	58.9	58.9
Language	74.5	74.5
Retrieval	75.8	75.7
Tools	86.9	86.9
Arts	60.9	60.3

All 140,178 requests of the public suite, scored by the official 0.3 kit; the index reads the 110,087 requests (272,229 questions) of its 37 benchmarks, and the other 30,091 rows (MMLU, RouterBench, SGD, ARC, SimpleBench) are not part of it. The d3-turbo column is d3-turbo v1.1's own run on an AMD Instinct MI325X, which d3-turbo's card reports as 71.7; scored by the kit this card's run used, its results give 71.62. This package gives d3-turbo's top answer on 99.19% of the index's questions (2,197 change); the lowest agreement is on ChessBench legal moves (92.8%), HLE text (93.8%) and GPQA Diamond (93.9%). KL(d3-turbo ‖ this package): mean 0.00091, median 0.000033, 95th percentile 0.0036, 99th percentile 0.0145.

Images: Jev Decision Index 0.3, public vision suite

	d3-turbo (BF16)	d3-turbo-FP8
Public vision index	76.64	76.81

By benchmark (accuracy)

Benchmark	d3-turbo (BF16)	d3-turbo-FP8
CV-Bench	0.893	0.892
BLINK	0.829	0.837
RealWorldQA	0.822	0.825
CharXiv	0.857	0.852
InfographicVQA	0.971	0.975
Mind2Web	0.889	0.888
Winoground	0.926	0.925
KIE (CORD+FUNSD)	0.951	0.951
Moderation (Hateful Memes)	0.753	0.756
R-Bench-M	0.495	0.496
MMMU-Pro vision	0.503	0.503

All 12,210 rows, both packages through the same engine on the same MI300X. d3-turbo's card gives its vision board estimate (72.9), which is not this public index. This package gives d3-turbo's top answer on 97.73% of the questions (277 change), most of the changes on MMMU-Pro vision (94.2% agreement) and R-Bench-M (94.3%), where d3-turbo itself is right about half the time; every other benchmark agrees on 97.7% to 99.7%. KL: mean 0.0037, median 0.00033, 99th percentile 0.045.

Images: Box and images in options

Measure	d3-turbo card	d3-turbo (BF16)	d3-turbo-FP8
Box: accuracy at IoU 0.5	0.90	0.8962	0.8948
Box: confidence AUROC	0.96	0.9610	0.9621
Images in questions and options: accuracy	0.96	0.9568	0.9575
The same, against d3 reading every image in the input	+0.31	+0.313	+0.314

d3-turbo's own Box test set (5,836 items) and images-in-options test set (2,752 questions), scored as d3-turbo's evaluation scores them. On Box this package hits or misses as d3-turbo does on 99.0% of the items (32 hits become misses, 24 misses become hits): −0.1 points at IoU 0.5 (95% interval −0.4 to +0.1), and at IoU 0.75 0.7887 against 0.7939 (mean IoU 0.8113 against 0.8152). Every family stays within 0.01 (COCO spatial 0.705 → 0.701, exemplar 0.731 → 0.725; charts, receipts, screens, shapes and tables within 0.003). On images in options it is right or wrong as d3-turbo is on 99.7% of the questions (3 right answers become wrong, 5 wrong ones right): +0.1 points (95% interval −0.1 to +0.3).

Videos: tracking

Measure	d3-turbo card	d3-turbo (BF16)	d3-turbo-FP8
Track, 1 update per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.79 / 0.89 / 0.97	0.796 / 0.893 / 0.969	0.792 / 0.894 / 0.969
Track, 2 updates per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.80 / 0.89 / 0.97	0.807 / 0.902 / 0.966	0.804 / 0.901 / 0.965

The 96 tracking test clips of d3-turbo of at most 60 s (2,690 moments at rate 1, 5,377 at rate 2), sent as their own JPEG frames muxed into Matroska at the clip's frame rate without re-encoding. At rate 1, the tracks of the three targets of one clip (120 moments) come back with times that do not line up with the test's moments for both packages and count as misses. This package hits or misses as d3-turbo does on 98.5% and 99.3% of the moments with a visible target, and calls the target visible or not as d3-turbo does on 99.6% and 99.2%.

Text: tasksource-jev-typed-decisions, test split

Type	Requests	Top-1 agreement with d3-turbo	Accuracy, d3-turbo (BF16)	Accuracy, d3-turbo-FP8
Choice	11,033	98.93% (118 changed)	82.35%	82.37%
Yes / No	2,057	99.12% (18 changed)	88.53%	88.62%
Score	941	97.02% (28 changed)	73.01%	73.01%

Every usable row of the test split of tasksource/tasksource-jev-typed-decisions at revision 28c39125 (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, 164 answers changed (98.83%), answered by the vLLM Semantic Router model runtime on one AMD Instinct MI300X. Agreement compares the top answer of every question with d3-turbo on the same requests (Choice and Score: the highest-probability option; Yes / No: p ≥ 0.5). An answer is correct when it is the request's target (Yes / No: the target's side of 0.5). Accuracy is given per type only, since the three types count a correct answer differently.

KL by type and Typed Decisions

KL(d3-turbo ‖ this package) on the tasksource test split, mean / median: Choice 0.00167 / 0.000090, Yes / No 0.00089 / 0.000031, Score 0.00176 / 0.000256.

Typed Decisions (LocalLLaMA/typed-decisions, all test split at d9eda7f3: 400 cases of five questions, one /v1/systemone request per case, scored as the dataset's bench/score_with_jobs.py scores them; the runtime gives a Yes / No answer as P(true), which is turned into the true / false pair the scorer reads). Both columns ran through the model runtime on the MI300X above, so they differ only in the weights; the dataset's leaderboard lists d3-turbo at 0.736 accuracy (KL 0.438) from its own submission.

	d3-turbo (BF16, model runtime)	d3-turbo-FP8 (model runtime)
Accuracy ↑	0.7260	0.7275
KL from gold ↓	0.4401	0.4376
Brier ↓	0.163	0.161
ECE ↓	0.0789	0.0806

_{Built on 2026-10-20 by fp8_package.py in bench/d3-quant-eval, the recipe of vllm-sr/d3-FP8: each output channel's scale is its largest absolute weight divided by 448, rounded to BF16. Evaluation runs, all on one AMD Instinct MI300X: the Decision Index text suite with the official 0.3 kit (decision-index f5a6e4f, as d3-turbo v1.1's own run) through d3q, one request at a time; the public vision suite with the same kit; Box, images in options, tracking, the speed benchmark and Typed Decisions through vllm-srun serve (model runtime from main as of 2026-10-20) at its default settings.}

License

Apache-2.0 (LICENSE), as d3-turbo. Built on vllm-sr/d3-turbo.

Citation

@misc{d3turbo_2026,
  title        = {{d3-turbo}: A Multimodal Decision Model that Reads Once},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-turbo}}
}