d3-ultra-FP8

d3-ultra-FP8 is d3-ultra, the newest 27B multimodal decision model of Decision 3.0, with its decoder's projections in FP8. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, rate on a scale, or point to where something is in an image or a video. It reads the input once per request, answers every question on it and returns a probability for every answer, as d3-ultra does, in 31.9 GB of weights instead of 52.2 GB.

Parameters	26.09B, including the 0.46B vision encoder
Base model	vllm-sr/d3-ultra, the release of 2026-10-24 (the weights its Decision Index runs used)
Quantization	FP8 W8A8 (compressed-tensors float-quantized, FP8_DYNAMIC): E4M3 weights with one BF16 scale per output channel, activations quantized to FP8 per token at run time. 304 projections; the Gated DeltaNet input projections, the vision tower, the embeddings and the readout stay as published
Calibration data	None: weight scales come from the weights, activation scales from each request
Weights on disk	31.9 GB (d3-ultra: 52.2 GB)
Needs	FP8 matrix units: AMD Instinct MI300 / MI350 series, or NVIDIA compute capability 8.9 or later. Measured on one AMD Instinct MI300X
Inputs	Text or JSON, plus images and videos (several per request); images can also sit inside questions and options
Decision types	Choice · Yes / No · Score · Box · Track, all measured against d3-ultra
License	Apache-2.0

Highlights

• 71.52 on the Jev Decision Index 0.3 public suite against d3-ultra's 71.57, every area within 0.3 points, with d3-ultra's top answer on 99.3% of 272,229 questions.
• 75.51 on the public vision suite against 75.35 for d3-ultra on the same GPU, with d3-ultra's top answer on 97.7% of 12,210 questions.
• Box and tracking hold: Box accuracy at IoU 0.5 is 0.871 against 0.870, tracking success AUC 0.774 / 0.780 against 0.778 / 0.782, and images in questions and options 0.952 against 0.952.
• 31.9 GB of weights instead of 52.2 GB, and 40 ms against 51 ms at the median for one text question at a time on one AMD Instinct MI300X.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.17" torch torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention  # optional: fast GPU kernels for the linear-attention layers

import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-ultra-FP8")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")


def asset(name):
    return hf_hub_download("vllm-sr/d3-ultra", f"assets/{name}")


# Text: several questions, one call
print(model.system_one(
    state="Ticket: the invoice total is wrong after the plan change.",
    questions={
        "route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "tech": "Technical support"}},
        "urgent": {"type": "noul", "instructions": "Is the customer asking for an urgent fix?"},
    },
))

# Box: who is in the canoe? Three questions, one read of the painting
print(model.system_one(
    state="A painting of fur traders on a river.",
    images=[asset("fur-traders.jpg")],  # local paths, http(s) URLs, PIL images or base64-encoded images
    questions={
        "cat": {"type": "box", "instructions": "The cat"},
        "canoe": {"type": "box", "instructions": "The canoe"},
        "dog": {"type": "box", "instructions": "A dog"},
    },
))

# Photo to etching: the question holds a photo, each option is an etching
print(model.system_one(
    state="An architecture quiz.",
    questions={"pick": {"type": "choice", "instructions": "Which 18th-century etching shows the same building as <building>?",
                        "criteria": {"a": "<a>", "b": "<b>", "c": "<c>", "d": "<d>"}}},
    assets={"<building>": asset("pantheon.jpg"), "<a>": asset("pantheon-a.jpg"), "<b>": asset("pantheon-b.jpg"),
            "<c>": asset("pantheon-c.jpg"), "<d>": asset("pantheon-d.jpg")},
))

load_d3 is d3-ultra's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its weights in FP8, and each quantized projection becomes a layer that quantizes its input to FP8 per row with one Triton kernel and multiplies in torch._scaled_mm. Prompting, reading the input once, the FP32 readout, Box and Track answers, images and videos are d3-ultra's own; the answers are formatted as on d3-ultra's card. AutoModel.from_pretrained(..., trust_remote_code=True) alone does not run FP8 weights.

Checked on one AMD Instinct MI300X with transformers 5.17.0 and compressed-tensors 0.17.0: the three requests above are answered as d3-ultra answers them through the same path. Every edge of the cat and canoe boxes is within 0.004 of d3-ultra's, the dog is not found by either (confidence 0.05 against 0.00), etching d gets 0.996 from both, and the ticket goes to billing with the same probability to five decimals.

Run with vLLM Semantic Router

The vLLM Semantic Router model runtime loads d3-ultra-FP8 as it loads d3-ultra, verifies the package against its MODEL_MANIFEST.json, and keeps the weights in FP8 on the GPU:

pip install -e "src/model-runtime[multimodal]" flash-linear-attention   # in a checkout of vllm-project/semantic-router
vllm-srun serve vllm-sr/d3-ultra-FP8 --revision 440be8e8509f157f2cb61950ea412bb3436a4ea6 --port 8899 --max-request-bytes 67108864

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-ultra-FP8",
  "state": "Write a Python function that reverses a string.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

This package was served with the model runtime from main as of 2026-10-29. A runtime that predates d3-ultra's prompt layout and per-request temperature refuses the package, since the model identity in its manifest differs. vllm-srun takes a Hub package that is not built in only at a full 40-hex --revision; requests with videos need --max-request-bytes above the default 8 MiB. The Box, images-in-options, tracking and tasksource results below come from the model runtime run on a local copy of this repository (the same weights and loaded files).

The full speed benchmark of the d3-turbo packages has not been run on this package. One question per request on the tasksource test split below, one request at a time through the model runtime on one AMD Instinct MI300X: median 40.4 ms against 51.3 ms for d3-ultra.

Evaluation

Every comparison is against d3-ultra (BF16) on the same questions. The text index compares with d3-ultra's own public run (vllm-sr/d3-ultra-decision-index); images, Box, images in options and tracking compare with d3-ultra run on the same MI300X through the same engine. d3-ultra's vision run on this MI300X gives 75.35 against 75.33 in its public run, so the differences below are the quantization's.

Text: Jev Decision Index 0.3, public suite

	d3-ultra (BF16)	d3-ultra-FP8
Decision Index (public)	71.57	71.52
Raw index	78.59	78.55
Knowledge	62.3	62.0
Language	72.4	72.5
Retrieval	71.8	71.8
Tools	87.8	87.8
Arts	63.5	63.3

The 140,178 requests of d3-ultra's public run, scored by the official 0.3 kit; the index reads the 110,087 requests (272,229 questions) of its 37 benchmarks. Re-scored by the same scorer as this package, d3-ultra's results give the 71.57 of its public run. This package's run, on another build of the same 0.3 suite, also answered 442 requests that d3-ultra's run does not have; they are left out, and scoring this package on all its own requests gives the same 71.52. This package gives d3-ultra's top answer on 99.26% of the index's questions (2,023 change); the lowest agreement is on HLE text (92.0%), ChessBench legal moves (94.8%) and GPQA Diamond (94.9%). KL(d3-ultra ‖ this package): mean 0.0012, median 0.000022, 95th percentile 0.0043, 99th percentile 0.022.

Images: Jev Decision Index 0.3, public vision suite

	d3-ultra (BF16)	d3-ultra-FP8
Public vision index	75.35	75.51

By benchmark (accuracy)

Benchmark	d3-ultra (BF16)	d3-ultra-FP8
CV-Bench	0.892	0.891
BLINK	0.816	0.816
RealWorldQA	0.815	0.815
CharXiv	0.852	0.844
InfographicVQA	0.971	0.980
Mind2Web	0.892	0.896
Winoground	0.920	0.921
KIE (CORD+FUNSD)	0.954	0.953
Moderation (Hateful Memes)	0.702	0.707
R-Bench-M	0.526	0.526
MMMU-Pro vision	0.513	0.512

All 12,210 rows, both packages through the same engine on the same MI300X. This package gives d3-ultra's top answer on 97.72% of the questions (278 change), most of the changes on R-Bench-M (91.9% agreement) and MMMU-Pro vision (93.0%), where d3-ultra itself is right about half the time; every other benchmark agrees on 97.9% to 99.8%. KL: mean 0.0036, median 0.00018, 99th percentile 0.051.

Images: Box and images in options

Measure	d3-ultra (BF16)	d3-ultra-FP8
Box: accuracy at IoU 0.5	0.8703	0.8706
Box: accuracy at IoU 0.75	0.7084	0.6964
Box: confidence AUROC	0.9639	0.9613
Images in questions and options: accuracy	0.9524	0.9524

The Box test set of d3-turbo (5,836 items) and its images-in-options test set (2,752 questions), scored as d3-turbo's evaluation scores them. On Box this package hits or misses as d3-ultra does on 98.9% of the items (32 hits become misses, 34 misses become hits): +0.03 points at IoU 0.5 (95% interval −0.2 to +0.3); at IoU 0.75 it is 1.2 points lower (mean IoU 0.766 against 0.771). Every family stays within 0.011 (COCO exemplar 0.679 → 0.688, spatial 0.650 → 0.655, unique 0.799 → 0.802, receipts 0.957 → 0.946; charts, screens, shapes and tables within 0.002). On images in options it is right or wrong as d3-ultra is on 99.5% of the questions (7 right answers become wrong, 7 wrong ones right).

Videos: tracking

Measure	d3-ultra (BF16)	d3-ultra-FP8
Track, 1 update per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.778 / 0.891 / 0.974	0.774 / 0.887 / 0.972
Track, 2 updates per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.782 / 0.887 / 0.968	0.780 / 0.886 / 0.964

The 96 tracking test clips of d3-turbo of at most 60 s (2,690 moments at rate 1, 5,377 at rate 2), sent as their own JPEG frames muxed into Matroska at the clip's frame rate without re-encoding. At rate 1, the tracks of the three targets of one clip (120 moments) come back with times that do not line up with the test's moments for both packages and count as misses. This package hits or misses as d3-ultra does on 99.1% and 99.2% of the moments with a visible target, and calls the target visible or not as d3-ultra does on 99.3% and 98.8%.

Text: tasksource-jev-typed-decisions, test split

Type	Requests	Top-1 agreement with d3-ultra	Accuracy, d3-ultra (BF16)	Accuracy, d3-ultra-FP8
Choice	11,033	98.46% (170 changed)	82.66%	82.76%
Yes / No	2,057	98.88% (23 changed)	88.53%	88.19%
Score	941	98.19% (17 changed)	74.71%	74.60%

Every usable row of the test split of tasksource/tasksource-jev-typed-decisions (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, answered by the vLLM Semantic Router model runtime on one AMD Instinct MI300X. Agreement compares the top answer of every question with d3-ultra on the same requests (Choice and Score: the highest-probability option; Yes / No: p ≥ 0.5). An answer is correct when it is the request's target (Yes / No: the target's side of 0.5). Accuracy is given per type only, since the three types count a correct answer differently. KL(d3-ultra ‖ this package), mean / median: Choice 0.0046 / 0.000021, Yes / No 0.0021 / 0.000016, Score 0.0045 / 0.00021.

_{Built on 2026-10-29 by fp8_package.py in bench/d3-quant-eval, the recipe of vllm-sr/d3-turbo-FP8: each output channel's scale is its largest absolute weight divided by 448, rounded to BF16. Evaluation runs, all on one AMD Instinct MI300X: the Decision Index text suite with the official 0.3 kit through d3q, one request at a time; the public vision suite with the same kit; Box, images in options, tracking and tasksource through the model runtime from main as of 2026-10-29 at its default settings.}

License

Apache-2.0 (LICENSE), as d3-ultra. Built on vllm-sr/d3-ultra.

Citation

@misc{d3ultra_2026,
  title        = {{d3-ultra}: A Multimodal Decision Model},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-ultra}}
}