API Try Docs Blog

d1-2B-ONNX

ONNX version of d1-2B for cross-platform inference with ONNX Runtime.

Find architecture and usage details in the original model card.

Files

---

Each precision pairs a decoder with a vision encoder.

Precision	Decoder	Vision encoder	Size	Device
q4	decoder_q4	vision_encoder_q8	1.69 GB	CPU
q8	decoder_q8	vision_encoder_fp16	2.51 GB	CPU
q4f16	decoder_q4f16	vision_encoder_fp16	1.69 GB	GPU
q8f16	decoder_q8f16	vision_encoder_fp16	2.34 GB	GPU
fp16	decoder_fp16	vision_encoder_fp16	3.90 GB	GPU
fp32	decoder	vision_encoder	7.80 GB	Reference

The fp16, q4f16 and q8f16 decoders run with fp16 activations, the others with fp32. q4 and q4f16 keep the embedding, the LM head and 27 of the 153 layer projections in int8 and the rest in int4.

Use with ONNX Runtime

---

The vision encoder maps each image, or each temporal patch of a video, to projected features. The decoder answers a state and up to 16 questions, packed into one row without padding, in a single forward pass with no cache. It takes input_ids, attention_mask, position_ids, the features at the <|image|> and <|video|> placeholders (image_features, video_features), question_ids (-1 on the state, k on question k) and the flat positions of the answer slots (slot_positions). It returns logits at the answer slots and the final-norm hidden_states.

The checkpoint's Python code in this repository builds the rows and turns these outputs into typed answers. For image and video preprocessing, use the values in processor_config.json.

On GPU, use fp16, q4f16 or q8f16. The CUDA execution provider has no fp32 GroupQueryAttention kernel, so the other graphs run the sliding-window layers on the CPU.

In the browser, use import * as ort from 'onnxruntime-web/webgpu': the default onnxruntime-web import cannot run these graphs. On WebGPU, pair decoder_q4f16 with the float32 side graph of the q4 bundle (vision_encoder_q8), and keep each packed row under about 8,000 tokens for q4.

d1-2B reads its answers from a single forward pass, so there are no sampling parameters to set. More options in the ONNX guide.