Qwen2.5-1.5B-Instruct-NVFP4

NVFP4 quantisation of Qwen/Qwen2.5-1.5B-Instruct, produced with NVIDIA TensorRT Model Optimizer on a 128 GB DGX Spark (GB10) so it runs through vLLM's Blackwell FP4 path.

No training or fine-tuning was involved - this is a numeric format conversion of somebody else's finished checkpoint, and it is a lossy one.

What is in this repo

Architecture	Qwen2ForCausalLM
Model type	qwen2
Quantisation	nvfp4
Tensors	926
Shards	1
Weights on disk	1.20 GB

Calibrated on 512 samples at 2048 tokens.

Speculative decoding (MTP)

This checkpoint has no MTP head, so multi-token-prediction speculative decoding is not available for it.

Measured performance

configuration	tokens/s
single stream	141.1
4 concurrent streams (aggregate)	593.5

Measured on this artifact on a DGX Spark (GB10, 128 GB unified) through vLLM 0.25.2 at 8192 context, max-num-seqs 4, gpu-memory-utilization 0.5, with the GPU clock capped at 2000 MHz. Token counts are taken from usage.completion_tokens, not from streamed chunk counts (1.004 tokens per chunk here, confirming no speculative decoding). Single-stream TTFT was 0.018 s on a short prompt. No quality evaluation was run: these are throughput numbers only.

Limitations

• 4-bit quantisation is lossy. No quality evaluation of this artifact against its bf16 source was run here.