Qwen3.5-9B-NVFP4A16

Quantized version of Qwen/Qwen3.5-9B using NVFP4A16 (4-bit floating point weights, 16-bit activations).

Quantization Details

Method: NVFP4A16 (weight-only, per group 16)
Tool: vllm-project/llm-compressor
Ignored modules: lm_head, visual encoder (re:.*visual.*), linear attention (re:.*linear_attn.*), MTP modules (re:.*mtp.*)

vllm serve 2imi9/Qwen3.5-9B-NVFP4A16

Safetensors

Model size

7B params

Tensor type

F32

BF16

F8_E4M3

Base model

Finetuned

Quantized

(166)

this model