nvidia/Qwen3.6-27B-NVFP4 outputs repeated "!" / "d" tokens on Blackwell SM120
I am seeing degenerate generation with nvidia/Qwen3.6-27B-NVFP4 on RTX PRO 6000 Blackwell SM120 using vLLM.
The model loads successfully, but even a minimal prompt like:
"Responde exactamente con la palabra: OK"
returns repeated tokens such as:
"d d d d d d d d d d"
or repeated exclamation marks.
Same server and GPU work correctly with unsloth/Qwen3.6-27B-NVFP4.
Hardware:
- GPU: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
- Compute capability: 12.0
- CUDA: 13.0
- VRAM: 96 GB
vLLM:
- Tested with vllm/vllm-openai:nightly
- Also tested cu130-nightly
- Required --quantization modelopt_fp4 to load
Minimal command and logs available if needed.
Confirm this but only on nightly build. On release 0.24.0 works well
Confirm this but only on nightly build. On release 0.24.0 works well
Update: I can confirm this issue only happens on the vLLM nightly build.
When using vllm/vllm-openai:nightly, nvidia/Qwen3.6-27B-NVFP4 produces degenerate outputs such as repeated ! or repeated d tokens.
However, with the stable vLLM release 0.24.0, the model works correctly.
So this appears to be a vLLM nightly regression rather than a model weight issue.
Same issue with both vllm==0.24.0 and the newest vllm-nightly.
:(
The problem should be solved now in the latest vllm/vllm-openai:v0.25.1.
same issue with vllm/vllm-openai:v0.25.1
Multi-GPU ? Add argument "--disable-custom-all-reduce"