Resources

vLLM 0.23.1rc1.dev774+g2b802ad5b.d20260706. . load this model encounter OOM !!

#18 opened 3 days ago by

xuefeizzzzzz

SGLang support?

➕ 5

#17 opened 6 days ago by

danielus

[need help] vllm 0.24.0 on dgx spark warn Your GPU does not have native support for FP4

#16 opened 7 days ago by

imshenshen

doubled TPS NVFP4 vs FP8

🔥 3

#15 opened 7 days ago by

sSeeMyRolexXx

Fallback to marlin kernel give wired/incorrect and sometime gable result

#14 opened 7 days ago by

sapjunior

this model hallucinates easily

#13 opened 8 days ago by

batsclamp

vLLM + HGX B200: Your GPU does not have native support for FP4 computation but FP4 quantization is being used.

👍 3

#12 opened 8 days ago by

cgelias

Works via Ollama, but the ~40 tok/s MTP speedup requires raw llama-server (Ollama has no --spec-type flag) - plus vLLM lockup findings on DGX Spark

#11 opened 8 days ago by

darkmatter2222

为什么全是！！！！！！！！！

#10 opened 8 days ago by

jiecong

Why does this say 18B?

#9 opened 8 days ago by

darkmatter2222

NVFP4 vs FP8 on an RTX PRO 6000 Blackwell (vLLM 0.24): sadly, FP4 is still not faster than FP8 - Qwen3.6‑27B

👀👍 11

#8 opened 8 days ago by

janreges3

CUDA error: an illegal memory access was encountered using vllm(sm120)

#7 opened 9 days ago by

shakhizat

Does it support MTP?

➕👍 3

#6 opened 9 days ago by

sharon8811

will it work on rtx 5090 32gb vram ?

#5 opened 9 days ago by

arunsahu44

nvidia/Qwen3.6-27B-NVFP4 outputs repeated "!" / "d" tokens on Blackwell SM120

➕ 1

#4 opened 9 days ago by

richgua

Could we include BF16 as well as FP8 in the benchmarks please?

➕👀 5

#3 opened 9 days ago by

haydonryan

Qwen-AgentWorld-35B-A3B-NVFP4?

👍 2

#2 opened 10 days ago by

nagelanping

Approved

👍 1

#1 opened 10 days ago by

jasionkajakub