RadixArk

/

Qwen3.8-Flash-Next-NVFP4

like 47

Follow

RadixArk 195

Has anyone been able to get this to startup ok with the vllm-openai:qwen38-flash-next docker image?

ValueError: There is no module or parameter named 'ngram_embedding.weight_scale' in Qwen3_8FlashNextNGramEmbedding. The available parameters belonging to ngram_embedding (VocabParallelEmbedding) are: {'ngram_embedding.weight'}

  --model RadixArk/Qwen3.8-Flash-Next-NVFP4
  --max-num-seqs 12
  --tensor-parallel-size 2
  --gpu-memory-utilization 0.96
  --reasoning-parser qwen3
  --enable-auto-tool-choice 
  --tool-call-parser qwen3_coder
  --safetensors-load-strategy lazy
  --enable-prefix-caching 
  --max-model-len auto
  --enable-chunked-prefill
  --speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}'