ValueError: There is no module or parameter named 'lm_head.biases' in Qwen3ForCausalLM
#1
by
the1dv
- opened
Trying to run on vLLM 0.9.2 with cuda 12.4
I don't think the vLLM supports MLX models. If you are using Mac, you can deploy it with mlx-lm or LM-Studio, which come with the mlx support.
Would have assumed as it was AWQ it would work but all good :)