error when run vllm serve Qwen/Qwen3-ASR-1.7B with vllm
#1
by phamcao - opened
pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=1972002) Value error, Model architectures ['Qwen3ASRForConditionalGeneration'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=1972002) For further information visit https://errors.pydantic.dev/2.12/v/value_error
I have error when run vllm serve Qwen/Qwen3-ASR-1.7B with vllm version" vllm 0.16.0rc1.dev2+g80b918f2b"
I am using older vllm version "0.15.2rc1.dev119+g7c233dbb3" (llm/vllm-openai:nightly) & it works for me via docker.
Here is a full guide steps:
Create a Dockerfile
FROM vllm/vllm-openai:nightly
RUN uv pip install --system mistral-common[soundfile] vllm[audio]
ENV VLLM_DISABLE_COMPILE_CACHE=1
Then run:
docker build -t vllm-qwen:latest .
docker run --gpus all -p 8000:8000 vllm-qwen Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000