Qwen3Guard-Gen-0.6B chat template error on vLLM despite tokenizer_config.json existing
#8
by
Chinofritz
- opened
I'm getting a chat template error when deploying Qwen3Guard-Gen-0.6B with vLLM, even though the model has a chat_template defined in tokenizer_config.json.
Setup:
- vLLM:
vllm/vllm-openai:v0.9.0.1 - Launch flags:
--max-model-len 2048 --trust-remote-code --served-model-name Qwen/Qwen3Guard-Gen-0.6B - Using LangChain's ChatOpenAI client
Error:
ValueError: As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one
The tokenizer_config.json clearly has a chat_template defined.
Has anyone successfully run Qwen3Guard-Gen with vLLM? Do these models need special handling for the chat template?
Thanks!