froggeric

/

Qwen-Fixed-Chat-Templates

like 1.29k

Summary

With v22.1, thinking can be disabled internally by the template without updating the state used by vLLM's Qwen3 reasoning parser.

This affects:

• Inline <|think_off|>
• auto_disable_thinking_with_tools=true
• chat_template_kwargs.reasoning_effort="none"

For non-streaming requests, a normal final answer may be returned entirely as reasoning, with content=null. If include_reasoning=false, the answer can effectively disappear.

Tested revisions

• Qwen-Fixed-Chat-Templates: 2b50d8ef73e9ba606680856aaeb46ab94702e788
• Template version: qwen3.8-froggeric-v22.1
• vLLM: 311b3513af33bc29b4acb2fde2e9313e5e9966a0
• Model: Qwen/Qwen3.8-27B-FP8

Reproduction

Start vLLM with the custom template and Qwen3 reasoning parser:

vllm serve Qwen/Qwen3.8-27B-FP8 \
  --chat-template /path/to/chat_template.jinja \
  --reasoning-parser qwen3

Send a non-streaming request:

{
  "model": "Qwen/Qwen3.8-27B-FP8",
  "messages": [
    {
      "role": "user",
      "content": "Answer in one short sentence. <|think_off|>"
    }
  ],
  "stream": false
}

The template resolves ns_state.thinking to false and prefills:

<think>

</think>

The generated completion therefore contains only the final answer, without a generated </think> marker.

Expected behavior

{
  "message": {
    "reasoning": null,
    "content": "The final answer."
  }
}

Actual behavior

The Qwen3 parser may treat the complete answer as reasoning:

{
  "message": {
    "reasoning": "The final answer.",
    "content": null
  }
}

With include_reasoning=false, the reasoning field is removed and the response may appear empty.

The reverse mismatch is also possible: when enable_thinking=false is passed explicitly but a historical inline tag re-enables thinking, vLLM starts in CONTENT state and reasoning can leak into content.

Root cause

The template changes its private state here:

• Inline controls: chat_template.jinja:35-80
• Automatic tool toggle: chat_template.jinja:32-34
• reasoning_effort="none": chat_template.jinja:18-23

However, vLLM's Qwen3 parser only reads the original chat_template_kwargs.enable_thinking value:

https://github.com/vllm-project/vllm/blob/311b3513af33bc29b4acb2fde2e9313e5e9966a0/vllm/parser/qwen3.py#L225-L236

The non-streaming serving path passes only output.text to the parser, so it cannot see the empty <think></think> block that was inserted into the prompt:

https://github.com/vllm-project/vllm/blob/311b3513af33bc29b4acb2fde2e9313e5e9966a0/vllm/entrypoints/openai/chat_completion/serving.py#L945-L951

Top-level vLLM reasoning_effort="none" is safe because vLLM also derives enable_thinking=false. Placing "none" only inside chat_template_kwargs does not provide that synchronization.

Suggested resolution

Because a Jinja template cannot update the parser's request state after rendering, possible resolutions include:

1. Document that inline thinking toggles and auto_disable_thinking_with_tools are incompatible with --reasoning-parser qwen3.
2. Require callers to pass an explicit enable_thinking value matching the final template state.
3. Avoid allowing historical inline tags to override the request-level enable_thinking setting.
4. Add vLLM integration tests covering both streaming and non-streaming requests.

The tests should verify at least:

• enable_thinking=false
• Inline <|think_off|>
• chat_template_kwargs.reasoning_effort="none"
• auto_disable_thinking_with_tools=true
• enable_thinking=false combined with a historical <|think_on|>

The current 34 tests validate rendered string fragments, but they do not perform a render-to-vLLM-parser round trip.

Generated by GPT-5.6-Sol