google

/

gemma-4-12B-it

like 336

Follow

Google 57k

Preserve the correct ordering for OpenAI-style assistant turns that include both visible content and tool_calls

• [{"role": "assistant", "content": ..., "tool_calls": ...}, {"role": "tool", ...}] was previously rendered as tool_calls, tool_responses, content with an extra <turn|> token, now renders as content, tool_calls, tool_responses.

Avoid emitting a duplicate <|tool_response> opener for the legacy assistant-embedded tool_responses

• [{"role": "assistant", "tool_calls": ...}, {"role": "assistant", "tool_responses": ...}] was previously rendered as <|tool_response><|tool_response> ... <tool_response|>. An extra <|tool_response> was emitted.

Example before vs after:

<bos><|turn>system
<|think|>
<|tool>declaration:get_current_pe_ratio{description:<|"|>Get the current price-to-earnings ratio for a stock ticker.<|"|>,parameters:{properties:{ticker:{description:<|"|>Stock ticker. Example: INTC for Intel Corporation.<|"|>,type:<|"|>STRING<|"|>}},required:[<|"|>ticker<|"|>],type:<|"|>OBJECT<|"|>}}<tool|><turn|>
<|turn>user
What does a P/E ratio mean, and what's NVIDIA's current one?<turn|>
<|turn>model
<|channel>thought
The user is asking two things:
1.  The meaning of a P/E ratio.
2.  The current P/E ratio for NVIDIA.

I can explain the P/E ratio based on general knowledge.
To get NVIDIA's current P/E ratio, I should use the `get_current_pe_ratio` tool.

Plan:
1.  Explain what a P/E ratio is.
2.  Call `get_current_pe_ratio` with the ticker 'NVDA' (NVIDIA's ticker).
3.  Provide the result to the user.
<channel|><|tool_call>call:get_current_pe_ratio{ticker:<|"|>NVDA<|"|>}<tool_call|><|tool_response>response:get_current_pe_ratio{value:<|"|>{"ticker": "NVDA", "pe_ratio": 35.13}<|"|>}<tool_response|>A **P/E ratio**, or Price-to-Earnings ratio, is a key financial metric used to value a company. It represents the relationship between a company's current share price and its earnings per share (EPS). 

Essentially, it tells you how much investors are willing to pay for every $1 of profit the company generates. 

*   **High P/E:** Often suggests that investors expect high growth in the future, or that the stock is overvalued.
*   **Low P/E:** May indicate that the company is undervalued, or that investors have low expectations for its future growth.

To get NVIDIA's current P/E ratio, let me look that up for you.<turn|>
<|channel>thought
The tool call has already been made and the result is available: `{"ticker": "NVDA", "pe_ratio": 35.13}`. I should now incorporate this value into the final response.
<channel|>NVIDIA's current P/E ratio is **35.13**.<turn|>

Note that an extra <turn|> was emitted. Although tool call was emitted after visible content, the template rendered tool call and tool result before content.

vs

<bos><|turn>system
<|think|>
<|tool>declaration:get_current_pe_ratio{description:<|"|>Get the current price-to-earnings ratio for a stock ticker.<|"|>,parameters:{properties:{ticker:{description:<|"|>Stock ticker. Example: INTC for Intel Corporation.<|"|>,type:<|"|>STRING<|"|>}},required:[<|"|>ticker<|"|>],type:<|"|>OBJECT<|"|>}}<tool|><turn|>
<|turn>user
What does a P/E ratio mean, and what's NVIDIA's current one?<turn|>
<|turn>model
<|channel>thought
The user is asking two things:
1.  The meaning of a P/E ratio.
2.  The current P/E ratio for NVIDIA.

I can explain the P/E ratio based on general knowledge.
To get NVIDIA's current P/E ratio, I should use the `get_current_pe_ratio` tool.

Plan:
1.  Explain what a P/E ratio is.
2.  Call `get_current_pe_ratio` with the ticker 'NVDA' (NVIDIA's ticker).
3.  Provide the result to the user.
<channel|><|tool_call>call:get_current_pe_ratio{ticker:<|"|>NVDA<|"|>}<tool_call|><|tool_response>response:get_current_pe_ratio{value:<|"|>{"ticker": "NVDA", "pe_ratio": 35.13}<|"|>}<tool_response|>A **P/E ratio**, or Price-to-Earnings ratio, is a key financial metric used to value a company. It represents the relationship between a company's current share price and its earnings per share (EPS). 

Essentially, it tells you how much investors are willing to pay for every $1 of profit the company generates. 

*   **High P/E:** Often suggests that investors expect high growth in the future, or that the stock is overvalued.
*   **Low P/E:** May indicate that the company is undervalued, or that investors have low expectations for its future growth.

To get NVIDIA's current P/E ratio, let me look that up for you.<turn|>
<|channel>thought
The tool call has already been made and the result is available: `{"ticker": "NVDA", "pe_ratio": 35.13}`. I should now incorporate this value into the final response.
<channel|>NVIDIA's current P/E ratio is **35.13**.<turn|>

To reproduce, render the following trace:

[
  {
    "content": "What does a P/E ratio mean, and what's NVIDIA's current one?",
    "role": "user"
  },
  {
    "content": "A **P/E ratio**, or Price-to-Earnings ratio, is a key financial metric used to value a company. It represents the relationship between a company's current share price and its earnings per share (EPS). \n\nEssentially, it tells you how much investors are willing to pay for every $1 of profit the company generates. \n\n*   **High P/E:** Often suggests that investors expect high growth in the future, or that the stock is overvalued.\n*   **Low P/E:** May indicate that the company is undervalued, or that investors have low expectations for its future growth.\n\nTo get NVIDIA's current P/E ratio, let me look that up for you.",
    "reasoning": "The user is asking two things:\n1.  The meaning of a P/E ratio.\n2.  The current P/E ratio for NVIDIA.\n\nI can explain the P/E ratio based on general knowledge.\nTo get NVIDIA's current P/E ratio, I should use the `get_current_pe_ratio` tool.\n\nPlan:\n1.  Explain what a P/E ratio is.\n2.  Call `get_current_pe_ratio` with the ticker 'NVDA' (NVIDIA's ticker).\n3.  Provide the result to the user.",
    "role": "assistant",
    "tool_calls": [
      {
        "function": {
          "arguments": "{\"ticker\": \"NVDA\"}",
          "name": "get_current_pe_ratio"
        },
        "id": "chatcmpl-tool-98b92c0758642b63",
        "type": "function"
      }
    ]
  },
  {
    "content": "{\"ticker\": \"NVDA\", \"pe_ratio\": 35.13}",
    "role": "tool",
    "tool_call_id": "chatcmpl-tool-98b92c0758642b63"
  },
  {
    "content": "NVIDIA's current P/E ratio is **35.13**.",
    "reasoning": "The tool call has already been made and the result is available: `{\"ticker\": \"NVDA\", \"pe_ratio\": 35.13}`. I should now incorporate this value into the final response.\n",
    "role": "assistant"
  }
]

https://github.com/vllm-project/vllm/pull/42776