added response_template

#1
by nico-martin HF Staff - opened

This PR adds support for the response_template in tokenizer_config.json. A response template tells the tokenizer how to parse the model's raw output into a structured assistant message (content, tool_calls), so tokenizer.parse_response() and chat pipelines return parsed messages instead of raw text. See https://huggingface.co/docs/transformers/en/chat_response_parsing

Template
Neither this repo nor the original model (google/functiongemma-270m-it) ships a response_template, so we built this one from the model's chat template and its real output:

  • Tool calls are <start_function_call>call:name{key:<escape>value<escape>,…}<end_function_call>, and several calls can follow each other.
  • After <end_function_response>, the model can continue in the same turn.
  • A turn ends with <end_of_turn>, <start_function_response> or <eos>.

The template only uses parsing features that upstream Python transformers supports.

How it was tested
We ran this checkpoint (q4f16) with Transformers.js, using its own chat template, on a single tool call, two parallel tool calls, a plain answer, a plain question with a tool available, and requests for Prism OS commands. Every well-formed output parsed correctly, both in one shot and when streamed. Upstream Python transformers gives the same results on the same outputs.

The template describes the checkpoint's output format. It does not mean that every response contains every region, and it does not guarantee tool-use quality.

nico-martin changed pull request status to open
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment