added response_template

#1
by nico-martin HF Staff - opened
ONNX Community org

This PR adds support for the response_template in tokenizer_config.json. A response template tells the tokenizer how to parse the model's raw output into a structured assistant message (reasoning_content, content), so tokenizer.parse_response() and chat pipelines return parsed messages instead of raw text. See https://huggingface.co/docs/transformers/en/chat_response_parsing

Template
This is the existing deepseek-r1 template, copied unchanged from onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX. This model is a DeepSeek-R1-Distill-Qwen-1.5B fine-tune that keeps its chat template and output format: <think>…</think> reasoning followed by the answer. The original model (agentica-org/DeepCoder-1.5B-Preview) has no response_template of its own.

How it was tested
We ran this checkpoint (q4f16) with Transformers.js, using its own chat template, on a tool request, a plain answer, a plain question with a tool available, and a run with thinking disabled. Every well-formed output parsed correctly, both in one shot and when streamed. Upstream Python transformers gives the same results on the same outputs.

The template describes the checkpoint's output format. It does not mean that every response contains every region, and it does not guarantee tool-use quality.

nico-martin changed pull request status to open
Xenova changed pull request status to merged

Sign up or log in to comment