Update tool use template to new format
#46
by Rocketknight1 HF Staff - opened
Do not merge this yet! This is a currently experimental PR to test a new template format for tool use, to go with the new JSON schema standard for passing tools. Please see the updated docs for more info.
To try it out, you should install the latest version of transformers
Then, load the tokenizer and template from this PR branch:
from transformers import AutoTokenizer
from typing import List, Dict
tokenizer = AutoTokenizer.from_pretrained("CohereForAI/c4ai-command-r-plus", revision="pr/46")
def internet_search(query: str):
"""Returns a list of relevant document snippets for a textual query retrieved from the internet
Args:
query: Query to search the internet with
"""
pass
def directly_answer():
"""Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history
"""
pass
conversation = [
{"role": "user", "content": "Whats the biggest penguin in the world?"}
]
tools = [internet_search, directly_answer]
new_tool_use_prompt = tokenizer.apply_chat_template(
conversation,
tools=tools,
chat_template="tool_use",
tokenize=False,
add_generation_prompt=True,
)
print(new_tool_use_prompt)
coherecode changed pull request status to merged