lxuechen/phi-2-tool-use

Model Summary

phi-2-tool-use is fine-tuned version of Phi-2 for function calling purposes. The model was fine-tuned on the public function call dataset glaiveai/glaive-function-calling-v2.

The purpose of the experiment is to understand the quality of the pre-trained Phi-2 model. phi-2-tool-use can generalize to call simple tools/functions not seen during fine-tuning.

Decoding

Format your prompt as

"""SYSTEM: {system_content}\n\nUSER: {user_content} {eos_token} ASSISTANT:"""

where system_content is the system message containing a description of the tool/function as a json schema, user_content is the user message, and eos_token is the EOS token. The model can handle multi-turn dialogue as it was trained on such data.

Here's a full-fledged example:

import torch
import transformers

model_name_or_path = "lxuechen/phi-2-tool-use"
model: transformers.PreTrainedModel = transformers.AutoModelForCausalLM.from_pretrained(
    model_name_or_path,
    low_cpu_mem_usage=True,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.float16
)
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name_or_path)

input_text = """SYSTEM: You are a helpful assistant with access to the following functions. Use them if required - { "name": "get_exchange_rate", "description": "Get the exchange rate between two currencies", "parameters": { "type": "object", "properties": { "base_currency": { "type": "string", "description": "The currency to convert from" }, "target_currency": { "type": "string", "description": "The currency to convert to" } }, "required": [ "base_currency", "target_currency" ] } }\n\nUSER: Convert 100 USD to CAD <|endoftext|> ASSISTANT:"""

outputs = model.generate(
    tokenizer(input_text, return_tensors="pt").to(model.device)['input_ids'],
    max_length=1024,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

The model was fine-tuned with SFT on glaiveai/glaive-function-calling-v2.

Hyperparameters:

• learning rate: 3% linear warmup, with a peak of 2e-5 and cosine decay
• epochs: 2
• batch size: 64
• context length: 2048