Instructions to use skshreyas714/tiny-aya-global-tool-calling-500steps with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use skshreyas714/tiny-aya-global-tool-calling-500steps with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global") model = PeftModel.from_pretrained(base_model, "skshreyas714/tiny-aya-global-tool-calling-500steps") - Notebooks
- Google Colab
- Kaggle
Tiny Aya Global Tool-Calling Adapter
This repository contains a PEFT LoRA adapter fine-tuned from CohereLabs/tiny-aya-global. It is an adapter, not a standalone copy of the base model. The base model must be loaded when using this adapter.
Training
The adapter was trained for 500 optimizer steps using TRL SFTTrainer and 4-bit NF4 QLoRA. Configuration: learning rate 1e-4, LoRA rank 16, alpha 32, dropout 0.05, per-device batch size 2, gradient accumulation 4, BF16 compute, gradient checkpointing, and maximum sequence length 8192.
The training pool was prepared from Salesforce xLAM Function Calling 60K, Team-ACE ToolACE, and Salesforce APIGen-MT-5K. The prepared training split contains 5,000 xLAM, 2,500 ToolACE, and 1,000 APIGen-MT examples. The run stopped after 500 steps, so it did not necessarily consume every training example. Dataset rows and test predictions are not included in this repository.
Inference
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
base_id = "CohereLabs/tiny-aya-global"
adapter_id = "skshreyas714/tiny-aya-global-tool-calling-500steps"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
compute_dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
quantization = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=compute_dtype,
)
base = AutoModelForCausalLM.from_pretrained(
base_id,
device_map={"": 0},
dtype=compute_dtype,
quantization_config=quantization,
attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(base, adapter_id).eval()
messages = [{"role": "user", "content": "Find movie theaters near Los Angeles."}]
tools = [{"type": "function", "function": {"name": "search_movie_theaters", "description": "Find nearby movie theaters", "parameters": {"type": "object", "properties": {"latitude": {"type": "number"}, "longitude": {"type": "number"}, "distance": {"type": "integer"}}, "required": ["latitude", "longitude", "distance"]}}}]
inputs = tokenizer.apply_chat_template(
messages, tools=tools, add_generation_prompt=True,
tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False))
License and use
The base Tiny Aya model is under Cohere Labs' CC-BY-NC license and acceptable-use policy. The training mixture includes APIGen-MT, whose dataset card specifies CC BY-NC 4.0 and additional use restrictions concerning GPT-generated data. Review and comply with the base-model and every source-dataset license before use. This adapter is intended for non-commercial research; do not assume commercial-use rights.
The generated tool call is a model output. Validate it against the application's tool schema and authorization rules before execution.
- Downloads last month
- 47