README.md

---
library_name: peft
license: mit
language:
- en
base_model:
- meta-llama/Llama-3.2-1B-Instruct
tags:
- function-calling
- tool-use
- qlora
- lora
---

# Llama-3.2-1B-Instruct — Function Calling (QLoRA)

A LoRA adapter that teaches Llama-3.2-1B-Instruct to emit well-formed function/tool calls. The base model understands tool-use intent but produces structurally incorrect JSON; this adapter fixes the output format.

## What it does

**Before (base model):**
```json
{"type": "function", "function": "get_weather", "parameters": {"city": "Nagoya", "unit": "celsius"}}
```

**After (with this adapter):**
```json
[{"name": "get_weather", "arguments": {"city": "Nagoya", "unit": "celsius"}}]
```

The base model used `"parameters"` (wrong key) and a flattened structure. The adapter corrects it to the standard `name`/`arguments` format, wrapped in a list.

## How to use

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "meta-llama/Llama-3.2-1B-Instruct"
adapter_id = "Thanush16/llama-3.2-1b-function-calling"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
```

Build prompts with `tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True)`.

## Training

- **Base model:** meta-llama/Llama-3.2-1B-Instruct
- **Method:** QLoRA (4-bit base, LoRA rank 16 on attention + MLP layers, ~0.9% of params trained)
- **Dataset:** [Salesforce/xlam-function-calling-60k](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k), 500-example subset
- **Config:** 2 epochs, effective batch size 16, learning rate 2e-4
- **Hardware:** single free Kaggle T4 GPU
- **Final mean token accuracy:** ~88%

## Limitations

Trained on 500 examples to fix output *format*, not to be a broadly capable tool-use model. It reliably produces well-formed calls but wasn't evaluated for tool-selection accuracy on hard or ambiguous queries. Only the LoRA adapter is released — load it alongside the base model.