File size: 3,247 Bytes
cac90d2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
base_model: unsloth/functiongemma-270m-it
library_name: gguf
license: gemma
tags:
- function-calling
- tool-use
- agent
- llama-cpp
- gguf
- unsloth
- llama-agent
datasets:
- victor/functiongemma-agent-sft
pipeline_tag: text-generation
---
# FunctionGemma Agent GGUF
A fine-tuned version of [FunctionGemma-270M](https://huggingface.co/unsloth/functiongemma-270m-it) for agentic tool-calling tasks, converted to GGUF format for use with llama.cpp and [llama-agent](https://github.com/ggml-org/llama.cpp/tree/master/tools/agent).
## Model Details
| Property | Value |
|----------|-------|
| Base Model | [unsloth/functiongemma-270m-it](https://huggingface.co/unsloth/functiongemma-270m-it) |
| Fine-tuned Model | [victor/functiongemma-agent-finetuned](https://huggingface.co/victor/functiongemma-agent-finetuned) |
| Training Dataset | [victor/functiongemma-agent-sft](https://huggingface.co/datasets/victor/functiongemma-agent-sft) |
| Quantization | Q4_K_M (4-bit) |
| Parameters | 270M |
## Training
Fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) with LoRA on HuggingFace Jobs infrastructure.
**Training Configuration:**
- LoRA rank: 128, alpha: 256
- Epochs: 3
- Learning rate: 2e-4
- Batch size: 4, gradient accumulation: 2
- Hardware: NVIDIA A100-80GB
- Training method: SFT with `train_on_responses_only`
**Dataset:** 7,500 synthetic examples covering:
- Multi-step tool chaining (glob → read → edit)
- Error recovery patterns
- Clarification dialogs
- No-tool responses
- Parallel tool calls
## Tools
The model is trained on 5 tools matching llama-agent:
| Tool | Description |
|------|-------------|
| `read_file` | Read file contents with line numbers |
| `write_file` | Create or overwrite a file |
| `edit_file` | Find and replace text in a file |
| `glob` | Find files matching pattern |
| `bash` | Execute shell command |
## Usage
### With llama.cpp
```bash
# Download
wget https://huggingface.co/victor/functiongemma-agent-gguf/resolve/main/functiongemma-270m-it.Q4_K_M.gguf
# Run inference
./llama-cli -m functiongemma-270m-it.Q4_K_M.gguf -p "<start_of_turn>user
Read the main.py file
<end_of_turn>
<start_of_turn>model"
```
### With llama-agent
```bash
./llama-agent -m functiongemma-270m-it.Q4_K_M.gguf
```
## Format
Uses FunctionGemma's native format with `<escape>` delimiters:
```
<start_of_turn>user
Fix the typo in config.json
<end_of_turn>
<start_of_turn>model
<think>I need to find and read the config file first.</think>
<start_function_call>call:glob{pattern:<escape>**/config.json<escape>}<end_function_call>
<end_of_turn>
<start_of_turn>developer
<start_function_response>response:glob{stdout:<escape>src/config.json<escape>,stderr:<escape><escape>,exit_code:0}<end_function_response>
<end_of_turn>
...
```
## License
This model inherits the [Gemma license](https://ai.google.dev/gemma/terms) from the base model.
## Links
- Training script: [victor/llama-agent-training](https://huggingface.co/victor/llama-agent-training)
- Dataset: [victor/functiongemma-agent-sft](https://huggingface.co/datasets/victor/functiongemma-agent-sft)
- llama-agent: [github.com/ggml-org/llama.cpp/tools/agent](https://github.com/ggml-org/llama.cpp/tree/master/tools/agent)
|