|
|
--- |
|
|
base_model: unsloth/functiongemma-270m-it |
|
|
library_name: gguf |
|
|
license: gemma |
|
|
tags: |
|
|
- function-calling |
|
|
- tool-use |
|
|
- agent |
|
|
- llama-cpp |
|
|
- gguf |
|
|
- unsloth |
|
|
- llama-agent |
|
|
datasets: |
|
|
- victor/functiongemma-agent-sft |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# FunctionGemma Agent GGUF |
|
|
|
|
|
A fine-tuned version of [FunctionGemma-270M](https://huggingface.co/unsloth/functiongemma-270m-it) for agentic tool-calling tasks, converted to GGUF format for use with llama.cpp and [llama-agent](https://github.com/ggml-org/llama.cpp/tree/master/tools/agent). |
|
|
|
|
|
## Model Details |
|
|
|
|
|
| Property | Value | |
|
|
|----------|-------| |
|
|
| Base Model | [unsloth/functiongemma-270m-it](https://huggingface.co/unsloth/functiongemma-270m-it) | |
|
|
| Fine-tuned Model | [victor/functiongemma-agent-finetuned](https://huggingface.co/victor/functiongemma-agent-finetuned) | |
|
|
| Training Dataset | [victor/functiongemma-agent-sft](https://huggingface.co/datasets/victor/functiongemma-agent-sft) | |
|
|
| Quantization | Q4_K_M (4-bit) | |
|
|
| Parameters | 270M | |
|
|
|
|
|
## Training |
|
|
|
|
|
Fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) with LoRA on HuggingFace Jobs infrastructure. |
|
|
|
|
|
**Training Configuration:** |
|
|
- LoRA rank: 128, alpha: 256 |
|
|
- Epochs: 3 |
|
|
- Learning rate: 2e-4 |
|
|
- Batch size: 4, gradient accumulation: 2 |
|
|
- Hardware: NVIDIA A100-80GB |
|
|
- Training method: SFT with `train_on_responses_only` |
|
|
|
|
|
**Dataset:** 7,500 synthetic examples covering: |
|
|
- Multi-step tool chaining (glob → read → edit) |
|
|
- Error recovery patterns |
|
|
- Clarification dialogs |
|
|
- No-tool responses |
|
|
- Parallel tool calls |
|
|
|
|
|
## Tools |
|
|
|
|
|
The model is trained on 5 tools matching llama-agent: |
|
|
|
|
|
| Tool | Description | |
|
|
|------|-------------| |
|
|
| `read_file` | Read file contents with line numbers | |
|
|
| `write_file` | Create or overwrite a file | |
|
|
| `edit_file` | Find and replace text in a file | |
|
|
| `glob` | Find files matching pattern | |
|
|
| `bash` | Execute shell command | |
|
|
|
|
|
## Usage |
|
|
|
|
|
### With llama.cpp |
|
|
|
|
|
```bash |
|
|
# Download |
|
|
wget https://huggingface.co/victor/functiongemma-agent-gguf/resolve/main/functiongemma-270m-it.Q4_K_M.gguf |
|
|
|
|
|
# Run inference |
|
|
./llama-cli -m functiongemma-270m-it.Q4_K_M.gguf -p "<start_of_turn>user |
|
|
Read the main.py file |
|
|
<end_of_turn> |
|
|
<start_of_turn>model" |
|
|
``` |
|
|
|
|
|
### With llama-agent |
|
|
|
|
|
```bash |
|
|
./llama-agent -m functiongemma-270m-it.Q4_K_M.gguf |
|
|
``` |
|
|
|
|
|
## Format |
|
|
|
|
|
Uses FunctionGemma's native format with `<escape>` delimiters: |
|
|
|
|
|
``` |
|
|
<start_of_turn>user |
|
|
Fix the typo in config.json |
|
|
<end_of_turn> |
|
|
<start_of_turn>model |
|
|
<think>I need to find and read the config file first.</think> |
|
|
<start_function_call>call:glob{pattern:<escape>**/config.json<escape>}<end_function_call> |
|
|
<end_of_turn> |
|
|
<start_of_turn>developer |
|
|
<start_function_response>response:glob{stdout:<escape>src/config.json<escape>,stderr:<escape><escape>,exit_code:0}<end_function_response> |
|
|
<end_of_turn> |
|
|
... |
|
|
``` |
|
|
|
|
|
## License |
|
|
|
|
|
This model inherits the [Gemma license](https://ai.google.dev/gemma/terms) from the base model. |
|
|
|
|
|
## Links |
|
|
|
|
|
- Training script: [victor/llama-agent-training](https://huggingface.co/victor/llama-agent-training) |
|
|
- Dataset: [victor/functiongemma-agent-sft](https://huggingface.co/datasets/victor/functiongemma-agent-sft) |
|
|
- llama-agent: [github.com/ggml-org/llama.cpp/tools/agent](https://github.com/ggml-org/llama.cpp/tree/master/tools/agent) |
|
|
|
