Hugging Face's logo

macmacmacmac
/
functiongemma-nextjs

Text Generation
English
function-calling
infrastructure
devops
litertlm
Model card Files
xet
Community
functiongemma-nextjs
File size: 4,133 Bytes 
e16caa5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
 
# FunctionGemma Infrastructure LiteRT-LM

A fine-tuned [FunctionGemma-270M](https://huggingface.co/google/functiongemma-270m-it) model converted to LiteRT-LM format for on-device inference. Designed for self-healing infrastructure and automatic error remediation.

## Model Details

| Property | Value |
|----------|-------|
| Base Model | google/functiongemma-270m-it |
| Format | LiteRT-LM (.litertlm) |
| Quantization | Dynamic INT8 |
| File Size | 272 MB |
| Parameters | 270M |

## Intended Use

This model is designed for [dad-express](https://github.com/anthropics/dad-express), a self-healing gateway that monitors HTTP traffic and automatically fixes infrastructure configuration issues by calling the appropriate tools.

## Supported Tools

The model was fine-tuned on 9 infrastructure tools:

| Tool | Description | Parameters |
|------|-------------|------------|
| `addProxyRoute` | Add reverse proxy route | path, upstream, port |
| `addCorsHeaders` | Configure CORS headers | origin, credentials |
| `configureSsl` | Configure SSL certificate | hostname, selfSigned |
| `setEnvVariable` | Set environment variable | name, value |
| `exposePort` | Expose port in Docker/firewall | service, port |
| `addHostEntry` | Add hostname to /etc/hosts | hostname, ip |
| `restartService` | Restart a service | service |
| `clearCache` | Clear cache | cacheType |
| `modifyConfig` | Modify config file | file, key, value |

## Training Details

### Dataset
- **Total Examples**: 10,216
- **Train/Eval Split**: 90/10 (9,194 train, 1,022 eval)
- **Format**: Prompt-completion pairs using FunctionGemma chat template
- **Distribution**: ~1,000-1,200 examples per tool (balanced)

### Training Configuration

Trained following [Google's official FunctionGemma fine-tuning notebook](https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/FunctionGemma/%5BFunctionGemma%5DFinetune_FunctionGemma_270M_for_Mobile_Actions_with_Hugging_Face.ipynb):

```python
SFTConfig(
    num_train_epochs=2,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=1e-5,
    lr_scheduler_type="cosine",
    gradient_checkpointing=True,
    packing=False,
    optim="adamw_torch_fused",
    bf16=True,
    completion_only_loss=True,  # Critical: only train on completion tokens
)
```

### Hardware
- **GPU**: NVIDIA L4 (24GB)
- **Training Time**: ~23 minutes
- **Conversion Time**: ~10 minutes

### Performance Metrics

| Metric | Value |
|--------|-------|
| Final Eval Loss | 0.034 |
| Token Accuracy | 98.6% |
| Training Steps | 576 |

## LiteRT-LM Conversion

Converted using [ai-edge-torch](https://github.com/google-ai-edge/ai-edge-torch):

```python
converter.convert_to_litert(
    pytorch_model,
    prefill_seq_len=256,
    kv_cache_max_len=1024,
    quantize="dynamic_int8",
    output_format="litertlm",
)
```

### LLM Metadata
```protobuf
start_token: { token_ids: { ids: [ 2 ] } }
stop_tokens: { token_str: "<end_of_turn>" }
stop_tokens: { token_str: "<start_function_response>" }
llm_model_type: { function_gemma: {} }
```

## Usage

### With LiteRT-LM Runtime
```typescript
import { LiteRTLM } from 'litert-lm';

const model = await LiteRTLM.load('functiongemma-infrastructure_q8_ekv1024.litertlm');
const response = await model.generate(prompt);
```

### Example Input/Output

**Input:**
```
Error: CORS - No 'Access-Control-Allow-Origin' header from http://localhost:3000
```

**Output:**
```
<start_function_call>call:addCorsHeaders{origin:<escape>http://localhost:3000<escape>,credentials:<escape>true<escape>}<end_function_call><start_function_response>
```

## Dependencies

- Python 3.11+
- transformers==4.57.1
- trl==0.25.1
- datasets==4.4.1
- ai-edge-torch-nightly
- ai-edge-litert-nightly

## License

This model inherits the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).

## Citation

```bibtex
@misc{functiongemma-infrastructure-litertlm,
  title={FunctionGemma Infrastructure LiteRT-LM},
  author={dad-express contributors},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/macmacmacmac/functiongemma-infrastructure-litertlm}
}
```