vijayk-huggingface commited on
Commit
e2afe60
·
verified ·
1 Parent(s): bfdaa0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +154 -40
README.md CHANGED
@@ -22,14 +22,13 @@ pipeline_tag: text-generation
22
  **Orchestrix** is a fine-tuned **FunctionGemma 270M** model designed for **on-device UI orchestration** using **function calling**.
23
  It converts natural language commands into **deterministic UI actions**, running fully **offline** via MediaPipe.
24
 
25
-
26
  ## Model Description
27
 
28
  This is a fine-tuned version of **FunctionGemma 270M Instruct** specifically trained for natural language-based UI control through function calling. The model interprets user commands and generates structured function calls to manipulate application UI elements.
29
 
30
  **Base Model**: `google/functiongemma-270m-it`
31
  **Fine-tuning Task**: Function calling for UI control
32
- **Format**: MediaPipe Task file (`.task`)
33
  **Deployment**: On-device inference via Flutter
34
 
35
  ## Intended Use
@@ -43,38 +42,40 @@ This is a fine-tuned version of **FunctionGemma 270M Instruct** specifically tra
43
 
44
  ### Supported Functions
45
 
46
- The model is trained to call the following functions:
47
 
48
  | Function | Description | Parameters |
49
  |----------|-------------|------------|
50
- | `change_theme` | Switch app theme | `theme`: "light", "dark", or "system" |
51
- | `change_background_color` | Modify background color | `color`: "red", "blue", "green", "black", "white", "yellow", "purple", "orange" |
52
- | `change_text_color` | Change text color | `color`: "white" or "black" |
53
- | `change_text_size` | Adjust text size | `size`: "small", "medium", or "large" |
 
 
54
 
55
  ## Training Details
56
 
57
  ### Training Data
58
 
59
- The model was fine-tuned on a custom dataset of ~1000 examples mapping natural language commands to function calls. Training data format:
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
  ```json
62
  {
63
- "messages": [
64
- {
65
- "role": "user",
66
- "content": "Make the background blue"
67
- },
68
- {
69
- "role": "assistant",
70
- "tool_calls": [
71
- {
72
- "name": "change_background_color",
73
- "arguments": {"color": "blue"}
74
- }
75
- ]
76
- }
77
- ]
78
  }
79
  ```
80
 
@@ -82,10 +83,10 @@ The model was fine-tuned on a custom dataset of ~1000 examples mapping natural l
82
 
83
  - **Base Model**: FunctionGemma 270M Instruct
84
  - **Fine-tuning Method**: Supervised fine-tuning on function calling examples
85
- - **Training Examples**: ~1000 user command → function call pairs
 
86
  - **Optimization**: Focused on accurate function name and parameter extraction
87
 
88
-
89
  ## Usage
90
 
91
  ### Flutter Integration
@@ -96,8 +97,6 @@ import 'package:flutter_gemma/flutter_gemma.dart';
96
  // Initialize Flutter Gemma
97
  await FlutterGemma.initialize();
98
 
99
- // Install the model
100
-
101
  // Create model instance
102
  final model = await FlutterGemma.getActiveModel(
103
  maxTokens: 1024,
@@ -114,13 +113,93 @@ final tools = [
114
  'properties': {
115
  'theme': {
116
  'type': 'string',
117
- 'enum': ['light', 'dark', 'system'],
 
118
  },
119
  },
120
  'required': ['theme'],
121
  },
122
  ),
123
- // ... other tools
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
  ];
125
 
126
  // Create chat session
@@ -146,9 +225,9 @@ Model Output: {
146
  }
147
  ```
148
 
149
- **Example 2: Color Change**
150
  ```
151
- User: "Make the background purple"
152
  Model Output: {
153
  "tool_calls": [{
154
  "name": "change_background_color",
@@ -157,36 +236,64 @@ Model Output: {
157
  }
158
  ```
159
 
160
- **Example 3: Multiple Commands**
161
  ```
162
- User: "Change to dark theme and make text large"
163
  Model Output: {
164
  "tool_calls": [
165
- {
166
- "name": "change_theme",
167
- "arguments": {"theme": "dark"}
168
- },
169
  {
170
  "name": "change_text_size",
171
  "arguments": {"size": "large"}
 
 
 
 
172
  }
173
  ]
174
  }
175
  ```
176
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
177
  ## Limitations
178
 
179
- 1. **Limited Function Set**: Only supports 4 predefined functions
180
- 2. **Parameter Constraints**: Parameters are restricted to predefined enums
181
  3. **Language**: English only
182
  4. **Context**: Optimized for UI control tasks, not general conversation
183
- 5. **Ambiguity**: May struggle with highly ambiguous or complex commands
 
184
 
185
  ## Bias and Ethical Considerations
186
 
187
  - The model is trained on synthetic data and may not represent diverse user populations
188
  - Function calling is deterministic and limited to safe UI operations
189
  - No personal data is collected or transmitted (fully on-device)
 
190
 
191
  ## Model Card Authors
192
 
@@ -205,3 +312,10 @@ This model is released under the [Gemma License](https://ai.google.dev/gemma/ter
205
  - **Google DeepMind** for the Gemma base model
206
  - **MediaPipe** for on-device inference capabilities
207
  - **Flutter Gemma** package maintainers
 
 
 
 
 
 
 
 
22
  **Orchestrix** is a fine-tuned **FunctionGemma 270M** model designed for **on-device UI orchestration** using **function calling**.
23
  It converts natural language commands into **deterministic UI actions**, running fully **offline** via MediaPipe.
24
 
 
25
  ## Model Description
26
 
27
  This is a fine-tuned version of **FunctionGemma 270M Instruct** specifically trained for natural language-based UI control through function calling. The model interprets user commands and generates structured function calls to manipulate application UI elements.
28
 
29
  **Base Model**: `google/functiongemma-270m-it`
30
  **Fine-tuning Task**: Function calling for UI control
31
+ **Format**: MediaPipe compatible formats (`.task`, `.tflite`, `.litert`)
32
  **Deployment**: On-device inference via Flutter
33
 
34
  ## Intended Use
 
42
 
43
  ### Supported Functions
44
 
45
+ The model is trained to call the following 6 functions:
46
 
47
  | Function | Description | Parameters |
48
  |----------|-------------|------------|
49
+ | `change_theme` | Switch app theme | `theme`: "light", "dark", "system", "blue", "green", "purple", "orange", "red", "material", "cupertino", "custom" |
50
+ | `change_background_color` | Modify background color | `color`: "red", "green", "blue", "yellow", "purple", "orange", "white", "black", "pink", "cyan", "amber", "teal", "indigo", "lime", "brown", "grey" |
51
+ | `change_text_color` | Change text color | `color`: "red", "green", "blue", "yellow", "purple", "orange", "white", "black", "pink", "cyan", "amber", "teal", "indigo", "lime", "brown", "grey" |
52
+ | `change_text_size` | Adjust text size | `size`: "small", "medium", "large", "extra large", "tiny", "12", "14", "16", "18", "20", "22", "24", "28", "32", "36", "40", "48" |
53
+ | `change_app_title` | Change application title | `title`: Any app name (200+ variations including "Dashboard", "Settings", "Task Manager", etc.) |
54
+ | `show_alert` | Display notification/alert | `title`: Alert title, `message`: Alert message |
55
 
56
  ## Training Details
57
 
58
  ### Training Data
59
 
60
+ The model was fine-tuned on a custom dataset of **3,300 examples** mapping natural language commands to function calls. The dataset was synthetically generated with diverse phrasing patterns to ensure robust understanding.
61
+
62
+ **Training data statistics**:
63
+ - **Total examples**: 3,300
64
+ - **Functions**: 6 (change_theme, change_background_color, change_text_color, change_text_size, change_app_title, show_alert)
65
+ - **Pattern variations**: 10-12 unique phrasings per function
66
+ - **Color options**: 16 colors
67
+ - **Theme options**: 11 themes
68
+ - **Text size options**: 17 size variations
69
+ - **App titles**: 200+ predefined titles
70
+ - **Alert variations**: 100+ titles and 150+ messages
71
+
72
+ Training data format (JSONL):
73
 
74
  ```json
75
  {
76
+ "user_content": "make the background purple",
77
+ "tool_name": "change_background_color",
78
+ "tool_arguments": "{\"color\": \"purple\"}"
 
 
 
 
 
 
 
 
 
 
 
 
79
  }
80
  ```
81
 
 
83
 
84
  - **Base Model**: FunctionGemma 270M Instruct
85
  - **Fine-tuning Method**: Supervised fine-tuning on function calling examples
86
+ - **Training Examples**: 3,300 user command → function call pairs
87
+ - **Data Generation**: Synthetic data with template-based variation
88
  - **Optimization**: Focused on accurate function name and parameter extraction
89
 
 
90
  ## Usage
91
 
92
  ### Flutter Integration
 
97
  // Initialize Flutter Gemma
98
  await FlutterGemma.initialize();
99
 
 
 
100
  // Create model instance
101
  final model = await FlutterGemma.getActiveModel(
102
  maxTokens: 1024,
 
113
  'properties': {
114
  'theme': {
115
  'type': 'string',
116
+ 'enum': ['light', 'dark', 'system', 'blue', 'green', 'purple',
117
+ 'orange', 'red', 'material', 'cupertino', 'custom'],
118
  },
119
  },
120
  'required': ['theme'],
121
  },
122
  ),
123
+ Tool(
124
+ name: 'change_background_color',
125
+ description: 'Change the background color of the app',
126
+ parameters: {
127
+ 'type': 'object',
128
+ 'properties': {
129
+ 'color': {
130
+ 'type': 'string',
131
+ 'enum': ['red', 'green', 'blue', 'yellow', 'purple', 'orange',
132
+ 'white', 'black', 'pink', 'cyan', 'amber', 'teal',
133
+ 'indigo', 'lime', 'brown', 'grey'],
134
+ },
135
+ },
136
+ 'required': ['color'],
137
+ },
138
+ ),
139
+ Tool(
140
+ name: 'change_text_color',
141
+ description: 'Change the text color in the app',
142
+ parameters: {
143
+ 'type': 'object',
144
+ 'properties': {
145
+ 'color': {
146
+ 'type': 'string',
147
+ 'enum': ['red', 'green', 'blue', 'yellow', 'purple', 'orange',
148
+ 'white', 'black', 'pink', 'cyan', 'amber', 'teal',
149
+ 'indigo', 'lime', 'brown', 'grey'],
150
+ },
151
+ },
152
+ 'required': ['color'],
153
+ },
154
+ ),
155
+ Tool(
156
+ name: 'change_text_size',
157
+ description: 'Adjust the text size',
158
+ parameters: {
159
+ 'type': 'object',
160
+ 'properties': {
161
+ 'size': {
162
+ 'type': 'string',
163
+ 'enum': ['small', 'medium', 'large', 'extra large', 'tiny',
164
+ '12', '14', '16', '18', '20', '22', '24', '28',
165
+ '32', '36', '40', '48'],
166
+ },
167
+ },
168
+ 'required': ['size'],
169
+ },
170
+ ),
171
+ Tool(
172
+ name: 'change_app_title',
173
+ description: 'Change the application title',
174
+ parameters: {
175
+ 'type': 'object',
176
+ 'properties': {
177
+ 'title': {
178
+ 'type': 'string',
179
+ 'description': 'The new title for the application',
180
+ },
181
+ },
182
+ 'required': ['title'],
183
+ },
184
+ ),
185
+ Tool(
186
+ name: 'show_alert',
187
+ description: 'Display an alert or notification',
188
+ parameters: {
189
+ 'type': 'object',
190
+ 'properties': {
191
+ 'title': {
192
+ 'type': 'string',
193
+ 'description': 'The alert title',
194
+ },
195
+ 'message': {
196
+ 'type': 'string',
197
+ 'description': 'The alert message',
198
+ },
199
+ },
200
+ 'required': ['title', 'message'],
201
+ },
202
+ ),
203
  ];
204
 
205
  // Create chat session
 
225
  }
226
  ```
227
 
228
+ **Example 2: Background Color**
229
  ```
230
+ User: "make it purple"
231
  Model Output: {
232
  "tool_calls": [{
233
  "name": "change_background_color",
 
236
  }
237
  ```
238
 
239
+ **Example 3: Text Styling**
240
  ```
241
+ User: "make text large and cyan"
242
  Model Output: {
243
  "tool_calls": [
 
 
 
 
244
  {
245
  "name": "change_text_size",
246
  "arguments": {"size": "large"}
247
+ },
248
+ {
249
+ "name": "change_text_color",
250
+ "arguments": {"color": "cyan"}
251
  }
252
  ]
253
  }
254
  ```
255
 
256
+ **Example 4: App Title**
257
+ ```
258
+ User: "rename app to Task Manager"
259
+ Model Output: {
260
+ "tool_calls": [{
261
+ "name": "change_app_title",
262
+ "arguments": {"title": "Task Manager"}
263
+ }]
264
+ }
265
+ ```
266
+
267
+ **Example 5: Alert Notification**
268
+ ```
269
+ User: "show alert Success with message File uploaded successfully"
270
+ Model Output: {
271
+ "tool_calls": [{
272
+ "name": "show_alert",
273
+ "arguments": {
274
+ "title": "Success",
275
+ "message": "File uploaded successfully"
276
+ }
277
+ }]
278
+ }
279
+ ```
280
+
281
+
282
  ## Limitations
283
 
284
+ 1. **Limited Function Set**: Only supports 6 predefined functions
285
+ 2. **Parameter Constraints**: Colors and themes are restricted to predefined enums
286
  3. **Language**: English only
287
  4. **Context**: Optimized for UI control tasks, not general conversation
288
+ 5. **Ambiguity**: May struggle with highly ambiguous commands not seen during training
289
+ 6. **Synthetic Training**: Trained on template-generated data, may differ from real user phrasing
290
 
291
  ## Bias and Ethical Considerations
292
 
293
  - The model is trained on synthetic data and may not represent diverse user populations
294
  - Function calling is deterministic and limited to safe UI operations
295
  - No personal data is collected or transmitted (fully on-device)
296
+ - Color names follow Western conventions and may not align with all cultural interpretations
297
 
298
  ## Model Card Authors
299
 
 
312
  - **Google DeepMind** for the Gemma base model
313
  - **MediaPipe** for on-device inference capabilities
314
  - **Flutter Gemma** package maintainers
315
+
316
+ ## Repository
317
+
318
+ Training scripts and dataset generation code available in the project repository:
319
+ - `generate_data.py`: Synthetic dataset generation script
320
+ - `training_data.jsonl`: 3,300 training examples
321
+ - `train.jsonl`: Converted training data for fine-tuning