Omartificial-Intelligence-Space commited on
Commit
cb123ca
·
verified ·
1 Parent(s): 95c1548

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +228 -15
README.md CHANGED
@@ -1,23 +1,236 @@
1
  ---
 
 
 
2
  base_model: unsloth/functiongemma-270m-it
3
  tags:
4
- - text-generation-inference
5
- - transformers
6
- - unsloth
7
- - gemma3_text
8
- - trl
9
- - sft
10
- license: apache-2.0
11
- language:
12
- - en
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ---
14
 
15
- # Uploaded model
 
 
16
 
17
- - **Developed by:** Sa74ll
18
- - **License:** apache-2.0
19
- - **Finetuned from model :** unsloth/functiongemma-270m-it
20
 
21
- This gemma3_text model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
22
 
23
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
2
+ language:
3
+ - ar
4
+ license: apache-2.0
5
  base_model: unsloth/functiongemma-270m-it
6
  tags:
7
+ - function-calling
8
+ - arabic
9
+ - tool-use
10
+ - agentic
11
+ - gemma
12
+ - fine-tuned
13
+ datasets:
14
+ - AISA-Framework/AISA-AR-FunctionCall
15
+ pipeline_tag: text-generation
16
+ library_name: transformers
17
+ ---
18
+
19
+ # AISA-AR-FunctionCall-FT
20
+
21
+ <p align="center">
22
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/628f7a71dd993507cfcbe587/vnL90Tybn1528x21dMNsd.png" width="700"/>
23
+ </p>
24
+
25
+
26
+ **Reliable Arabic Structured Tool Calling via Data-Centric Fine-Tuning**
27
+
28
+ `AISA-AR-FunctionCall-FT` is a fully fine-tuned Arabic function-calling model built on top of [FunctionGemma (Gemma 3 270M)](https://huggingface.co/unsloth/functiongemma-270m-it) and optimized for structured tool invocation in Arabic agentic systems.
29
+
30
+ The model converts natural Arabic requests into structured executable API calls, enabling reliable integration between language models and external tools.
31
+
32
+ > This model is part of the **AISA** (Agentic AI Systems Architecture) initiative.
33
+
34
+ ---
35
+
36
+ ## Model Overview
37
+
38
+ | Field | Value |
39
+ |---|---|
40
+ | **Model name** | AISA-AR-FunctionCall-FT |
41
+ | **Base model** | unsloth/functiongemma-270m-it |
42
+ | **Architecture** | Gemma 3 (270M parameters) |
43
+ | **Fine-tuning type** | Full-parameter supervised fine-tuning |
44
+ | **Primary task** | Arabic function calling / tool invocation |
45
+
46
+ The model is designed to translate Arabic natural language requests into structured tool calls following the FunctionGemma tool-calling format.
47
+
48
+ ---
49
+
50
+ ## Key Capabilities
51
+
52
+ - Arabic natural language → structured API calls
53
+ - Multi-dialect Arabic understanding
54
+ - Tool selection and argument extraction
55
+ - Structured execution environments
56
+
57
+ **Supported domains:**
58
+
59
+ | Domain |
60
+ |---|
61
+ | Travel |
62
+ | Utilities |
63
+ | Islamic services |
64
+ | Weather |
65
+ | Healthcare |
66
+ | Banking & finance |
67
+ | E-commerce |
68
+ | Government services |
69
+
70
+ ---
71
+
72
+ ## Dataset
73
+
74
+ The model is trained on **AISA-AR-FunctionCall** — a production-ready Arabic function-calling dataset built through a rigorous data-centric pipeline:
75
+
76
+ - Dataset auditing
77
+ - Schema normalization
78
+ - Enum correction
79
+ - Tool pruning
80
+ - Prompt restructuring
81
+ - Tool sampling
82
+
83
+ **Dataset splits:**
84
+
85
+ | Split | Samples |
86
+ |---|---|
87
+ | Train | 41,104 |
88
+ | Validation | 4,568 |
89
+ | Test | 5,079 |
90
+
91
+ **Dataset includes:**
92
+ - 5 Arabic dialects
93
+ - 8 real-world domains
94
+ - 27 tool schemas
95
+ - Structured tool-call annotations
96
+
97
+ Dataset: [AISA-Framework/AISA-AR-FunctionCall](https://huggingface.co/datasets/AISA-Framework/AISA-AR-FunctionCall)
98
+
99
+ ---
100
+
101
+ ## Training Methodology
102
+
103
+ The model was trained using a **data-centric fine-tuning pipeline** designed to stabilize structured execution.
104
+
105
+ **Key pipeline steps:**
106
+
107
+ 1. Structural dataset auditing
108
+ 2. Enum constraint repair
109
+ 3. Tool schema normalization
110
+ 4. Tool pruning (36 → 27 tools)
111
+ 5. Tool sampling to prevent prompt truncation
112
+ 6. FunctionGemma-compatible chat serialization
113
+ 7. Completion-only supervised fine-tuning
114
+
115
+ **Training configuration:**
116
+
117
+ | Parameter | Value |
118
+ |---|---|
119
+ | Model size | 270M |
120
+ | Training type | Full fine-tuning |
121
+ | Epochs | 2 |
122
+ | Effective batch size | 32 |
123
+ | Learning rate | 2e-5 |
124
+ | Optimizer | 8-bit AdamW |
125
+ | Scheduler | Cosine |
126
+ | Precision | BF16 |
127
+ | Gradient checkpointing | Enabled |
128
+
129
+ ---
130
+
131
+ ## Evaluation Results
132
+
133
+ Evaluation was performed on a held-out test set of **5,079 samples**.
134
+
135
+ ### Clean Positive Evaluation (n = 2,873)
136
+
137
+ | Metric | Baseline | AISA-AR-FunctionCall-FT |
138
+ |---|---|---|
139
+ | Function Name Accuracy | 0.0804 | **0.6547** |
140
+ | Full Tool-Call Match | 0.0056 | **0.3362** |
141
+ | Argument Key F1 | 0.0600 | **0.5728** |
142
+ | Argument Exact Match | 0.0422 | **0.6377** |
143
+ | Parse Failure Rate | 0.8726 | **0.0084** |
144
+ | Format Validity | 0.1274 | **0.9916** |
145
+ | Hallucination Rate | 0.0003 | 0.0226 |
146
+
147
+ > **Key improvement:** Parse failure reduced from **87% → <1%**
148
+
149
+ ### Dialect Performance
150
+
151
+ | Dialect | Function Accuracy |
152
+ |---|---|
153
+ | MSA | 0.761 |
154
+ | Gulf | 0.697 |
155
+ | Egyptian | 0.683 |
156
+ | Levantine | 0.694 |
157
+ | Maghrebi | 0.616 |
158
+
159
+ Fine-tuning significantly reduces dialect disparity compared to the baseline model.
160
+
161
+ ---
162
+
163
+ ## Known Limitations
164
+
165
+ Remaining errors are primarily **semantic**, including:
166
+
167
+ - Tool selection ambiguity
168
+ - Argument mismatches
169
+ - Domain overlap (e.g., weather vs. air quality)
170
+
171
+ Structured formatting errors are largely eliminated.
172
+
173
+ ---
174
+
175
+ ## Example Usage
176
+
177
+ **Prompt:**
178
+
179
+ ```
180
+ ما حالة الطقس في الرياض اليوم؟
181
+ ```
182
+
183
+ **Model output:**
184
+
185
+ ```
186
+ <start_function_call>
187
+ call:get_weather{
188
+ city:<escape>الرياض<escape>,
189
+ days:1
190
+ }
191
+ <end_function_call>
192
+ ```
193
+
194
+ The structured call can then be executed by the application runtime.
195
+
196
+ ---
197
+
198
+ ## Intended Use
199
+
200
+ This model is designed for:
201
+
202
+ - Arabic AI assistants
203
+ - Tool-based agents
204
+ - Structured API orchestration
205
+ - Arabic enterprise automation
206
+ - Research on multilingual tool calling
207
+
208
+ ### Out-of-Scope Uses
209
+
210
+ This model is **not** designed for:
211
+
212
+ - General chatbots or open-ended conversation
213
+ - Sensitive decision-making systems
214
+ - Safety-critical deployments without additional validation
215
+
216
+ ---
217
+
218
+ ## Related Models
219
+
220
+ | Model | Description |
221
+ |---|---|
222
+ | [AISA-AR-FunctionCall-Think](https://huggingface.co/AISA-Framework/AISA-AR-FunctionCall-Think) | Reasoning-augmented tool-calling model |
223
+
224
  ---
225
 
226
+ ## AISA Framework
227
+
228
+ This model is part of the AISA initiative for building reliable agentic AI systems.
229
 
230
+ Model collection: [AISA-Framework/aisa-arabic-functioncall-datasets-and-models](https://huggingface.co/collections/AISA-Framework/aisa-arabic-functioncall-datasets-and-models)
231
+
232
+ ---
233
 
234
+ ## License
235
 
236
+ [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)