fix newlines
Browse files
README.md
CHANGED
|
@@ -21,20 +21,20 @@ This model is a fine-tuned version of [INSAIT-Institute/MamayLM-Gemma-3-4B-IT-v1
|
|
| 21 |
It has been trained using [TRL](https://github.com/huggingface/trl).
|
| 22 |
|
| 23 |
## Evaluation
|
| 24 |
-
This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training.
|
| 25 |
-
During this phase new tokens (including tool_call) were introduced to the model and we evaluated how well it uses and understands the purpose of tool_call.
|
| 26 |
|
| 27 |
### Metrics
|
| 28 |
-
Accuracy in function calling (if response contains tool_call token) - find_longest_common_sequence_length(ground_truth_tokens, generated_tokens) / len(ground_truth_tokens)
|
| 29 |
-
Match in helpful exchange (if response does not contain tool_call token) - Computes the percentage of matching elements between generated tokens and ground truth tokens
|
| 30 |
|
| 31 |
### Performance before fine-tuning:
|
| 32 |
-
Accuracy in function calling: 0.38107
|
| 33 |
-
Match in helpful exchange: 0.07440
|
| 34 |
|
| 35 |
### Performance after fine-tuning:
|
| 36 |
-
Accuracy in function calling: 0.95415
|
| 37 |
-
Match in helpful exchange: 0.09937
|
| 38 |
|
| 39 |
## Quick start
|
| 40 |
|
|
|
|
| 21 |
It has been trained using [TRL](https://github.com/huggingface/trl).
|
| 22 |
|
| 23 |
## Evaluation
|
| 24 |
+
This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training. <br>
|
| 25 |
+
During this phase new tokens (including tool_call) were introduced to the model and we evaluated how well it uses and understands the purpose of tool_call. <br>
|
| 26 |
|
| 27 |
### Metrics
|
| 28 |
+
- Accuracy in function calling (if response contains tool_call token) - ```find_longest_common_sequence_length(ground_truth_tokens, generated_tokens) / len(ground_truth_tokens)```
|
| 29 |
+
- Match in helpful exchange (if response does not contain tool_call token) - Computes the percentage of matching elements between generated tokens and ground truth tokens
|
| 30 |
|
| 31 |
### Performance before fine-tuning:
|
| 32 |
+
Accuracy in function calling: 0.38107 <br>
|
| 33 |
+
Match in helpful exchange: 0.07440 <br>
|
| 34 |
|
| 35 |
### Performance after fine-tuning:
|
| 36 |
+
Accuracy in function calling: 0.95415 <br>
|
| 37 |
+
Match in helpful exchange: 0.09937 <br>
|
| 38 |
|
| 39 |
## Quick start
|
| 40 |
|