fix token display
Browse files
README.md
CHANGED
|
@@ -22,11 +22,11 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
|
|
| 22 |
|
| 23 |
## Evaluation
|
| 24 |
This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training.
|
| 25 |
-
During this phase new tokens (including
|
| 26 |
|
| 27 |
### Metrics
|
| 28 |
-
Accuracy in function calling (if response contains
|
| 29 |
-
Match in helpful exchange (if response does not contain
|
| 30 |
|
| 31 |
### Performance before fine-tuning:
|
| 32 |
Accuracy in function calling: 0.38107
|
|
|
|
| 22 |
|
| 23 |
## Evaluation
|
| 24 |
This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training.
|
| 25 |
+
During this phase new tokens (including tool_call) were introduced to the model and we evaluated how well it uses and understands the purpose of tool_call.
|
| 26 |
|
| 27 |
### Metrics
|
| 28 |
+
Accuracy in function calling (if response contains tool_call token) - find_longest_common_sequence_length(ground_truth_tokens, generated_tokens) / len(ground_truth_tokens)
|
| 29 |
+
Match in helpful exchange (if response does not contain tool_call token) - Computes the percentage of matching elements between generated tokens and ground truth tokens
|
| 30 |
|
| 31 |
### Performance before fine-tuning:
|
| 32 |
Accuracy in function calling: 0.38107
|