TymofiiNasobko commited on
Commit
0688c18
·
verified ·
1 Parent(s): 930a5f5

fix newlines

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -21,20 +21,20 @@ This model is a fine-tuned version of [INSAIT-Institute/MamayLM-Gemma-3-4B-IT-v1
21
  It has been trained using [TRL](https://github.com/huggingface/trl).
22
 
23
  ## Evaluation
24
- This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training.
25
- During this phase new tokens (including tool_call) were introduced to the model and we evaluated how well it uses and understands the purpose of tool_call.
26
 
27
  ### Metrics
28
- Accuracy in function calling (if response contains tool_call token) - find_longest_common_sequence_length(ground_truth_tokens, generated_tokens) / len(ground_truth_tokens)
29
- Match in helpful exchange (if response does not contain tool_call token) - Computes the percentage of matching elements between generated tokens and ground truth tokens
30
 
31
  ### Performance before fine-tuning:
32
- Accuracy in function calling: 0.38107
33
- Match in helpful exchange: 0.07440
34
 
35
  ### Performance after fine-tuning:
36
- Accuracy in function calling: 0.95415
37
- Match in helpful exchange: 0.09937
38
 
39
  ## Quick start
40
 
 
21
  It has been trained using [TRL](https://github.com/huggingface/trl).
22
 
23
  ## Evaluation
24
+ This is the first iteration of fine-tuning MamayLM for function calling. In the future, we plan to add metrics and improve training. <br>
25
+ During this phase new tokens (including tool_call) were introduced to the model and we evaluated how well it uses and understands the purpose of tool_call. <br>
26
 
27
  ### Metrics
28
+ - Accuracy in function calling (if response contains tool_call token) - ```find_longest_common_sequence_length(ground_truth_tokens, generated_tokens) / len(ground_truth_tokens)```
29
+ - Match in helpful exchange (if response does not contain tool_call token) - Computes the percentage of matching elements between generated tokens and ground truth tokens
30
 
31
  ### Performance before fine-tuning:
32
+ Accuracy in function calling: 0.38107 <br>
33
+ Match in helpful exchange: 0.07440 <br>
34
 
35
  ### Performance after fine-tuning:
36
+ Accuracy in function calling: 0.95415 <br>
37
+ Match in helpful exchange: 0.09937 <br>
38
 
39
  ## Quick start
40