Update README.md
Browse files
README.md
CHANGED
|
@@ -43,11 +43,11 @@ This model was trained with QLoRa in bfloat16 with flash attention 2 on oen A100
|
|
| 43 |
|
| 44 |
## Evaluation results
|
| 45 |
|
| 46 |
-
The model was evaluated using [scandeval](https://scandeval.com/dutch-nlg/).
|
| 47 |
|
| 48 |
| Model| conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl |
|
| 49 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:
|
| 50 |
-
Reynaerde-7B-Chat | 56.40 / 38.13 | 10.83 / 27.67 | 20.02 / 55.40 | 53.56 / 65.29 |
|
| 51 |
Mistral-7B-v0.3 | 57.08 / 42.65 | 14.05 / 39.13 | 8.08 / 43.07 | 45.57 / 55.20 | 62.28 / 16.46 | 20.39 / 40.03 | 13.28 / 34.13
|
| 52 |
Mistral-7B-v0.3-Instruct | 60.76 / 45.39 | 13.20 / 34.26 | 23.23 / 59.26 | 48.94 / 60.13 | 66.09 / 18.02 | 24.95 / 43.67 | 24.86 / 43.57
|
| 53 |
|
|
|
|
| 43 |
|
| 44 |
## Evaluation results
|
| 45 |
|
| 46 |
+
The model was evaluated using [scandeval](https://scandeval.com/dutch-nlg/). There are improvements in 4/7 benchmarks compared to the Mistral-7B-v0.3-Instruct model.
|
| 47 |
|
| 48 |
| Model| conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl |
|
| 49 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:
|
| 50 |
+
Reynaerde-7B-Chat | 56.40 / 38.13 | 10.83 / 27.67 | 20.02 / 55.40 | 53.56 / 65.29 | 68.13 / 20.85 | 32.50 / 49.10 | 31.36 / 47.79
|
| 51 |
Mistral-7B-v0.3 | 57.08 / 42.65 | 14.05 / 39.13 | 8.08 / 43.07 | 45.57 / 55.20 | 62.28 / 16.46 | 20.39 / 40.03 | 13.28 / 34.13
|
| 52 |
Mistral-7B-v0.3-Instruct | 60.76 / 45.39 | 13.20 / 34.26 | 23.23 / 59.26 | 48.94 / 60.13 | 66.09 / 18.02 | 24.95 / 43.67 | 24.86 / 43.57
|
| 53 |
|