Added disclaimer
Browse files
README.md
CHANGED
|
@@ -9,6 +9,25 @@ tags:
|
|
| 9 |
- argilla/distilabeled-Marcoro14-7B-slerp
|
| 10 |
---
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
# MergeTrix-7B
|
| 13 |
|
| 14 |
MergeTrix-7B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
|
|
|
| 9 |
- argilla/distilabeled-Marcoro14-7B-slerp
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# IMPORTANT NOTE | READ ME! #
|
| 13 |
+
This model uses udkai/Turdus which may produce inaccurate results for the Winogrande evaluation scores.
|
| 14 |
+
The following is a quote directly taken from that models page:
|
| 15 |
+
- "A less contaminated version of udkai/Garrulus and the second model to be discussed in the paper Subtle DPO-Contamination with modified Winogrande increases TruthfulQA, Hellaswag & ARC."
|
| 16 |
+
- "Subtle DPO-Contamination with modified Winogrande causes the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) to be 0.2% higher than the underlying model."
|
| 17 |
+
|
| 18 |
+
In my understanding the Winogrande scores are only slightly influenced by the DPO-Contamination, that has the "side-effect" of increasing the scores on the other benchmarks.
|
| 19 |
+
Since the effect on the Winogrande scores was subtle in the udkai/Turdus benchmarking results, and this model combines it with other models (probably making this effect even less pronounced),
|
| 20 |
+
I still believe that this model can be of value to the community as it's overall performance is quite impressive.
|
| 21 |
+
|
| 22 |
+
However I do not want to mislead anybody or produce any unfair scores, hence this note!
|
| 23 |
+
The full training configuration is also fully transparant and can be found below.
|
| 24 |
+
|
| 25 |
+
Hope this model will prove useful.
|
| 26 |
+
There's GGUF versions available here: https://huggingface.co/CultriX/MergeTrix-7B-GGUF
|
| 27 |
+
|
| 28 |
+
Kind regards,
|
| 29 |
+
CultriX
|
| 30 |
+
|
| 31 |
# MergeTrix-7B
|
| 32 |
|
| 33 |
MergeTrix-7B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|