How do you have a score of 88 on winogrande?
#7
by distantquant - opened
For reference it is 2-4 points above all other top models, which makes no sense unless you had some insane new method of training. This is especially suspect for only a 7B MoE.
How was this trained?
Otherwise I will assume contamination.
TomGrc changed discussion status to closed
Otherwise I will assume contamination.
i feel that too , their is no info on the model whtsoever and he closes the discussions which are problamatic.
TomGrc changed discussion status to open
If you assume this model is contaminated, please provide data contamination check results to confirm that. Otherwise, I don't have the responsibility to do a test due to the MIT license.
TomGrc changed discussion status to closed