MNLI benchmark results
#2
by chefexperte - opened
Hey,
I ran the model on MNLI 1.0 dev mismatched set and got an accuracy of 0.8853.
How was the claimed accuracy of 90% achieved? I saw there also was a reupload (or something) of the MNLI dataset linked (https://huggingface.co/datasets/nyu-mll/multi_nli/), which contains fewer pairs (9.83k pairs) instead of the 10000 in the original MNLI dataset from https://cims.nyu.edu/~sbowman/multinli/.
Why is there a difference between those datasets?
Besides that, great work and great model!