DeBERTa v3 xsmall SQuAD 2.0
Microsoft reports that this model can get 84.8/82.0 on f1/em on the dev set.
I got 81.5/78.3 but I only did one run and I didn't use the official squad2 evaluation script. I will do some more runs and show the results on the official script soon.
- Downloads last month
- 2
Dataset used to train nbroad/deberta-v3-xsmall-squad2
Evaluation results
- f1 on SQuAD2.0self-reported81.500
- exact on SQuAD2.0self-reported78.300
- Exact Match on squad_v2validation set verified78.534
- F1 on squad_v2validation set verified81.641
- total on squad_v2validation set verified11870.000
- Exact Match on squadvalidation set verified84.174
- F1 on squadvalidation set verified91.077