Update README.md
Browse files
README.md
CHANGED
|
@@ -84,6 +84,26 @@ Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://works
|
|
| 84 |
"NoAns_total": 5945
|
| 85 |
```
|
| 86 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
## Authors
|
| 88 |
**Branden Chan:** branden.chan@deepset.ai
|
| 89 |
**Timo M枚ller:** timo.moeller@deepset.ai
|
|
|
|
| 84 |
"NoAns_total": 5945
|
| 85 |
```
|
| 86 |
|
| 87 |
+
Using the official [question answering notebook](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb) from `transformers` yields:
|
| 88 |
+
|
| 89 |
+
```
|
| 90 |
+
{'HasAns_exact': 77.93522267206478,
|
| 91 |
+
'HasAns_f1': 83.93715663402219,
|
| 92 |
+
'HasAns_total': 5928,
|
| 93 |
+
'NoAns_exact': 81.90075693860386,
|
| 94 |
+
'NoAns_f1': 81.90075693860386,
|
| 95 |
+
'NoAns_total': 5945,
|
| 96 |
+
'best_exact': 79.92082877116145,
|
| 97 |
+
'best_exact_thresh': 0.0,
|
| 98 |
+
'best_f1': 82.91749890730902,
|
| 99 |
+
'best_f1_thresh': 0.0,
|
| 100 |
+
'exact': 79.92082877116145,
|
| 101 |
+
'f1': 82.91749890730917,
|
| 102 |
+
'total': 11873}
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
which is consistent with the officially reported results.
|
| 106 |
+
|
| 107 |
## Authors
|
| 108 |
**Branden Chan:** branden.chan@deepset.ai
|
| 109 |
**Timo M枚ller:** timo.moeller@deepset.ai
|