Commit
·
18c01a6
1
Parent(s):
930271a
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
---
|
| 2 |
language: en
|
| 3 |
datasets:
|
| 4 |
-
-
|
| 5 |
license: mit
|
| 6 |
thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
|
| 7 |
tags:
|
|
@@ -19,12 +19,7 @@ tags:
|
|
| 19 |
**Published**: Apr 21st, 2021
|
| 20 |
|
| 21 |
## Details
|
| 22 |
-
-
|
| 23 |
-
- The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published [online](https://deepset.ai/germanquad).
|
| 24 |
-
- The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536answers, because we removed 76 wrong answers.
|
| 25 |
-
- In addition to the annotations in GermanQuAD, haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model.
|
| 26 |
-
|
| 27 |
-
See https://deepset.ai/germanquad for more details and dataset download in SQuAD format.
|
| 28 |
|
| 29 |
## Hyperparameters
|
| 30 |
```
|
|
@@ -38,11 +33,6 @@ temperature = 1.5
|
|
| 38 |
distillation_loss_weight = 0.75
|
| 39 |
```
|
| 40 |
## Performance
|
| 41 |
-
We evaluated the extractive question answering performance on the SQuAD v2 dev set.
|
| 42 |
-
Model types and training data are included in the model name.
|
| 43 |
-
For finetuning XLM-Roberta, we use the English SQuAD v2.0 dataset.
|
| 44 |
-
The GELECTRA models are warm started on the German translation of SQuAD v1.1 and finetuned on \\\\germanquad.
|
| 45 |
-
The human baseline was computed for the 3-way test set by taking one answer as prediction and the other two as ground truth.
|
| 46 |
```
|
| 47 |
"exact": 79.8366040596311
|
| 48 |
"f1": 83.916407079888
|
|
|
|
| 1 |
---
|
| 2 |
language: en
|
| 3 |
datasets:
|
| 4 |
+
- squad_v2
|
| 5 |
license: mit
|
| 6 |
thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
|
| 7 |
tags:
|
|
|
|
| 19 |
**Published**: Apr 21st, 2021
|
| 20 |
|
| 21 |
## Details
|
| 22 |
+
- haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Hyperparameters
|
| 25 |
```
|
|
|
|
| 33 |
distillation_loss_weight = 0.75
|
| 34 |
```
|
| 35 |
## Performance
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
```
|
| 37 |
"exact": 79.8366040596311
|
| 38 |
"f1": 83.916407079888
|