Update README.md
Browse files
README.md
CHANGED
|
@@ -3,17 +3,21 @@ Give the model a passage and it will generate a question about the passage.
|
|
| 3 |
|
| 4 |
## Trained on the following datasets:
|
| 5 |
|
| 6 |
-
- SQuAD (English)
|
| 7 |
-
- TyDiQA-GoldP (Arabic, Bengali, Finnish, Japanese, Indonesian, Kiswahili, Korean, Russian, Telugu, Thai)
|
| 8 |
-
- MLQA (Arabic, Chinese, English, German, Hindi, Spanish, Vietnames)
|
| 9 |
-
- XQuAD (Arabic, Chinese, German, Greek, Hindi, Russian, Spanish, Thai, Turkish Vietnamese)
|
| 10 |
-
- GermanQuAD (German)
|
| 11 |
-
- Persian QA (Persian)
|
| 12 |
-
- Bengali QA (Bengali)
|
| 13 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
|
| 16 |
There is no guarantee that it will produce a question in the language of the passage, but it usually does.
|
| 17 |
|
| 18 |
|
| 19 |
-
Model
|
|
|
|
| 3 |
|
| 4 |
## Trained on the following datasets:
|
| 5 |
|
| 6 |
+
- [SQuAD (English)](https://rajpurkar.github.io/SQuAD-explorer/)
|
| 7 |
+
- [TyDiQA-GoldP (Arabic, Bengali, Finnish, Japanese, Indonesian, Kiswahili, Korean, Russian, Telugu, Thai)](https://github.com/google-research-datasets/tydiqa)
|
| 8 |
+
- [MLQA (Arabic, Chinese, English, German, Hindi, Spanish, Vietnames)](https://github.com/facebookresearch/MLQA)
|
| 9 |
+
- [XQuAD (Arabic, Chinese, German, Greek, Hindi, Russian, Spanish, Thai, Turkish Vietnamese)](https://github.com/deepmind/xquad)
|
| 10 |
+
- [GermanQuAD (German)](https://huggingface.co/datasets/deepset/germanquad)
|
| 11 |
+
- [Persian QA (Persian)](https://www.kaggle.com/sajjadayobi360/persianqa)
|
| 12 |
+
- [Bengali QA (Bengali)](https://www.kaggle.com/mayeesha/bengali-question-answering-dataset)
|
| 13 |
+
- [chaii (Hindi, Tamil)](https://www.kaggle.com/c/chaii-hindi-and-tamil-question-answering/data)
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
## Training details
|
| 17 |
+
I used [flax summarization script](https://github.com/huggingface/transformers/tree/master/examples/flax/summarization) and a TPU v3-8. Summarization expects a text column and a summary column. For question generation training, use the context column instead of text column and question instead of summary column.
|
| 18 |
|
| 19 |
|
| 20 |
There is no guarantee that it will produce a question in the language of the passage, but it usually does.
|
| 21 |
|
| 22 |
|
| 23 |
+
Model trained on Cloud TPUs from Google's TPU Research Cloud (TRC)
|