Commit ·
cc9f031
1
Parent(s): 25113a1
Delete README.md
Browse files
README.md
DELETED
|
@@ -1,57 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
language: multilingual
|
| 3 |
-
datasets:
|
| 4 |
-
- NQ
|
| 5 |
-
- Trivia
|
| 6 |
-
- SQuAD
|
| 7 |
-
- MLQA
|
| 8 |
-
- DRCD
|
| 9 |
-
---
|
| 10 |
-
|
| 11 |
-
# dpr-ctx_encoder-bert-base-multilingual
|
| 12 |
-
|
| 13 |
-
## Description
|
| 14 |
-
|
| 15 |
-
Multilingual DPR Model base on bert-base-multilingual-cased.
|
| 16 |
-
[DPR model](https://arxiv.org/abs/2004.04906)
|
| 17 |
-
[DPR repo](https://github.com/facebookresearch/DPR)
|
| 18 |
-
|
| 19 |
-
## Data
|
| 20 |
-
1. [NQ](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
|
| 21 |
-
2. [Trivia](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
|
| 22 |
-
3. [SQuAD](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
|
| 23 |
-
4. [DRCD*](https://github.com/DRCKnowledgeTeam/DRCD)
|
| 24 |
-
5. [MLQA*](https://github.com/facebookresearch/MLQA)
|
| 25 |
-
|
| 26 |
-
`question pairs for train`: 644,217
|
| 27 |
-
`question pairs for dev`: 73,710
|
| 28 |
-
|
| 29 |
-
*DRCD and MLQA are converted using script from haystack [squad_to_dpr.py](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)
|
| 30 |
-
|
| 31 |
-
## Training Script
|
| 32 |
-
I use the script from [haystack](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.ipynb)
|
| 33 |
-
|
| 34 |
-
## Usage
|
| 35 |
-
|
| 36 |
-
```python
|
| 37 |
-
from transformers import DPRContextEncoder, DPRContextEncoderTokenizer
|
| 38 |
-
tokenizer = DPRContextEncoderTokenizer.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
|
| 39 |
-
model = DPRContextEncoder.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
|
| 40 |
-
input_ids = tokenizer("Hello, is my dog cute ?", return_tensors='pt')["input_ids"]
|
| 41 |
-
embeddings = model(input_ids).pooler_output
|
| 42 |
-
```
|
| 43 |
-
|
| 44 |
-
Follow the tutorial from `haystack`:
|
| 45 |
-
[Better Retrievers via "Dense Passage Retrieval"](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial6_Better_Retrieval_via_DPR.ipynb)
|
| 46 |
-
```
|
| 47 |
-
from haystack.retriever.dense import DensePassageRetriever
|
| 48 |
-
retriever = DensePassageRetriever(document_store=document_store,
|
| 49 |
-
query_embedding_model="voidful/dpr-question_encoder-bert-base-multilingual",
|
| 50 |
-
passage_embedding_model="voidful/dpr-ctx_encoder-bert-base-multilingual",
|
| 51 |
-
max_seq_len_query=64,
|
| 52 |
-
max_seq_len_passage=256,
|
| 53 |
-
batch_size=16,
|
| 54 |
-
use_gpu=True,
|
| 55 |
-
embed_title=True,
|
| 56 |
-
use_fast_tokenizers=True)
|
| 57 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|