Hugging Face's logo

cross-encoder
/
ms-marco-electra-base

Text Ranking
sentence-transformers
PyTorch
ONNX
Safetensors
OpenVINO
Transformers
English
electra
text-classification
Model card Files
xet
 Community
3
Tom Aarsen commited on
Commit
d720fd7
·
1 Parent(s): 0333c86

Revert inadvertent config, tokenizer updates

Browse files

This reverts commit 9d005c539e13071a715a557a7299e2a4ee22b303.

Files changed (3) hide show
  1. README.md +78 -78
  2. config.json +31 -38
  3. special_tokens_map.json +5 -35
README.md CHANGED
@@ -1,79 +1,79 @@
1
- ---
2
- license: apache-2.0
3
- datasets:
4
- - sentence-transformers/msmarco
5
- language:
6
- - en
7
- base_model:
8
- - google/electra-base-discriminator
9
- pipeline_tag: text-ranking
10
- library_name: sentence-transformers
11
- tags:
12
- - transformers
13
- ---
14
- # Cross-Encoder for MS Marco
15
-
16
- This model was trained on the [MS Marco Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking) task.
17
-
18
- The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) for more details. The training code is available here: [SBERT.net Training MS Marco](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco)
19
-
20
-
21
- ## Usage with SentenceTransformers
22
-
23
- The usage is easy when you have [SentenceTransformers](https://www.sbert.net/) installed. Then you can use the pre-trained models like this:
24
- ```python
25
- from sentence_transformers import CrossEncoder
26
-
27
- model = CrossEncoder('cross-encoder/ms-marco-electra-base')
28
- scores = model.predict([
29
- ("How many people live in Berlin?", "Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers."),
30
- ("How many people live in Berlin?", "Berlin is well known for its museums."),
31
- ])
32
- print(scores)
33
- # [9.9227107e-01 2.0136760e-05]
34
- ```
35
-
36
- ## Usage with Transformers
37
-
38
- ```python
39
- from transformers import AutoTokenizer, AutoModelForSequenceClassification
40
- import torch
41
-
42
- model = AutoModelForSequenceClassification.from_pretrained('model_name')
43
- tokenizer = AutoTokenizer.from_pretrained('model_name')
44
-
45
- features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
46
-
47
- model.eval()
48
- with torch.no_grad():
49
- scores = model(**features).logits
50
- print(scores)
51
- ```
52
-
53
-
54
- ## Performance
55
- In the following table, we provide various pre-trained Cross-Encoders together with their performance on the [TREC Deep Learning 2019](https://microsoft.github.io/TREC-2019-Deep-Learning/) and the [MS Marco Passage Reranking](https://github.com/microsoft/MSMARCO-Passage-Ranking/) dataset.
56
-
57
-
58
- | Model-Name | NDCG@10 (TREC DL 19) | MRR@10 (MS Marco Dev) | Docs / Sec |
59
- | ------------- |:-------------| -----| --- |
60
- | **Version 2 models** | | |
61
- | cross-encoder/ms-marco-TinyBERT-L-2-v2 | 69.84 | 32.56 | 9000
62
- | cross-encoder/ms-marco-MiniLM-L-2-v2 | 71.01 | 34.85 | 4100
63
- | cross-encoder/ms-marco-MiniLM-L-4-v2 | 73.04 | 37.70 | 2500
64
- | cross-encoder/ms-marco-MiniLM-L-6-v2 | 74.30 | 39.01 | 1800
65
- | cross-encoder/ms-marco-MiniLM-L-12-v2 | 74.31 | 39.02 | 960
66
- | **Version 1 models** | | |
67
- | cross-encoder/ms-marco-TinyBERT-L-2 | 67.43 | 30.15 | 9000
68
- | cross-encoder/ms-marco-TinyBERT-L-4 | 68.09 | 34.50 | 2900
69
- | cross-encoder/ms-marco-TinyBERT-L-6 | 69.57 | 36.13 | 680
70
- | cross-encoder/ms-marco-electra-base | 71.99 | 36.41 | 340
71
- | **Other models** | | |
72
- | nboost/pt-tinybert-msmarco | 63.63 | 28.80 | 2900
73
- | nboost/pt-bert-base-uncased-msmarco | 70.94 | 34.75 | 340
74
- | nboost/pt-bert-large-msmarco | 73.36 | 36.48 | 100
75
- | Capreolus/electra-base-msmarco | 71.23 | 36.89 | 340
76
- | amberoad/bert-multilingual-passage-reranking-msmarco | 68.40 | 35.54 | 330
77
- | sebastian-hofstaetter/distilbert-cat-margin_mse-T2-msmarco | 72.82 | 37.88 | 720
78
-
79
  Note: Runtime was computed on a V100 GPU.
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - sentence-transformers/msmarco
5
+ language:
6
+ - en
7
+ base_model:
8
+ - google/electra-base-discriminator
9
+ pipeline_tag: text-ranking
10
+ library_name: sentence-transformers
11
+ tags:
12
+ - transformers
13
+ ---
14
+ # Cross-Encoder for MS Marco
15
+
16
+ This model was trained on the [MS Marco Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking) task.
17
+
18
+ The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) for more details. The training code is available here: [SBERT.net Training MS Marco](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco)
19
+
20
+
21
+ ## Usage with SentenceTransformers
22
+
23
+ The usage is easy when you have [SentenceTransformers](https://www.sbert.net/) installed. Then you can use the pre-trained models like this:
24
+ ```python
25
+ from sentence_transformers import CrossEncoder
26
+
27
+ model = CrossEncoder('cross-encoder/ms-marco-electra-base')
28
+ scores = model.predict([
29
+ ("How many people live in Berlin?", "Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers."),
30
+ ("How many people live in Berlin?", "Berlin is well known for its museums."),
31
+ ])
32
+ print(scores)
33
+ # [9.9227107e-01 2.0136760e-05]
34
+ ```
35
+
36
+ ## Usage with Transformers
37
+
38
+ ```python
39
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
40
+ import torch
41
+
42
+ model = AutoModelForSequenceClassification.from_pretrained('model_name')
43
+ tokenizer = AutoTokenizer.from_pretrained('model_name')
44
+
45
+ features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
46
+
47
+ model.eval()
48
+ with torch.no_grad():
49
+ scores = model(**features).logits
50
+ print(scores)
51
+ ```
52
+
53
+
54
+ ## Performance
55
+ In the following table, we provide various pre-trained Cross-Encoders together with their performance on the [TREC Deep Learning 2019](https://microsoft.github.io/TREC-2019-Deep-Learning/) and the [MS Marco Passage Reranking](https://github.com/microsoft/MSMARCO-Passage-Ranking/) dataset.
56
+
57
+
58
+ | Model-Name | NDCG@10 (TREC DL 19) | MRR@10 (MS Marco Dev) | Docs / Sec |
59
+ | ------------- |:-------------| -----| --- |
60
+ | **Version 2 models** | | |
61
+ | cross-encoder/ms-marco-TinyBERT-L-2-v2 | 69.84 | 32.56 | 9000
62
+ | cross-encoder/ms-marco-MiniLM-L-2-v2 | 71.01 | 34.85 | 4100
63
+ | cross-encoder/ms-marco-MiniLM-L-4-v2 | 73.04 | 37.70 | 2500
64
+ | cross-encoder/ms-marco-MiniLM-L-6-v2 | 74.30 | 39.01 | 1800
65
+ | cross-encoder/ms-marco-MiniLM-L-12-v2 | 74.31 | 39.02 | 960
66
+ | **Version 1 models** | | |
67
+ | cross-encoder/ms-marco-TinyBERT-L-2 | 67.43 | 30.15 | 9000
68
+ | cross-encoder/ms-marco-TinyBERT-L-4 | 68.09 | 34.50 | 2900
69
+ | cross-encoder/ms-marco-TinyBERT-L-6 | 69.57 | 36.13 | 680
70
+ | cross-encoder/ms-marco-electra-base | 71.99 | 36.41 | 340
71
+ | **Other models** | | |
72
+ | nboost/pt-tinybert-msmarco | 63.63 | 28.80 | 2900
73
+ | nboost/pt-bert-base-uncased-msmarco | 70.94 | 34.75 | 340
74
+ | nboost/pt-bert-large-msmarco | 73.36 | 36.48 | 100
75
+ | Capreolus/electra-base-msmarco | 71.23 | 36.89 | 340
76
+ | amberoad/bert-multilingual-passage-reranking-msmarco | 68.40 | 35.54 | 330
77
+ | sebastian-hofstaetter/distilbert-cat-margin_mse-T2-msmarco | 72.82 | 37.88 | 720
78
+
79
  Note: Runtime was computed on a V100 GPU.
config.json CHANGED
@@ -1,38 +1,31 @@
1
- {
2
- "architectures": [
3
- "ElectraForSequenceClassification"
4
- ],
5
- "attention_probs_dropout_prob": 0.1,
6
- "classifier_dropout": null,
7
- "embedding_size": 768,
8
- "hidden_act": "gelu",
9
- "hidden_dropout_prob": 0.1,
10
- "hidden_size": 768,
11
- "id2label": {
12
- "0": "LABEL_0"
13
- },
14
- "initializer_range": 0.02,
15
- "intermediate_size": 3072,
16
- "label2id": {
17
- "LABEL_0": 0
18
- },
19
- "layer_norm_eps": 1e-12,
20
- "max_position_embeddings": 512,
21
- "model_type": "electra",
22
- "num_attention_heads": 12,
23
- "num_hidden_layers": 12,
24
- "pad_token_id": 0,
25
- "position_embedding_type": "absolute",
26
- "sentence_transformers": {
27
- "activation_fn": "torch.nn.modules.activation.Sigmoid",
28
- "version": "4.1.0.dev0"
29
- },
30
- "summary_activation": "gelu",
31
- "summary_last_dropout": 0.1,
32
- "summary_type": "first",
33
- "summary_use_proj": true,
34
- "transformers_version": "4.52.0.dev0",
35
- "type_vocab_size": 2,
36
- "use_cache": true,
37
- "vocab_size": 30522
38
- }
 
1
+ {
2
+ "_name_or_path": "google/electra-base-discriminator",
3
+ "architectures": [
4
+ "ElectraForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "embedding_size": 768,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 768,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 3072,
16
+ "label2id": {
17
+ "LABEL_0": 0
18
+ },
19
+ "layer_norm_eps": 1e-12,
20
+ "max_position_embeddings": 512,
21
+ "model_type": "electra",
22
+ "num_attention_heads": 12,
23
+ "num_hidden_layers": 12,
24
+ "pad_token_id": 0,
25
+ "summary_activation": "gelu",
26
+ "summary_last_dropout": 0.1,
27
+ "summary_type": "first",
28
+ "summary_use_proj": true,
29
+ "type_vocab_size": 2,
30
+ "vocab_size": 30522
31
+ }
 
 
 
 
 
 
 
special_tokens_map.json CHANGED
@@ -1,37 +1,7 @@
1
  {
2
- "cls_token": {
3
- "content": "[CLS]",
4
- "lstrip": false,
5
- "normalized": false,
6
- "rstrip": false,
7
- "single_word": false
8
- },
9
- "mask_token": {
10
- "content": "[MASK]",
11
- "lstrip": false,
12
- "normalized": false,
13
- "rstrip": false,
14
- "single_word": false
15
- },
16
- "pad_token": {
17
- "content": "[PAD]",
18
- "lstrip": false,
19
- "normalized": false,
20
- "rstrip": false,
21
- "single_word": false
22
- },
23
- "sep_token": {
24
- "content": "[SEP]",
25
- "lstrip": false,
26
- "normalized": false,
27
- "rstrip": false,
28
- "single_word": false
29
- },
30
- "unk_token": {
31
- "content": "[UNK]",
32
- "lstrip": false,
33
- "normalized": false,
34
- "rstrip": false,
35
- "single_word": false
36
- }
37
  }
 
1
  {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  }