Hugging Face's logo

jangedoo
/
multilingual-e5-small-pruned

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
token-pruning
text-embeddings-inference
Model card Files
xet
 Community
jangedoo commited on
Commit
696e4b6
·
verified ·
1 Parent(s): 3d0c5ff

Add new SentenceTransformer model

Browse files
Files changed (11) hide show
  1. .gitattributes +1 -0
  2. 1_Pooling/config.json +5 -0
  3. README.md +482 -0
  4. config.json +30 -0
  5. config_sentence_transformers.json +14 -0
  6. model.safetensors +3 -0
  7. modules.json +20 -0
  8. pruning_tokenizer.py +62 -0
  9. sentence_bert_config.json +10 -0
  10. tokenizer.json +3 -0
  11. tokenizer_config.json +0 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
1_Pooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "embedding_dimension": 384,
3
+ "pooling_mode": "mean",
4
+ "include_prompt": true
5
+ }
README.md ADDED
@@ -0,0 +1,482 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: intfloat/multilingual-e5-small
3
+ pipeline_tag: sentence-similarity
4
+ library_name: sentence-transformers
5
+ tags:
6
+ - sentence-transformers
7
+ - sentence-similarity
8
+ - feature-extraction
9
+ - token-pruning
10
+ ---
11
+
12
+ # multilingual-e5-small-pruned
13
+
14
+ This model is a **token-embedding pruned** version of
15
+ [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small).
16
+
17
+ Token-embedding pruning clusters semantically similar tokens in the embedding
18
+ space (using DBSCAN) and merges each cluster into a single shared embedding,
19
+ shrinking the vocabulary and reducing memory without retraining the transformer
20
+ layers.
21
+
22
+ ## How to use
23
+
24
+ ```python
25
+ from sentence_transformers import SentenceTransformer
26
+
27
+ model = SentenceTransformer("jangedoo/multilingual-e5-small-pruned", trust_remote_code=True)
28
+ embeddings = model.encode(["Hello world", "How are you?"])
29
+ ```
30
+
31
+ > **Note:** `trust_remote_code=True` is required because the model ships a small
32
+ > custom tokenizer class (`pruned_tokenizer.py`) that applies the id remapping
33
+ > after tokenization. No additional package installation is needed.
34
+
35
+ ## Pruning statistics
36
+
37
+ | | Base | Pruned | Reduction |
38
+ |---|---:|---:|---:|
39
+ | Vocab size | 250,037 | 172,569 | 30.98% |
40
+ | Total parameters | 117,653,760 | 87,906,048 | 25.28% |
41
+ | Embedding parameters | 96,014,208 | 66,266,496 | 30.98% |
42
+ | Embedding size (MB) | 366.3 | 252.8 | 113.5 MB saved |
43
+
44
+ ## Evaluation
45
+
46
+ | Dataset / Metric | Base | Pruned | Relative (base = 1.0) |
47
+ |---|---:|---:|---:|
48
+ | stsb / stsb_pearson_cosine | 0.8092 | 0.7925 | 0.9794 |
49
+ | stsb / stsb_spearman_cosine | 0.8359 | 0.8014 | 0.9588 |
50
+ | nanobeir / NanoClimateFEVER_cosine_accuracy@1 | 0.3000 | 0.2600 | 0.8667 |
51
+ | nanobeir / NanoClimateFEVER_cosine_accuracy@3 | 0.4200 | 0.3600 | 0.8571 |
52
+ | nanobeir / NanoClimateFEVER_cosine_accuracy@5 | 0.5000 | 0.3800 | 0.7600 |
53
+ | nanobeir / NanoClimateFEVER_cosine_accuracy@10 | 0.6600 | 0.5400 | 0.8182 |
54
+ | nanobeir / NanoClimateFEVER_cosine_precision@1 | 0.3000 | 0.2600 | 0.8667 |
55
+ | nanobeir / NanoClimateFEVER_cosine_precision@3 | 0.1533 | 0.1333 | 0.8696 |
56
+ | nanobeir / NanoClimateFEVER_cosine_precision@5 | 0.1160 | 0.0880 | 0.7586 |
57
+ | nanobeir / NanoClimateFEVER_cosine_precision@10 | 0.0880 | 0.0680 | 0.7727 |
58
+ | nanobeir / NanoClimateFEVER_cosine_recall@1 | 0.1500 | 0.1283 | 0.8556 |
59
+ | nanobeir / NanoClimateFEVER_cosine_recall@3 | 0.2000 | 0.1717 | 0.8583 |
60
+ | nanobeir / NanoClimateFEVER_cosine_recall@5 | 0.2433 | 0.1817 | 0.7466 |
61
+ | nanobeir / NanoClimateFEVER_cosine_recall@10 | 0.3530 | 0.2667 | 0.7554 |
62
+ | nanobeir / NanoClimateFEVER_cosine_ndcg@10 | 0.2927 | 0.2364 | 0.8076 |
63
+ | nanobeir / NanoClimateFEVER_cosine_mrr@10 | 0.3906 | 0.3305 | 0.8464 |
64
+ | nanobeir / NanoClimateFEVER_cosine_map@100 | 0.2358 | 0.1934 | 0.8202 |
65
+ | nanobeir / NanoDBPedia_cosine_accuracy@1 | 0.5800 | 0.6400 | 1.1034 |
66
+ | nanobeir / NanoDBPedia_cosine_accuracy@3 | 0.8400 | 0.7800 | 0.9286 |
67
+ | nanobeir / NanoDBPedia_cosine_accuracy@5 | 0.8800 | 0.8400 | 0.9545 |
68
+ | nanobeir / NanoDBPedia_cosine_accuracy@10 | 0.9600 | 0.9200 | 0.9583 |
69
+ | nanobeir / NanoDBPedia_cosine_precision@1 | 0.5800 | 0.6400 | 1.1034 |
70
+ | nanobeir / NanoDBPedia_cosine_precision@3 | 0.5400 | 0.5200 | 0.9630 |
71
+ | nanobeir / NanoDBPedia_cosine_precision@5 | 0.5200 | 0.4920 | 0.9462 |
72
+ | nanobeir / NanoDBPedia_cosine_precision@10 | 0.4300 | 0.3980 | 0.9256 |
73
+ | nanobeir / NanoDBPedia_cosine_recall@1 | 0.0755 | 0.0895 | 1.1861 |
74
+ | nanobeir / NanoDBPedia_cosine_recall@3 | 0.1534 | 0.1405 | 0.9156 |
75
+ | nanobeir / NanoDBPedia_cosine_recall@5 | 0.2049 | 0.1976 | 0.9641 |
76
+ | nanobeir / NanoDBPedia_cosine_recall@10 | 0.3126 | 0.2802 | 0.8963 |
77
+ | nanobeir / NanoDBPedia_cosine_ndcg@10 | 0.5371 | 0.5108 | 0.9510 |
78
+ | nanobeir / NanoDBPedia_cosine_mrr@10 | 0.7175 | 0.7211 | 1.0050 |
79
+ | nanobeir / NanoDBPedia_cosine_map@100 | 0.3988 | 0.3692 | 0.9256 |
80
+ | nanobeir / NanoFEVER_cosine_accuracy@1 | 0.6200 | 0.5400 | 0.8710 |
81
+ | nanobeir / NanoFEVER_cosine_accuracy@3 | 0.8800 | 0.8200 | 0.9318 |
82
+ | nanobeir / NanoFEVER_cosine_accuracy@5 | 0.9400 | 0.8800 | 0.9362 |
83
+ | nanobeir / NanoFEVER_cosine_accuracy@10 | 0.9800 | 0.9600 | 0.9796 |
84
+ | nanobeir / NanoFEVER_cosine_precision@1 | 0.6200 | 0.5400 | 0.8710 |
85
+ | nanobeir / NanoFEVER_cosine_precision@3 | 0.3000 | 0.2800 | 0.9333 |
86
+ | nanobeir / NanoFEVER_cosine_precision@5 | 0.1960 | 0.1800 | 0.9184 |
87
+ | nanobeir / NanoFEVER_cosine_precision@10 | 0.1020 | 0.1000 | 0.9804 |
88
+ | nanobeir / NanoFEVER_cosine_recall@1 | 0.5867 | 0.5067 | 0.8636 |
89
+ | nanobeir / NanoFEVER_cosine_recall@3 | 0.8433 | 0.7967 | 0.9447 |
90
+ | nanobeir / NanoFEVER_cosine_recall@5 | 0.9033 | 0.8567 | 0.9483 |
91
+ | nanobeir / NanoFEVER_cosine_recall@10 | 0.9333 | 0.9233 | 0.9893 |
92
+ | nanobeir / NanoFEVER_cosine_ndcg@10 | 0.7897 | 0.7353 | 0.9310 |
93
+ | nanobeir / NanoFEVER_cosine_mrr@10 | 0.7592 | 0.6909 | 0.9100 |
94
+ | nanobeir / NanoFEVER_cosine_map@100 | 0.7338 | 0.6652 | 0.9066 |
95
+ | nanobeir / NanoFiQA2018_cosine_accuracy@1 | 0.3600 | 0.3200 | 0.8889 |
96
+ | nanobeir / NanoFiQA2018_cosine_accuracy@3 | 0.5600 | 0.5200 | 0.9286 |
97
+ | nanobeir / NanoFiQA2018_cosine_accuracy@5 | 0.6200 | 0.5800 | 0.9355 |
98
+ | nanobeir / NanoFiQA2018_cosine_accuracy@10 | 0.6600 | 0.6800 | 1.0303 |
99
+ | nanobeir / NanoFiQA2018_cosine_precision@1 | 0.3600 | 0.3200 | 0.8889 |
100
+ | nanobeir / NanoFiQA2018_cosine_precision@3 | 0.2400 | 0.1933 | 0.8056 |
101
+ | nanobeir / NanoFiQA2018_cosine_precision@5 | 0.1800 | 0.1560 | 0.8667 |
102
+ | nanobeir / NanoFiQA2018_cosine_precision@10 | 0.1060 | 0.0960 | 0.9057 |
103
+ | nanobeir / NanoFiQA2018_cosine_recall@1 | 0.1801 | 0.1687 | 0.9371 |
104
+ | nanobeir / NanoFiQA2018_cosine_recall@3 | 0.3545 | 0.3174 | 0.8954 |
105
+ | nanobeir / NanoFiQA2018_cosine_recall@5 | 0.4403 | 0.3816 | 0.8666 |
106
+ | nanobeir / NanoFiQA2018_cosine_recall@10 | 0.4878 | 0.4738 | 0.9713 |
107
+ | nanobeir / NanoFiQA2018_cosine_ndcg@10 | 0.3956 | 0.3655 | 0.9240 |
108
+ | nanobeir / NanoFiQA2018_cosine_mrr@10 | 0.4630 | 0.4302 | 0.9292 |
109
+ | nanobeir / NanoFiQA2018_cosine_map@100 | 0.3380 | 0.2928 | 0.8664 |
110
+ | nanobeir / NanoHotpotQA_cosine_accuracy@1 | 0.7800 | 0.6800 | 0.8718 |
111
+ | nanobeir / NanoHotpotQA_cosine_accuracy@3 | 0.9200 | 0.9000 | 0.9783 |
112
+ | nanobeir / NanoHotpotQA_cosine_accuracy@5 | 0.9600 | 0.9200 | 0.9583 |
113
+ | nanobeir / NanoHotpotQA_cosine_accuracy@10 | 0.9800 | 0.9400 | 0.9592 |
114
+ | nanobeir / NanoHotpotQA_cosine_precision@1 | 0.7800 | 0.6800 | 0.8718 |
115
+ | nanobeir / NanoHotpotQA_cosine_precision@3 | 0.5000 | 0.4533 | 0.9067 |
116
+ | nanobeir / NanoHotpotQA_cosine_precision@5 | 0.3240 | 0.3040 | 0.9383 |
117
+ | nanobeir / NanoHotpotQA_cosine_precision@10 | 0.1720 | 0.1600 | 0.9302 |
118
+ | nanobeir / NanoHotpotQA_cosine_recall@1 | 0.3900 | 0.3400 | 0.8718 |
119
+ | nanobeir / NanoHotpotQA_cosine_recall@3 | 0.7500 | 0.6800 | 0.9067 |
120
+ | nanobeir / NanoHotpotQA_cosine_recall@5 | 0.8100 | 0.7600 | 0.9383 |
121
+ | nanobeir / NanoHotpotQA_cosine_recall@10 | 0.8600 | 0.8000 | 0.9302 |
122
+ | nanobeir / NanoHotpotQA_cosine_ndcg@10 | 0.7997 | 0.7254 | 0.9072 |
123
+ | nanobeir / NanoHotpotQA_cosine_mrr@10 | 0.8600 | 0.7879 | 0.9161 |
124
+ | nanobeir / NanoHotpotQA_cosine_map@100 | 0.7435 | 0.6629 | 0.8916 |
125
+ | nanobeir / NanoMSMARCO_cosine_accuracy@1 | 0.4200 | 0.4200 | 1.0000 |
126
+ | nanobeir / NanoMSMARCO_cosine_accuracy@3 | 0.5800 | 0.6000 | 1.0345 |
127
+ | nanobeir / NanoMSMARCO_cosine_accuracy@5 | 0.7600 | 0.6800 | 0.8947 |
128
+ | nanobeir / NanoMSMARCO_cosine_accuracy@10 | 0.8600 | 0.7800 | 0.9070 |
129
+ | nanobeir / NanoMSMARCO_cosine_precision@1 | 0.4200 | 0.4200 | 1.0000 |
130
+ | nanobeir / NanoMSMARCO_cosine_precision@3 | 0.1933 | 0.2000 | 1.0345 |
131
+ | nanobeir / NanoMSMARCO_cosine_precision@5 | 0.1520 | 0.1360 | 0.8947 |
132
+ | nanobeir / NanoMSMARCO_cosine_precision@10 | 0.0860 | 0.0780 | 0.9070 |
133
+ | nanobeir / NanoMSMARCO_cosine_recall@1 | 0.4200 | 0.4200 | 1.0000 |
134
+ | nanobeir / NanoMSMARCO_cosine_recall@3 | 0.5800 | 0.6000 | 1.0345 |
135
+ | nanobeir / NanoMSMARCO_cosine_recall@5 | 0.7600 | 0.6800 | 0.8947 |
136
+ | nanobeir / NanoMSMARCO_cosine_recall@10 | 0.8600 | 0.7800 | 0.9070 |
137
+ | nanobeir / NanoMSMARCO_cosine_ndcg@10 | 0.6187 | 0.5920 | 0.9568 |
138
+ | nanobeir / NanoMSMARCO_cosine_mrr@10 | 0.5436 | 0.5332 | 0.9808 |
139
+ | nanobeir / NanoMSMARCO_cosine_map@100 | 0.5517 | 0.5444 | 0.9868 |
140
+ | nanobeir / NanoNFCorpus_cosine_accuracy@1 | 0.4200 | 0.4000 | 0.9524 |
141
+ | nanobeir / NanoNFCorpus_cosine_accuracy@3 | 0.5000 | 0.5000 | 1.0000 |
142
+ | nanobeir / NanoNFCorpus_cosine_accuracy@5 | 0.5600 | 0.5600 | 1.0000 |
143
+ | nanobeir / NanoNFCorpus_cosine_accuracy@10 | 0.6400 | 0.6000 | 0.9375 |
144
+ | nanobeir / NanoNFCorpus_cosine_precision@1 | 0.4200 | 0.4000 | 0.9524 |
145
+ | nanobeir / NanoNFCorpus_cosine_precision@3 | 0.3267 | 0.3333 | 1.0204 |
146
+ | nanobeir / NanoNFCorpus_cosine_precision@5 | 0.3280 | 0.3040 | 0.9268 |
147
+ | nanobeir / NanoNFCorpus_cosine_precision@10 | 0.2520 | 0.2460 | 0.9762 |
148
+ | nanobeir / NanoNFCorpus_cosine_recall@1 | 0.0148 | 0.0233 | 1.5744 |
149
+ | nanobeir / NanoNFCorpus_cosine_recall@3 | 0.0442 | 0.0428 | 0.9684 |
150
+ | nanobeir / NanoNFCorpus_cosine_recall@5 | 0.0772 | 0.0685 | 0.8880 |
151
+ | nanobeir / NanoNFCorpus_cosine_recall@10 | 0.0999 | 0.0938 | 0.9389 |
152
+ | nanobeir / NanoNFCorpus_cosine_ndcg@10 | 0.2937 | 0.2873 | 0.9783 |
153
+ | nanobeir / NanoNFCorpus_cosine_mrr@10 | 0.4829 | 0.4625 | 0.9577 |
154
+ | nanobeir / NanoNFCorpus_cosine_map@100 | 0.1046 | 0.1013 | 0.9678 |
155
+ | nanobeir / NanoNQ_cosine_accuracy@1 | 0.5400 | 0.3400 | 0.6296 |
156
+ | nanobeir / NanoNQ_cosine_accuracy@3 | 0.6400 | 0.5200 | 0.8125 |
157
+ | nanobeir / NanoNQ_cosine_accuracy@5 | 0.7000 | 0.6000 | 0.8571 |
158
+ | nanobeir / NanoNQ_cosine_accuracy@10 | 0.8200 | 0.7200 | 0.8780 |
159
+ | nanobeir / NanoNQ_cosine_precision@1 | 0.5400 | 0.3400 | 0.6296 |
160
+ | nanobeir / NanoNQ_cosine_precision@3 | 0.2133 | 0.1733 | 0.8125 |
161
+ | nanobeir / NanoNQ_cosine_precision@5 | 0.1480 | 0.1240 | 0.8378 |
162
+ | nanobeir / NanoNQ_cosine_precision@10 | 0.0900 | 0.0760 | 0.8444 |
163
+ | nanobeir / NanoNQ_cosine_recall@1 | 0.4900 | 0.3400 | 0.6939 |
164
+ | nanobeir / NanoNQ_cosine_recall@3 | 0.5900 | 0.5000 | 0.8475 |
165
+ | nanobeir / NanoNQ_cosine_recall@5 | 0.6700 | 0.5900 | 0.8806 |
166
+ | nanobeir / NanoNQ_cosine_recall@10 | 0.8000 | 0.7000 | 0.8750 |
167
+ | nanobeir / NanoNQ_cosine_ndcg@10 | 0.6371 | 0.5086 | 0.7983 |
168
+ | nanobeir / NanoNQ_cosine_mrr@10 | 0.6107 | 0.4496 | 0.7362 |
169
+ | nanobeir / NanoNQ_cosine_map@100 | 0.5816 | 0.4546 | 0.7816 |
170
+ | nanobeir / NanoQuoraRetrieval_cosine_accuracy@1 | 0.8800 | 0.8400 | 0.9545 |
171
+ | nanobeir / NanoQuoraRetrieval_cosine_accuracy@3 | 1.0000 | 0.9600 | 0.9600 |
172
+ | nanobeir / NanoQuoraRetrieval_cosine_accuracy@5 | 1.0000 | 0.9600 | 0.9600 |
173
+ | nanobeir / NanoQuoraRetrieval_cosine_accuracy@10 | 1.0000 | 0.9600 | 0.9600 |
174
+ | nanobeir / NanoQuoraRetrieval_cosine_precision@1 | 0.8800 | 0.8400 | 0.9545 |
175
+ | nanobeir / NanoQuoraRetrieval_cosine_precision@3 | 0.4067 | 0.3733 | 0.9180 |
176
+ | nanobeir / NanoQuoraRetrieval_cosine_precision@5 | 0.2520 | 0.2280 | 0.9048 |
177
+ | nanobeir / NanoQuoraRetrieval_cosine_precision@10 | 0.1320 | 0.1180 | 0.8939 |
178
+ | nanobeir / NanoQuoraRetrieval_cosine_recall@1 | 0.7807 | 0.7540 | 0.9658 |
179
+ | nanobeir / NanoQuoraRetrieval_cosine_recall@3 | 0.9587 | 0.9253 | 0.9652 |
180
+ | nanobeir / NanoQuoraRetrieval_cosine_recall@5 | 0.9693 | 0.9320 | 0.9615 |
181
+ | nanobeir / NanoQuoraRetrieval_cosine_recall@10 | 0.9833 | 0.9393 | 0.9553 |
182
+ | nanobeir / NanoQuoraRetrieval_cosine_ndcg@10 | 0.9359 | 0.8947 | 0.9560 |
183
+ | nanobeir / NanoQuoraRetrieval_cosine_mrr@10 | 0.9333 | 0.8967 | 0.9607 |
184
+ | nanobeir / NanoQuoraRetrieval_cosine_map@100 | 0.9123 | 0.8732 | 0.9572 |
185
+ | nanobeir / NanoSCIDOCS_cosine_accuracy@1 | 0.4000 | 0.3000 | 0.7500 |
186
+ | nanobeir / NanoSCIDOCS_cosine_accuracy@3 | 0.6400 | 0.5200 | 0.8125 |
187
+ | nanobeir / NanoSCIDOCS_cosine_accuracy@5 | 0.7400 | 0.6000 | 0.8108 |
188
+ | nanobeir / NanoSCIDOCS_cosine_accuracy@10 | 0.8200 | 0.7800 | 0.9512 |
189
+ | nanobeir / NanoSCIDOCS_cosine_precision@1 | 0.4000 | 0.3000 | 0.7500 |
190
+ | nanobeir / NanoSCIDOCS_cosine_precision@3 | 0.3067 | 0.2333 | 0.7609 |
191
+ | nanobeir / NanoSCIDOCS_cosine_precision@5 | 0.2600 | 0.2000 | 0.7692 |
192
+ | nanobeir / NanoSCIDOCS_cosine_precision@10 | 0.1560 | 0.1400 | 0.8974 |
193
+ | nanobeir / NanoSCIDOCS_cosine_recall@1 | 0.0847 | 0.0627 | 0.7402 |
194
+ | nanobeir / NanoSCIDOCS_cosine_recall@3 | 0.1897 | 0.1437 | 0.7575 |
195
+ | nanobeir / NanoSCIDOCS_cosine_recall@5 | 0.2667 | 0.2057 | 0.7712 |
196
+ | nanobeir / NanoSCIDOCS_cosine_recall@10 | 0.3187 | 0.2887 | 0.9059 |
197
+ | nanobeir / NanoSCIDOCS_cosine_ndcg@10 | 0.3225 | 0.2703 | 0.8380 |
198
+ | nanobeir / NanoSCIDOCS_cosine_mrr@10 | 0.5353 | 0.4398 | 0.8216 |
199
+ | nanobeir / NanoSCIDOCS_cosine_map@100 | 0.2448 | 0.1997 | 0.8155 |
200
+ | nanobeir / NanoArguAna_cosine_accuracy@1 | 0.1000 | 0.1000 | 1.0000 |
201
+ | nanobeir / NanoArguAna_cosine_accuracy@3 | 0.4800 | 0.4400 | 0.9167 |
202
+ | nanobeir / NanoArguAna_cosine_accuracy@5 | 0.6200 | 0.4800 | 0.7742 |
203
+ | nanobeir / NanoArguAna_cosine_accuracy@10 | 0.7200 | 0.6200 | 0.8611 |
204
+ | nanobeir / NanoArguAna_cosine_precision@1 | 0.1000 | 0.1000 | 1.0000 |
205
+ | nanobeir / NanoArguAna_cosine_precision@3 | 0.1600 | 0.1467 | 0.9167 |
206
+ | nanobeir / NanoArguAna_cosine_precision@5 | 0.1240 | 0.0960 | 0.7742 |
207
+ | nanobeir / NanoArguAna_cosine_precision@10 | 0.0720 | 0.0620 | 0.8611 |
208
+ | nanobeir / NanoArguAna_cosine_recall@1 | 0.1000 | 0.1000 | 1.0000 |
209
+ | nanobeir / NanoArguAna_cosine_recall@3 | 0.4800 | 0.4400 | 0.9167 |
210
+ | nanobeir / NanoArguAna_cosine_recall@5 | 0.6200 | 0.4800 | 0.7742 |
211
+ | nanobeir / NanoArguAna_cosine_recall@10 | 0.7200 | 0.6200 | 0.8611 |
212
+ | nanobeir / NanoArguAna_cosine_ndcg@10 | 0.4121 | 0.3676 | 0.8920 |
213
+ | nanobeir / NanoArguAna_cosine_mrr@10 | 0.3128 | 0.2864 | 0.9156 |
214
+ | nanobeir / NanoArguAna_cosine_map@100 | 0.3267 | 0.2962 | 0.9067 |
215
+ | nanobeir / NanoSciFact_cosine_accuracy@1 | 0.6800 | 0.5200 | 0.7647 |
216
+ | nanobeir / NanoSciFact_cosine_accuracy@3 | 0.7400 | 0.6800 | 0.9189 |
217
+ | nanobeir / NanoSciFact_cosine_accuracy@5 | 0.7400 | 0.7400 | 1.0000 |
218
+ | nanobeir / NanoSciFact_cosine_accuracy@10 | 0.7800 | 0.7800 | 1.0000 |
219
+ | nanobeir / NanoSciFact_cosine_precision@1 | 0.6800 | 0.5200 | 0.7647 |
220
+ | nanobeir / NanoSciFact_cosine_precision@3 | 0.2533 | 0.2400 | 0.9474 |
221
+ | nanobeir / NanoSciFact_cosine_precision@5 | 0.1600 | 0.1600 | 1.0000 |
222
+ | nanobeir / NanoSciFact_cosine_precision@10 | 0.0880 | 0.0860 | 0.9773 |
223
+ | nanobeir / NanoSciFact_cosine_recall@1 | 0.6450 | 0.5000 | 0.7752 |
224
+ | nanobeir / NanoSciFact_cosine_recall@3 | 0.7150 | 0.6600 | 0.9231 |
225
+ | nanobeir / NanoSciFact_cosine_recall@5 | 0.7250 | 0.7250 | 1.0000 |
226
+ | nanobeir / NanoSciFact_cosine_recall@10 | 0.7800 | 0.7700 | 0.9872 |
227
+ | nanobeir / NanoSciFact_cosine_ndcg@10 | 0.7209 | 0.6455 | 0.8955 |
228
+ | nanobeir / NanoSciFact_cosine_mrr@10 | 0.7117 | 0.6116 | 0.8592 |
229
+ | nanobeir / NanoSciFact_cosine_map@100 | 0.7011 | 0.6058 | 0.8640 |
230
+ | nanobeir / NanoTouche2020_cosine_accuracy@1 | 0.4898 | 0.4082 | 0.8333 |
231
+ | nanobeir / NanoTouche2020_cosine_accuracy@3 | 0.8980 | 0.7959 | 0.8864 |
232
+ | nanobeir / NanoTouche2020_cosine_accuracy@5 | 0.9388 | 0.8776 | 0.9348 |
233
+ | nanobeir / NanoTouche2020_cosine_accuracy@10 | 0.9796 | 0.9592 | 0.9792 |
234
+ | nanobeir / NanoTouche2020_cosine_precision@1 | 0.4898 | 0.4082 | 0.8333 |
235
+ | nanobeir / NanoTouche2020_cosine_precision@3 | 0.5442 | 0.4150 | 0.7625 |
236
+ | nanobeir / NanoTouche2020_cosine_precision@5 | 0.4816 | 0.4163 | 0.8644 |
237
+ | nanobeir / NanoTouche2020_cosine_precision@10 | 0.4000 | 0.3429 | 0.8571 |
238
+ | nanobeir / NanoTouche2020_cosine_recall@1 | 0.0309 | 0.0251 | 0.8101 |
239
+ | nanobeir / NanoTouche2020_cosine_recall@3 | 0.1093 | 0.0844 | 0.7719 |
240
+ | nanobeir / NanoTouche2020_cosine_recall@5 | 0.1638 | 0.1409 | 0.8598 |
241
+ | nanobeir / NanoTouche2020_cosine_recall@10 | 0.2602 | 0.2243 | 0.8621 |
242
+ | nanobeir / NanoTouche2020_cosine_ndcg@10 | 0.4483 | 0.3753 | 0.8372 |
243
+ | nanobeir / NanoTouche2020_cosine_mrr@10 | 0.6885 | 0.5975 | 0.8679 |
244
+ | nanobeir / NanoTouche2020_cosine_map@100 | 0.3263 | 0.2599 | 0.7967 |
245
+ | nanobeir / NanoBEIR_mean_cosine_accuracy@1 | 0.5054 | 0.4437 | 0.8780 |
246
+ | nanobeir / NanoBEIR_mean_cosine_accuracy@3 | 0.6998 | 0.6458 | 0.9228 |
247
+ | nanobeir / NanoBEIR_mean_cosine_accuracy@5 | 0.7661 | 0.6998 | 0.9135 |
248
+ | nanobeir / NanoBEIR_mean_cosine_accuracy@10 | 0.8354 | 0.7876 | 0.9429 |
249
+ | nanobeir / NanoBEIR_mean_cosine_precision@1 | 0.5054 | 0.4437 | 0.8780 |
250
+ | nanobeir / NanoBEIR_mean_cosine_precision@3 | 0.3183 | 0.2842 | 0.8930 |
251
+ | nanobeir / NanoBEIR_mean_cosine_precision@5 | 0.2494 | 0.2219 | 0.8898 |
252
+ | nanobeir / NanoBEIR_mean_cosine_precision@10 | 0.1672 | 0.1516 | 0.9066 |
253
+ | nanobeir / NanoBEIR_mean_cosine_recall@1 | 0.3037 | 0.2660 | 0.8759 |
254
+ | nanobeir / NanoBEIR_mean_cosine_recall@3 | 0.4591 | 0.4233 | 0.9220 |
255
+ | nanobeir / NanoBEIR_mean_cosine_recall@5 | 0.5272 | 0.4769 | 0.9045 |
256
+ | nanobeir / NanoBEIR_mean_cosine_recall@10 | 0.5976 | 0.5508 | 0.9216 |
257
+ | nanobeir / NanoBEIR_mean_cosine_ndcg@10 | 0.5542 | 0.5011 | 0.9043 |
258
+ | nanobeir / NanoBEIR_mean_cosine_mrr@10 | 0.6161 | 0.5568 | 0.9037 |
259
+ | nanobeir / NanoBEIR_mean_cosine_map@100 | 0.4769 | 0.4245 | 0.8902 |
260
+ | nanobeir_ne / NanoClimateFEVER_cosine_accuracy@1 | 0.1000 | 0.0600 | 0.6000 |
261
+ | nanobeir_ne / NanoClimateFEVER_cosine_accuracy@3 | 0.3000 | 0.1000 | 0.3333 |
262
+ | nanobeir_ne / NanoClimateFEVER_cosine_accuracy@5 | 0.4200 | 0.2000 | 0.4762 |
263
+ | nanobeir_ne / NanoClimateFEVER_cosine_accuracy@10 | 0.5400 | 0.3400 | 0.6296 |
264
+ | nanobeir_ne / NanoClimateFEVER_cosine_precision@1 | 0.1000 | 0.0600 | 0.6000 |
265
+ | nanobeir_ne / NanoClimateFEVER_cosine_precision@3 | 0.1000 | 0.0333 | 0.3333 |
266
+ | nanobeir_ne / NanoClimateFEVER_cosine_precision@5 | 0.1000 | 0.0400 | 0.4000 |
267
+ | nanobeir_ne / NanoClimateFEVER_cosine_precision@10 | 0.0700 | 0.0380 | 0.5429 |
268
+ | nanobeir_ne / NanoClimateFEVER_cosine_recall@1 | 0.0300 | 0.0350 | 1.1667 |
269
+ | nanobeir_ne / NanoClimateFEVER_cosine_recall@3 | 0.1400 | 0.0450 | 0.3214 |
270
+ | nanobeir_ne / NanoClimateFEVER_cosine_recall@5 | 0.2073 | 0.0967 | 0.4662 |
271
+ | nanobeir_ne / NanoClimateFEVER_cosine_recall@10 | 0.2747 | 0.1533 | 0.5583 |
272
+ | nanobeir_ne / NanoClimateFEVER_cosine_ndcg@10 | 0.1836 | 0.0970 | 0.5282 |
273
+ | nanobeir_ne / NanoClimateFEVER_cosine_mrr@10 | 0.2279 | 0.1112 | 0.4881 |
274
+ | nanobeir_ne / NanoClimateFEVER_cosine_map@100 | 0.1241 | 0.0700 | 0.5636 |
275
+ | nanobeir_ne / NanoDBPedia_cosine_accuracy@1 | 0.4000 | 0.3800 | 0.9500 |
276
+ | nanobeir_ne / NanoDBPedia_cosine_accuracy@3 | 0.7600 | 0.6200 | 0.8158 |
277
+ | nanobeir_ne / NanoDBPedia_cosine_accuracy@5 | 0.8000 | 0.7400 | 0.9250 |
278
+ | nanobeir_ne / NanoDBPedia_cosine_accuracy@10 | 0.8200 | 0.8400 | 1.0244 |
279
+ | nanobeir_ne / NanoDBPedia_cosine_precision@1 | 0.4000 | 0.3800 | 0.9500 |
280
+ | nanobeir_ne / NanoDBPedia_cosine_precision@3 | 0.4133 | 0.3333 | 0.8065 |
281
+ | nanobeir_ne / NanoDBPedia_cosine_precision@5 | 0.3680 | 0.3360 | 0.9130 |
282
+ | nanobeir_ne / NanoDBPedia_cosine_precision@10 | 0.3260 | 0.2860 | 0.8773 |
283
+ | nanobeir_ne / NanoDBPedia_cosine_recall@1 | 0.0736 | 0.0736 | 1.0002 |
284
+ | nanobeir_ne / NanoDBPedia_cosine_recall@3 | 0.1475 | 0.1168 | 0.7921 |
285
+ | nanobeir_ne / NanoDBPedia_cosine_recall@5 | 0.1746 | 0.1629 | 0.9329 |
286
+ | nanobeir_ne / NanoDBPedia_cosine_recall@10 | 0.2453 | 0.2255 | 0.9193 |
287
+ | nanobeir_ne / NanoDBPedia_cosine_ndcg@10 | 0.4156 | 0.3676 | 0.8845 |
288
+ | nanobeir_ne / NanoDBPedia_cosine_mrr@10 | 0.5748 | 0.5297 | 0.9215 |
289
+ | nanobeir_ne / NanoDBPedia_cosine_map@100 | 0.3034 | 0.2661 | 0.8770 |
290
+ | nanobeir_ne / NanoFEVER_cosine_accuracy@1 | 0.3400 | 0.1800 | 0.5294 |
291
+ | nanobeir_ne / NanoFEVER_cosine_accuracy@3 | 0.5800 | 0.4600 | 0.7931 |
292
+ | nanobeir_ne / NanoFEVER_cosine_accuracy@5 | 0.6600 | 0.5800 | 0.8788 |
293
+ | nanobeir_ne / NanoFEVER_cosine_accuracy@10 | 0.8000 | 0.7000 | 0.8750 |
294
+ | nanobeir_ne / NanoFEVER_cosine_precision@1 | 0.3400 | 0.1800 | 0.5294 |
295
+ | nanobeir_ne / NanoFEVER_cosine_precision@3 | 0.1933 | 0.1533 | 0.7931 |
296
+ | nanobeir_ne / NanoFEVER_cosine_precision@5 | 0.1360 | 0.1200 | 0.8824 |
297
+ | nanobeir_ne / NanoFEVER_cosine_precision@10 | 0.0820 | 0.0720 | 0.8780 |
298
+ | nanobeir_ne / NanoFEVER_cosine_recall@1 | 0.3267 | 0.1800 | 0.5510 |
299
+ | nanobeir_ne / NanoFEVER_cosine_recall@3 | 0.5567 | 0.4500 | 0.8084 |
300
+ | nanobeir_ne / NanoFEVER_cosine_recall@5 | 0.6467 | 0.5567 | 0.8608 |
301
+ | nanobeir_ne / NanoFEVER_cosine_recall@10 | 0.7767 | 0.6767 | 0.8712 |
302
+ | nanobeir_ne / NanoFEVER_cosine_ndcg@10 | 0.5473 | 0.4206 | 0.7685 |
303
+ | nanobeir_ne / NanoFEVER_cosine_mrr@10 | 0.4854 | 0.3419 | 0.7043 |
304
+ | nanobeir_ne / NanoFEVER_cosine_map@100 | 0.4794 | 0.3463 | 0.7223 |
305
+ | nanobeir_ne / NanoFiQA2018_cosine_accuracy@1 | 0.2600 | 0.1000 | 0.3846 |
306
+ | nanobeir_ne / NanoFiQA2018_cosine_accuracy@3 | 0.4200 | 0.2400 | 0.5714 |
307
+ | nanobeir_ne / NanoFiQA2018_cosine_accuracy@5 | 0.4600 | 0.2600 | 0.5652 |
308
+ | nanobeir_ne / NanoFiQA2018_cosine_accuracy@10 | 0.5400 | 0.3600 | 0.6667 |
309
+ | nanobeir_ne / NanoFiQA2018_cosine_precision@1 | 0.2600 | 0.1000 | 0.3846 |
310
+ | nanobeir_ne / NanoFiQA2018_cosine_precision@3 | 0.1600 | 0.0867 | 0.5417 |
311
+ | nanobeir_ne / NanoFiQA2018_cosine_precision@5 | 0.1240 | 0.0600 | 0.4839 |
312
+ | nanobeir_ne / NanoFiQA2018_cosine_precision@10 | 0.0800 | 0.0400 | 0.5000 |
313
+ | nanobeir_ne / NanoFiQA2018_cosine_recall@1 | 0.1287 | 0.0640 | 0.4971 |
314
+ | nanobeir_ne / NanoFiQA2018_cosine_recall@3 | 0.2288 | 0.1807 | 0.7897 |
315
+ | nanobeir_ne / NanoFiQA2018_cosine_recall@5 | 0.2893 | 0.1872 | 0.6470 |
316
+ | nanobeir_ne / NanoFiQA2018_cosine_recall@10 | 0.3780 | 0.2352 | 0.6221 |
317
+ | nanobeir_ne / NanoFiQA2018_cosine_ndcg@10 | 0.2912 | 0.1687 | 0.5793 |
318
+ | nanobeir_ne / NanoFiQA2018_cosine_mrr@10 | 0.3572 | 0.1806 | 0.5056 |
319
+ | nanobeir_ne / NanoFiQA2018_cosine_map@100 | 0.2275 | 0.1385 | 0.6088 |
320
+ | nanobeir_ne / NanoHotpotQA_cosine_accuracy@1 | 0.7800 | 0.6600 | 0.8462 |
321
+ | nanobeir_ne / NanoHotpotQA_cosine_accuracy@3 | 0.8400 | 0.8000 | 0.9524 |
322
+ | nanobeir_ne / NanoHotpotQA_cosine_accuracy@5 | 0.8600 | 0.8200 | 0.9535 |
323
+ | nanobeir_ne / NanoHotpotQA_cosine_accuracy@10 | 0.9000 | 0.8400 | 0.9333 |
324
+ | nanobeir_ne / NanoHotpotQA_cosine_precision@1 | 0.7800 | 0.6600 | 0.8462 |
325
+ | nanobeir_ne / NanoHotpotQA_cosine_precision@3 | 0.3800 | 0.3467 | 0.9123 |
326
+ | nanobeir_ne / NanoHotpotQA_cosine_precision@5 | 0.2520 | 0.2200 | 0.8730 |
327
+ | nanobeir_ne / NanoHotpotQA_cosine_precision@10 | 0.1380 | 0.1180 | 0.8551 |
328
+ | nanobeir_ne / NanoHotpotQA_cosine_recall@1 | 0.3900 | 0.3300 | 0.8462 |
329
+ | nanobeir_ne / NanoHotpotQA_cosine_recall@3 | 0.5700 | 0.5200 | 0.9123 |
330
+ | nanobeir_ne / NanoHotpotQA_cosine_recall@5 | 0.6300 | 0.5500 | 0.8730 |
331
+ | nanobeir_ne / NanoHotpotQA_cosine_recall@10 | 0.6900 | 0.5900 | 0.8551 |
332
+ | nanobeir_ne / NanoHotpotQA_cosine_ndcg@10 | 0.6636 | 0.5728 | 0.8631 |
333
+ | nanobeir_ne / NanoHotpotQA_cosine_mrr@10 | 0.8132 | 0.7269 | 0.8938 |
334
+ | nanobeir_ne / NanoHotpotQA_cosine_map@100 | 0.5941 | 0.5034 | 0.8473 |
335
+ | nanobeir_ne / NanoMSMARCO_cosine_accuracy@1 | 0.2600 | 0.1800 | 0.6923 |
336
+ | nanobeir_ne / NanoMSMARCO_cosine_accuracy@3 | 0.5800 | 0.4400 | 0.7586 |
337
+ | nanobeir_ne / NanoMSMARCO_cosine_accuracy@5 | 0.6600 | 0.5200 | 0.7879 |
338
+ | nanobeir_ne / NanoMSMARCO_cosine_accuracy@10 | 0.7400 | 0.6800 | 0.9189 |
339
+ | nanobeir_ne / NanoMSMARCO_cosine_precision@1 | 0.2600 | 0.1800 | 0.6923 |
340
+ | nanobeir_ne / NanoMSMARCO_cosine_precision@3 | 0.1933 | 0.1467 | 0.7586 |
341
+ | nanobeir_ne / NanoMSMARCO_cosine_precision@5 | 0.1320 | 0.1040 | 0.7879 |
342
+ | nanobeir_ne / NanoMSMARCO_cosine_precision@10 | 0.0740 | 0.0680 | 0.9189 |
343
+ | nanobeir_ne / NanoMSMARCO_cosine_recall@1 | 0.2600 | 0.1800 | 0.6923 |
344
+ | nanobeir_ne / NanoMSMARCO_cosine_recall@3 | 0.5800 | 0.4400 | 0.7586 |
345
+ | nanobeir_ne / NanoMSMARCO_cosine_recall@5 | 0.6600 | 0.5200 | 0.7879 |
346
+ | nanobeir_ne / NanoMSMARCO_cosine_recall@10 | 0.7400 | 0.6800 | 0.9189 |
347
+ | nanobeir_ne / NanoMSMARCO_cosine_ndcg@10 | 0.4955 | 0.4125 | 0.8325 |
348
+ | nanobeir_ne / NanoMSMARCO_cosine_mrr@10 | 0.4174 | 0.3298 | 0.7901 |
349
+ | nanobeir_ne / NanoMSMARCO_cosine_map@100 | 0.4252 | 0.3412 | 0.8024 |
350
+ | nanobeir_ne / NanoNFCorpus_cosine_accuracy@1 | 0.2800 | 0.1600 | 0.5714 |
351
+ | nanobeir_ne / NanoNFCorpus_cosine_accuracy@3 | 0.4400 | 0.3400 | 0.7727 |
352
+ | nanobeir_ne / NanoNFCorpus_cosine_accuracy@5 | 0.4400 | 0.4800 | 1.0909 |
353
+ | nanobeir_ne / NanoNFCorpus_cosine_accuracy@10 | 0.4400 | 0.5600 | 1.2727 |
354
+ | nanobeir_ne / NanoNFCorpus_cosine_precision@1 | 0.2800 | 0.1600 | 0.5714 |
355
+ | nanobeir_ne / NanoNFCorpus_cosine_precision@3 | 0.2600 | 0.1933 | 0.7436 |
356
+ | nanobeir_ne / NanoNFCorpus_cosine_precision@5 | 0.2120 | 0.2240 | 1.0566 |
357
+ | nanobeir_ne / NanoNFCorpus_cosine_precision@10 | 0.1600 | 0.1840 | 1.1500 |
358
+ | nanobeir_ne / NanoNFCorpus_cosine_recall@1 | 0.0084 | 0.0054 | 0.6468 |
359
+ | nanobeir_ne / NanoNFCorpus_cosine_recall@3 | 0.0413 | 0.0296 | 0.7165 |
360
+ | nanobeir_ne / NanoNFCorpus_cosine_recall@5 | 0.0486 | 0.0483 | 0.9939 |
361
+ | nanobeir_ne / NanoNFCorpus_cosine_recall@10 | 0.0617 | 0.0802 | 1.3003 |
362
+ | nanobeir_ne / NanoNFCorpus_cosine_ndcg@10 | 0.1975 | 0.1976 | 1.0005 |
363
+ | nanobeir_ne / NanoNFCorpus_cosine_mrr@10 | 0.3500 | 0.2882 | 0.8235 |
364
+ | nanobeir_ne / NanoNFCorpus_cosine_map@100 | 0.0701 | 0.0660 | 0.9418 |
365
+ | nanobeir_ne / NanoNQ_cosine_accuracy@1 | 0.2000 | 0.1600 | 0.8000 |
366
+ | nanobeir_ne / NanoNQ_cosine_accuracy@3 | 0.3400 | 0.3000 | 0.8824 |
367
+ | nanobeir_ne / NanoNQ_cosine_accuracy@5 | 0.3400 | 0.3200 | 0.9412 |
368
+ | nanobeir_ne / NanoNQ_cosine_accuracy@10 | 0.4400 | 0.4600 | 1.0455 |
369
+ | nanobeir_ne / NanoNQ_cosine_precision@1 | 0.2000 | 0.1600 | 0.8000 |
370
+ | nanobeir_ne / NanoNQ_cosine_precision@3 | 0.1133 | 0.1000 | 0.8824 |
371
+ | nanobeir_ne / NanoNQ_cosine_precision@5 | 0.0680 | 0.0640 | 0.9412 |
372
+ | nanobeir_ne / NanoNQ_cosine_precision@10 | 0.0440 | 0.0460 | 1.0455 |
373
+ | nanobeir_ne / NanoNQ_cosine_recall@1 | 0.1800 | 0.1500 | 0.8333 |
374
+ | nanobeir_ne / NanoNQ_cosine_recall@3 | 0.3100 | 0.2800 | 0.9032 |
375
+ | nanobeir_ne / NanoNQ_cosine_recall@5 | 0.3100 | 0.3000 | 0.9677 |
376
+ | nanobeir_ne / NanoNQ_cosine_recall@10 | 0.4100 | 0.4200 | 1.0244 |
377
+ | nanobeir_ne / NanoNQ_cosine_ndcg@10 | 0.2951 | 0.2817 | 0.9545 |
378
+ | nanobeir_ne / NanoNQ_cosine_mrr@10 | 0.2767 | 0.2495 | 0.9016 |
379
+ | nanobeir_ne / NanoNQ_cosine_map@100 | 0.2685 | 0.2439 | 0.9087 |
380
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_accuracy@1 | 0.8200 | 0.7400 | 0.9024 |
381
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_accuracy@3 | 0.9000 | 0.8200 | 0.9111 |
382
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_accuracy@5 | 0.9200 | 0.8600 | 0.9348 |
383
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_accuracy@10 | 0.9800 | 0.9000 | 0.9184 |
384
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_precision@1 | 0.8200 | 0.7400 | 0.9024 |
385
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_precision@3 | 0.3533 | 0.3000 | 0.8491 |
386
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_precision@5 | 0.2360 | 0.2040 | 0.8644 |
387
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_precision@10 | 0.1320 | 0.1140 | 0.8636 |
388
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_recall@1 | 0.7240 | 0.6773 | 0.9355 |
389
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_recall@3 | 0.8380 | 0.7713 | 0.9204 |
390
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_recall@5 | 0.8860 | 0.8260 | 0.9323 |
391
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_recall@10 | 0.9660 | 0.8727 | 0.9034 |
392
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_ndcg@10 | 0.8797 | 0.7971 | 0.9061 |
393
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_mrr@10 | 0.8707 | 0.7867 | 0.9035 |
394
+ | nanobeir_ne / NanoQuoraRetrieval_cosine_map@100 | 0.8452 | 0.7688 | 0.9096 |
395
+ | nanobeir_ne / NanoSCIDOCS_cosine_accuracy@1 | 0.2000 | 0.1600 | 0.8000 |
396
+ | nanobeir_ne / NanoSCIDOCS_cosine_accuracy@3 | 0.3600 | 0.2800 | 0.7778 |
397
+ | nanobeir_ne / NanoSCIDOCS_cosine_accuracy@5 | 0.4600 | 0.4200 | 0.9130 |
398
+ | nanobeir_ne / NanoSCIDOCS_cosine_accuracy@10 | 0.5800 | 0.5000 | 0.8621 |
399
+ | nanobeir_ne / NanoSCIDOCS_cosine_precision@1 | 0.2000 | 0.1600 | 0.8000 |
400
+ | nanobeir_ne / NanoSCIDOCS_cosine_precision@3 | 0.1733 | 0.1267 | 0.7308 |
401
+ | nanobeir_ne / NanoSCIDOCS_cosine_precision@5 | 0.1480 | 0.1200 | 0.8108 |
402
+ | nanobeir_ne / NanoSCIDOCS_cosine_precision@10 | 0.0980 | 0.0840 | 0.8571 |
403
+ | nanobeir_ne / NanoSCIDOCS_cosine_recall@1 | 0.0420 | 0.0330 | 0.7857 |
404
+ | nanobeir_ne / NanoSCIDOCS_cosine_recall@3 | 0.1080 | 0.0770 | 0.7130 |
405
+ | nanobeir_ne / NanoSCIDOCS_cosine_recall@5 | 0.1557 | 0.1240 | 0.7966 |
406
+ | nanobeir_ne / NanoSCIDOCS_cosine_recall@10 | 0.2047 | 0.1730 | 0.8453 |
407
+ | nanobeir_ne / NanoSCIDOCS_cosine_ndcg@10 | 0.1883 | 0.1565 | 0.8312 |
408
+ | nanobeir_ne / NanoSCIDOCS_cosine_mrr@10 | 0.3002 | 0.2527 | 0.8416 |
409
+ | nanobeir_ne / NanoSCIDOCS_cosine_map@100 | 0.1343 | 0.1044 | 0.7773 |
410
+ | nanobeir_ne / NanoArguAna_cosine_accuracy@1 | 0.1200 | 0.0800 | 0.6667 |
411
+ | nanobeir_ne / NanoArguAna_cosine_accuracy@3 | 0.5200 | 0.4000 | 0.7692 |
412
+ | nanobeir_ne / NanoArguAna_cosine_accuracy@5 | 0.5800 | 0.5200 | 0.8966 |
413
+ | nanobeir_ne / NanoArguAna_cosine_accuracy@10 | 0.7400 | 0.6200 | 0.8378 |
414
+ | nanobeir_ne / NanoArguAna_cosine_precision@1 | 0.1200 | 0.0800 | 0.6667 |
415
+ | nanobeir_ne / NanoArguAna_cosine_precision@3 | 0.1733 | 0.1333 | 0.7692 |
416
+ | nanobeir_ne / NanoArguAna_cosine_precision@5 | 0.1160 | 0.1040 | 0.8966 |
417
+ | nanobeir_ne / NanoArguAna_cosine_precision@10 | 0.0740 | 0.0620 | 0.8378 |
418
+ | nanobeir_ne / NanoArguAna_cosine_recall@1 | 0.1200 | 0.0800 | 0.6667 |
419
+ | nanobeir_ne / NanoArguAna_cosine_recall@3 | 0.5200 | 0.4000 | 0.7692 |
420
+ | nanobeir_ne / NanoArguAna_cosine_recall@5 | 0.5800 | 0.5200 | 0.8966 |
421
+ | nanobeir_ne / NanoArguAna_cosine_recall@10 | 0.7400 | 0.6200 | 0.8378 |
422
+ | nanobeir_ne / NanoArguAna_cosine_ndcg@10 | 0.4276 | 0.3476 | 0.8130 |
423
+ | nanobeir_ne / NanoArguAna_cosine_mrr@10 | 0.3276 | 0.2602 | 0.7945 |
424
+ | nanobeir_ne / NanoArguAna_cosine_map@100 | 0.3367 | 0.2650 | 0.7870 |
425
+ | nanobeir_ne / NanoSciFact_cosine_accuracy@1 | 0.3200 | 0.2400 | 0.7500 |
426
+ | nanobeir_ne / NanoSciFact_cosine_accuracy@3 | 0.4800 | 0.4600 | 0.9583 |
427
+ | nanobeir_ne / NanoSciFact_cosine_accuracy@5 | 0.5400 | 0.5200 | 0.9630 |
428
+ | nanobeir_ne / NanoSciFact_cosine_accuracy@10 | 0.6600 | 0.5800 | 0.8788 |
429
+ | nanobeir_ne / NanoSciFact_cosine_precision@1 | 0.3200 | 0.2400 | 0.7500 |
430
+ | nanobeir_ne / NanoSciFact_cosine_precision@3 | 0.1667 | 0.1600 | 0.9600 |
431
+ | nanobeir_ne / NanoSciFact_cosine_precision@5 | 0.1120 | 0.1080 | 0.9643 |
432
+ | nanobeir_ne / NanoSciFact_cosine_precision@10 | 0.0700 | 0.0620 | 0.8857 |
433
+ | nanobeir_ne / NanoSciFact_cosine_recall@1 | 0.3050 | 0.2250 | 0.7377 |
434
+ | nanobeir_ne / NanoSciFact_cosine_recall@3 | 0.4600 | 0.4400 | 0.9565 |
435
+ | nanobeir_ne / NanoSciFact_cosine_recall@5 | 0.5100 | 0.5000 | 0.9804 |
436
+ | nanobeir_ne / NanoSciFact_cosine_recall@10 | 0.6250 | 0.5650 | 0.9040 |
437
+ | nanobeir_ne / NanoSciFact_cosine_ndcg@10 | 0.4596 | 0.4007 | 0.8720 |
438
+ | nanobeir_ne / NanoSciFact_cosine_mrr@10 | 0.4155 | 0.3574 | 0.8600 |
439
+ | nanobeir_ne / NanoSciFact_cosine_map@100 | 0.4093 | 0.3533 | 0.8632 |
440
+ | nanobeir_ne / NanoTouche2020_cosine_accuracy@1 | 0.3469 | 0.1224 | 0.3529 |
441
+ | nanobeir_ne / NanoTouche2020_cosine_accuracy@3 | 0.5510 | 0.3469 | 0.6296 |
442
+ | nanobeir_ne / NanoTouche2020_cosine_accuracy@5 | 0.6735 | 0.5510 | 0.8182 |
443
+ | nanobeir_ne / NanoTouche2020_cosine_accuracy@10 | 0.8571 | 0.7143 | 0.8333 |
444
+ | nanobeir_ne / NanoTouche2020_cosine_precision@1 | 0.3469 | 0.1224 | 0.3529 |
445
+ | nanobeir_ne / NanoTouche2020_cosine_precision@3 | 0.3333 | 0.1633 | 0.4898 |
446
+ | nanobeir_ne / NanoTouche2020_cosine_precision@5 | 0.3224 | 0.2122 | 0.6582 |
447
+ | nanobeir_ne / NanoTouche2020_cosine_precision@10 | 0.2939 | 0.1857 | 0.6319 |
448
+ | nanobeir_ne / NanoTouche2020_cosine_recall@1 | 0.0216 | 0.0091 | 0.4228 |
449
+ | nanobeir_ne / NanoTouche2020_cosine_recall@3 | 0.0678 | 0.0325 | 0.4793 |
450
+ | nanobeir_ne / NanoTouche2020_cosine_recall@5 | 0.1083 | 0.0700 | 0.6466 |
451
+ | nanobeir_ne / NanoTouche2020_cosine_recall@10 | 0.1892 | 0.1182 | 0.6244 |
452
+ | nanobeir_ne / NanoTouche2020_cosine_ndcg@10 | 0.3143 | 0.1835 | 0.5838 |
453
+ | nanobeir_ne / NanoTouche2020_cosine_mrr@10 | 0.4881 | 0.2811 | 0.5759 |
454
+ | nanobeir_ne / NanoTouche2020_cosine_map@100 | 0.2267 | 0.1326 | 0.5847 |
455
+ | nanobeir_ne / NanoBEIR_mean_cosine_accuracy@1 | 0.3405 | 0.2479 | 0.7279 |
456
+ | nanobeir_ne / NanoBEIR_mean_cosine_accuracy@3 | 0.5439 | 0.4313 | 0.7929 |
457
+ | nanobeir_ne / NanoBEIR_mean_cosine_accuracy@5 | 0.6010 | 0.5224 | 0.8691 |
458
+ | nanobeir_ne / NanoBEIR_mean_cosine_accuracy@10 | 0.6952 | 0.6226 | 0.8957 |
459
+ | nanobeir_ne / NanoBEIR_mean_cosine_precision@1 | 0.3405 | 0.2479 | 0.7279 |
460
+ | nanobeir_ne / NanoBEIR_mean_cosine_precision@3 | 0.2318 | 0.1751 | 0.7555 |
461
+ | nanobeir_ne / NanoBEIR_mean_cosine_precision@5 | 0.1790 | 0.1474 | 0.8237 |
462
+ | nanobeir_ne / NanoBEIR_mean_cosine_precision@10 | 0.1263 | 0.1046 | 0.8281 |
463
+ | nanobeir_ne / NanoBEIR_mean_cosine_recall@1 | 0.2008 | 0.1571 | 0.7826 |
464
+ | nanobeir_ne / NanoBEIR_mean_cosine_recall@3 | 0.3514 | 0.2910 | 0.8281 |
465
+ | nanobeir_ne / NanoBEIR_mean_cosine_recall@5 | 0.4005 | 0.3432 | 0.8570 |
466
+ | nanobeir_ne / NanoBEIR_mean_cosine_recall@10 | 0.4847 | 0.4161 | 0.8585 |
467
+ | nanobeir_ne / NanoBEIR_mean_cosine_ndcg@10 | 0.4122 | 0.3388 | 0.8218 |
468
+ | nanobeir_ne / NanoBEIR_mean_cosine_mrr@10 | 0.4542 | 0.3612 | 0.7953 |
469
+ | nanobeir_ne / NanoBEIR_mean_cosine_map@100 | 0.3419 | 0.2769 | 0.8098 |
470
+
471
+ ## Citation
472
+
473
+ If you use this model or the pruning approach, please cite:
474
+
475
+ ```bibtex
476
+ @misc{subedi2025tokenpruning,
477
+ author = {Sanjaya Subedi},
478
+ title = {Token Embedding Pruning for Sentence Transformers},
479
+ year = {2026},
480
+ note = {Available at: [link to be added upon publication]}
481
+ }
482
+ ```
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "dtype": "float32",
10
+ "eos_token_id": null,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 384,
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 1536,
16
+ "is_decoder": false,
17
+ "layer_norm_eps": 1e-12,
18
+ "max_position_embeddings": 512,
19
+ "model_type": "bert",
20
+ "num_attention_heads": 12,
21
+ "num_hidden_layers": 12,
22
+ "pad_token_id": 0,
23
+ "position_embedding_type": "absolute",
24
+ "tie_word_embeddings": true,
25
+ "tokenizer_class": "XLMRobertaTokenizer",
26
+ "transformers_version": "5.12.1",
27
+ "type_vocab_size": 2,
28
+ "use_cache": true,
29
+ "vocab_size": 172569
30
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.12.1+cu130",
4
+ "sentence_transformers": "5.6.0",
5
+ "transformers": "5.12.1"
6
+ },
7
+ "default_prompt_name": null,
8
+ "model_type": "SentenceTransformer",
9
+ "prompts": {
10
+ "document": "",
11
+ "query": ""
12
+ },
13
+ "similarity_fn_name": "cosine"
14
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8de21ecee219f55eb9a29e3f0ba7a6167756b6109512d201895b3158e97ae3e
3
+ size 351646568
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.sentence_transformer.modules.normalize.Normalize"
19
+ }
20
+ ]
pruning_tokenizer.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import os
3
+ import shutil
4
+
5
+ import torch
6
+ from transformers import PreTrainedTokenizerFast
7
+
8
+
9
+ class PrunedTokenizer(PreTrainedTokenizerFast):
10
+ def set_id_map(self, old_to_new: dict[int, int]) -> None:
11
+ size = max(int(k) for k in old_to_new) + 1
12
+ id_list = [0] * size
13
+ for old_id, new_id in old_to_new.items():
14
+ id_list[int(old_id)] = int(new_id)
15
+ self.init_kwargs["pruned_id_map"] = id_list
16
+ self._map_tensor = torch.tensor(id_list, dtype=torch.long)
17
+
18
+ def _ensure_map(self) -> None:
19
+ if getattr(self, "_map_tensor", None) is not None:
20
+ return
21
+ id_list = self.init_kwargs.get("pruned_id_map")
22
+ self._map_tensor = torch.tensor(id_list, dtype=torch.long) if id_list else None
23
+
24
+ def _encode_plus(self, *args, **kwargs):
25
+ encoding = super()._encode_plus(*args, **kwargs)
26
+ self._ensure_map()
27
+ if self._map_tensor is not None and "input_ids" in encoding:
28
+ ids = encoding["input_ids"]
29
+ if isinstance(ids, torch.Tensor):
30
+ remapped = self._map_tensor.to(ids.device)[ids.long()].to(ids.dtype)
31
+ else: # list / nested list / numpy array
32
+ remapped = self._map_tensor[torch.as_tensor(ids, dtype=torch.long)].tolist()
33
+ encoding["input_ids"] = remapped
34
+ return encoding
35
+
36
+ def save_pretrained(
37
+ self,
38
+ save_directory,
39
+ legacy_format=None,
40
+ filename_prefix=None,
41
+ push_to_hub=False,
42
+ **kwargs,
43
+ ):
44
+ result = super().save_pretrained(
45
+ save_directory,
46
+ legacy_format=legacy_format,
47
+ filename_prefix=filename_prefix,
48
+ push_to_hub=push_to_hub,
49
+ **kwargs,
50
+ )
51
+
52
+ config_path = os.path.join(save_directory, "tokenizer_config.json")
53
+ with open(config_path) as f:
54
+ cfg = json.load(f)
55
+ cfg["auto_map"] = {"AutoTokenizer": [None, "pruning_tokenizer.PrunedTokenizer"]}
56
+ with open(config_path, "w") as f:
57
+ json.dump(cfg, f, indent=2)
58
+
59
+ # ship this file alongside the model so trust_remote_code=True finds the class
60
+ shutil.copy(__file__, os.path.join(save_directory, "pruning_tokenizer.py"))
61
+
62
+ return result
sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "feature-extraction",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "last_hidden_state"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6040ba36e3e2f7b2fa6ae076b69d024a08666bea4c345105a32e542900fcc7e7
3
+ size 17082735
tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff