Update README.md
Browse files
README.md
CHANGED
|
@@ -11,12 +11,26 @@ GitHub: https://github.com/RAIVNLab/AdANNS
|
|
| 11 |
|
| 12 |
Arxiv: https://arxiv.org/abs/2305.19435
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
We provide four BERT-Base models finetuned on Natural Questions with [Matryoshka Representation Learning](https://github.com/RAIVNLab/MRL) (MRL).
|
| 15 |
|
| 16 |
A vanilla pretrained BERT-Base has a 768-d representation (information bottleneck). As we train with MRL, we enforce the network to learn representations at
|
| 17 |
multiple granularities nested within a 768-d embedding. The granularities at which we finetune BERT-Base with Matroyshka Loss are specified in the folder name,
|
| 18 |
e.g. for `dpr-nq-d768_384_192_96_48`, we have d=[48, 96, 192, 384, 768].
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
## Citation
|
| 21 |
If you find this project useful in your research, please consider citing:
|
| 22 |
```
|
|
@@ -28,4 +42,4 @@ If you find this project useful in your research, please consider citing:
|
|
| 28 |
month = {December},
|
| 29 |
year = {2023},
|
| 30 |
}
|
| 31 |
-
```
|
|
|
|
| 11 |
|
| 12 |
Arxiv: https://arxiv.org/abs/2305.19435
|
| 13 |
|
| 14 |
+
<p align="center">
|
| 15 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/64af72d4a609b29cc7b5919b/QYOqZ25qc9pTWlFR0D0VF.png" alt="drawing" width="600"/>
|
| 16 |
+
Adaptive representations can be utilized effectively in the decoupled components of clustering and
|
| 17 |
+
searching for a better accuracy-compute trade-off (AdANNS-IVF).
|
| 18 |
+
</p>
|
| 19 |
+
|
| 20 |
We provide four BERT-Base models finetuned on Natural Questions with [Matryoshka Representation Learning](https://github.com/RAIVNLab/MRL) (MRL).
|
| 21 |
|
| 22 |
A vanilla pretrained BERT-Base has a 768-d representation (information bottleneck). As we train with MRL, we enforce the network to learn representations at
|
| 23 |
multiple granularities nested within a 768-d embedding. The granularities at which we finetune BERT-Base with Matroyshka Loss are specified in the folder name,
|
| 24 |
e.g. for `dpr-nq-d768_384_192_96_48`, we have d=[48, 96, 192, 384, 768].
|
| 25 |
|
| 26 |
+
You can easily load an mrl-nq model as follows:
|
| 27 |
+
```
|
| 28 |
+
from transformers import BertModel
|
| 29 |
+
import torch
|
| 30 |
+
|
| 31 |
+
model = BertModel.from_pretrained('dpr-nq-d768_384_192_96_48')
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
## Citation
|
| 35 |
If you find this project useful in your research, please consider citing:
|
| 36 |
```
|
|
|
|
| 42 |
month = {December},
|
| 43 |
year = {2023},
|
| 44 |
}
|
| 45 |
+
```
|