Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ metrics:
|
|
| 11 |
|
| 12 |
<!-- Provide a quick summary of what the model is/does. -->
|
| 13 |
|
| 14 |
-
This medium-sized BERT model was created using the [Masked Latent Semantic Modeling] (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
|
| 15 |
During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
|
| 16 |
The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
|
| 17 |
|
|
|
|
| 11 |
|
| 12 |
<!-- Provide a quick summary of what the model is/does. -->
|
| 13 |
|
| 14 |
+
This medium-sized BERT model was created using the [Masked Latent Semantic Modeling](https://aclanthology.org/2023.findings-acl.876/) (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
|
| 15 |
During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
|
| 16 |
The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
|
| 17 |
|