Commit
·
9f87d7d
1
Parent(s):
cda4b61
Update README.md
Browse files
README.md
CHANGED
|
@@ -46,6 +46,8 @@ There are 2 versions of the pre-trained model,
|
|
| 46 |
* v1 - Random Masking: We randomly choose tokens to mask in the MLM task;
|
| 47 |
* v2 - Selective Masking: As we want the model to learn more about the financial context, we selectively choose tokens to mask in the MLM task. We rely on the Loughran-McDonald dictionary to choose the important tokens to masked.
|
| 48 |
|
|
|
|
|
|
|
| 49 |
# Instruction to load the pre-trained model
|
| 50 |
* Clone the git repo
|
| 51 |
```
|
|
@@ -69,7 +71,7 @@ backbone = 'minhtriphan/LongFinBERT-base'
|
|
| 69 |
|
| 70 |
tokenizer = LongBERTTokenizer.from_pretrained(backbone)
|
| 71 |
config = LongBERTConfig.from_pretrained(backbone)
|
| 72 |
-
model = LongBERTModel.from_pretrained(backbone)
|
| 73 |
```
|
| 74 |
|
| 75 |
# Model usage
|
|
|
|
| 46 |
* v1 - Random Masking: We randomly choose tokens to mask in the MLM task;
|
| 47 |
* v2 - Selective Masking: As we want the model to learn more about the financial context, we selectively choose tokens to mask in the MLM task. We rely on the Loughran-McDonald dictionary to choose the important tokens to masked.
|
| 48 |
|
| 49 |
+
The argument `version` in the method `load_pretrained` of the `LongBERTModel` allows one to choose which version is loaded.
|
| 50 |
+
|
| 51 |
# Instruction to load the pre-trained model
|
| 52 |
* Clone the git repo
|
| 53 |
```
|
|
|
|
| 71 |
|
| 72 |
tokenizer = LongBERTTokenizer.from_pretrained(backbone)
|
| 73 |
config = LongBERTConfig.from_pretrained(backbone)
|
| 74 |
+
model = LongBERTModel.from_pretrained(backbone, version = 'v1')
|
| 75 |
```
|
| 76 |
|
| 77 |
# Model usage
|