Instructions to use nsadeq/InformBERT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nsadeq/InformBERT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="nsadeq/InformBERT")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("nsadeq/InformBERT") model = AutoModelForMaskedLM.from_pretrained("nsadeq/InformBERT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
## Introduction
|
| 8 |
|
| 9 |
-
InformBERT is pretrained
|
| 10 |
|
| 11 |
More detail: https://arxiv.org/abs/2210.11771
|
| 12 |
|
|
|
|
| 6 |
|
| 7 |
## Introduction
|
| 8 |
|
| 9 |
+
InformBERT is pretrained using variable masking strategy, where informative tokens are masked more frequently compared to other tokens. InformBERT outperforms random masking based pretrained models on the factual recall benchmark LAMA and extractive question answering benchmark SQuAD.
|
| 10 |
|
| 11 |
More detail: https://arxiv.org/abs/2210.11771
|
| 12 |
|