Instructions to use nasa-impact/bert-e-base-mlm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nasa-impact/bert-e-base-mlm with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="nasa-impact/bert-e-base-mlm")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("nasa-impact/bert-e-base-mlm") model = AutoModelForMaskedLM.from_pretrained("nasa-impact/bert-e-base-mlm") - Notebooks
- Google Colab
- Kaggle
update number of papers in readme file
Browse files
README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
This model is further trained on top of scibert-base using masked language modeling loss (MLM). The corpus is roughly
|
| 2 |
|
| 3 |
The tokenizer used is AutoTokenizer, which is trained on the same corpus.
|
| 4 |
|
|
@@ -7,4 +7,4 @@ Stay tuned for further downstream task tests and updates to the model.
|
|
| 7 |
in the works
|
| 8 |
- MLM + NSP task loss
|
| 9 |
- Add more data sources for training
|
| 10 |
-
- Test using downstream tasks
|
|
|
|
| 1 |
+
This model is further trained on top of scibert-base using masked language modeling loss (MLM). The corpus is roughly 270,000 earth science-based publications.
|
| 2 |
|
| 3 |
The tokenizer used is AutoTokenizer, which is trained on the same corpus.
|
| 4 |
|
|
|
|
| 7 |
in the works
|
| 8 |
- MLM + NSP task loss
|
| 9 |
- Add more data sources for training
|
| 10 |
+
- Test using downstream tasks
|