Fill-Mask
Transformers
Safetensors
English
modernbert
biomedical
clinical
encoder
Eval Results (legacy)
Instructions to use almanach/ModernBERT-bio-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use almanach/ModernBERT-bio-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="almanach/ModernBERT-bio-base")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("almanach/ModernBERT-bio-base") model = AutoModelForMaskedLM.from_pretrained("almanach/ModernBERT-bio-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update citation with arXiv ID 2605.12438
Browse files
README.md
CHANGED
|
@@ -264,11 +264,14 @@ Apache 2.0
|
|
| 264 |
## Citation
|
| 265 |
|
| 266 |
```bibtex
|
| 267 |
-
@
|
| 268 |
-
|
| 269 |
-
|
| 270 |
-
|
| 271 |
-
|
|
|
|
|
|
|
|
|
|
| 272 |
}
|
| 273 |
```
|
| 274 |
|
|
|
|
| 264 |
## Citation
|
| 265 |
|
| 266 |
```bibtex
|
| 267 |
+
@misc{touchent2026causallanguagemodelingdetour,
|
| 268 |
+
title={A Causal Language Modeling Detour Improves Encoder Continued Pretraining},
|
| 269 |
+
author={Rian Touchent and Eric de la Clergerie},
|
| 270 |
+
year={2026},
|
| 271 |
+
eprint={2605.12438},
|
| 272 |
+
archivePrefix={arXiv},
|
| 273 |
+
primaryClass={cs.CL},
|
| 274 |
+
url={https://arxiv.org/abs/2605.12438},
|
| 275 |
}
|
| 276 |
```
|
| 277 |
|