File size: 994 Bytes
117b80c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | ---
language: en
license: apache-2.0
---
# H.G. BERT
This is a `22.7M` parameter [BERT](https://arxiv.org/abs/1810.04805) encoder-only model trained on [Historical English Books](https://huggingface.co/datasets/NeuML/historical-english-books) from 1700 - 1899.
This is a domain-specialized small model that often performs as good as models 10-100x larger. It demonstrates that narrowing down a model to a small domain requires less overall parameters than models generalized for all problems.
## Usage
`hgbert-small` can be loaded using [Hugging Face Transformers](https://huggingface.co/docs/transformers/en/index) as follows.
```python
from transformers import AutoModel
model = AutoModel.from_pretrained("neuml/hgbert-small")
```
The model is intended to be further fine-tuned for a specific task such as Text Classification, Entity Extraction, Sentence Embeddings and so on.
## More Information
Read more about the model in [this article](https://hf.co/blog/NeuML/hgbert-small).
|