Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -24,12 +24,15 @@ All the HPLT GPT-BERT models use the same hyper-parameters:
|
|
| 24 |
- hidden size: 640
|
| 25 |
- attention heads: 10
|
| 26 |
- layers: 24
|
| 27 |
-
- vocabulary size:
|
| 28 |
|
| 29 |
Every model uses its own tokenizer trained on language-specific HPLT data.
|
| 30 |
|
| 31 |
[The training code](https://github.com/ltgoslo/NorBERT/tree/main/norbert4).
|
| 32 |
|
|
|
|
|
|
|
|
|
|
| 33 |
## Example usage (bidirectional encoding)
|
| 34 |
|
| 35 |
This model currently needs a custom wrapper from `modeling_gptbert.py`, you should therefore load the model with `trust_remote_code=True`.
|
|
@@ -157,4 +160,8 @@ print([b.name for b in out.branches])
|
|
| 157 |
url={https://arxiv.org/abs/2511.01066},
|
| 158 |
}
|
| 159 |
```
|
| 160 |
-
[.
|
| 32 |
|
| 33 |
+
```
|
| 34 |
+
pip install transformers==4.57.6
|
| 35 |
+
```
|
| 36 |
## Example usage (bidirectional encoding)
|
| 37 |
|
| 38 |
This model currently needs a custom wrapper from `modeling_gptbert.py`, you should therefore load the model with `trust_remote_code=True`.
|
|
|
|
| 160 |
url={https://arxiv.org/abs/2511.01066},
|
| 161 |
}
|
| 162 |
```
|
| 163 |
+
[](https://arxiv.org/abs/2410.24159)
|
| 164 |
+
|
| 165 |
+
[](https://arxiv.org/abs/2511.01066)
|
| 166 |
+
|
| 167 |
+
This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070350 and from UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee [grant number 10052546].
|