MariaFjodorowa commited on
Commit
89912a9
·
verified ·
1 Parent(s): 03808c7

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -24,12 +24,15 @@ All the HPLT GPT-BERT models use the same hyper-parameters:
24
  - hidden size: 640
25
  - attention heads: 10
26
  - layers: 24
27
- - vocabulary size: 51200
28
 
29
  Every model uses its own tokenizer trained on language-specific HPLT data.
30
 
31
  [The training code](https://github.com/ltgoslo/NorBERT/tree/main/norbert4).
32
 
 
 
 
33
  ## Example usage (bidirectional encoding)
34
 
35
  This model currently needs a custom wrapper from `modeling_gptbert.py`, you should therefore load the model with `trust_remote_code=True`.
@@ -157,4 +160,8 @@ print([b.name for b in out.branches])
157
  url={https://arxiv.org/abs/2511.01066},
158
  }
159
  ```
160
- [![arXiv](https://img.shields.io/badge/arXiv-2406.14167-b31b1b.svg)](https://arxiv.org/abs/2410.24159)
 
 
 
 
 
24
  - hidden size: 640
25
  - attention heads: 10
26
  - layers: 24
27
+ - vocabulary size: 32768
28
 
29
  Every model uses its own tokenizer trained on language-specific HPLT data.
30
 
31
  [The training code](https://github.com/ltgoslo/NorBERT/tree/main/norbert4).
32
 
33
+ ```
34
+ pip install transformers==4.57.6
35
+ ```
36
  ## Example usage (bidirectional encoding)
37
 
38
  This model currently needs a custom wrapper from `modeling_gptbert.py`, you should therefore load the model with `trust_remote_code=True`.
 
160
  url={https://arxiv.org/abs/2511.01066},
161
  }
162
  ```
163
+ [![arXiv](https://img.shields.io/badge/arXiv-2410.24159-b31b1b.svg)](https://arxiv.org/abs/2410.24159)
164
+
165
+ [![arXiv](https://img.shields.io/badge/arXiv-2511.01066-b31b1b.svg)](https://arxiv.org/abs/2511.01066)
166
+
167
+ This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070350 and from UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee [grant number 10052546].