Update README.md
Browse files
README.md
CHANGED
|
@@ -116,7 +116,18 @@ The training corpus consists of several corpora gathered from web crawling and p
|
|
| 116 |
### Training procedure
|
| 117 |
|
| 118 |
The training corpus has been tokenized using a byte version of [Byte-Pair Encoding (BPE)](https://github.com/openai/gpt-2)
|
| 119 |
-
used in the original [RoBERTA](https://github.com/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
The RoBERTa-ca-v2 pretraining consists of a masked language model training that follows the approach employed for the RoBERTa base model
|
| 121 |
with the same hyperparameters as in the original work.
|
| 122 |
The training lasted a total of 96 hours with 16 NVIDIA V100 GPUs of 16GB DDRAM.
|
|
@@ -233,9 +244,6 @@ If you use any of these resources (datasets or models) in your work, please cite
|
|
| 233 |
|
| 234 |
### Disclaimer
|
| 235 |
|
| 236 |
-
<details>
|
| 237 |
-
<summary>Click to expand</summary>
|
| 238 |
-
|
| 239 |
The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
|
| 240 |
|
| 241 |
When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
|
|
|
|
| 116 |
### Training procedure
|
| 117 |
|
| 118 |
The training corpus has been tokenized using a byte version of [Byte-Pair Encoding (BPE)](https://github.com/openai/gpt-2)
|
| 119 |
+
used in the original [RoBERTA](https://github.com/p
|
| 120 |
+
|
| 121 |
+
### Author
|
| 122 |
+
Text Mining Unit (TeMU) at the Barcelona Supercomputing Center (bsc-temu@bsc.es)
|
| 123 |
+
|
| 124 |
+
### Contact information
|
| 125 |
+
For further information, send an email to <plantl-gob-es@bsc.es>
|
| 126 |
+
|
| 127 |
+
### Copyright
|
| 128 |
+
Copyright by the Spanish State Secretariat for Digitalization and Artificial Intelligence (SEDIA) (2022)
|
| 129 |
+
|
| 130 |
+
### Licensing informationytorch/fairseq/tree/master/examples/roberta) model with a vocabulary size of 50,262 tokens.
|
| 131 |
The RoBERTa-ca-v2 pretraining consists of a masked language model training that follows the approach employed for the RoBERTa base model
|
| 132 |
with the same hyperparameters as in the original work.
|
| 133 |
The training lasted a total of 96 hours with 16 NVIDIA V100 GPUs of 16GB DDRAM.
|
|
|
|
| 244 |
|
| 245 |
### Disclaimer
|
| 246 |
|
|
|
|
|
|
|
|
|
|
| 247 |
The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
|
| 248 |
|
| 249 |
When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
|