Vít Novotný
commited on
Commit
·
204c0b9
1
Parent(s):
6b90090
Update `README.md`
Browse files
README.md
CHANGED
|
@@ -8,10 +8,10 @@ datasets:
|
|
| 8 |
|
| 9 |
# MathBERTa base model
|
| 10 |
|
| 11 |
-
Pretrained model on English language using a masked language modeling
|
| 12 |
-
objective. It was developed for [the ARQMath-3 shared task evaluation][1]
|
| 13 |
-
CLEF 2022 and first released in [this repository][2]. This model is
|
| 14 |
-
it makes a difference between english and English.
|
| 15 |
|
| 16 |
[1]: https://www.cs.rit.edu/~dprl/ARQMath/
|
| 17 |
[2]: https://github.com/witiko/scm-at-arqmath3
|
|
@@ -26,8 +26,8 @@ Like RoBERTa, MathBERTa has been fine-tuned with the Masked language modeling
|
|
| 26 |
(MLM) objective. Taking a sentence, the model randomly masks 15% of the words
|
| 27 |
and math symbols in the input then run the entire masked sentence through the
|
| 28 |
model and has to predict the masked words and symbols. This way, the model
|
| 29 |
-
learns an inner representation of the English language and
|
| 30 |
-
|
| 31 |
|
| 32 |
[3]: https://huggingface.co/roberta-base
|
| 33 |
[7]: https://github.com/Witiko/scm-at-arqmath3/blob/main/02-train-tokenizers.ipynb
|
|
|
|
| 8 |
|
| 9 |
# MathBERTa base model
|
| 10 |
|
| 11 |
+
Pretrained model on English language and LaTeX using a masked language modeling
|
| 12 |
+
(MLM) objective. It was developed for [the ARQMath-3 shared task evaluation][1]
|
| 13 |
+
at CLEF 2022 and first released in [this repository][2]. This model is
|
| 14 |
+
case-sensitive: it makes a difference between english and English.
|
| 15 |
|
| 16 |
[1]: https://www.cs.rit.edu/~dprl/ARQMath/
|
| 17 |
[2]: https://github.com/witiko/scm-at-arqmath3
|
|
|
|
| 26 |
(MLM) objective. Taking a sentence, the model randomly masks 15% of the words
|
| 27 |
and math symbols in the input then run the entire masked sentence through the
|
| 28 |
model and has to predict the masked words and symbols. This way, the model
|
| 29 |
+
learns an inner representation of the English language and LaTeX that can then
|
| 30 |
+
be used to extract features useful for downstream tasks.
|
| 31 |
|
| 32 |
[3]: https://huggingface.co/roberta-base
|
| 33 |
[7]: https://github.com/Witiko/scm-at-arqmath3/blob/main/02-train-tokenizers.ipynb
|