Added usage instructions
#2
by
alekkeersmaekers
- opened
README.md
CHANGED
|
@@ -8,7 +8,14 @@ tags:
|
|
| 8 |
|
| 9 |
|
| 10 |
|
| 11 |
-
An ELECTRA-small model for Ancient Greek, trained on texts from Homer up until the 4th century AD.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
## Citation
|
| 14 |
|
|
|
|
| 8 |
|
| 9 |
|
| 10 |
|
| 11 |
+
An ELECTRA-small model for Ancient Greek, trained on texts from Homer up until the 4th century AD from the literary [GLAUx](https://github.com/alekkeersmaekers/glaux) corpus and the [DukeNLP](https://github.com/alekkeersmaekers/duke-nlp) papyrus corpus.
|
| 12 |
+
|
| 13 |
+
The model has some design choices made to combat data sparsity:
|
| 14 |
+
* Its input should always be in Unicode NFD (so separate Unicode signs for diacritics).
|
| 15 |
+
* All grave accents should be replaced with acute accents (καί, not καὶ).
|
| 16 |
+
* When a word contains two accents, the second one should be removed (εἶπε μοι, not εἶπέ μοι).
|
| 17 |
+
|
| 18 |
+
If you use it in conjunction with [glaux-nlp](https://github.com/alekkeersmaekers/glaux-nlp), you can pass the tokenized sentence to normalize_tokens from tokenization.Tokenization, using normalization_rule=greek_glaux, which will do all these normalizations for you.
|
| 19 |
|
| 20 |
## Citation
|
| 21 |
|