Update README.md
Browse files
README.md
CHANGED
|
@@ -14,10 +14,12 @@ widget:
|
|
| 14 |
|
| 15 |
---
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
The project is still a work in progress, new versions will come with time.
|
| 20 |
|
|
|
|
|
|
|
| 21 |
# Training
|
| 22 |
|
| 23 |
- epochs: 250
|
|
@@ -27,4 +29,4 @@ The project is still a work in progress, new versions will come with time.
|
|
| 27 |
|
| 28 |
# Eval
|
| 29 |
|
| 30 |
-
- perplexity: 45 (it's a 12MB model
|
|
|
|
| 14 |
|
| 15 |
---
|
| 16 |
|
| 17 |
+
bert-tiny-ita is an italian foundational model (based on [bert-tiny](https://huggingface.co/google/bert_uncased_L-2_H-128_A-2)) pretrained from scratch on 20k italian Wikipedia articles and on a wide collection of italian words and dictionary definitions. It uses 512 context window size.
|
| 18 |
|
| 19 |
The project is still a work in progress, new versions will come with time.
|
| 20 |
|
| 21 |
+
Use it as a foundational model to be finetuned for specific italian tasks.
|
| 22 |
+
|
| 23 |
# Training
|
| 24 |
|
| 25 |
- epochs: 250
|
|
|
|
| 29 |
|
| 30 |
# Eval
|
| 31 |
|
| 32 |
+
- perplexity: 45 (it's a 12MB model!)
|