|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- fa |
|
|
--- |
|
|
|
|
|
## Persian Phoneme-Level BERT (training) |
|
|
> This model is based on [Vaguye](https://github.com/SadeghKrmi/vaguye) phonemizer for Persian! |
|
|
|
|
|
GPU RTX A6000 (1x) is used for training, dataset of 1.3 milion sentences, chunked, normalized from wikipedia dataset for Farsi. |
|
|
```bash |
|
|
ds = load_dataset("wikimedia/wikipedia", "20231101.fa", split="train", streaming=True) |
|
|
``` |
|
|
|
|
|
Training run for 305K steps with the following result (around the following values) |
|
|
```bash |
|
|
Step [305000/2000000], Loss: 1.38343, Vocab Loss: 0.34593, Token Loss: 1.25434 |
|
|
``` |
|
|
|
|
|
dataset and model is stored in HF: `SadeghK/FaPLBERT` |
|
|
|
|
|
|
|
|
For dataset preparation from wikipedia, refer to `wikipedia-dataset-styletts2-preparation` |
|
|
```bash |
|
|
wikipedia entry -> [normalize] -> [chunk] -> [hamnevise] -> [Zirneshane] |
|
|
``` |
|
|
|
|
|
For further details of preprocessing, training refer to `preprocess_fa.ipynb` and `train_fa.ipynb` of [SadeghKrmi/PL-BERT](https://github.com/SadeghKrmi/FaPLBERT.git) |
