Language
Collection
Dhivehi Natural Language Processing: Text analysis, translation, sentiment analysis, and language generation tools for Thaana • 27 items • Updated
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("alakxender/bert-base-dv")
model = AutoModelForMaskedLM.from_pretrained("alakxender/bert-base-dv")Quick Links
BERT base for Dhivehi
Pretrained model on Dhivehi language using masked language modeling (MLM).
Tokenizer
The WordPiece tokenizer uses several components:
- Normalization: lowercase and then NFKD unicode normalization.
- Pretokenization: splits by whitespace and punctuation.
- Postprocessing: single sentences are output in format
[CLS] sentence A [SEP]and pair sentences in format[CLS] sentence A [SEP] sentence B [SEP].
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
- Downloads last month
- 5
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="alakxender/bert-base-dv")