metadata
library_name: transformers
language:
- mt
license: cc-by-nc-sa-4.0
base_model: MLRS/BERTu
datasets:
- MLRS/SentiMalti
model-index:
- name: BERTu_SentiMalti
results:
- task:
type: sentiment-analysis
name: Sentiment Analysis
dataset:
type: sentimalti
name: SentiMalti
metrics:
- type: f1
args: macro
value: 69.07
name: Macro-averaged F1
extra_gated_fields:
Name: text
Surname: text
Date of Birth: date_picker
Organisation: text
Country: country
I agree to use this model in accordance to the license and for non-commercial use ONLY: checkbox
BERTu (Maltese Sentiment Analysis)
This model is a fine-tuned version of MLRS/BERTu on SentiMalti. It achieves the following results on the test set:
- Loss: 1.9082
- F1: 0.6907
Intended uses & limitations
The model is fine-tuned on a specific task and it should be used on the same or similar task. Any limitations present in the base model are inherited.
Training procedure
The model was fine-tuned using a customised script.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 3
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: inverse_sqrt
- lr_scheduler_warmup_ratio: 0.005
- num_epochs: 200.0
- early_stopping_patience: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | F1 |
|---|---|---|---|---|
| 0.9723 | 1.0 | 165 | 0.7271 | 0.6184 |
| 0.6386 | 2.0 | 330 | 0.6306 | 0.6998 |
| 0.3595 | 3.0 | 495 | 0.7487 | 0.6893 |
| 0.172 | 4.0 | 660 | 1.0045 | 0.7018 |
| 0.0827 | 5.0 | 825 | 1.2351 | 0.6827 |
| 0.035 | 6.0 | 990 | 1.3912 | 0.7184 |
| 0.0195 | 7.0 | 1155 | 1.5390 | 0.7162 |
| 0.0078 | 8.0 | 1320 | 1.7268 | 0.7061 |
| 0.0038 | 9.0 | 1485 | 1.7315 | 0.7188 |
| 0.0027 | 10.0 | 1650 | 1.7508 | 0.7153 |
| 0.0036 | 11.0 | 1815 | 1.8214 | 0.7069 |
| 0.0004 | 12.0 | 1980 | 1.8759 | 0.7163 |
| 0.0004 | 13.0 | 2145 | 1.9154 | 0.7058 |
| 0.0004 | 14.0 | 2310 | 1.9490 | 0.7072 |
| 0.0002 | 15.0 | 2475 | 1.9955 | 0.7116 |
| 0.0002 | 16.0 | 2640 | 2.0237 | 0.7116 |
| 0.0003 | 17.0 | 2805 | 2.0499 | 0.6976 |
| 0.0002 | 18.0 | 2970 | 2.0582 | 0.7071 |
| 0.0013 | 19.0 | 3135 | 2.1837 | 0.6997 |
| 0.0003 | 20.0 | 3300 | 2.1333 | 0.7078 |
| 0.0002 | 21.0 | 3465 | 2.1226 | 0.7131 |
| 0.0021 | 22.0 | 3630 | 2.1613 | 0.7085 |
| 0.0001 | 23.0 | 3795 | 2.2053 | 0.7116 |
| 0.0001 | 24.0 | 3960 | 2.2182 | 0.7154 |
| 0.0013 | 25.0 | 4125 | 2.1939 | 0.7119 |
| 0.0001 | 26.0 | 4290 | 2.2181 | 0.7134 |
| 0.0001 | 27.0 | 4455 | 2.2612 | 0.7185 |
| 0.0018 | 28.0 | 4620 | 2.2793 | 0.7120 |
| 0.0001 | 29.0 | 4785 | 2.3041 | 0.7123 |
Framework versions
- Transformers 4.51.1
- Pytorch 2.8.0+cu128
- Datasets 3.2.0
- Tokenizers 0.21.4
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at https://mlrs.research.um.edu.mt/.
Citation
This work was first presented in SentiMalti: A Maltese Sentiment Analysis Dataset and Models. Cite it as follows:
@inproceedings{SentiMalti,
title = "{S}enti{M}alti: A {M}altese Sentiment Analysis Dataset and Models",
author = "Caruana, Ian and
Vella, Matthew and
Zammit, Fabio and
Micallef, Kurt and
Claudia Borg",
booktitle = {Proceedings of the Fifteenth Biennial Language Resources and Evaluation Conference (LREC)},
month = may,
year = "2026",
address = "Palma, Mallorca, Spain",
publisher = "European Language Resources Association",
}