ania3000/ossbert-lemm-v2.1

trainer_output

This model is a fine-tuned version of AlexeySorokin/ossbert-onc-unlab-from_multilingual-bs64-5epochs on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1199
• Lemma accuracy: 98.6715
• Sentence accuracy (lemmas): 84.0367

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Lemma accuracy	Sentence accuracy (lemmas)
0.7308	1.0	546	0.2709	95.4255	58.3486
0.2327	2.0	1092	0.1774	97.0422	69.5413
0.1375	3.0	1638	0.1404	97.6939	75.0459
0.0967	4.0	2184	0.1269	97.9070	77.2477
0.0716	5.0	2730	0.1131	98.3206	80.9174
0.0492	6.0	3276	0.1140	98.2454	80.0
0.0427	7.0	3822	0.1168	98.0574	78.8991
0.0311	8.0	4368	0.1273	98.1827	79.2661
0.0238	9.0	4914	0.1085	98.5462	82.7523
0.0184	10.0	5460	0.1047	98.5963	83.1193
0.0104	11.0	6006	0.1105	98.5211	82.9358
0.0079	12.0	6552	0.1083	98.6214	84.2202
0.0055	13.0	7098	0.1133	98.5838	83.3028
0.0045	14.0	7644	0.1134	98.5337	82.9358
0.0035	15.0	8190	0.1148	98.5587	83.4862
0.0037	16.0	8736	0.1126	98.6339	84.2202
0.003	17.0	9282	0.1112	98.6464	83.6697
0.0022	18.0	9828	0.1174	98.6966	84.4037
0.0013	19.0	10374	0.1193	98.6590	84.0367
0.0012	20.0	10920	0.1186	98.6715	84.0367
0.0011	21.0	11466	0.1220	98.6590	83.8532
0.0008	22.0	12012	0.1199	98.6715	84.0367

Framework versions

• Transformers 4.57.3
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2