MASRIBERTV4

This model is a fine-tuned version of UBC-NLP/MARBERTv2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4905

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10000
• num_epochs: 2
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
49.3326	0.0427	500	5.4990
42.0818	0.0854	1000	4.8453
38.9605	0.1281	1500	4.5192
36.9452	0.1709	2000	4.3034
35.4721	0.2136	2500	4.1469
17.1800	0.2563	3000	2.0153
16.6866	0.2990	3500	1.9642
16.3991	0.3417	4000	1.9193
16.0343	0.3844	4500	1.8890
15.7752	0.4271	5000	1.8541
15.5412	0.4699	5500	1.8296
15.3676	0.5126	6000	1.8037
15.1004	0.5553	6500	1.7819
14.9579	0.5980	7000	1.7591
14.8206	0.6407	7500	1.7475
14.5731	0.6834	8000	1.7312
14.4853	0.7262	8500	1.7171
14.4356	0.7689	9000	1.7090
14.2340	0.8116	9500	1.6929
14.2497	0.8543	10000	1.6809
14.1660	0.8970	10500	1.6745
14.0404	0.9397	11000	1.6584
13.8540	0.9824	11500	1.6474
13.7538	1.0251	12000	1.6362
13.6777	1.0678	12500	1.6223
13.5928	1.1105	13000	1.6111
13.4528	1.1533	13500	1.6016
13.3583	1.1960	14000	1.5926
13.3129	1.2387	14500	1.5797
13.2261	1.2814	15000	1.5714
13.2270	1.3241	15500	1.5613
13.0836	1.3668	16000	1.5562
13.0298	1.4096	16500	1.5463
12.9826	1.4523	17000	1.5360
12.9178	1.4950	17500	1.5267
12.8210	1.5377	18000	1.5218
12.7591	1.5804	18500	1.5172
12.7104	1.6231	19000	1.5090
12.6183	1.6658	19500	1.5056
12.5794	1.7086	20000	1.5013
12.5963	1.7513	20500	1.4976
12.5140	1.7940	21000	1.4922
12.4895	1.8367	21500	1.4932
12.4931	1.8794	22000	1.4914
12.5685	1.9221	22500	1.4908
12.4823	1.9648	23000	1.4929

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2