ArabicNewSplits6_FineTuningAraBERT_run2_AugV5_k1_task1_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7138
• Qwk: 0.6868
• Mse: 0.7138
• Rmse: 0.8448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.25	2	5.5849	-0.0661	5.5849	2.3632
No log	0.5	4	3.2072	0.0686	3.2072	1.7909
No log	0.75	6	2.1896	0.1589	2.1896	1.4797
No log	1.0	8	1.3887	0.1353	1.3887	1.1784
No log	1.25	10	1.2000	0.2638	1.2000	1.0955
No log	1.5	12	1.1120	0.2089	1.1120	1.0545
No log	1.75	14	1.1249	0.3363	1.1249	1.0606
No log	2.0	16	1.1679	0.3703	1.1679	1.0807
No log	2.25	18	1.3504	0.2172	1.3504	1.1621
No log	2.5	20	1.0517	0.4626	1.0517	1.0255
No log	2.75	22	0.7976	0.5684	0.7976	0.8931
No log	3.0	24	0.7608	0.6109	0.7608	0.8723
No log	3.25	26	0.7082	0.6334	0.7082	0.8415
No log	3.5	28	0.7392	0.6711	0.7392	0.8598
No log	3.75	30	1.2294	0.4645	1.2294	1.1088
No log	4.0	32	1.7204	0.4051	1.7204	1.3117
No log	4.25	34	1.7432	0.4076	1.7432	1.3203
No log	4.5	36	1.2673	0.4555	1.2673	1.1257
No log	4.75	38	0.8273	0.6830	0.8273	0.9096
No log	5.0	40	0.6898	0.6648	0.6898	0.8305
No log	5.25	42	0.7209	0.6549	0.7209	0.8491
No log	5.5	44	0.7079	0.6541	0.7079	0.8414
No log	5.75	46	0.6902	0.6849	0.6902	0.8308
No log	6.0	48	0.7021	0.6804	0.7021	0.8379
No log	6.25	50	0.7313	0.6652	0.7313	0.8551
No log	6.5	52	0.7503	0.6696	0.7503	0.8662
No log	6.75	54	0.7305	0.6671	0.7305	0.8547
No log	7.0	56	0.7406	0.6752	0.7406	0.8606
No log	7.25	58	0.7466	0.6779	0.7466	0.8641
No log	7.5	60	0.7721	0.6903	0.7721	0.8787
No log	7.75	62	0.7737	0.6884	0.7737	0.8796
No log	8.0	64	0.7433	0.6791	0.7433	0.8622
No log	8.25	66	0.7302	0.6745	0.7302	0.8545
No log	8.5	68	0.7125	0.6750	0.7125	0.8441
No log	8.75	70	0.7063	0.6750	0.7063	0.8404
No log	9.0	72	0.7060	0.6691	0.7060	0.8402
No log	9.25	74	0.7059	0.6691	0.7059	0.8402
No log	9.5	76	0.7089	0.6868	0.7089	0.8419
No log	9.75	78	0.7126	0.6868	0.7126	0.8442
No log	10.0	80	0.7138	0.6868	0.7138	0.8448

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1