ArabicNewSplits7_FineTuningAraBERT_run2_AugV5_k2_task5_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5357
• Qwk: 0.6447
• Mse: 0.5357
• Rmse: 0.7319

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.25	2	4.1135	0.0130	4.1135	2.0282
No log	0.5	4	2.2846	0.1047	2.2846	1.5115
No log	0.75	6	1.3250	-0.0030	1.3250	1.1511
No log	1.0	8	1.0522	0.1837	1.0522	1.0258
No log	1.25	10	1.0442	0.1360	1.0442	1.0219
No log	1.5	12	1.0736	0.1545	1.0736	1.0362
No log	1.75	14	1.1683	0.1454	1.1683	1.0809
No log	2.0	16	1.3520	-0.0320	1.3520	1.1628
No log	2.25	18	1.4032	0.0	1.4032	1.1846
No log	2.5	20	1.0958	0.2125	1.0958	1.0468
No log	2.75	22	0.8511	0.3476	0.8511	0.9226
No log	3.0	24	0.8826	0.3432	0.8826	0.9395
No log	3.25	26	0.8140	0.3922	0.8140	0.9022
No log	3.5	28	0.8160	0.3353	0.8160	0.9033
No log	3.75	30	0.7580	0.4409	0.7580	0.8706
No log	4.0	32	0.7589	0.4980	0.7589	0.8712
No log	4.25	34	0.7522	0.4613	0.7522	0.8673
No log	4.5	36	0.7229	0.5879	0.7229	0.8502
No log	4.75	38	0.7666	0.5674	0.7666	0.8755
No log	5.0	40	0.6977	0.5763	0.6977	0.8353
No log	5.25	42	0.6845	0.6187	0.6845	0.8273
No log	5.5	44	0.7104	0.6144	0.7104	0.8428
No log	5.75	46	0.7129	0.6052	0.7129	0.8443
No log	6.0	48	0.6838	0.5816	0.6838	0.8269
No log	6.25	50	0.6372	0.6186	0.6372	0.7982
No log	6.5	52	0.6234	0.6310	0.6234	0.7895
No log	6.75	54	0.6458	0.6215	0.6458	0.8036
No log	7.0	56	0.6598	0.6206	0.6598	0.8123
No log	7.25	58	0.6649	0.6365	0.6649	0.8154
No log	7.5	60	0.6388	0.6011	0.6388	0.7993
No log	7.75	62	0.6563	0.6564	0.6563	0.8101
No log	8.0	64	0.6015	0.6555	0.6015	0.7755
No log	8.25	66	0.6031	0.6593	0.6031	0.7766
No log	8.5	68	0.7642	0.5833	0.7642	0.8742
No log	8.75	70	0.8268	0.6312	0.8268	0.9093
No log	9.0	72	0.7127	0.6193	0.7127	0.8442
No log	9.25	74	0.6450	0.6272	0.6450	0.8031
No log	9.5	76	0.6335	0.6598	0.6335	0.7959
No log	9.75	78	0.6357	0.6570	0.6357	0.7973
No log	10.0	80	0.7572	0.6304	0.7572	0.8702
No log	10.25	82	0.9212	0.5504	0.9212	0.9598
No log	10.5	84	0.7210	0.6831	0.7210	0.8491
No log	10.75	86	0.6260	0.6988	0.6260	0.7912
No log	11.0	88	0.6695	0.6259	0.6695	0.8182
No log	11.25	90	0.6167	0.6681	0.6167	0.7853
No log	11.5	92	0.7750	0.6312	0.7750	0.8803
No log	11.75	94	0.8945	0.4995	0.8945	0.9458
No log	12.0	96	0.7172	0.6082	0.7172	0.8469
No log	12.25	98	0.6394	0.6424	0.6394	0.7996
No log	12.5	100	0.6456	0.6066	0.6456	0.8035
No log	12.75	102	0.6289	0.6649	0.6289	0.7930
No log	13.0	104	0.6888	0.6062	0.6888	0.8299
No log	13.25	106	0.6350	0.6426	0.6350	0.7969
No log	13.5	108	0.6074	0.6649	0.6074	0.7794
No log	13.75	110	0.6064	0.6349	0.6064	0.7787
No log	14.0	112	0.6658	0.6184	0.6658	0.8159
No log	14.25	114	0.6647	0.5653	0.6647	0.8153
No log	14.5	116	0.5837	0.7089	0.5837	0.7640
No log	14.75	118	0.5879	0.7239	0.5879	0.7667
No log	15.0	120	0.6079	0.6677	0.6079	0.7797
No log	15.25	122	0.5616	0.7303	0.5616	0.7494
No log	15.5	124	0.5601	0.6805	0.5601	0.7484
No log	15.75	126	0.5788	0.6460	0.5788	0.7608
No log	16.0	128	0.6200	0.6460	0.6200	0.7874
No log	16.25	130	0.6486	0.5635	0.6486	0.8054
No log	16.5	132	0.6612	0.5794	0.6612	0.8131
No log	16.75	134	0.6112	0.6460	0.6112	0.7818
No log	17.0	136	0.5857	0.6788	0.5857	0.7653
No log	17.25	138	0.5840	0.6924	0.5840	0.7642
No log	17.5	140	0.5983	0.6491	0.5983	0.7735
No log	17.75	142	0.6557	0.6811	0.6557	0.8098
No log	18.0	144	0.6277	0.6254	0.6277	0.7923
No log	18.25	146	0.6655	0.6012	0.6655	0.8158
No log	18.5	148	0.6445	0.6177	0.6445	0.8028
No log	18.75	150	0.6370	0.5714	0.6370	0.7981
No log	19.0	152	0.6244	0.5274	0.6244	0.7902
No log	19.25	154	0.6406	0.6138	0.6406	0.8004
No log	19.5	156	0.6242	0.6256	0.6242	0.7901
No log	19.75	158	0.6336	0.6444	0.6336	0.7960
No log	20.0	160	0.7066	0.6745	0.7066	0.8406
No log	20.25	162	0.6679	0.6570	0.6679	0.8173
No log	20.5	164	0.5981	0.6452	0.5981	0.7733
No log	20.75	166	0.6486	0.6282	0.6486	0.8053
No log	21.0	168	0.6551	0.6464	0.6551	0.8094
No log	21.25	170	0.6058	0.6333	0.6058	0.7784
No log	21.5	172	0.6247	0.6526	0.6247	0.7904
No log	21.75	174	0.6674	0.6154	0.6674	0.8170
No log	22.0	176	0.6874	0.5826	0.6874	0.8291
No log	22.25	178	0.6613	0.6032	0.6613	0.8132
No log	22.5	180	0.6207	0.6509	0.6207	0.7878
No log	22.75	182	0.6530	0.6218	0.6530	0.8081
No log	23.0	184	0.6115	0.6177	0.6115	0.7820
No log	23.25	186	0.6006	0.6426	0.6006	0.7750
No log	23.5	188	0.6534	0.5873	0.6534	0.8083
No log	23.75	190	0.6374	0.6194	0.6374	0.7984
No log	24.0	192	0.5724	0.6636	0.5724	0.7566
No log	24.25	194	0.5672	0.6509	0.5672	0.7531
No log	24.5	196	0.6089	0.6664	0.6089	0.7803
No log	24.75	198	0.6031	0.6536	0.6031	0.7766
No log	25.0	200	0.6090	0.6452	0.6090	0.7804
No log	25.25	202	0.6431	0.5966	0.6431	0.8019
No log	25.5	204	0.6303	0.6269	0.6303	0.7939
No log	25.75	206	0.6107	0.6546	0.6107	0.7814
No log	26.0	208	0.6726	0.6199	0.6726	0.8201
No log	26.25	210	0.7147	0.6101	0.7147	0.8454
No log	26.5	212	0.6684	0.6255	0.6684	0.8176
No log	26.75	214	0.6083	0.6164	0.6083	0.7799
No log	27.0	216	0.6256	0.6174	0.6256	0.7910
No log	27.25	218	0.6103	0.6426	0.6103	0.7812
No log	27.5	220	0.5870	0.6729	0.5870	0.7661
No log	27.75	222	0.5820	0.6788	0.5820	0.7629
No log	28.0	224	0.5924	0.6602	0.5924	0.7696
No log	28.25	226	0.5878	0.6681	0.5878	0.7667
No log	28.5	228	0.5894	0.6858	0.5894	0.7677
No log	28.75	230	0.6073	0.5770	0.6073	0.7793
No log	29.0	232	0.6080	0.5326	0.6080	0.7798
No log	29.25	234	0.5739	0.7033	0.5739	0.7575
No log	29.5	236	0.5511	0.7041	0.5511	0.7424
No log	29.75	238	0.5632	0.6623	0.5632	0.7505
No log	30.0	240	0.5789	0.6575	0.5789	0.7609
No log	30.25	242	0.5533	0.6548	0.5533	0.7438
No log	30.5	244	0.5355	0.7176	0.5355	0.7318
No log	30.75	246	0.5391	0.6958	0.5391	0.7343
No log	31.0	248	0.5416	0.6958	0.5416	0.7360
No log	31.25	250	0.5432	0.6850	0.5432	0.7370
No log	31.5	252	0.5409	0.6978	0.5409	0.7355
No log	31.75	254	0.5620	0.6519	0.5620	0.7497
No log	32.0	256	0.5814	0.6157	0.5814	0.7625
No log	32.25	258	0.5593	0.6519	0.5593	0.7478
No log	32.5	260	0.5587	0.6846	0.5587	0.7474
No log	32.75	262	0.5567	0.6748	0.5567	0.7461
No log	33.0	264	0.5530	0.7103	0.5530	0.7437
No log	33.25	266	0.5605	0.6602	0.5605	0.7487
No log	33.5	268	0.5640	0.6602	0.5640	0.7510
No log	33.75	270	0.5590	0.6927	0.5590	0.7477
No log	34.0	272	0.5653	0.6927	0.5653	0.7519
No log	34.25	274	0.5884	0.6215	0.5884	0.7671
No log	34.5	276	0.5799	0.6402	0.5799	0.7615
No log	34.75	278	0.5680	0.6725	0.5680	0.7537
No log	35.0	280	0.5216	0.7041	0.5216	0.7222
No log	35.25	282	0.5126	0.7049	0.5126	0.7159
No log	35.5	284	0.5197	0.6890	0.5197	0.7209
No log	35.75	286	0.5123	0.6778	0.5123	0.7157
No log	36.0	288	0.5054	0.7151	0.5054	0.7109
No log	36.25	290	0.5110	0.6888	0.5110	0.7149
No log	36.5	292	0.5119	0.6888	0.5119	0.7155
No log	36.75	294	0.5100	0.6846	0.5100	0.7141
No log	37.0	296	0.5269	0.6890	0.5269	0.7259
No log	37.25	298	0.5356	0.6890	0.5356	0.7319
No log	37.5	300	0.5193	0.6890	0.5193	0.7207
No log	37.75	302	0.5095	0.6995	0.5095	0.7138
No log	38.0	304	0.5152	0.6909	0.5152	0.7177
No log	38.25	306	0.5151	0.6995	0.5151	0.7177
No log	38.5	308	0.5196	0.7253	0.5196	0.7208
No log	38.75	310	0.5238	0.7097	0.5238	0.7237
No log	39.0	312	0.5314	0.6778	0.5314	0.7289
No log	39.25	314	0.5322	0.6890	0.5322	0.7295
No log	39.5	316	0.5280	0.6778	0.5280	0.7267
No log	39.75	318	0.5232	0.6737	0.5232	0.7233
No log	40.0	320	0.5245	0.7253	0.5245	0.7242
No log	40.25	322	0.5349	0.7285	0.5349	0.7314
No log	40.5	324	0.5310	0.7102	0.5310	0.7287
No log	40.75	326	0.5233	0.7352	0.5233	0.7234
No log	41.0	328	0.5314	0.7104	0.5314	0.7289
No log	41.25	330	0.5405	0.6854	0.5405	0.7352
No log	41.5	332	0.5505	0.6570	0.5505	0.7420
No log	41.75	334	0.5528	0.6465	0.5528	0.7435
No log	42.0	336	0.5455	0.6553	0.5455	0.7386
No log	42.25	338	0.5394	0.6709	0.5394	0.7344
No log	42.5	340	0.5322	0.6581	0.5322	0.7295
No log	42.75	342	0.5262	0.6882	0.5262	0.7254
No log	43.0	344	0.5281	0.6882	0.5281	0.7267
No log	43.25	346	0.5422	0.6716	0.5422	0.7363
No log	43.5	348	0.5687	0.7222	0.5687	0.7541
No log	43.75	350	0.5932	0.7056	0.5932	0.7702
No log	44.0	352	0.6069	0.7056	0.6069	0.7791
No log	44.25	354	0.6244	0.6883	0.6244	0.7902
No log	44.5	356	0.6117	0.7009	0.6117	0.7821
No log	44.75	358	0.5960	0.7009	0.5960	0.7720
No log	45.0	360	0.5869	0.7009	0.5869	0.7661
No log	45.25	362	0.6063	0.6883	0.6063	0.7787
No log	45.5	364	0.5805	0.7056	0.5805	0.7619
No log	45.75	366	0.5791	0.7056	0.5791	0.7610
No log	46.0	368	0.5625	0.7222	0.5625	0.7500
No log	46.25	370	0.5310	0.7326	0.5310	0.7287
No log	46.5	372	0.5310	0.7001	0.5310	0.7287
No log	46.75	374	0.5318	0.7001	0.5318	0.7293
No log	47.0	376	0.5146	0.6882	0.5146	0.7174
No log	47.25	378	0.5117	0.7089	0.5117	0.7153
No log	47.5	380	0.5115	0.7143	0.5115	0.7152
No log	47.75	382	0.5050	0.7201	0.5050	0.7106
No log	48.0	384	0.5053	0.7201	0.5053	0.7108
No log	48.25	386	0.5050	0.6882	0.5050	0.7107
No log	48.5	388	0.5043	0.7201	0.5043	0.7101
No log	48.75	390	0.5042	0.7201	0.5042	0.7101
No log	49.0	392	0.5068	0.6940	0.5068	0.7119
No log	49.25	394	0.5033	0.6853	0.5033	0.7095
No log	49.5	396	0.5127	0.6805	0.5127	0.7160
No log	49.75	398	0.5168	0.6805	0.5168	0.7189
No log	50.0	400	0.5040	0.6853	0.5040	0.7099
No log	50.25	402	0.5015	0.7201	0.5015	0.7082
No log	50.5	404	0.5058	0.7033	0.5058	0.7112
No log	50.75	406	0.5075	0.7033	0.5075	0.7124
No log	51.0	408	0.5142	0.7033	0.5142	0.7171
No log	51.25	410	0.5192	0.7033	0.5192	0.7206
No log	51.5	412	0.5160	0.7352	0.5160	0.7183
No log	51.75	414	0.5230	0.7124	0.5230	0.7232
No log	52.0	416	0.5680	0.6422	0.5680	0.7537
No log	52.25	418	0.6134	0.6420	0.6134	0.7832
No log	52.5	420	0.6066	0.6420	0.6066	0.7788
No log	52.75	422	0.5575	0.6797	0.5575	0.7467
No log	53.0	424	0.5310	0.6553	0.5310	0.7287
No log	53.25	426	0.5437	0.6598	0.5437	0.7374
No log	53.5	428	0.5810	0.7179	0.5810	0.7622
No log	53.75	430	0.6171	0.6308	0.6171	0.7856
No log	54.0	432	0.5979	0.6332	0.5979	0.7732
No log	54.25	434	0.5733	0.7056	0.5733	0.7572
No log	54.5	436	0.5667	0.6429	0.5667	0.7528
No log	54.75	438	0.5510	0.6993	0.5510	0.7423
No log	55.0	440	0.5423	0.6830	0.5423	0.7364
No log	55.25	442	0.5446	0.6424	0.5446	0.7380
No log	55.5	444	0.5401	0.6424	0.5401	0.7349
No log	55.75	446	0.5312	0.6896	0.5312	0.7288
No log	56.0	448	0.5305	0.6838	0.5305	0.7284
No log	56.25	450	0.5291	0.6737	0.5291	0.7274
No log	56.5	452	0.5217	0.6589	0.5217	0.7223
No log	56.75	454	0.5187	0.6589	0.5187	0.7202
No log	57.0	456	0.5163	0.6896	0.5163	0.7185
No log	57.25	458	0.5149	0.6896	0.5149	0.7176
No log	57.5	460	0.5161	0.6896	0.5161	0.7184
No log	57.75	462	0.5130	0.6896	0.5130	0.7162
No log	58.0	464	0.5123	0.6940	0.5123	0.7158
No log	58.25	466	0.5119	0.7201	0.5119	0.7155
No log	58.5	468	0.5118	0.6940	0.5118	0.7154
No log	58.75	470	0.5158	0.6762	0.5158	0.7182
No log	59.0	472	0.5127	0.6896	0.5127	0.7160
No log	59.25	474	0.5132	0.6813	0.5132	0.7164
No log	59.5	476	0.5064	0.6813	0.5064	0.7116
No log	59.75	478	0.5004	0.6853	0.5004	0.7074
No log	60.0	480	0.5000	0.6940	0.5000	0.7071
No log	60.25	482	0.5023	0.6813	0.5023	0.7087
No log	60.5	484	0.5114	0.6969	0.5114	0.7152
No log	60.75	486	0.5141	0.6969	0.5141	0.7170
No log	61.0	488	0.5134	0.6969	0.5134	0.7165
No log	61.25	490	0.5135	0.6456	0.5135	0.7166
No log	61.5	492	0.5159	0.6491	0.5159	0.7183
No log	61.75	494	0.5128	0.6526	0.5128	0.7161
No log	62.0	496	0.5105	0.6778	0.5105	0.7145
No log	62.25	498	0.5107	0.6778	0.5107	0.7146
0.2005	62.5	500	0.5114	0.6882	0.5114	0.7151
0.2005	62.75	502	0.5152	0.7201	0.5152	0.7178
0.2005	63.0	504	0.5155	0.7201	0.5155	0.7179
0.2005	63.25	506	0.5258	0.6537	0.5258	0.7251
0.2005	63.5	508	0.5348	0.6147	0.5348	0.7313
0.2005	63.75	510	0.5363	0.6147	0.5363	0.7323
0.2005	64.0	512	0.5304	0.6424	0.5304	0.7283
0.2005	64.25	514	0.5176	0.6911	0.5176	0.7194
0.2005	64.5	516	0.5122	0.6882	0.5122	0.7157
0.2005	64.75	518	0.5215	0.6993	0.5215	0.7221
0.2005	65.0	520	0.5357	0.7104	0.5357	0.7319
0.2005	65.25	522	0.5432	0.6935	0.5432	0.7370
0.2005	65.5	524	0.5368	0.6882	0.5368	0.7327
0.2005	65.75	526	0.5285	0.6770	0.5285	0.7270
0.2005	66.0	528	0.5313	0.6770	0.5313	0.7289
0.2005	66.25	530	0.5361	0.6138	0.5361	0.7322
0.2005	66.5	532	0.5357	0.6447	0.5357	0.7319

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1