ArabicNewSplits7_FineTuningAraBERT_run1_AugV5_k5_task1_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9058
• Qwk: 0.6533
• Mse: 0.9058
• Rmse: 0.9518

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.08	2	7.2065	-0.0107	7.2065	2.6845
No log	0.16	4	4.3899	0.0750	4.3899	2.0952
No log	0.24	6	3.3815	-0.0333	3.3815	1.8389
No log	0.32	8	3.8914	-0.0591	3.8914	1.9727
No log	0.4	10	2.9041	0.0690	2.9041	1.7042
No log	0.48	12	2.5009	0.0288	2.5009	1.5814
No log	0.56	14	2.1516	0.0650	2.1516	1.4668
No log	0.64	16	1.8521	0.1524	1.8521	1.3609
No log	0.72	18	1.8463	0.1165	1.8463	1.3588
No log	0.8	20	1.8896	0.0784	1.8896	1.3746
No log	0.88	22	2.0011	0.0962	2.0011	1.4146
No log	0.96	24	2.0892	0.1481	2.0892	1.4454
No log	1.04	26	2.0258	0.2182	2.0258	1.4233
No log	1.12	28	1.8887	0.2018	1.8887	1.3743
No log	1.2	30	1.7794	0.1321	1.7794	1.3339
No log	1.28	32	1.8148	0.2037	1.8148	1.3471
No log	1.3600	34	1.8695	0.2222	1.8695	1.3673
No log	1.44	36	1.7654	0.0784	1.7654	1.3287
No log	1.52	38	1.7339	0.0784	1.7339	1.3168
No log	1.6	40	1.6444	0.0962	1.6444	1.2823
No log	1.6800	42	1.6920	0.2364	1.6920	1.3008
No log	1.76	44	1.7382	0.2667	1.7382	1.3184
No log	1.8400	46	2.1034	0.2406	2.1034	1.4503
No log	1.92	48	2.5822	0.0816	2.5822	1.6069
No log	2.0	50	2.1975	0.1571	2.1975	1.4824
No log	2.08	52	1.5850	0.1441	1.5850	1.2590
No log	2.16	54	1.4786	0.1538	1.4786	1.2160
No log	2.24	56	1.6676	0.1538	1.6676	1.2914
No log	2.32	58	1.7849	0.1538	1.7849	1.3360
No log	2.4	60	1.6961	0.1714	1.6961	1.3023
No log	2.48	62	1.5486	0.2385	1.5486	1.2444
No log	2.56	64	1.5478	0.2182	1.5478	1.2441
No log	2.64	66	1.4984	0.1509	1.4984	1.2241
No log	2.7200	68	1.4294	0.1165	1.4294	1.1956
No log	2.8	70	1.3543	0.2264	1.3543	1.1637
No log	2.88	72	1.3952	0.1887	1.3952	1.1812
No log	2.96	74	1.9198	0.1935	1.9198	1.3856
No log	3.04	76	2.6848	-0.0576	2.6848	1.6385
No log	3.12	78	2.7482	-0.0426	2.7482	1.6578
No log	3.2	80	1.9130	0.2154	1.9130	1.3831
No log	3.2800	82	1.2864	0.4390	1.2864	1.1342
No log	3.36	84	1.1578	0.4333	1.1578	1.0760
No log	3.44	86	1.1520	0.5760	1.1520	1.0733
No log	3.52	88	1.3254	0.4754	1.3254	1.1512
No log	3.6	90	1.3710	0.4640	1.3710	1.1709
No log	3.68	92	1.2911	0.5075	1.2911	1.1363
No log	3.76	94	1.3562	0.4167	1.3562	1.1646
No log	3.84	96	1.6018	0.3810	1.6018	1.2656
No log	3.92	98	1.5157	0.3770	1.5157	1.2311
No log	4.0	100	1.3283	0.3390	1.3283	1.1525
No log	4.08	102	1.2439	0.512	1.2439	1.1153
No log	4.16	104	1.2479	0.5255	1.2479	1.1171
No log	4.24	106	1.3204	0.4714	1.3204	1.1491
No log	4.32	108	1.2492	0.4964	1.2492	1.1177
No log	4.4	110	1.1072	0.5874	1.1072	1.0522
No log	4.48	112	1.0925	0.5674	1.0925	1.0452
No log	4.5600	114	1.1991	0.5	1.1991	1.0950
No log	4.64	116	1.2597	0.5191	1.2597	1.1224
No log	4.72	118	1.2373	0.5039	1.2373	1.1123
No log	4.8	120	1.1497	0.5692	1.1497	1.0723
No log	4.88	122	1.0215	0.6260	1.0215	1.0107
No log	4.96	124	1.0691	0.5512	1.0691	1.0340
No log	5.04	126	1.0956	0.5385	1.0956	1.0467
No log	5.12	128	1.2058	0.5263	1.2058	1.0981
No log	5.2	130	1.1350	0.5496	1.1350	1.0654
No log	5.28	132	1.0405	0.5496	1.0405	1.0200
No log	5.36	134	1.0300	0.5496	1.0300	1.0149
No log	5.44	136	0.9463	0.6377	0.9463	0.9728
No log	5.52	138	0.9599	0.6418	0.9599	0.9797
No log	5.6	140	1.0140	0.5538	1.0140	1.0070
No log	5.68	142	0.9408	0.6308	0.9408	0.9700
No log	5.76	144	0.9339	0.6015	0.9339	0.9664
No log	5.84	146	0.9676	0.5781	0.9676	0.9836
No log	5.92	148	1.0516	0.5827	1.0516	1.0255
No log	6.0	150	1.1124	0.5625	1.1124	1.0547
No log	6.08	152	0.9819	0.5938	0.9819	0.9909
No log	6.16	154	0.9668	0.6047	0.9668	0.9833
No log	6.24	156	1.1109	0.5191	1.1109	1.0540
No log	6.32	158	1.1977	0.4769	1.1977	1.0944
No log	6.4	160	1.0271	0.5649	1.0271	1.0135
No log	6.48	162	0.9888	0.6316	0.9888	0.9944
No log	6.5600	164	1.1125	0.5564	1.1125	1.0548
No log	6.64	166	1.1139	0.5581	1.1139	1.0554
No log	6.72	168	0.9513	0.6767	0.9513	0.9753
No log	6.8	170	0.9870	0.6061	0.9870	0.9935
No log	6.88	172	0.9941	0.5954	0.9941	0.9971
No log	6.96	174	0.9270	0.6619	0.9270	0.9628
No log	7.04	176	0.8655	0.7050	0.8655	0.9303
No log	7.12	178	0.9267	0.6519	0.9267	0.9627
No log	7.2	180	1.0204	0.6165	1.0204	1.0101
No log	7.28	182	1.1704	0.5414	1.1704	1.0818
No log	7.36	184	1.2059	0.5538	1.2059	1.0981
No log	7.44	186	1.1131	0.5846	1.1131	1.0550
No log	7.52	188	1.0590	0.5556	1.0590	1.0291
No log	7.6	190	1.0154	0.6119	1.0154	1.0077
No log	7.68	192	1.0704	0.5821	1.0704	1.0346
No log	7.76	194	1.1249	0.5429	1.1249	1.0606
No log	7.84	196	1.1227	0.5522	1.1227	1.0596
No log	7.92	198	1.1060	0.5606	1.1060	1.0516
No log	8.0	200	1.0575	0.5778	1.0575	1.0283
No log	8.08	202	1.0000	0.6377	1.0000	1.0000
No log	8.16	204	1.0172	0.6074	1.0172	1.0085
No log	8.24	206	1.0017	0.5758	1.0017	1.0009
No log	8.32	208	1.1009	0.5846	1.1009	1.0492
No log	8.4	210	1.1522	0.5344	1.1522	1.0734
No log	8.48	212	1.1013	0.5538	1.1013	1.0494
No log	8.56	214	1.1155	0.5606	1.1155	1.0562
No log	8.64	216	1.2406	0.4925	1.2406	1.1138
No log	8.72	218	1.3007	0.4317	1.3007	1.1405
No log	8.8	220	1.2528	0.4638	1.2528	1.1193
No log	8.88	222	1.2381	0.5263	1.2381	1.1127
No log	8.96	224	1.2001	0.5556	1.2001	1.0955
No log	9.04	226	1.1051	0.6047	1.1051	1.0512
No log	9.12	228	1.0703	0.6	1.0703	1.0345
No log	9.2	230	1.0946	0.5909	1.0946	1.0463
No log	9.28	232	1.1239	0.5362	1.1239	1.0601
No log	9.36	234	1.0795	0.5401	1.0795	1.0390
No log	9.44	236	0.9392	0.6714	0.9392	0.9691
No log	9.52	238	0.8355	0.7101	0.8355	0.9141
No log	9.6	240	0.8590	0.7	0.8590	0.9268
No log	9.68	242	1.0264	0.6027	1.0264	1.0131
No log	9.76	244	1.1035	0.5986	1.1035	1.0505
No log	9.84	246	0.9091	0.6571	0.9091	0.9535
No log	9.92	248	0.8673	0.6667	0.8673	0.9313
No log	10.0	250	0.8808	0.6667	0.8808	0.9385
No log	10.08	252	0.9684	0.6345	0.9684	0.9841
No log	10.16	254	0.9734	0.6351	0.9734	0.9866
No log	10.24	256	0.9085	0.6761	0.9085	0.9532
No log	10.32	258	0.9045	0.6667	0.9045	0.9511
No log	10.4	260	0.9935	0.6331	0.9935	0.9967
No log	10.48	262	1.0764	0.5775	1.0764	1.0375
No log	10.56	264	1.0671	0.5775	1.0671	1.0330
No log	10.64	266	0.9976	0.6176	0.9976	0.9988
No log	10.72	268	0.9532	0.6406	0.9532	0.9763
No log	10.8	270	0.9820	0.6277	0.9820	0.9910
No log	10.88	272	1.0649	0.5652	1.0649	1.0320
No log	10.96	274	1.0864	0.5547	1.0864	1.0423
No log	11.04	276	1.0509	0.6032	1.0509	1.0251
No log	11.12	278	1.0499	0.576	1.0499	1.0247
No log	11.2	280	1.0303	0.576	1.0303	1.0150
No log	11.28	282	1.0346	0.6202	1.0346	1.0172
No log	11.36	284	1.1251	0.5954	1.1251	1.0607
No log	11.44	286	1.1600	0.5333	1.1600	1.0770
No log	11.52	288	1.0838	0.5891	1.0838	1.0411
No log	11.6	290	1.0048	0.6412	1.0048	1.0024
No log	11.68	292	0.9524	0.6047	0.9524	0.9759
No log	11.76	294	1.0013	0.5954	1.0013	1.0006
No log	11.84	296	1.0803	0.5693	1.0803	1.0394
No log	11.92	298	1.0560	0.5693	1.0560	1.0276
No log	12.0	300	0.9531	0.6423	0.9531	0.9763
No log	12.08	302	0.9694	0.6351	0.9694	0.9846
No log	12.16	304	1.0120	0.6259	1.0120	1.0060
No log	12.24	306	1.0330	0.5882	1.0330	1.0164
No log	12.32	308	0.9337	0.5882	0.9337	0.9663
No log	12.4	310	0.8142	0.6370	0.8142	0.9024
No log	12.48	312	0.7679	0.6406	0.7679	0.8763
No log	12.56	314	0.7702	0.6615	0.7702	0.8776
No log	12.64	316	0.8229	0.6667	0.8229	0.9071
No log	12.72	318	0.9973	0.6753	0.9973	0.9986
No log	12.8	320	1.1280	0.6335	1.1280	1.0621
No log	12.88	322	1.0566	0.6420	1.0566	1.0279
No log	12.96	324	0.9178	0.6835	0.9178	0.9580
No log	13.04	326	0.8482	0.7075	0.8482	0.9210
No log	13.12	328	0.8583	0.6623	0.8583	0.9264
No log	13.2	330	0.9762	0.6536	0.9762	0.9880
No log	13.28	332	1.1105	0.6329	1.1105	1.0538
No log	13.36	334	1.0227	0.6184	1.0227	1.0113
No log	13.44	336	0.8471	0.6522	0.8471	0.9204
No log	13.52	338	0.6972	0.7338	0.6972	0.8350
No log	13.6	340	0.6681	0.7429	0.6681	0.8174
No log	13.68	342	0.6849	0.7338	0.6849	0.8276
No log	13.76	344	0.7800	0.6619	0.7800	0.8832
No log	13.84	346	0.8928	0.5985	0.8928	0.9449
No log	13.92	348	0.9491	0.6040	0.9491	0.9742
No log	14.0	350	0.8714	0.6571	0.8714	0.9335
No log	14.08	352	0.8212	0.6763	0.8212	0.9062
No log	14.16	354	0.8173	0.6714	0.8173	0.9040
No log	14.24	356	0.8050	0.6714	0.8050	0.8972
No log	14.32	358	0.8714	0.6897	0.8714	0.9335
No log	14.4	360	0.9980	0.6184	0.9980	0.9990
No log	14.48	362	0.9856	0.6093	0.9856	0.9928
No log	14.56	364	0.8942	0.6338	0.8942	0.9456
No log	14.64	366	0.8701	0.6377	0.8701	0.9328
No log	14.72	368	0.8730	0.6370	0.8730	0.9343
No log	14.8	370	0.8718	0.6047	0.8718	0.9337
No log	14.88	372	0.9415	0.5758	0.9415	0.9703
No log	14.96	374	1.0012	0.5547	1.0012	1.0006
No log	15.04	376	0.9758	0.5547	0.9758	0.9878
No log	15.12	378	0.8761	0.6187	0.8761	0.9360
No log	15.2	380	0.7972	0.6861	0.7972	0.8928
No log	15.28	382	0.8012	0.7083	0.8012	0.8951
No log	15.36	384	0.9151	0.6533	0.9151	0.9566
No log	15.44	386	1.0256	0.5986	1.0256	1.0127
No log	15.52	388	0.9796	0.6164	0.9796	0.9898
No log	15.6	390	0.8615	0.6525	0.8615	0.9282
No log	15.68	392	0.8459	0.6222	0.8459	0.9197
No log	15.76	394	0.8875	0.6483	0.8875	0.9421
No log	15.84	396	0.9574	0.6389	0.9574	0.9785
No log	15.92	398	0.9336	0.6713	0.9336	0.9662
No log	16.0	400	0.8541	0.6620	0.8541	0.9242
No log	16.08	402	0.7713	0.7260	0.7713	0.8782
No log	16.16	404	0.7633	0.7246	0.7633	0.8737
No log	16.24	406	0.8128	0.6912	0.8128	0.9016
No log	16.32	408	0.9698	0.64	0.9698	0.9848
No log	16.4	410	1.1654	0.5912	1.1654	1.0795
No log	16.48	412	1.2077	0.5767	1.2077	1.0989
No log	16.56	414	1.0718	0.6225	1.0718	1.0353
No log	16.64	416	0.9274	0.6763	0.9274	0.9630
No log	16.72	418	0.8809	0.6957	0.8809	0.9385
No log	16.8	420	0.8765	0.6906	0.8765	0.9362
No log	16.88	422	0.8893	0.6901	0.8893	0.9430
No log	16.96	424	0.9652	0.6232	0.9652	0.9824
No log	17.04	426	0.9925	0.6232	0.9925	0.9962
No log	17.12	428	0.9179	0.6260	0.9179	0.9581
No log	17.2	430	0.7938	0.6357	0.7938	0.8909
No log	17.28	432	0.7404	0.7015	0.7404	0.8605
No log	17.36	434	0.7279	0.7015	0.7279	0.8532
No log	17.44	436	0.8239	0.6939	0.8239	0.9077
No log	17.52	438	0.9908	0.6623	0.9908	0.9954
No log	17.6	440	1.0875	0.625	1.0875	1.0429
No log	17.68	442	1.0192	0.6579	1.0192	1.0096
No log	17.76	444	0.8711	0.6887	0.8711	0.9333
No log	17.84	446	0.7333	0.7273	0.7333	0.8564
No log	17.92	448	0.7107	0.7724	0.7107	0.8430
No log	18.0	450	0.7371	0.7234	0.7371	0.8586
No log	18.08	452	0.7880	0.7143	0.7880	0.8877
No log	18.16	454	0.9472	0.6269	0.9472	0.9733
No log	18.24	456	1.0827	0.5970	1.0827	1.0405
No log	18.32	458	1.1307	0.5758	1.1307	1.0634
No log	18.4	460	1.0578	0.5846	1.0578	1.0285
No log	18.48	462	0.9370	0.625	0.9370	0.9680
No log	18.56	464	0.8929	0.6357	0.8929	0.9449
No log	18.64	466	0.8371	0.7015	0.8371	0.9149
No log	18.72	468	0.8099	0.7206	0.8099	0.8999
No log	18.8	470	0.8334	0.6617	0.8334	0.9129
No log	18.88	472	0.9196	0.6277	0.9196	0.9590
No log	18.96	474	1.0200	0.5547	1.0200	1.0100
No log	19.04	476	1.0080	0.5588	1.0080	1.0040
No log	19.12	478	0.9190	0.5984	0.9190	0.9586
No log	19.2	480	0.8413	0.6094	0.8413	0.9172
No log	19.28	482	0.8137	0.6667	0.8137	0.9021
No log	19.36	484	0.8200	0.6357	0.8200	0.9055
No log	19.44	486	0.8550	0.6364	0.8550	0.9246
No log	19.52	488	0.8898	0.6466	0.8898	0.9433
No log	19.6	490	0.9532	0.6370	0.9532	0.9763
No log	19.68	492	0.9558	0.6269	0.9558	0.9777
No log	19.76	494	0.9596	0.6074	0.9596	0.9796
No log	19.84	496	0.9756	0.6087	0.9756	0.9877
No log	19.92	498	0.9671	0.6316	0.9671	0.9834
0.4829	20.0	500	0.9185	0.6870	0.9185	0.9584
0.4829	20.08	502	0.9076	0.6870	0.9076	0.9527
0.4829	20.16	504	0.9370	0.6212	0.9370	0.9680
0.4829	20.24	506	1.0246	0.6122	1.0246	1.0122
0.4829	20.32	508	0.9984	0.6351	0.9984	0.9992
0.4829	20.4	510	0.9058	0.6533	0.9058	0.9518

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1