Arabic_CrossPrompt_FineTuningAraBERT_noAug_TestTask5_grammar

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4960
• Qwk: 0.5793
• Mse: 0.4960
• Rmse: 0.7043

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.0198	2	4.3214	-0.0067	4.3214	2.0788
No log	0.0396	4	2.9172	0.0635	2.9172	1.7080
No log	0.0594	6	1.5062	0.0019	1.5062	1.2273
No log	0.0792	8	1.0377	0.0355	1.0377	1.0187
No log	0.0990	10	0.8053	0.1398	0.8053	0.8974
No log	0.1188	12	0.8507	0.0841	0.8507	0.9223
No log	0.1386	14	0.8907	0.1137	0.8907	0.9438
No log	0.1584	16	0.7907	0.1793	0.7907	0.8892
No log	0.1782	18	0.6838	0.3145	0.6838	0.8269
No log	0.1980	20	0.6677	0.3585	0.6677	0.8171
No log	0.2178	22	0.6864	0.2340	0.6864	0.8285
No log	0.2376	24	0.7205	0.1808	0.7205	0.8488
No log	0.2574	26	0.7187	0.1657	0.7187	0.8478
No log	0.2772	28	0.6942	0.1957	0.6942	0.8332
No log	0.2970	30	0.6759	0.3128	0.6759	0.8221
No log	0.3168	32	0.6765	0.3796	0.6765	0.8225
No log	0.3366	34	0.6537	0.4037	0.6537	0.8085
No log	0.3564	36	0.5938	0.3794	0.5938	0.7706
No log	0.3762	38	0.6528	0.3029	0.6528	0.8079
No log	0.3960	40	0.7335	0.3070	0.7335	0.8564
No log	0.4158	42	0.7371	0.3172	0.7371	0.8585
No log	0.4356	44	0.6799	0.3768	0.6799	0.8245
No log	0.4554	46	0.5893	0.4440	0.5893	0.7676
No log	0.4752	48	0.5767	0.4800	0.5767	0.7594
No log	0.4950	50	0.5700	0.4605	0.5700	0.7550
No log	0.5149	52	0.5702	0.4662	0.5702	0.7551
No log	0.5347	54	0.5748	0.4026	0.5748	0.7582
No log	0.5545	56	0.6172	0.3677	0.6172	0.7856
No log	0.5743	58	0.6126	0.3491	0.6126	0.7827
No log	0.5941	60	0.6277	0.3003	0.6277	0.7923
No log	0.6139	62	0.5893	0.4110	0.5893	0.7677
No log	0.6337	64	0.5511	0.4678	0.5511	0.7423
No log	0.6535	66	0.5633	0.5058	0.5633	0.7505
No log	0.6733	68	0.6189	0.5007	0.6189	0.7867
No log	0.6931	70	0.6102	0.5174	0.6102	0.7811
No log	0.7129	72	0.5809	0.5142	0.5809	0.7622
No log	0.7327	74	0.6551	0.4910	0.6551	0.8094
No log	0.7525	76	0.5688	0.5201	0.5688	0.7542
No log	0.7723	78	0.5110	0.5429	0.5110	0.7149
No log	0.7921	80	0.5380	0.3901	0.5380	0.7335
No log	0.8119	82	0.6138	0.3229	0.6138	0.7834
No log	0.8317	84	0.6101	0.3231	0.6101	0.7811
No log	0.8515	86	0.5300	0.4620	0.5300	0.7280
No log	0.8713	88	0.5203	0.5345	0.5203	0.7213
No log	0.8911	90	0.5034	0.5517	0.5034	0.7095
No log	0.9109	92	0.6162	0.3668	0.6162	0.7850
No log	0.9307	94	0.7100	0.3035	0.7100	0.8426
No log	0.9505	96	0.6769	0.3292	0.6769	0.8227
No log	0.9703	98	0.5643	0.4905	0.5643	0.7512
No log	0.9901	100	0.5014	0.5022	0.5014	0.7081
No log	1.0099	102	0.4908	0.5146	0.4908	0.7006
No log	1.0297	104	0.4970	0.4629	0.4970	0.7050
No log	1.0495	106	0.5592	0.3769	0.5592	0.7478
No log	1.0693	108	0.6568	0.4085	0.6568	0.8104
No log	1.0891	110	0.7098	0.4463	0.7098	0.8425
No log	1.1089	112	0.6890	0.4918	0.6890	0.8301
No log	1.1287	114	0.5379	0.5552	0.5379	0.7334
No log	1.1485	116	0.5224	0.5883	0.5224	0.7228
No log	1.1683	118	0.5042	0.6115	0.5042	0.7101
No log	1.1881	120	0.6940	0.5222	0.6940	0.8331
No log	1.2079	122	0.8069	0.3473	0.8069	0.8982
No log	1.2277	124	0.8173	0.2675	0.8173	0.9040
No log	1.2475	126	0.7003	0.4258	0.7003	0.8369
No log	1.2673	128	0.5189	0.5775	0.5189	0.7203
No log	1.2871	130	0.5131	0.5372	0.5131	0.7163
No log	1.3069	132	0.5103	0.5715	0.5103	0.7143
No log	1.3267	134	0.5104	0.4942	0.5104	0.7144
No log	1.3465	136	0.6700	0.4089	0.6700	0.8185
No log	1.3663	138	0.7466	0.3296	0.7466	0.8640
No log	1.3861	140	0.6965	0.4316	0.6965	0.8346
No log	1.4059	142	0.6386	0.5585	0.6386	0.7991
No log	1.4257	144	0.6027	0.5620	0.6027	0.7763
No log	1.4455	146	0.4979	0.5620	0.4979	0.7056
No log	1.4653	148	0.4714	0.5272	0.4714	0.6866
No log	1.4851	150	0.4736	0.5024	0.4736	0.6882
No log	1.5050	152	0.4853	0.4033	0.4853	0.6966
No log	1.5248	154	0.5246	0.3924	0.5246	0.7243
No log	1.5446	156	0.6435	0.4319	0.6435	0.8022
No log	1.5644	158	0.6377	0.4026	0.6377	0.7986
No log	1.5842	160	0.5481	0.4043	0.5481	0.7403
No log	1.6040	162	0.5065	0.3718	0.5065	0.7117
No log	1.6238	164	0.5086	0.4257	0.5086	0.7132
No log	1.6436	166	0.5509	0.4817	0.5509	0.7422
No log	1.6634	168	0.6131	0.5389	0.6131	0.7830
No log	1.6832	170	0.8426	0.4808	0.8426	0.9179
No log	1.7030	172	0.9014	0.4595	0.9014	0.9494
No log	1.7228	174	0.7099	0.4987	0.7099	0.8426
No log	1.7426	176	0.6223	0.5521	0.6223	0.7889
No log	1.7624	178	0.6193	0.5599	0.6193	0.7869
No log	1.7822	180	0.7295	0.4730	0.7295	0.8541
No log	1.8020	182	0.7522	0.4658	0.7522	0.8673
No log	1.8218	184	0.6219	0.5308	0.6219	0.7886
No log	1.8416	186	0.5378	0.5054	0.5378	0.7334
No log	1.8614	188	0.5580	0.5189	0.5580	0.7470
No log	1.8812	190	0.5325	0.4654	0.5325	0.7297
No log	1.9010	192	0.5347	0.4226	0.5347	0.7312
No log	1.9208	194	0.5795	0.4631	0.5795	0.7612
No log	1.9406	196	0.6169	0.4687	0.6169	0.7854
No log	1.9604	198	0.6850	0.4003	0.6850	0.8276
No log	1.9802	200	0.7071	0.4247	0.7071	0.8409
No log	2.0	202	0.8268	0.4295	0.8268	0.9093
No log	2.0198	204	0.8451	0.4717	0.8451	0.9193
No log	2.0396	206	0.7749	0.5098	0.7749	0.8803
No log	2.0594	208	0.6879	0.5427	0.6879	0.8294
No log	2.0792	210	0.6468	0.5373	0.6468	0.8042
No log	2.0990	212	0.5832	0.5442	0.5832	0.7637
No log	2.1188	214	0.5547	0.5476	0.5547	0.7448
No log	2.1386	216	0.6863	0.4893	0.6863	0.8284
No log	2.1584	218	0.7505	0.4176	0.7505	0.8663
No log	2.1782	220	0.6569	0.4044	0.6569	0.8105
No log	2.1980	222	0.5207	0.5029	0.5207	0.7216
No log	2.2178	224	0.4929	0.5183	0.4929	0.7020
No log	2.2376	226	0.5028	0.5392	0.5028	0.7091
No log	2.2574	228	0.6333	0.4736	0.6333	0.7958
No log	2.2772	230	0.7494	0.4754	0.7494	0.8657
No log	2.2970	232	0.6745	0.5546	0.6745	0.8213
No log	2.3168	234	0.5450	0.5775	0.5450	0.7382
No log	2.3366	236	0.5198	0.5461	0.5198	0.7210
No log	2.3564	238	0.4995	0.5423	0.4995	0.7068
No log	2.3762	240	0.4971	0.5364	0.4971	0.7051
No log	2.3960	242	0.4948	0.5284	0.4948	0.7034
No log	2.4158	244	0.5098	0.4742	0.5098	0.7140
No log	2.4356	246	0.4943	0.4964	0.4943	0.7031
No log	2.4554	248	0.4904	0.4865	0.4904	0.7003
No log	2.4752	250	0.5070	0.4865	0.5070	0.7120
No log	2.4950	252	0.6019	0.5224	0.6019	0.7758
No log	2.5149	254	0.7326	0.4532	0.7326	0.8559
No log	2.5347	256	0.8288	0.4272	0.8288	0.9104
No log	2.5545	258	0.7715	0.4900	0.7715	0.8783
No log	2.5743	260	0.6758	0.5568	0.6758	0.8221
No log	2.5941	262	0.6080	0.6088	0.6080	0.7798
No log	2.6139	264	0.6009	0.6006	0.6009	0.7752
No log	2.6337	266	0.5798	0.6130	0.5798	0.7615
No log	2.6535	268	0.5962	0.5761	0.5962	0.7721
No log	2.6733	270	0.6342	0.4635	0.6342	0.7964
No log	2.6931	272	0.6725	0.2463	0.6725	0.8201
No log	2.7129	274	0.6584	0.2377	0.6584	0.8114
No log	2.7327	276	0.6294	0.2870	0.6294	0.7934
No log	2.7525	278	0.6059	0.2870	0.6059	0.7784
No log	2.7723	280	0.5971	0.2957	0.5971	0.7727
No log	2.7921	282	0.6375	0.2710	0.6375	0.7984
No log	2.8119	284	0.6941	0.3000	0.6941	0.8331
No log	2.8317	286	0.6663	0.4606	0.6663	0.8162
No log	2.8515	288	0.5939	0.5822	0.5939	0.7707
No log	2.8713	290	0.6253	0.5926	0.6253	0.7907
No log	2.8911	292	0.5989	0.5948	0.5989	0.7739
No log	2.9109	294	0.5802	0.6007	0.5802	0.7617
No log	2.9307	296	0.6774	0.5268	0.6774	0.8231
No log	2.9505	298	0.6966	0.4976	0.6966	0.8346
No log	2.9703	300	0.6021	0.5328	0.6021	0.7760
No log	2.9901	302	0.4904	0.5367	0.4904	0.7003
No log	3.0099	304	0.4959	0.4902	0.4959	0.7042
No log	3.0297	306	0.4931	0.4585	0.4931	0.7022
No log	3.0495	308	0.4977	0.4957	0.4977	0.7055
No log	3.0693	310	0.5766	0.5460	0.5766	0.7593
No log	3.0891	312	0.6484	0.5177	0.6484	0.8052
No log	3.1089	314	0.6354	0.5230	0.6354	0.7971
No log	3.1287	316	0.5811	0.5177	0.5811	0.7623
No log	3.1485	318	0.5403	0.5195	0.5403	0.7351
No log	3.1683	320	0.5572	0.5243	0.5572	0.7465
No log	3.1881	322	0.5900	0.5432	0.5900	0.7681
No log	3.2079	324	0.6617	0.4795	0.6617	0.8134
No log	3.2277	326	0.6676	0.4534	0.6676	0.8171
No log	3.2475	328	0.6030	0.4787	0.6030	0.7766
No log	3.2673	330	0.5358	0.5630	0.5358	0.7320
No log	3.2871	332	0.4971	0.5573	0.4971	0.7050
No log	3.3069	334	0.5111	0.5993	0.5111	0.7149
No log	3.3267	336	0.5067	0.6072	0.5067	0.7118
No log	3.3465	338	0.5683	0.5969	0.5683	0.7539
No log	3.3663	340	0.7460	0.4823	0.7460	0.8637
No log	3.3861	342	0.8828	0.4145	0.8828	0.9396
No log	3.4059	344	0.8907	0.4153	0.8907	0.9438
No log	3.4257	346	0.7572	0.4594	0.7572	0.8702
No log	3.4455	348	0.6012	0.5598	0.6012	0.7754
No log	3.4653	350	0.5851	0.5488	0.5851	0.7649
No log	3.4851	352	0.5497	0.5623	0.5497	0.7414
No log	3.5050	354	0.5605	0.5773	0.5605	0.7487
No log	3.5248	356	0.6674	0.5558	0.6674	0.8170
No log	3.5446	358	0.7834	0.4663	0.7834	0.8851
No log	3.5644	360	0.7836	0.4805	0.7836	0.8852
No log	3.5842	362	0.7023	0.5345	0.7023	0.8380
No log	3.6040	364	0.5420	0.6189	0.5420	0.7362
No log	3.6238	366	0.5198	0.6031	0.5198	0.7210
No log	3.6436	368	0.5154	0.5974	0.5154	0.7179
No log	3.6634	370	0.5909	0.5966	0.5909	0.7687
No log	3.6832	372	0.7003	0.4983	0.7003	0.8368
No log	3.7030	374	0.6886	0.4809	0.6886	0.8298
No log	3.7228	376	0.5734	0.5264	0.5734	0.7572
No log	3.7426	378	0.5256	0.5879	0.5256	0.7250
No log	3.7624	380	0.5182	0.5897	0.5182	0.7198
No log	3.7822	382	0.5598	0.5668	0.5598	0.7482
No log	3.8020	384	0.6724	0.5205	0.6724	0.8200
No log	3.8218	386	0.7733	0.4747	0.7733	0.8794
No log	3.8416	388	0.8037	0.4765	0.8037	0.8965
No log	3.8614	390	0.7182	0.4836	0.7182	0.8475
No log	3.8812	392	0.6395	0.5337	0.6395	0.7997
No log	3.9010	394	0.5742	0.5832	0.5742	0.7577
No log	3.9208	396	0.5875	0.5721	0.5875	0.7665
No log	3.9406	398	0.6721	0.5291	0.6721	0.8198
No log	3.9604	400	0.7364	0.5316	0.7364	0.8581
No log	3.9802	402	0.7351	0.5588	0.7351	0.8574
No log	4.0	404	0.7558	0.5398	0.7558	0.8694
No log	4.0198	406	0.7428	0.5288	0.7428	0.8619
No log	4.0396	408	0.6595	0.5607	0.6595	0.8121
No log	4.0594	410	0.5730	0.5518	0.5730	0.7570
No log	4.0792	412	0.5627	0.5360	0.5627	0.7501
No log	4.0990	414	0.6115	0.5104	0.6115	0.7820
No log	4.1188	416	0.7058	0.4445	0.7058	0.8401
No log	4.1386	418	0.7630	0.4347	0.7630	0.8735
No log	4.1584	420	0.7869	0.4618	0.7869	0.8871
No log	4.1782	422	0.6505	0.5437	0.6505	0.8065
No log	4.1980	424	0.5968	0.5814	0.5968	0.7726
No log	4.2178	426	0.6656	0.5341	0.6656	0.8158
No log	4.2376	428	0.8111	0.4616	0.8111	0.9006
No log	4.2574	430	0.7756	0.4812	0.7756	0.8807
No log	4.2772	432	0.5990	0.5883	0.5990	0.7739
No log	4.2970	434	0.5469	0.5647	0.5469	0.7395
No log	4.3168	436	0.5644	0.5139	0.5644	0.7513
No log	4.3366	438	0.5172	0.5424	0.5172	0.7192
No log	4.3564	440	0.5641	0.5331	0.5641	0.7510
No log	4.3762	442	0.8089	0.4315	0.8089	0.8994
No log	4.3960	444	0.8810	0.4041	0.8810	0.9386
No log	4.4158	446	0.7841	0.4075	0.7841	0.8855
No log	4.4356	448	0.6109	0.5324	0.6109	0.7816
No log	4.4554	450	0.5221	0.5467	0.5221	0.7226
No log	4.4752	452	0.5011	0.5492	0.5011	0.7079
No log	4.4950	454	0.4949	0.5866	0.4949	0.7035
No log	4.5149	456	0.5824	0.5652	0.5824	0.7632
No log	4.5347	458	0.7049	0.5323	0.7049	0.8396
No log	4.5545	460	0.7355	0.5253	0.7355	0.8576
No log	4.5743	462	0.6313	0.5639	0.6313	0.7945
No log	4.5941	464	0.5840	0.5911	0.5840	0.7642
No log	4.6139	466	0.5819	0.5897	0.5819	0.7628
No log	4.6337	468	0.6567	0.5372	0.6567	0.8104
No log	4.6535	470	0.7490	0.4798	0.7490	0.8654
No log	4.6733	472	0.8477	0.4355	0.8477	0.9207
No log	4.6931	474	0.7648	0.4907	0.7648	0.8745
No log	4.7129	476	0.6175	0.5585	0.6175	0.7858
No log	4.7327	478	0.5142	0.6049	0.5142	0.7171
No log	4.7525	480	0.5148	0.5908	0.5148	0.7175
No log	4.7723	482	0.5935	0.5651	0.5935	0.7704
No log	4.7921	484	0.5711	0.5678	0.5711	0.7557
No log	4.8119	486	0.6065	0.5353	0.6065	0.7788
No log	4.8317	488	0.5842	0.5413	0.5842	0.7643
No log	4.8515	490	0.6870	0.5324	0.6870	0.8289
No log	4.8713	492	0.6627	0.5408	0.6627	0.8140
No log	4.8911	494	0.6137	0.5463	0.6137	0.7834
No log	4.9109	496	0.5093	0.5944	0.5093	0.7137
No log	4.9307	498	0.4654	0.6641	0.4654	0.6822
0.5253	4.9505	500	0.4715	0.6245	0.4715	0.6867
0.5253	4.9703	502	0.5949	0.5558	0.5949	0.7713
0.5253	4.9901	504	0.7813	0.4918	0.7813	0.8839
0.5253	5.0099	506	0.7283	0.4850	0.7283	0.8534
0.5253	5.0297	508	0.5528	0.5614	0.5528	0.7435
0.5253	5.0495	510	0.4960	0.5793	0.4960	0.7043

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1