roberta-2020-Q2-filtered

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5868

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1400
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.9727
3.1662	0.07	16000	2.8727
3.1662	0.1	24000	2.8134
2.914	0.14	32000	2.7663
2.914	0.17	40000	2.7392
2.8319	0.21	48000	2.7003
2.8319	0.24	56000	2.6796
2.7892	0.28	64000	2.6640
2.7892	0.31	72000	2.6422
2.7471	0.35	80000	2.6159
2.7471	0.38	88000	2.6056
2.727	0.42	96000	2.6007
2.727	0.45	104000	2.6046
2.7185	0.49	112000	2.6009
2.7185	0.52	120000	2.5992
2.7121	0.56	128000	2.5893
2.7121	0.59	136000	2.5993
2.722	0.63	144000	2.5935
2.722	0.66	152000	2.5808
2.724	0.7	160000	2.6009
2.724	0.73	168000	2.6015
2.7192	0.77	176000	2.5918
2.7192	0.8	184000	2.6031
2.7234	0.84	192000	2.5920
2.7234	0.87	200000	2.6065
2.7369	0.91	208000	2.6125
2.7369	0.94	216000	2.6040
2.7282	0.98	224000	2.6042
2.7282	1.01	232000	2.6186
2.7322	1.05	240000	2.6130
2.7322	1.08	248000	2.6214
2.7361	1.12	256000	2.6168
2.7361	1.15	264000	2.6266
2.7514	1.19	272000	2.6223
2.7514	1.22	280000	2.6240
2.7557	1.26	288000	2.6301
2.7557	1.29	296000	2.6284
2.7591	1.33	304000	2.6443
2.7591	1.36	312000	2.6485
2.7675	1.4	320000	2.6457
2.7675	1.43	328000	2.6444
2.7732	1.47	336000	2.6537
2.7732	1.5	344000	2.6632
2.7922	1.54	352000	2.6641
2.7922	1.57	360000	2.6715
2.7908	1.61	368000	2.6743
2.7908	1.64	376000	2.6686
2.8148	1.68	384000	2.6779
2.8148	1.71	392000	2.6765
2.8103	1.75	400000	2.6829
2.8103	1.78	408000	2.6941
2.8192	1.82	416000	2.6800
2.8192	1.85	424000	2.6954
2.8298	1.89	432000	2.7022
2.8298	1.92	440000	2.6993
2.8298	1.96	448000	2.7028
2.8298	1.99	456000	2.7213
2.8546	2.03	464000	2.7187
2.8546	2.06	472000	2.7107
2.8334	2.09	480000	2.7094
2.8334	2.13	488000	2.7309
2.854	2.16	496000	2.7340
2.854	2.2	504000	2.7264
2.8539	2.23	512000	2.7456
2.8539	2.27	520000	2.7412
2.8717	2.3	528000	2.7517
2.8717	2.34	536000	2.7474
2.8733	2.37	544000	2.7649
2.8733	2.41	552000	2.7536
2.8876	2.44	560000	2.7602
2.8876	2.48	568000	2.7617
2.905	2.51	576000	2.7663
2.905	2.55	584000	2.7840
2.8964	2.58	592000	2.7827
2.8964	2.62	600000	2.7769
2.9118	2.65	608000	2.7880
2.9118	2.69	616000	2.7923
2.9222	2.72	624000	2.7897
2.9222	2.76	632000	2.8131
2.9311	2.79	640000	2.8014
2.9311	2.83	648000	2.8287
2.9469	2.86	656000	2.8267
2.9469	2.9	664000	2.8234
2.9449	2.93	672000	2.8258
2.9449	2.97	680000	2.8252
2.9608	3.0	688000	2.8328
2.9608	3.04	696000	2.8387
2.9499	3.07	704000	2.8425
2.9499	3.11	712000	2.8431
2.9662	3.14	720000	2.8575
2.9662	3.18	728000	2.8588
2.9779	3.21	736000	2.8636
2.9779	3.25	744000	2.8631
2.9787	3.28	752000	2.8736
2.9787	3.32	760000	2.8701
3.0025	3.35	768000	2.8815
3.0025	3.39	776000	2.8750
2.999	3.42	784000	2.8860
2.999	3.46	792000	2.8876
3.0012	3.49	800000	2.9017
3.0012	3.53	808000	2.8898
3.0076	3.56	816000	2.9074
3.0076	3.6	824000	2.8906
3.0122	3.63	832000	2.9073
3.0122	3.67	840000	2.9154
3.0209	3.7	848000	2.9111
3.0209	3.74	856000	2.9094
3.0383	3.77	864000	2.9132
3.0383	3.81	872000	2.9201
3.043	3.84	880000	2.9280
3.043	3.88	888000	2.9231
3.0469	3.91	896000	2.9240
3.0469	3.95	904000	2.9272
3.0386	3.98	912000	2.9258
3.0386	4.02	920000	2.9494
3.0479	4.05	928000	2.9389
3.0479	4.08	936000	2.9377
3.0473	4.12	944000	2.9467
3.0473	4.15	952000	2.9495
3.0509	4.19	960000	2.9501
3.0509	4.22	968000	2.9470
3.0414	4.26	976000	2.9405
3.0414	4.29	984000	2.9444
3.0529	4.33	992000	2.9393
3.0529	4.36	1000000	2.9435
3.0594	4.4	1008000	2.9583
3.0594	4.43	1016000	2.9457
3.0479	4.47	1024000	2.9435
3.0479	4.5	1032000	2.9527
3.0564	4.54	1040000	2.9500
3.0564	4.57	1048000	2.9550
3.0554	4.61	1056000	2.9578
3.0554	4.64	1064000	2.9628
3.0626	4.68	1072000	2.9580
3.0626	4.71	1080000	2.9667
3.0722	4.75	1088000	2.9734
3.0722	4.78	1096000	2.9653
3.0731	4.82	1104000	2.9689
3.0731	4.85	1112000	2.9739
3.0724	4.89	1120000	2.9875
3.0724	4.92	1128000	2.9849
3.0656	4.96	1136000	2.9752
3.0656	4.99	1144000	2.9751
3.0829	5.03	1152000	2.9768
3.0829	5.06	1160000	2.9835
3.0785	5.1	1168000	2.9843
3.0785	5.13	1176000	3.0001
3.0704	5.17	1184000	2.9906
3.0704	5.2	1192000	2.9850
3.075	5.24	1200000	2.9931
3.075	5.27	1208000	2.9986
3.083	5.31	1216000	3.0008
3.083	5.34	1224000	3.0009
3.0708	5.38	1232000	3.0017
3.0708	5.41	1240000	2.9932
3.0896	5.45	1248000	2.9970
3.0896	5.48	1256000	3.0027
3.092	5.52	1264000	3.0002
3.092	5.55	1272000	2.9967
3.0916	5.59	1280000	2.9987
3.0916	5.62	1288000	2.9990
3.0938	5.66	1296000	3.0035
3.0938	5.69	1304000	2.9999
3.1039	5.73	1312000	3.0097
3.1039	5.76	1320000	3.0022
3.1059	5.8	1328000	3.0161
3.1059	5.83	1336000	3.0071
3.1014	5.87	1344000	3.0150
3.1014	5.9	1352000	2.9986
3.1048	5.94	1360000	3.0096
3.1048	5.97	1368000	3.0063
3.1099	6.01	1376000	3.0095
3.1099	6.04	1384000	3.0152
3.0891	6.08	1392000	3.0179
3.0891	6.11	1400000	3.0299
3.0979	6.14	1408000	3.0127
3.0979	6.18	1416000	3.0260
3.099	6.21	1424000	3.0187
3.099	6.25	1432000	3.0114
3.103	6.28	1440000	3.0191
3.103	6.32	1448000	3.0168
3.1066	6.35	1456000	3.0174
3.1066	6.39	1464000	3.0256
3.1164	6.42	1472000	3.0192
3.1164	6.46	1480000	3.0066
3.1066	6.49	1488000	3.0160
3.1066	6.53	1496000	3.0187
3.1014	6.56	1504000	3.0213
3.1014	6.6	1512000	3.0170
3.1043	6.63	1520000	3.0251
3.1043	6.67	1528000	3.0157
3.1073	6.7	1536000	3.0193
3.1073	6.74	1544000	3.0174
3.1131	6.77	1552000	3.0244
3.1131	6.81	1560000	3.0210
3.1033	6.84	1568000	3.0235
3.1033	6.88	1576000	3.0189
3.1087	6.91	1584000	3.0213
3.1087	6.95	1592000	3.0196
3.1065	6.98	1600000	3.0123
3.1065	7.02	1608000	3.0229
3.1019	7.05	1616000	3.0206
3.1019	7.09	1624000	3.0216
3.1023	7.12	1632000	3.0147
3.1023	7.16	1640000	3.0227
3.0969	7.19	1648000	3.0306
3.0969	7.23	1656000	3.0179
3.1034	7.26	1664000	3.0259
3.1034	7.3	1672000	3.0237
3.1077	7.33	1680000	3.0165
3.1077	7.37	1688000	3.0213
3.0983	7.4	1696000	3.0233
3.0983	7.44	1704000	3.0224
3.1014	7.47	1712000	3.0187
3.1014	7.51	1720000	3.0207
3.1052	7.54	1728000	3.0070
3.1052	7.58	1736000	3.0236
3.1062	7.61	1744000	3.0230
3.1062	7.65	1752000	3.0190
3.0941	7.68	1760000	3.0235
3.0941	7.72	1768000	3.0134
3.0942	7.75	1776000	3.0254
3.0942	7.79	1784000	3.0154
3.1089	7.82	1792000	3.0075
3.1089	7.86	1800000	3.0065
3.1117	7.89	1808000	3.0241
3.1117	7.93	1816000	3.0098
3.0958	7.96	1824000	3.0017
3.0958	8.0	1832000	3.0100
3.1177	8.03	1840000	3.0163
3.1177	8.07	1848000	3.0100
3.097	8.1	1856000	3.0099
3.097	8.13	1864000	3.0287
3.1039	8.17	1872000	3.0107
3.1039	8.2	1880000	3.0103
3.0987	8.24	1888000	3.0200
3.0987	8.27	1896000	3.0197
3.1029	8.31	1904000	3.0141
3.1029	8.34	1912000	3.0254
3.1053	8.38	1920000	3.0128
3.1053	8.41	1928000	3.0140
3.1042	8.45	1936000	3.0233
3.1042	8.48	1944000	3.0156
3.1039	8.52	1952000	3.0125
3.1039	8.55	1960000	3.0144
3.1044	8.59	1968000	3.0247
3.1044	8.62	1976000	3.0140
3.1172	8.66	1984000	3.0106
3.1172	8.69	1992000	3.0161
3.1106	8.73	2000000	3.0168
3.1106	8.76	2008000	3.0230
3.107	8.8	2016000	3.0207
3.107	8.83	2024000	3.0218
3.1153	8.87	2032000	3.0157
3.1153	8.9	2040000	3.0326
3.1104	8.94	2048000	3.0194
3.1104	8.97	2056000	3.0211
3.1206	9.01	2064000	3.0197
3.1206	9.04	2072000	3.0311
3.1101	9.08	2080000	3.0218
3.1101	9.11	2088000	3.0224
3.1166	9.15	2096000	3.0326
3.1166	9.18	2104000	3.0252
3.106	9.22	2112000	3.0259
3.106	9.25	2120000	3.0116
3.1067	9.29	2128000	3.0312
3.1067	9.32	2136000	3.0125
3.1152	9.36	2144000	3.0147
3.1152	9.39	2152000	3.0210
3.1122	9.43	2160000	3.0388
3.1122	9.46	2168000	3.0409
3.1092	9.5	2176000	3.0364
3.1092	9.53	2184000	3.0270
3.1117	9.57	2192000	3.0326
3.1117	9.6	2200000	3.0381
3.1089	9.64	2208000	3.0258
3.1089	9.67	2216000	3.0287
3.1195	9.71	2224000	3.0326
3.1195	9.74	2232000	3.0374
3.1172	9.78	2240000	3.0227
3.1172	9.81	2248000	3.0281
3.1271	9.85	2256000	3.0274
3.1271	9.88	2264000	3.0225
3.1112	9.92	2272000	3.0248
3.1112	9.95	2280000	3.0188
3.1179	9.99	2288000	3.0227
3.1179	10.02	2296000	3.0337
3.1178	10.06	2304000	3.0241
3.1178	10.09	2312000	3.0247
3.1148	10.13	2320000	3.0342
3.1148	10.16	2328000	3.0202
3.1153	10.19	2336000	3.0294
3.1153	10.23	2344000	3.0282
3.1097	10.26	2352000	3.0198
3.1097	10.3	2360000	3.0188
3.1041	10.33	2368000	3.0225
3.1041	10.37	2376000	3.0212
3.0992	10.4	2384000	3.0208
3.0992	10.44	2392000	3.0191
3.1135	10.47	2400000	3.0245

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0