StatementOfWork_Generator_Omega_BS_512

This model is a fine-tuned version of distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7165

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 50
• eval_batch_size: 50
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	4	0.8137
No log	2.0	8	0.7930
No log	3.0	12	0.7915
No log	4.0	16	0.7846
No log	5.0	20	0.7785
No log	6.0	24	0.7751
No log	7.0	28	0.7724
No log	8.0	32	0.7687
No log	9.0	36	0.7654
No log	10.0	40	0.7630
No log	11.0	44	0.7615
No log	12.0	48	0.7604
No log	13.0	52	0.7595
No log	14.0	56	0.7572
No log	15.0	60	0.7559
No log	16.0	64	0.7549
No log	17.0	68	0.7541
No log	18.0	72	0.7541
No log	19.0	76	0.7522
No log	20.0	80	0.7496
No log	21.0	84	0.7485
No log	22.0	88	0.7492
No log	23.0	92	0.7481
No log	24.0	96	0.7469
No log	25.0	100	0.7447
No log	26.0	104	0.7436
No log	27.0	108	0.7435
No log	28.0	112	0.7431
No log	29.0	116	0.7426
No log	30.0	120	0.7418
No log	31.0	124	0.7414
No log	32.0	128	0.7409
No log	33.0	132	0.7393
No log	34.0	136	0.7393
No log	35.0	140	0.7392
No log	36.0	144	0.7368
No log	37.0	148	0.7361
No log	38.0	152	0.7366
No log	39.0	156	0.7367
No log	40.0	160	0.7349
No log	41.0	164	0.7344
No log	42.0	168	0.7341
No log	43.0	172	0.7328
No log	44.0	176	0.7317
No log	45.0	180	0.7314
No log	46.0	184	0.7325
No log	47.0	188	0.7325
No log	48.0	192	0.7311
No log	49.0	196	0.7301
No log	50.0	200	0.7306
No log	51.0	204	0.7305
No log	52.0	208	0.7301
No log	53.0	212	0.7286
No log	54.0	216	0.7266
No log	55.0	220	0.7275
No log	56.0	224	0.7292
No log	57.0	228	0.7288
No log	58.0	232	0.7275
No log	59.0	236	0.7263
No log	60.0	240	0.7265
No log	61.0	244	0.7269
No log	62.0	248	0.7271
No log	63.0	252	0.7262
No log	64.0	256	0.7252
No log	65.0	260	0.7262
No log	66.0	264	0.7251
No log	67.0	268	0.7243
No log	68.0	272	0.7241
No log	69.0	276	0.7248
No log	70.0	280	0.7238
No log	71.0	284	0.7227
No log	72.0	288	0.7234
No log	73.0	292	0.7233
No log	74.0	296	0.7222
No log	75.0	300	0.7213
No log	76.0	304	0.7223
No log	77.0	308	0.7230
No log	78.0	312	0.7226
No log	79.0	316	0.7222
No log	80.0	320	0.7220
No log	81.0	324	0.7222
No log	82.0	328	0.7220
No log	83.0	332	0.7209
No log	84.0	336	0.7202
No log	85.0	340	0.7208
No log	86.0	344	0.7218
No log	87.0	348	0.7214
No log	88.0	352	0.7195
No log	89.0	356	0.7190
No log	90.0	360	0.7199
No log	91.0	364	0.7213
No log	92.0	368	0.7207
No log	93.0	372	0.7198
No log	94.0	376	0.7198
No log	95.0	380	0.7200
No log	96.0	384	0.7199
No log	97.0	388	0.7198
No log	98.0	392	0.7198
No log	99.0	396	0.7194
No log	100.0	400	0.7192
No log	101.0	404	0.7187
No log	102.0	408	0.7181
No log	103.0	412	0.7186
No log	104.0	416	0.7192
No log	105.0	420	0.7184
No log	106.0	424	0.7178
No log	107.0	428	0.7181
No log	108.0	432	0.7188
No log	109.0	436	0.7192
No log	110.0	440	0.7189
No log	111.0	444	0.7186
No log	112.0	448	0.7183
No log	113.0	452	0.7181
No log	114.0	456	0.7184
No log	115.0	460	0.7187
No log	116.0	464	0.7186
No log	117.0	468	0.7180
No log	118.0	472	0.7178
No log	119.0	476	0.7176
No log	120.0	480	0.7175
No log	121.0	484	0.7171
No log	122.0	488	0.7170
No log	123.0	492	0.7172
No log	124.0	496	0.7173
0.2218	125.0	500	0.7174
0.2218	126.0	504	0.7173
0.2218	127.0	508	0.7174
0.2218	128.0	512	0.7174
0.2218	129.0	516	0.7174
0.2218	130.0	520	0.7171
0.2218	131.0	524	0.7167
0.2218	132.0	528	0.7166
0.2218	133.0	532	0.7169
0.2218	134.0	536	0.7171
0.2218	135.0	540	0.7172
0.2218	136.0	544	0.7171
0.2218	137.0	548	0.7168
0.2218	138.0	552	0.7166
0.2218	139.0	556	0.7165
0.2218	140.0	560	0.7165
0.2218	141.0	564	0.7165
0.2218	142.0	568	0.7165
0.2218	143.0	572	0.7166
0.2218	144.0	576	0.7166
0.2218	145.0	580	0.7166
0.2218	146.0	584	0.7166
0.2218	147.0	588	0.7166
0.2218	148.0	592	0.7166
0.2218	149.0	596	0.7165
0.2218	150.0	600	0.7165

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2