modernbert-CGEdit-AAE_last

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5197

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 50
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss
0.7929	1.0	26	0.7579
0.6599	2.0	52	0.6395
0.5991	3.0	78	0.5794
0.5559	4.0	104	0.5509
0.5343	5.0	130	0.5384
0.5255	6.0	156	0.5280
0.5176	7.0	182	0.5264
0.5198	8.0	208	0.5236
0.5166	9.0	234	0.5228
0.5161	10.0	260	0.5218
0.5154	11.0	286	0.5218
0.5191	12.0	312	0.5211
0.5241	13.0	338	0.5244
0.5191	14.0	364	0.5241
0.5246	15.0	390	0.5217
0.5131	16.0	416	0.5209
0.5158	17.0	442	0.5207
0.5135	18.0	468	0.5204
0.5124	19.0	494	0.5204
0.5092	20.0	520	0.5205
0.5199	21.0	546	0.5201
0.5240	22.0	572	0.5201
0.5252	23.0	598	0.5200
0.5134	24.0	624	0.5209
0.5182	25.0	650	0.5212
0.5106	26.0	676	0.5198
0.5166	27.0	702	0.5198
0.5146	28.0	728	0.5196
0.5127	29.0	754	0.5197
0.5190	30.0	780	0.5198
0.5198	31.0	806	0.5197
0.5082	32.0	832	0.5198
0.5185	33.0	858	0.5198
0.5177	34.0	884	0.5198
0.5174	35.0	910	0.5197
0.5171	36.0	936	0.5197
0.5178	37.0	962	0.5197
0.5183	38.0	988	0.5197
0.5205	39.0	1014	0.5197
0.5228	40.0	1040	0.5197
0.5197	41.0	1066	0.5197
0.5182	42.0	1092	0.5197
0.5170	43.0	1118	0.5197
0.5171	44.0	1144	0.5197
0.5246	45.0	1170	0.5197
0.5191	46.0	1196	0.5197
0.5131	47.0	1222	0.5197
0.5130	48.0	1248	0.5197
0.5121	49.0	1274	0.5197
0.5223	50.0	1300	0.5197

Framework versions

• Transformers 5.0.0
• Pytorch 2.5.1+cu121
• Tokenizers 0.22.1