roberta-base-atomic-anion-1e-06-256

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
0.5668	1.0	1152	0.5247
0.5001	2.0	2304	0.4849
0.4711	3.0	3456	0.4622
0.4543	4.0	4608	0.4461
0.4415	5.0	5760	0.4351
0.4256	6.0	6912	0.4262
0.4139	7.0	8064	0.4184
0.4082	8.0	9216	0.4153
0.3978	9.0	10368	0.4096
0.391	10.0	11520	0.4041
0.3809	11.0	12672	0.4000
0.379	12.0	13824	0.3965
0.3766	13.0	14976	0.3988
0.3676	14.0	16128	0.3920
0.3658	15.0	17280	0.3913
0.3587	16.0	18432	0.3872
0.3584	17.0	19584	0.3856
0.3554	18.0	20736	0.3832
0.3531	19.0	21888	0.3865
0.3503	20.0	23040	0.3828
0.3475	21.0	24192	0.3817
0.3428	22.0	25344	0.3807
0.3413	23.0	26496	0.3785
0.3376	24.0	27648	0.3811
0.3388	25.0	28800	0.3784
0.3356	26.0	29952	0.3797
0.3352	27.0	31104	0.3805
0.3374	28.0	32256	0.3782
0.3327	29.0	33408	0.3789
0.338	30.0	34560	0.3782

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1