tapt_helpfulness_unipelt_pretraining_model

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 21
• eval_batch_size: 21
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
4.7846	1.0	232	2.5050
2.4019	2.0	465	2.0958
2.1426	3.0	697	1.9449
2.0218	4.0	930	1.8522
1.9507	5.0	1162	1.8076
1.8903	6.0	1395	1.7539
1.8596	7.0	1627	1.7311
1.8213	8.0	1860	1.7060
1.8026	9.0	2092	1.6792
1.7766	10.0	2325	1.6698
1.7661	11.0	2557	1.6504
1.7469	12.0	2790	1.6404
1.7332	13.0	3022	1.6280
1.72	14.0	3255	1.6189
1.7135	15.0	3487	1.6213
1.6895	16.0	3720	1.6018
1.6974	17.0	3952	1.5918
1.6777	18.0	4185	1.5808
1.6794	19.0	4417	1.5978
1.6657	20.0	4650	1.5771
1.6661	21.0	4882	1.5759
1.6508	22.0	5115	1.5634
1.6535	23.0	5347	1.5830
1.649	24.0	5580	1.5624
1.6454	25.0	5812	1.5526
1.629	26.0	6045	1.5491
1.6356	27.0	6277	1.5479
1.6186	28.0	6510	1.5614
1.621	29.0	6742	1.5604
1.6255	30.0	6975	1.5376
1.6178	31.0	7207	1.5383
1.6096	32.0	7440	1.5390
1.6128	33.0	7672	1.5304
1.6068	34.0	7905	1.5242
1.6073	35.0	8137	1.5331
1.5983	36.0	8370	1.5167
1.6041	37.0	8602	1.5245
1.5926	38.0	8835	1.5294
1.6024	39.0	9067	1.5280
1.5868	40.0	9300	1.5183
1.592	41.0	9532	1.5302

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2