tapt_helpfulness_pretraining_model_final

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5048

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 21
• eval_batch_size: 21
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
4.7974	1.0	232	2.5728
2.4546	2.0	465	2.1447
2.1813	3.0	697	1.9812
2.0532	4.0	930	1.8776
1.9775	5.0	1162	1.8295
1.9073	6.0	1395	1.7695
1.88	7.0	1627	1.7444
1.8393	8.0	1860	1.7227
1.8182	9.0	2092	1.6952
1.7903	10.0	2325	1.6860
1.7805	11.0	2557	1.6647
1.7604	12.0	2790	1.6551
1.7472	13.0	3022	1.6403
1.7328	14.0	3255	1.6328
1.7251	15.0	3487	1.6339
1.704	16.0	3720	1.6132
1.7113	17.0	3952	1.6026
1.6898	18.0	4185	1.5934
1.692	19.0	4417	1.6081
1.6787	20.0	4650	1.5891
1.679	21.0	4882	1.5877
1.6632	22.0	5115	1.5764
1.6674	23.0	5347	1.5962
1.6627	24.0	5580	1.5759
1.6613	25.0	5812	1.5627
1.6421	26.0	6045	1.5636
1.6495	27.0	6277	1.5589
1.632	28.0	6510	1.5722
1.6343	29.0	6742	1.5717
1.638	30.0	6975	1.5477
1.6327	31.0	7207	1.5498
1.6218	32.0	7440	1.5496
1.6258	33.0	7672	1.5408
1.6205	34.0	7905	1.5361
1.6208	35.0	8137	1.5435
1.6105	36.0	8370	1.5285
1.6168	37.0	8602	1.5367
1.605	38.0	8835	1.5390
1.6139	39.0	9067	1.5382
1.599	40.0	9300	1.5283
1.6045	41.0	9532	1.5415
1.5947	42.0	9765	1.5284
1.6015	43.0	9997	1.5302
1.5952	44.0	10230	1.5434
1.5996	45.0	10462	1.5372
1.5924	46.0	10695	1.5147
1.5924	47.0	10927	1.5154
1.5867	48.0	11160	1.5339
1.5872	49.0	11392	1.5285
1.5829	50.0	11625	1.5139
1.5907	51.0	11857	1.5197
1.5841	52.0	12090	1.5263
1.5862	53.0	12322	1.4993
1.5728	54.0	12555	1.5045
1.5825	55.0	12787	1.5078
1.5729	56.0	13020	1.5030
1.5826	57.0	13252	1.5112
1.5718	58.0	13485	1.5157
1.5807	59.0	13717	1.5248
1.5725	60.0	13950	1.5161
1.5735	61.0	14182	1.5059
1.5694	62.0	14415	1.5198
1.5709	63.0	14647	1.5046

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2