tapt_helpfulness_base_pretraining_model_final

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4543

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 21
• eval_batch_size: 21
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
1.7697	1.0	232	1.5904
1.6633	2.0	465	1.5650
1.6314	3.0	697	1.5461
1.594	4.0	930	1.5243
1.5766	5.0	1162	1.5312
1.5451	6.0	1395	1.5194
1.5271	7.0	1627	1.5034
1.5038	8.0	1860	1.5080
1.4906	9.0	2092	1.4942
1.4801	10.0	2325	1.4783
1.4638	11.0	2557	1.4900
1.4407	12.0	2790	1.4820
1.4285	13.0	3022	1.4692
1.4177	14.0	3255	1.4698
1.4051	15.0	3487	1.4790
1.3899	16.0	3720	1.4800
1.3832	17.0	3952	1.4730
1.3706	18.0	4185	1.4656
1.3617	19.0	4417	1.4625
1.3464	20.0	4650	1.4699
1.3449	21.0	4882	1.4641
1.3258	22.0	5115	1.4554
1.3248	23.0	5347	1.4595
1.3119	24.0	5580	1.4643
1.3087	25.0	5812	1.4589
1.2942	26.0	6045	1.4633
1.2875	27.0	6277	1.4517
1.2731	28.0	6510	1.4506
1.2727	29.0	6742	1.4501
1.261	30.0	6975	1.4492
1.2559	31.0	7207	1.4553
1.2437	32.0	7440	1.4429
1.2404	33.0	7672	1.4456
1.2301	34.0	7905	1.4497
1.2277	35.0	8137	1.4400
1.2154	36.0	8370	1.4491
1.2118	37.0	8602	1.4521
1.2022	38.0	8835	1.4362
1.2027	39.0	9067	1.4431
1.1883	40.0	9300	1.4526
1.1861	41.0	9532	1.4596
1.1747	42.0	9765	1.4390
1.1708	43.0	9997	1.4501
1.1636	44.0	10230	1.4549
1.1623	45.0	10462	1.4616
1.1569	46.0	10695	1.4379
1.149	47.0	10927	1.4492
1.1401	48.0	11160	1.4502

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2