tapt_helpfulness_unipelt_pretraining_model_fix_train

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4789

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 21
• eval_batch_size: 21
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
2.7536	1.0	1068	1.8161
1.855	2.0	2137	1.6743
1.7578	3.0	3205	1.6134
1.7056	4.0	4274	1.5780
1.6755	5.0	5342	1.5660
1.6508	6.0	6411	1.5507
1.6402	7.0	7479	1.5236
1.6226	8.0	8548	1.5272
1.6145	9.0	9616	1.4970
1.6034	10.0	10685	1.4999
1.6004	11.0	11753	1.5120
1.5916	12.0	12822	1.4882
1.5888	13.0	13890	1.4974
1.5801	14.0	14959	1.4703
1.5784	15.0	16027	1.4767
1.5738	16.0	17096	1.4668
1.5717	17.0	18164	1.4776
1.5696	18.0	19233	1.4691
1.5681	19.0	20301	1.4756
1.5658	19.99	21360	1.4789

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.15.2