svenbl80/roberta-base-finetuned-mnli

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0037
• Validation Loss: 0.9286
• Train Accuracy: 0.8700
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 736290, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Accuracy	Epoch
0.4482	0.3946	0.8503	0
0.3320	0.3731	0.8635	1
0.2567	0.4062	0.8669	2
0.1970	0.4162	0.8693	3
0.1533	0.5317	0.8555	4
0.1237	0.5096	0.8651	5
0.1004	0.4955	0.8685	6
0.0832	0.5596	0.8606	7
0.0709	0.6180	0.8614	8
0.0613	0.6332	0.8613	9
0.0518	0.6511	0.8659	10
0.0449	0.6879	0.8642	11
0.0398	0.6564	0.8648	12
0.0351	0.6766	0.8697	13
0.0308	0.6751	0.8679	14
0.0271	0.7071	0.8608	15
0.0240	0.7776	0.8672	16
0.0212	0.7626	0.8654	17
0.0187	0.7747	0.8673	18
0.0166	0.7900	0.8681	19
0.0141	0.7816	0.8651	20
0.0125	0.8098	0.8683	21
0.0108	0.8284	0.8706	22
0.0091	0.8558	0.8654	23
0.0079	0.8508	0.8716	24
0.0066	0.8415	0.8705	25
0.0056	0.8841	0.8680	26
0.0049	0.8849	0.8698	27
0.0040	0.9275	0.8693	28
0.0037	0.9286	0.8700	29

Framework versions

• Transformers 4.28.0
• TensorFlow 2.7.0
• Datasets 2.3.2
• Tokenizers 0.12.1