svenbl80/roberta-base-finetuned-chatdoc-V3

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.6956
• Validation Loss: 0.4497
• Train Accuracy: 0.8652
• Epoch: 28

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-06, 'decay_steps': 330, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Accuracy	Epoch
1.1344	1.1124	0.1236	0
1.0978	1.0641	0.8652	1
1.0575	1.0020	0.8652	2
0.9999	0.9336	0.8652	3
0.9391	0.8170	0.8652	4
0.8501	0.6621	0.8652	5
0.7780	0.5321	0.8652	6
0.7866	0.4850	0.8652	7
0.7613	0.4796	0.8652	8
0.7512	0.4847	0.8652	9
0.7432	0.4933	0.8652	10
0.7474	0.4919	0.8652	11
0.7580	0.4863	0.8652	12
0.7253	0.4840	0.8652	13
0.7166	0.4724	0.8652	14
0.7245	0.4725	0.8652	15
0.7144	0.4706	0.8652	16
0.6870	0.4628	0.8652	17
0.6925	0.4583	0.8652	18
0.6945	0.4620	0.8652	19
0.6930	0.4564	0.8652	20
0.6737	0.4572	0.8652	21
0.6809	0.4496	0.8652	22
0.6766	0.4523	0.8652	23
0.7007	0.4525	0.8652	24
0.6945	0.4538	0.8652	25
0.6980	0.4521	0.8652	26
0.6769	0.4508	0.8652	27
0.6956	0.4497	0.8652	28

Framework versions

• Transformers 4.28.0
• TensorFlow 2.9.1
• Datasets 2.15.0
• Tokenizers 0.13.3