eraydikyologlu/bert_ayt_turkce_less

eraydikyologlu/bert_ayt_turkce

This model is a fine-tuned version of dbmdz/bert-base-turkish-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.8116
• Train Accuracy: 0.8616
• Validation Loss: 0.6930
• Validation Accuracy: 0.8594
• Epoch: 14

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'transformers.optimization_tf', 'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 1161, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'warmup_steps': 129, 'power': 1.0, 'name': None}, 'registered_name': 'WarmUp'}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train Accuracy	Validation Loss	Validation Accuracy	Epoch
4.0832	0.0233	3.9402	0.0508	0
3.7484	0.1148	3.2844	0.1992	1
3.0821	0.2376	2.6225	0.3711	2
2.5369	0.3728	2.1793	0.4453	3
2.1473	0.4680	1.8233	0.5508	4
1.8572	0.5603	1.5471	0.6172	5
1.6207	0.6170	1.3739	0.6758	6
1.4234	0.6784	1.1823	0.7109	7
1.2607	0.7198	1.0351	0.7461	8
1.1361	0.7653	0.9157	0.8008	9
1.0315	0.8012	0.8493	0.8047	10
0.9402	0.8307	0.7903	0.8477	11
0.8793	0.8463	0.7331	0.8477	12
0.8410	0.8565	0.6951	0.8516	13
0.8116	0.8616	0.6930	0.8594	14

Framework versions

• Transformers 4.52.4
• TensorFlow 2.18.0
• Datasets 2.14.4
• Tokenizers 0.21.1