README.md

metadata

license: mit
tags:
  - generated_from_keras_callback
model-index:
  - name: Ashraf-kasem/custom_gpt2_frames_text_continue
    results: []

Ashraf-kasem/custom_gpt2_frames_text_continue

This model is a fine-tuned version of Ashraf-kasem/custom_gpt2_frames_text_continue on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.0317
• Validation Loss: 2.1324
• Epoch: 34

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': {'class_name': 'LinearWarmup', 'config': {'after_warmup_lr_sched': {'initial_learning_rate': 5e-05, 'decay_steps': 628900, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'warmup_steps': 125780, 'warmup_learning_rate': 0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: mixed_float16

Training results

Train Loss	Validation Loss	Epoch
1.0060	2.0768	0
1.0147	2.0771	1
1.0238	2.0821	2
1.0331	2.0851	3
1.0422	2.0870	4
1.0525	2.0945	5
1.0618	2.1005	6
1.0718	2.1014	7
1.0823	2.1056	8
1.0921	2.1099	9
1.1028	2.1106	10
1.1127	2.1127	11
1.1230	2.1183	12
1.1329	2.1207	13
1.1423	2.1270	14
1.1521	2.1234	15
1.1614	2.1283	16
1.1700	2.1236	17
1.1784	2.1320	18
1.1864	2.1359	19
1.1873	2.1272	20
1.1766	2.1250	21
1.1652	2.1260	22
1.1537	2.1224	23
1.1415	2.1278	24
1.1296	2.1254	25
1.1178	2.1213	26
1.1059	2.1301	27
1.0950	2.1253	28
1.0838	2.1264	29
1.0729	2.1273	30
1.0625	2.1355	31
1.0519	2.1345	32
1.0414	2.1364	33
1.0317	2.1324	34

Framework versions

• Transformers 4.25.1
• TensorFlow 2.9.0
• Datasets 2.8.0
• Tokenizers 0.13.2