EricPeter/distilbert-base-cased-distilled-squad

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.1445
• Epoch: 45

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'inner_optimizer': {'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3650, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}}, 'dynamic': True, 'initial_scale': 32768.0, 'dynamic_growth_steps': 2000}
• training_precision: mixed_float16

Training results

Train Loss	Epoch
0.1520	0
0.1786	1
0.1719	2
0.1664	3
0.1531	4
0.1482	5
0.1488	6
0.1515	7
0.1520	8
0.1661	9
0.1510	10
0.1485	11
0.1489	12
0.1487	13
0.1487	14
0.1488	15
0.1516	16
0.1474	17
0.1494	18
0.1468	19
0.1466	20
0.1525	21
0.1494	22
0.1462	23
0.1466	24
0.1457	25
0.1463	26
0.1460	27
0.1450	28
0.1476	29
0.1464	30
0.1458	31
0.1482	32
0.1462	33
0.1479	34
0.1479	35
0.1468	36
0.1485	37
0.1466	38
0.1476	39
0.1461	40
0.1479	41
0.1475	42
0.1454	43
0.1468	44
0.1445	45

Framework versions

• Transformers 4.31.0
• TensorFlow 2.12.0
• Datasets 2.14.4
• Tokenizers 0.13.3