bert-finetuned-combo-textbook

This model is a fine-tuned version of google-bert/bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6464

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
2.3141	1.0	2468	2.0738
2.1063	2.0	4936	2.0016
2.0241	3.0	7404	1.9165
1.9769	4.0	9872	nan
1.897	5.0	12340	1.8370
1.8501	6.0	14808	1.8046
1.8182	7.0	17276	1.7854
1.7648	8.0	19744	1.7328
1.7409	9.0	22212	1.7238
1.7486	10.0	24680	1.7043
1.6753	11.0	27148	1.7076
1.6833	12.0	29616	1.6704
1.6684	13.0	32084	1.6873
1.6092	14.0	34552	1.6702
1.6077	15.0	37020	1.6972
1.6271	16.0	39488	1.6379
1.6159	17.0	41956	1.6624
1.5678	18.0	44424	1.6646
1.5857	19.0	46892	1.6290
1.5741	20.0	49360	1.6464

Framework versions

• Transformers 4.38.2
• Pytorch 1.13.1+cu116
• Datasets 2.18.0
• Tokenizers 0.15.2