mitre-bert-base-uncased

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1257

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
2.0301	2.72	1000	1.6360
1.4454	5.45	2000	1.4483
1.2259	8.17	3000	1.3658
1.0811	10.9	4000	1.2872
0.9744	13.62	5000	1.2763
0.8601	16.35	6000	1.2343
0.7918	19.07	7000	1.1792
0.7375	21.8	8000	1.1792
0.6806	24.52	9000	1.1332
0.6274	27.25	10000	1.1129
0.5803	29.97	11000	1.1239
0.5403	32.7	12000	1.1224
0.5028	35.42	13000	1.1344
0.4661	38.15	14000	1.1215
0.4463	40.87	15000	1.1453
0.4102	43.6	16000	1.0828
0.3866	46.32	17000	1.1044
0.3749	49.05	18000	1.1199
0.3554	51.77	19000	1.0722
0.3288	54.5	20000	1.1167
0.319	57.22	21000	1.1110
0.3064	59.95	22000	1.0821
0.2938	62.67	23000	1.0702
0.2741	65.4	24000	1.0373
0.2626	68.12	25000	1.1236
0.2525	70.84	26000	1.0831
0.2433	73.57	27000	1.0691
0.2376	76.29	28000	1.0647
0.2303	79.02	29000	1.0851
0.2164	81.74	30000	1.0864
0.2103	84.47	31000	1.0502
0.2049	87.19	32000	1.0644
0.2011	89.92	33000	1.0262
0.2024	92.64	34000	1.0738
0.1889	95.37	35000	1.0640
0.1846	98.09	36000	1.1257

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2