distil_task_C

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 462.2090
• Mae: 10.1956

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 18
• eval_batch_size: 18
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Mae
No log	1.0	203	7191.6465	55.0685
No log	2.0	406	5555.1792	43.4994
7733.7615	3.0	609	4179.3813	35.0103
7733.7615	4.0	812	3130.1318	27.5621
4115.3178	5.0	1015	2453.7505	23.4578
4115.3178	6.0	1218	1842.0889	19.4926
4115.3178	7.0	1421	1518.6580	17.3858
2159.565	8.0	1624	1207.9269	14.8793
2159.565	9.0	1827	1017.0433	13.6738
1328.2381	10.0	2030	1088.5262	15.4227
1328.2381	11.0	2233	758.3786	11.8081
1328.2381	12.0	2436	789.0392	12.4503
902.9826	13.0	2639	640.3048	11.4809
902.9826	14.0	2842	568.1979	10.7808
702.5015	15.0	3045	532.1036	10.4821
702.5015	16.0	3248	505.8835	10.4420
702.5015	17.0	3451	511.9482	10.7153
595.909	18.0	3654	503.0000	10.6231
595.909	19.0	3857	483.5316	10.4885
485.5158	20.0	4060	462.2090	10.1956

Framework versions

• Transformers 4.36.2
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0