bert_base_for_whole_train_result_Spam-Ham_farshad_1_2

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0515
• Accuracy: 0.9922
• F1: 0.9924

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 4096
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.648	2.9250	50	0.4089	0.8854	0.8855
0.2137	5.8501	100	0.0941	0.9701	0.9706
0.05	8.7751	150	0.0435	0.9861	0.9865
0.0216	11.7002	200	0.0404	0.9884	0.9888
0.012	14.6252	250	0.0633	0.9823	0.9827
0.0075	17.5503	300	0.0397	0.9916	0.9919
0.0071	20.4753	350	0.0582	0.9861	0.9864
0.0047	23.4004	400	0.0486	0.9884	0.9887
0.0026	26.3254	450	0.0483	0.9913	0.9916
0.0044	29.2505	500	0.0453	0.9916	0.9919
0.002	32.1755	550	0.0452	0.9913	0.9916
0.0023	35.1005	600	0.0381	0.9927	0.9930
0.0033	38.0256	650	0.1895	0.9646	0.9647
0.0029	40.9506	700	0.0453	0.9916	0.9919
0.0012	43.8757	750	0.0497	0.9907	0.9910
0.0011	46.8007	800	0.0509	0.9919	0.9921
0.0012	49.7258	850	0.0524	0.9916	0.9918
0.0008	52.6508	900	0.0377	0.9925	0.9927
0.0019	55.5759	950	0.0580	0.9887	0.9890
0.0007	58.5009	1000	0.0608	0.9913	0.9916
0.0008	61.4260	1050	0.0584	0.9916	0.9919
0.0006	64.3510	1100	0.0515	0.9922	0.9924

Framework versions

• Transformers 4.40.0
• Pytorch 2.4.1+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1