README.md

metadata

license: apache-2.0
base_model: bert-base-uncased
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
model-index:
  - name: bert_base_for_whole_train_result_Spam-Ham2_2
    results: []

bert_base_for_whole_train_result_Spam-Ham2_2

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0280
• Accuracy: 0.996
• F1: 0.9962

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 4096
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.7292	6.8817	50	0.5027	0.876	0.8813
0.2104	13.7634	100	0.0612	0.98	0.9809
0.0129	20.6452	150	0.0336	0.9915	0.9920
0.004	27.5269	200	0.0500	0.987	0.9877
0.0023	34.4086	250	0.0293	0.9955	0.9958
0.0017	41.2903	300	0.0339	0.9945	0.9948
0.0017	48.1720	350	0.0327	0.9945	0.9948
0.001	55.0538	400	0.0345	0.995	0.9953
0.0006	61.9355	450	0.0357	0.9945	0.9948
0.0002	68.8172	500	0.0390	0.9945	0.9948
0.0001	75.6989	550	0.0402	0.9945	0.9948
0.0001	82.5806	600	0.0408	0.9945	0.9948
0.0015	89.4624	650	0.0353	0.995	0.9953
0.0011	96.3441	700	0.0280	0.996	0.9962

Framework versions

• Transformers 4.40.0
• Pytorch 2.4.1+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1