README.md

metadata

license: apache-2.0
base_model: distilbert-base-uncased
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: bert_sql_classfication
    results: []

bert_sql_classfication

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7151
• Accuracy: 0.8624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.5598	1.0	62	0.5919	0.8073
0.2769	2.0	124	0.5313	0.8349
0.3082	3.0	186	0.3577	0.9083
0.0916	4.0	248	0.6712	0.8349
0.0708	5.0	310	0.5667	0.8257
0.0715	6.0	372	0.5633	0.8807
0.0589	7.0	434	0.8306	0.8440
0.0022	8.0	496	0.6568	0.8624
0.005	9.0	558	0.6176	0.8716
0.0017	10.0	620	0.6524	0.8440
0.001	11.0	682	0.8272	0.8532
0.0008	12.0	744	0.5695	0.8991
0.0006	13.0	806	0.5963	0.8991
0.0005	14.0	868	0.6273	0.8899
0.0006	15.0	930	0.6720	0.8899
0.0005	16.0	992	0.6864	0.8716
0.0003	17.0	1054	0.6928	0.8624
0.0004	18.0	1116	0.6931	0.8624
0.0003	19.0	1178	0.6874	0.8624
0.0003	20.0	1240	0.6688	0.8624
0.0003	21.0	1302	0.6989	0.8624
0.0003	22.0	1364	0.7050	0.8624
0.0003	23.0	1426	0.7038	0.8624
0.0002	24.0	1488	0.7083	0.8624
0.0002	25.0	1550	0.7134	0.8624
0.0002	26.0	1612	0.7133	0.8624
0.0002	27.0	1674	0.7151	0.8624
0.0002	28.0	1736	0.7155	0.8624
0.0002	29.0	1798	0.7135	0.8624
0.0002	30.0	1860	0.7151	0.8624

Framework versions

• Transformers 4.36.2
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0