metadata
license: other
base_model: microsoft/phi-1_5
tags:
- generated_from_trainer
model-index:
- name: phi-1_5-finetuned-SQL
results: []
phi-1_5-finetuned-SQL
This model is a fine-tuned version of microsoft/phi-1_5 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.3403
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 6000
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.4485 | 0.4 | 100 | 2.0478 |
| 2.0521 | 0.8 | 200 | 1.9223 |
| 1.9626 | 1.2 | 300 | 1.8386 |
| 1.8707 | 1.6 | 400 | 1.7702 |
| 1.79 | 2.0 | 500 | 1.7149 |
| 1.7197 | 2.4 | 600 | 1.6567 |
| 1.6904 | 2.8 | 700 | 1.6055 |
| 1.6379 | 3.2 | 800 | 1.5583 |
| 1.5794 | 3.6 | 900 | 1.5267 |
| 1.5977 | 4.0 | 1000 | 1.4928 |
| 1.4773 | 4.4 | 1100 | 1.4638 |
| 1.5185 | 4.8 | 1200 | 1.4446 |
| 1.4476 | 5.2 | 1300 | 1.4337 |
| 1.4321 | 5.6 | 1400 | 1.4287 |
| 1.4393 | 6.0 | 1500 | 1.4282 |
| 1.4956 | 6.4 | 1600 | 1.4504 |
| 1.5252 | 6.8 | 1700 | 1.4311 |
| 1.4864 | 7.2 | 1800 | 1.3654 |
| 1.4092 | 7.6 | 1900 | 1.3112 |
| 1.4063 | 8.0 | 2000 | 1.2925 |
| 1.2657 | 8.4 | 2100 | 1.2123 |
| 1.312 | 8.8 | 2200 | 1.1824 |
| 1.2451 | 9.2 | 2300 | 1.1223 |
| 1.1777 | 9.6 | 2400 | 1.0857 |
| 1.1913 | 10.0 | 2500 | 1.0422 |
| 1.0452 | 10.4 | 2600 | 0.9842 |
| 1.082 | 10.8 | 2700 | 0.9442 |
| 0.9814 | 11.2 | 2800 | 0.9002 |
| 0.9496 | 11.6 | 2900 | 0.8559 |
| 0.9639 | 12.0 | 3000 | 0.8163 |
| 0.823 | 12.4 | 3100 | 0.7827 |
| 0.8395 | 12.8 | 3200 | 0.7384 |
| 0.8038 | 13.2 | 3300 | 0.6971 |
| 0.7458 | 13.6 | 3400 | 0.6641 |
| 0.7495 | 14.0 | 3500 | 0.6328 |
| 0.6575 | 14.4 | 3600 | 0.6017 |
| 0.6448 | 14.8 | 3700 | 0.5829 |
| 0.6268 | 15.2 | 3800 | 0.5412 |
| 0.5738 | 15.6 | 3900 | 0.5233 |
| 0.5989 | 16.0 | 4000 | 0.5008 |
| 0.5033 | 16.4 | 4100 | 0.4781 |
| 0.5343 | 16.8 | 4200 | 0.4572 |
| 0.4881 | 17.2 | 4300 | 0.4390 |
| 0.4676 | 17.6 | 4400 | 0.4254 |
| 0.4683 | 18.0 | 4500 | 0.4171 |
| 0.4188 | 18.4 | 4600 | 0.3987 |
| 0.4245 | 18.8 | 4700 | 0.3869 |
| 0.4136 | 19.2 | 4800 | 0.3777 |
| 0.3938 | 19.6 | 4900 | 0.3694 |
| 0.3986 | 20.0 | 5000 | 0.3627 |
| 0.3661 | 20.4 | 5100 | 0.3571 |
| 0.3743 | 20.8 | 5200 | 0.3516 |
| 0.3668 | 21.2 | 5300 | 0.3482 |
| 0.3613 | 21.6 | 5400 | 0.3455 |
| 0.3542 | 22.0 | 5500 | 0.3430 |
| 0.3505 | 22.4 | 5600 | 0.3419 |
| 0.3495 | 22.8 | 5700 | 0.3410 |
| 0.3396 | 23.2 | 5800 | 0.3405 |
| 0.3481 | 23.6 | 5900 | 0.3403 |
| 0.3444 | 24.0 | 6000 | 0.3403 |
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1