---
license: other
base_model: microsoft/phi-1_5
tags:
- generated_from_trainer
model-index:
- name: phi-1_5-finetuned-SQL
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# phi-1_5-finetuned-SQL

This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3403

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 6000

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.4485        | 0.4   | 100  | 2.0478          |
| 2.0521        | 0.8   | 200  | 1.9223          |
| 1.9626        | 1.2   | 300  | 1.8386          |
| 1.8707        | 1.6   | 400  | 1.7702          |
| 1.79          | 2.0   | 500  | 1.7149          |
| 1.7197        | 2.4   | 600  | 1.6567          |
| 1.6904        | 2.8   | 700  | 1.6055          |
| 1.6379        | 3.2   | 800  | 1.5583          |
| 1.5794        | 3.6   | 900  | 1.5267          |
| 1.5977        | 4.0   | 1000 | 1.4928          |
| 1.4773        | 4.4   | 1100 | 1.4638          |
| 1.5185        | 4.8   | 1200 | 1.4446          |
| 1.4476        | 5.2   | 1300 | 1.4337          |
| 1.4321        | 5.6   | 1400 | 1.4287          |
| 1.4393        | 6.0   | 1500 | 1.4282          |
| 1.4956        | 6.4   | 1600 | 1.4504          |
| 1.5252        | 6.8   | 1700 | 1.4311          |
| 1.4864        | 7.2   | 1800 | 1.3654          |
| 1.4092        | 7.6   | 1900 | 1.3112          |
| 1.4063        | 8.0   | 2000 | 1.2925          |
| 1.2657        | 8.4   | 2100 | 1.2123          |
| 1.312         | 8.8   | 2200 | 1.1824          |
| 1.2451        | 9.2   | 2300 | 1.1223          |
| 1.1777        | 9.6   | 2400 | 1.0857          |
| 1.1913        | 10.0  | 2500 | 1.0422          |
| 1.0452        | 10.4  | 2600 | 0.9842          |
| 1.082         | 10.8  | 2700 | 0.9442          |
| 0.9814        | 11.2  | 2800 | 0.9002          |
| 0.9496        | 11.6  | 2900 | 0.8559          |
| 0.9639        | 12.0  | 3000 | 0.8163          |
| 0.823         | 12.4  | 3100 | 0.7827          |
| 0.8395        | 12.8  | 3200 | 0.7384          |
| 0.8038        | 13.2  | 3300 | 0.6971          |
| 0.7458        | 13.6  | 3400 | 0.6641          |
| 0.7495        | 14.0  | 3500 | 0.6328          |
| 0.6575        | 14.4  | 3600 | 0.6017          |
| 0.6448        | 14.8  | 3700 | 0.5829          |
| 0.6268        | 15.2  | 3800 | 0.5412          |
| 0.5738        | 15.6  | 3900 | 0.5233          |
| 0.5989        | 16.0  | 4000 | 0.5008          |
| 0.5033        | 16.4  | 4100 | 0.4781          |
| 0.5343        | 16.8  | 4200 | 0.4572          |
| 0.4881        | 17.2  | 4300 | 0.4390          |
| 0.4676        | 17.6  | 4400 | 0.4254          |
| 0.4683        | 18.0  | 4500 | 0.4171          |
| 0.4188        | 18.4  | 4600 | 0.3987          |
| 0.4245        | 18.8  | 4700 | 0.3869          |
| 0.4136        | 19.2  | 4800 | 0.3777          |
| 0.3938        | 19.6  | 4900 | 0.3694          |
| 0.3986        | 20.0  | 5000 | 0.3627          |
| 0.3661        | 20.4  | 5100 | 0.3571          |
| 0.3743        | 20.8  | 5200 | 0.3516          |
| 0.3668        | 21.2  | 5300 | 0.3482          |
| 0.3613        | 21.6  | 5400 | 0.3455          |
| 0.3542        | 22.0  | 5500 | 0.3430          |
| 0.3505        | 22.4  | 5600 | 0.3419          |
| 0.3495        | 22.8  | 5700 | 0.3410          |
| 0.3396        | 23.2  | 5800 | 0.3405          |
| 0.3481        | 23.6  | 5900 | 0.3403          |
| 0.3444        | 24.0  | 6000 | 0.3403          |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1