train-dir

This model is a fine-tuned version of microsoft/phi-2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.6200

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
3.7486	0.0190	25	3.4551
3.4581	0.0379	50	3.2213
3.239	0.0569	75	3.0629
3.1169	0.0758	100	2.9433
2.9742	0.0948	125	2.8331
2.8096	0.1137	150	2.7640
2.7135	0.1327	175	2.7206
2.7755	0.1516	200	2.6963
2.7801	0.1706	225	2.6765
2.7129	0.1895	250	2.6682
2.7635	0.2085	275	2.6622
2.7921	0.2274	300	2.6552
2.7238	0.2464	325	2.6459
2.6943	0.2654	350	2.6383
2.6689	0.2843	375	2.6321
2.7476	0.3033	400	2.6276
2.6384	0.3222	425	2.6254
2.7073	0.3412	450	2.6211
2.6845	0.3601	475	2.6210
2.7525	0.3791	500	2.6200

Framework versions

• PEFT 0.14.0
• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0