fine_tune_e2e

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3 on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 1.2216

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00025
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
1.3113	0.0926	50	1.2471
1.2364	0.1852	100	1.2382
1.2241	0.2778	150	1.2332
1.2608	0.3704	200	1.2313
1.2333	0.4630	250	1.2288
1.2352	0.5556	300	1.2272
1.2323	0.6481	350	1.2256
1.2482	0.7407	400	1.2238
1.2104	0.8333	450	1.2227
1.2348	0.9259	500	1.2208
1.2176	1.0185	550	1.2208
1.1914	1.1111	600	1.2219
1.1972	1.2037	650	1.2230
1.1815	1.2963	700	1.2226
1.1838	1.3889	750	1.2230
1.2029	1.4815	800	1.2225
1.1571	1.5741	850	1.2224
1.1575	1.6667	900	1.2221
1.18	1.7593	950	1.2218
1.1708	1.8519	1000	1.2217
1.1513	1.9444	1050	1.2216

Framework versions

• PEFT 0.11.1
• Transformers 4.40.1
• Pytorch 2.3.1+cu121
• Datasets 4.0.0
• Tokenizers 0.19.1