Mistral-7B-v0.1_cola_ignore_none

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5376
• Accuracy: 0.8318

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 2
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 2
• total_train_batch_size: 512
• total_eval_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 750

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.2085	1.61	25	1.2708	0.5695
0.7114	3.23	50	0.7877	0.6913
0.6189	4.84	75	0.6380	0.7258
0.6101	6.45	100	0.6412	0.7411
0.4509	8.06	125	0.5499	0.7622
0.4873	9.68	150	0.5194	0.7756
0.4272	11.29	175	0.5000	0.7929
0.3804	12.9	200	0.4769	0.8073
0.3547	14.52	225	0.4611	0.8121
0.3394	16.13	250	0.4567	0.8130
0.3945	17.74	275	0.4237	0.8351
0.313	19.35	300	0.4182	0.8332
0.2822	20.97	325	0.4205	0.8389
0.293	22.58	350	0.4153	0.8370
0.2449	24.19	375	0.4229	0.8341
0.1968	25.81	400	0.4439	0.8341
0.1868	27.42	425	0.4428	0.8360
0.1395	29.03	450	0.4913	0.8284
0.1583	30.65	475	0.4841	0.8245
0.2198	32.26	500	0.5471	0.8207
0.0953	33.87	525	0.5189	0.8351
0.1583	35.48	550	0.5668	0.8265
0.0988	37.1	575	0.5778	0.8217
0.0451	38.71	600	0.6536	0.8313

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0