train_copa_1745950329

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1666
• Num Input Tokens Seen: 11206480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2143	2.2222	200	0.2376	56064
0.0428	4.4444	400	0.2171	112064
0.0493	6.6667	600	0.2003	168096
0.0733	8.8889	800	0.1899	224048
0.0795	11.1111	1000	0.1809	280048
0.0173	13.3333	1200	0.1703	336032
0.0645	15.5556	1400	0.1712	392032
0.0182	17.7778	1600	0.1666	448128
0.0316	20.0	1800	0.1777	503904
0.1075	22.2222	2000	0.1836	559936
0.0114	24.4444	2200	0.2092	615968
0.0245	26.6667	2400	0.2190	672064
0.0032	28.8889	2600	0.2499	728128
0.0017	31.1111	2800	0.2805	784032
0.0034	33.3333	3000	0.3116	839984
0.0003	35.5556	3200	0.3440	896288
0.001	37.7778	3400	0.3697	952128
0.0026	40.0	3600	0.3941	1008096
0.0004	42.2222	3800	0.4013	1063984
0.0003	44.4444	4000	0.4117	1120080
0.0002	46.6667	4200	0.4236	1176240
0.0015	48.8889	4400	0.4326	1232160
0.0009	51.1111	4600	0.4346	1288160
0.0002	53.3333	4800	0.4417	1344160
0.0001	55.5556	5000	0.4468	1400368
0.0001	57.7778	5200	0.4541	1456368
0.0002	60.0	5400	0.4568	1512336
0.0001	62.2222	5600	0.4609	1568192
0.0001	64.4444	5800	0.4665	1624288
0.0	66.6667	6000	0.4666	1680352
0.0001	68.8889	6200	0.4691	1736384
0.0001	71.1111	6400	0.4751	1792480
0.0	73.3333	6600	0.4761	1848416
0.0001	75.5556	6800	0.4826	1904480
0.0	77.7778	7000	0.4834	1960496
0.0001	80.0	7200	0.4857	2016368
0.0	82.2222	7400	0.4890	2072400
0.0001	84.4444	7600	0.4896	2128384
0.0	86.6667	7800	0.4960	2184416
0.0	88.8889	8000	0.4954	2240512
0.0	91.1111	8200	0.5020	2296496
0.0	93.3333	8400	0.5017	2352560
0.0	95.5556	8600	0.5012	2408640
0.0	97.7778	8800	0.5042	2464672
0.0	100.0	9000	0.5018	2520688
0.0	102.2222	9200	0.5055	2576656
0.0	104.4444	9400	0.5114	2632720
0.0	106.6667	9600	0.5129	2688704
0.0	108.8889	9800	0.5120	2744768
0.0	111.1111	10000	0.5117	2800768
0.0	113.3333	10200	0.5164	2856768
0.0	115.5556	10400	0.5140	2912640
0.0	117.7778	10600	0.5191	2968832
0.0	120.0	10800	0.5196	3024896
0.0	122.2222	11000	0.5214	3081056
0.0	124.4444	11200	0.5252	3136944
0.0	126.6667	11400	0.5255	3192960
0.0	128.8889	11600	0.5231	3248976
0.0	131.1111	11800	0.5255	3305024
0.0	133.3333	12000	0.5284	3361008
0.0	135.5556	12200	0.5276	3417152
0.0	137.7778	12400	0.5304	3472832
0.0	140.0	12600	0.5347	3529008
0.0	142.2222	12800	0.5350	3585200
0.0	144.4444	13000	0.5368	3641200
0.0	146.6667	13200	0.5370	3697232
0.0	148.8889	13400	0.5378	3753168
0.0	151.1111	13600	0.5391	3809136
0.0	153.3333	13800	0.5414	3865216
0.0	155.5556	14000	0.5389	3921216
0.0	157.7778	14200	0.5419	3977312
0.0	160.0	14400	0.5420	4033488
0.0	162.2222	14600	0.5429	4089504
0.0	164.4444	14800	0.5490	4145504
0.0	166.6667	15000	0.5452	4201440
0.0	168.8889	15200	0.5472	4257504
0.0	171.1111	15400	0.5527	4313408
0.0	173.3333	15600	0.5525	4369488
0.0	175.5556	15800	0.5555	4425536
0.0	177.7778	16000	0.5523	4481568
0.0	180.0	16200	0.5524	4537616
0.0	182.2222	16400	0.5524	4593600
0.0	184.4444	16600	0.5498	4649664
0.0	186.6667	16800	0.5551	4705600
0.0	188.8889	17000	0.5546	4761760
0.0	191.1111	17200	0.5605	4817728
0.0	193.3333	17400	0.5628	4873856
0.0	195.5556	17600	0.5610	4929936
0.0	197.7778	17800	0.5632	4985840
0.0	200.0	18000	0.5627	5041920
0.0	202.2222	18200	0.5630	5097872
0.0	204.4444	18400	0.5646	5154064
0.0	206.6667	18600	0.5638	5210112
0.0	208.8889	18800	0.5702	5266064
0.0	211.1111	19000	0.5698	5322160
0.0	213.3333	19200	0.5704	5378224
0.0	215.5556	19400	0.5750	5434432
0.0	217.7778	19600	0.5732	5490352
0.0	220.0	19800	0.5726	5546432
0.0	222.2222	20000	0.5745	5602400
0.0	224.4444	20200	0.5741	5658464
0.0	226.6667	20400	0.5741	5714352
0.0	228.8889	20600	0.5769	5770416
0.0	231.1111	20800	0.5763	5826496
0.0	233.3333	21000	0.5732	5882496
0.0	235.5556	21200	0.5835	5938432
0.0	237.7778	21400	0.5815	5994480
0.0	240.0	21600	0.5835	6050656
0.0	242.2222	21800	0.5869	6106736
0.0	244.4444	22000	0.5928	6162896
0.0	246.6667	22200	0.5898	6218976
0.0	248.8889	22400	0.5861	6274960
0.0	251.1111	22600	0.5979	6331008
0.0	253.3333	22800	0.5831	6387152
0.0	255.5556	23000	0.5921	6443200
0.0	257.7778	23200	0.6070	6499088
0.0	260.0	23400	0.5994	6555184
0.0	262.2222	23600	0.5919	6611312
0.0	264.4444	23800	0.5900	6667104
0.0	266.6667	24000	0.6003	6723024
0.0	268.8889	24200	0.6093	6779376
0.0	271.1111	24400	0.5960	6835232
0.0	273.3333	24600	0.5964	6891104
0.0	275.5556	24800	0.6063	6947456
0.0	277.7778	25000	0.6019	7003408
0.0	280.0	25200	0.6047	7059536
0.0	282.2222	25400	0.5979	7115504
0.0	284.4444	25600	0.6192	7171744
0.0	286.6667	25800	0.6264	7227712
0.0	288.8889	26000	0.6053	7283856
0.0	291.1111	26200	0.6174	7339872
0.0	293.3333	26400	0.6198	7395808
0.0	295.5556	26600	0.6126	7451904
0.0	297.7778	26800	0.6094	7507792
0.0	300.0	27000	0.6202	7563888
0.0	302.2222	27200	0.6175	7619872
0.0	304.4444	27400	0.6100	7676016
0.0	306.6667	27600	0.6189	7731872
0.0	308.8889	27800	0.6271	7787920
0.0	311.1111	28000	0.6351	7844080
0.0	313.3333	28200	0.6254	7900064
0.0	315.5556	28400	0.6216	7956016
0.0	317.7778	28600	0.6143	8012160
0.0	320.0	28800	0.6363	8068256
0.0	322.2222	29000	0.6423	8124112
0.0	324.4444	29200	0.6250	8180192
0.0	326.6667	29400	0.6246	8236304
0.0	328.8889	29600	0.6353	8292272
0.0	331.1111	29800	0.6413	8348416
0.0	333.3333	30000	0.6229	8404432
0.0	335.5556	30200	0.6219	8460384
0.0	337.7778	30400	0.6248	8516432
0.0	340.0	30600	0.6258	8572496
0.0	342.2222	30800	0.6229	8628448
0.0	344.4444	31000	0.6496	8684672
0.0	346.6667	31200	0.6263	8740800
0.0	348.8889	31400	0.6195	8796784
0.0	351.1111	31600	0.6348	8852784
0.0	353.3333	31800	0.6384	8909040
0.0	355.5556	32000	0.6456	8965104
0.0	357.7778	32200	0.6348	9021344
0.0	360.0	32400	0.6414	9077456
0.0	362.2222	32600	0.6277	9133648
0.0	364.4444	32800	0.6210	9189616
0.0	366.6667	33000	0.6307	9245504
0.0	368.8889	33200	0.6389	9301520
0.0	371.1111	33400	0.6404	9357712
0.0	373.3333	33600	0.6496	9413712
0.0	375.5556	33800	0.6209	9469696
0.0	377.7778	34000	0.6399	9525760
0.0	380.0	34200	0.6415	9581648
0.0	382.2222	34400	0.6450	9637632
0.0	384.4444	34600	0.6461	9693568
0.0	386.6667	34800	0.6339	9749792
0.0	388.8889	35000	0.6284	9805840
0.0	391.1111	35200	0.6371	9861856
0.0	393.3333	35400	0.6395	9917904
0.0	395.5556	35600	0.6395	9973888
0.0	397.7778	35800	0.6400	10030096
0.0	400.0	36000	0.6258	10086192
0.0	402.2222	36200	0.6269	10142304
0.0	404.4444	36400	0.6466	10198320
0.0	406.6667	36600	0.6441	10254256
0.0	408.8889	36800	0.6385	10310096
0.0	411.1111	37000	0.6475	10366160
0.0	413.3333	37200	0.6409	10422192
0.0	415.5556	37400	0.6314	10478368
0.0	417.7778	37600	0.6568	10534240
0.0	420.0	37800	0.6367	10590208
0.0	422.2222	38000	0.6496	10646384
0.0	424.4444	38200	0.6610	10702336
0.0	426.6667	38400	0.6584	10758400
0.0	428.8889	38600	0.6359	10814480
0.0	431.1111	38800	0.6359	10870400
0.0	433.3333	39000	0.6359	10926320
0.0	435.5556	39200	0.6359	10982240
0.0	437.7778	39400	0.6359	11038352
0.0	440.0	39600	0.6359	11094352
0.0	442.2222	39800	0.6359	11150400
0.0	444.4444	40000	0.6359	11206480

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1