train_wsc_1745950307

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 1.3411
• Num Input Tokens Seen: 13676608

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.38	1.6024	200	1.5036	68480
1.8434	3.2008	400	1.4091	137040
1.6131	4.8032	600	1.3973	205344
1.9089	6.4016	800	1.3924	273648
2.4603	8.0	1000	1.3827	342192
2.3568	9.6024	1200	1.3743	410624
2.6209	11.2008	1400	1.4002	479392
1.9548	12.8032	1600	1.3998	547360
2.3111	14.4016	1800	1.3934	616128
1.5327	16.0	2000	1.3905	683616
1.5612	17.6024	2200	1.3936	751520
2.6575	19.2008	2400	1.3690	820000
1.8487	20.8032	2600	1.3929	888576
1.5663	22.4016	2800	1.3796	956480
1.7485	24.0	3000	1.3739	1024784
1.7832	25.6024	3200	1.3798	1093536
1.84	27.2008	3400	1.3878	1161248
2.2367	28.8032	3600	1.3837	1229760
2.2398	30.4016	3800	1.3837	1298112
2.5764	32.0	4000	1.3718	1366864
2.3443	33.6024	4200	1.3770	1435664
1.5648	35.2008	4400	1.3796	1503408
2.418	36.8032	4600	1.3792	1572288
2.2188	38.4016	4800	1.3876	1640848
1.9725	40.0	5000	1.3678	1708416
2.2863	41.6024	5200	1.3855	1776416
1.4944	43.2008	5400	1.3841	1845088
1.4354	44.8032	5600	1.3899	1913360
1.2547	46.4016	5800	1.3720	1981136
2.239	48.0	6000	1.3813	2050304
2.0904	49.6024	6200	1.3658	2118640
2.3447	51.2008	6400	1.3733	2186992
2.5379	52.8032	6600	1.3852	2255392
2.2955	54.4016	6800	1.3662	2324240
1.7964	56.0	7000	1.3752	2391840
1.9475	57.6024	7200	1.3890	2460464
2.0349	59.2008	7400	1.3970	2528416
2.0624	60.8032	7600	1.3765	2597008
1.9295	62.4016	7800	1.3909	2664720
2.6086	64.0	8000	1.3866	2733360
1.4825	65.6024	8200	1.3820	2801792
1.4862	67.2008	8400	1.3797	2870768
1.311	68.8032	8600	1.3816	2939344
1.5444	70.4016	8800	1.3809	3007936
1.6452	72.0	9000	1.3795	3076384
1.2808	73.6024	9200	1.3821	3144624
1.4038	75.2008	9400	1.3899	3212896
2.0719	76.8032	9600	1.3870	3281408
2.1484	78.4016	9800	1.3826	3349872
1.3604	80.0	10000	1.3719	3418368
1.9583	81.6024	10200	1.3645	3486640
1.9835	83.2008	10400	1.3874	3555456
2.271	84.8032	10600	1.3794	3623440
1.738	86.4016	10800	1.3925	3691760
1.5836	88.0	11000	1.3949	3760416
2.4306	89.6024	11200	1.3694	3829184
1.8857	91.2008	11400	1.3643	3897520
1.915	92.8032	11600	1.3843	3965568
1.8895	94.4016	11800	1.3623	4033904
1.7895	96.0	12000	1.3735	4102480
2.4102	97.6024	12200	1.3713	4170912
1.6993	99.2008	12400	1.3902	4238208
1.9568	100.8032	12600	1.3873	4307408
2.684	102.4016	12800	1.3784	4375136
1.7455	104.0	13000	1.3705	4443232
1.8316	105.6024	13200	1.3711	4511824
1.8675	107.2008	13400	1.3649	4580464
1.998	108.8032	13600	1.3803	4648752
1.8191	110.4016	13800	1.3605	4717136
1.6216	112.0	14000	1.3584	4785328
1.359	113.6024	14200	1.3578	4853616
1.5381	115.2008	14400	1.3633	4922160
1.4842	116.8032	14600	1.3836	4990880
1.9195	118.4016	14800	1.3861	5059200
2.2324	120.0	15000	1.4003	5127856
2.6831	121.6024	15200	1.3824	5196320
2.9063	123.2008	15400	1.3948	5264752
2.2375	124.8032	15600	1.3633	5333360
1.4379	126.4016	15800	1.3655	5401648
2.022	128.0	16000	1.3829	5470144
1.5	129.6024	16200	1.3639	5539584
2.0066	131.2008	16400	1.3695	5606896
2.0182	132.8032	16600	1.3684	5675392
1.8751	134.4016	16800	1.3700	5743824
1.6614	136.0	17000	1.3650	5812000
1.621	137.6024	17200	1.4032	5880400
2.3474	139.2008	17400	1.3793	5949456
1.5025	140.8032	17600	1.3786	6017584
1.8176	142.4016	17800	1.3833	6086352
2.5774	144.0	18000	1.3774	6153776
1.6388	145.6024	18200	1.3680	6222672
2.3709	147.2008	18400	1.3629	6291168
1.7972	148.8032	18600	1.3776	6359136
2.2769	150.4016	18800	1.3718	6426976
2.4199	152.0	19000	1.3809	6495568
2.334	153.6024	19200	1.3765	6564224
1.3356	155.2008	19400	1.3622	6632768
1.5932	156.8032	19600	1.4022	6701376
2.132	158.4016	19800	1.3923	6769520
1.243	160.0	20000	1.3735	6837904
1.9679	161.6024	20200	1.3769	6905904
1.2943	163.2008	20400	1.3794	6974368
1.5976	164.8032	20600	1.3860	7043152
2.4079	166.4016	20800	1.3839	7112192
1.902	168.0	21000	1.3712	7179920
2.4094	169.6024	21200	1.3693	7248608
1.8267	171.2008	21400	1.3882	7316928
1.3429	172.8032	21600	1.3781	7385216
1.9929	174.4016	21800	1.3723	7453728
1.5492	176.0	22000	1.3745	7521888
2.029	177.6024	22200	1.3866	7590256
1.0526	179.2008	22400	1.3728	7658736
2.1402	180.8032	22600	1.3733	7727488
2.1717	182.4016	22800	1.3580	7796416
1.0474	184.0	23000	1.3782	7864592
2.6908	185.6024	23200	1.3840	7933232
1.6581	187.2008	23400	1.3909	8001808
1.737	188.8032	23600	1.3631	8070240
2.1513	190.4016	23800	1.3719	8138688
2.9168	192.0	24000	1.3730	8206576
1.3348	193.6024	24200	1.3669	8274800
1.8642	195.2008	24400	1.3766	8342976
1.8082	196.8032	24600	1.3738	8411584
1.9464	198.4016	24800	1.3706	8479856
1.3418	200.0	25000	1.3411	8548304
1.0372	201.6024	25200	1.3819	8617520
1.4196	203.2008	25400	1.3806	8685328
2.6419	204.8032	25600	1.3815	8753696
1.6081	206.4016	25800	1.3642	8821840
1.0938	208.0	26000	1.3757	8889904
1.7548	209.6024	26200	1.3723	8958528
0.4627	211.2008	26400	1.3632	9026416
1.8565	212.8032	26600	1.3725	9094992
1.8041	214.4016	26800	1.3807	9162896
2.2034	216.0	27000	1.3971	9231632
1.2453	217.6024	27200	1.3777	9299920
1.3627	219.2008	27400	1.3901	9368176
1.761	220.8032	27600	1.3853	9437280
2.4811	222.4016	27800	1.3785	9505712
1.2036	224.0	28000	1.3843	9573776
1.6312	225.6024	28200	1.3712	9641744
2.7126	227.2008	28400	1.3745	9710672
1.8068	228.8032	28600	1.3504	9778976
2.0016	230.4016	28800	1.3559	9846768
2.4666	232.0	29000	1.3740	9915328
2.3197	233.6024	29200	1.3657	9984304
2.2468	235.2008	29400	1.3896	10052656
2.5254	236.8032	29600	1.3690	10121152
1.7327	238.4016	29800	1.3695	10188944
2.1135	240.0	30000	1.3644	10257280
2.4984	241.6024	30200	1.3745	10326160
2.6298	243.2008	30400	1.3582	10393920
1.9454	244.8032	30600	1.3769	10462528
1.6705	246.4016	30800	1.3858	10530528
1.0821	248.0	31000	1.3936	10599104
0.9083	249.6024	31200	1.3718	10667920
2.1352	251.2008	31400	1.3749	10736624
2.3122	252.8032	31600	1.3664	10804624
1.9733	254.4016	31800	1.3804	10873200
1.9106	256.0	32000	1.3715	10941264
1.0159	257.6024	32200	1.3682	11010000
1.7219	259.2008	32400	1.3720	11077280
2.02	260.8032	32600	1.3732	11145744
1.8695	262.4016	32800	1.3925	11214112
0.979	264.0	33000	1.3623	11282096
1.4244	265.6024	33200	1.3711	11350608
1.158	267.2008	33400	1.3865	11418608
2.2895	268.8032	33600	1.3829	11487936
2.3647	270.4016	33800	1.3672	11556272
1.8704	272.0	34000	1.3715	11624208
2.0451	273.6024	34200	1.3936	11693424
1.933	275.2008	34400	1.3798	11761200
1.1507	276.8032	34600	1.3821	11830208
1.6466	278.4016	34800	1.3833	11898240
1.6418	280.0	35000	1.3833	11966432
2.0597	281.6024	35200	1.3833	12035232
1.4621	283.2008	35400	1.3833	12103232
2.1253	284.8032	35600	1.3833	12171376
1.6358	286.4016	35800	1.3833	12240128
2.2219	288.0	36000	1.3833	12308016
1.2531	289.6024	36200	1.3833	12375936
1.9197	291.2008	36400	1.3833	12444880
2.2558	292.8032	36600	1.3833	12513664
1.9804	294.4016	36800	1.3833	12581616
1.8568	296.0	37000	1.3833	12650688
1.4651	297.6024	37200	1.3833	12718976
1.8077	299.2008	37400	1.3833	12787680
1.3443	300.8032	37600	1.3833	12856448
2.9669	302.4016	37800	1.3833	12924128
1.9458	304.0	38000	1.3833	12992944
1.2774	305.6024	38200	1.3833	13060928
2.3874	307.2008	38400	1.3833	13129472
2.9663	308.8032	38600	1.3833	13198064
1.9925	310.4016	38800	1.3833	13266304
2.5085	312.0	39000	1.3833	13334832
1.2625	313.6024	39200	1.3833	13402912
1.2102	315.2008	39400	1.3833	13470656
1.5315	316.8032	39600	1.3833	13539984
1.4806	318.4016	39800	1.3833	13608768
1.8959	320.0	40000	1.3833	13676608

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1