train_wsc_1745950305

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3465
• Num Input Tokens Seen: 13676608

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.3
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3325	1.6024	200	0.3605	68480
0.3379	3.2008	400	0.3765	137040
0.3508	4.8032	600	0.3483	205344
0.3381	6.4016	800	0.3567	273648
0.3404	8.0	1000	0.3533	342192
0.3268	9.6024	1200	0.3606	410624
0.3476	11.2008	1400	0.3527	479392
0.3407	12.8032	1600	0.3513	547360
0.455	14.4016	1800	0.4226	616128
0.3578	16.0	2000	0.3499	683616
0.3816	17.6024	2200	0.3683	751520
0.3329	19.2008	2400	0.3862	820000
0.3521	20.8032	2600	0.3503	888576
0.3601	22.4016	2800	0.3551	956480
0.371	24.0	3000	0.3517	1024784
0.3627	25.6024	3200	0.3602	1093536
0.3825	27.2008	3400	0.3662	1161248
0.3475	28.8032	3600	0.3634	1229760
0.3052	30.4016	3800	0.3920	1298112
0.3438	32.0	4000	0.3651	1366864
0.3207	33.6024	4200	0.4171	1435664
0.3271	35.2008	4400	0.3586	1503408
0.3249	36.8032	4600	0.3873	1572288
0.34	38.4016	4800	0.3727	1640848
0.3863	40.0	5000	0.3465	1708416
0.342	41.6024	5200	0.4012	1776416
0.3973	43.2008	5400	0.3653	1845088
0.3478	44.8032	5600	0.3515	1913360
0.3739	46.4016	5800	0.3639	1981136
0.3474	48.0	6000	0.3589	2050304
0.3357	49.6024	6200	0.3597	2118640
0.3441	51.2008	6400	0.3493	2186992
0.3389	52.8032	6600	0.3626	2255392
0.3369	54.4016	6800	0.3573	2324240
0.3405	56.0	7000	0.3498	2391840
0.4696	57.6024	7200	0.4197	2460464
0.3696	59.2008	7400	0.3568	2528416
0.3539	60.8032	7600	0.3624	2597008
0.3473	62.4016	7800	0.3542	2664720
0.407	64.0	8000	0.3540	2733360
0.3636	65.6024	8200	0.3860	2801792
0.3871	67.2008	8400	0.3540	2870768
0.3408	68.8032	8600	0.3515	2939344
0.3282	70.4016	8800	0.3686	3007936
0.3514	72.0	9000	0.3518	3076384
0.3398	73.6024	9200	0.3750	3144624
0.3516	75.2008	9400	0.3529	3212896
0.3249	76.8032	9600	0.3582	3281408
0.3239	78.4016	9800	0.3675	3349872
0.3616	80.0	10000	0.3526	3418368
0.3467	81.6024	10200	0.3633	3486640
0.3863	83.2008	10400	0.3823	3555456
0.2734	84.8032	10600	0.4335	3623440
0.3603	86.4016	10800	0.3589	3691760
0.361	88.0	11000	0.3542	3760416
0.3533	89.6024	11200	0.3507	3829184
0.3363	91.2008	11400	0.3524	3897520
0.3575	92.8032	11600	0.3531	3965568
0.364	94.4016	11800	0.3505	4033904
0.3393	96.0	12000	0.3676	4102480
0.2965	97.6024	12200	0.4004	4170912
0.3537	99.2008	12400	0.3585	4238208
0.3467	100.8032	12600	0.3618	4307408
0.3177	102.4016	12800	0.3566	4375136
0.3552	104.0	13000	0.3670	4443232
0.356	105.6024	13200	0.3606	4511824
0.384	107.2008	13400	0.3608	4580464
0.3675	108.8032	13600	0.3677	4648752
0.3317	110.4016	13800	0.3689	4717136
0.3446	112.0	14000	0.3579	4785328
0.3484	113.6024	14200	0.3578	4853616
0.3572	115.2008	14400	0.3665	4922160
0.3744	116.8032	14600	0.3712	4990880
0.3334	118.4016	14800	0.3621	5059200
0.349	120.0	15000	0.3630	5127856
0.3289	121.6024	15200	0.3662	5196320
0.3287	123.2008	15400	0.3851	5264752
0.3354	124.8032	15600	0.3629	5333360
0.3293	126.4016	15800	0.3642	5401648
0.327	128.0	16000	0.3861	5470144
0.3302	129.6024	16200	0.3584	5539584
0.3457	131.2008	16400	0.3675	5606896
0.3378	132.8032	16600	0.3589	5675392
0.36	134.4016	16800	0.3743	5743824
0.3566	136.0	17000	0.3707	5812000
0.3488	137.6024	17200	0.3776	5880400
0.3693	139.2008	17400	0.3654	5949456
0.3659	140.8032	17600	0.3797	6017584
0.3796	142.4016	17800	0.3899	6086352
0.3124	144.0	18000	0.3803	6153776
0.3336	145.6024	18200	0.3745	6222672
0.3405	147.2008	18400	0.3724	6291168
0.335	148.8032	18600	0.3796	6359136
0.3566	150.4016	18800	0.3824	6426976
0.3467	152.0	19000	0.3783	6495568
0.3422	153.6024	19200	0.3934	6564224
0.3908	155.2008	19400	0.4015	6632768
0.3349	156.8032	19600	0.3737	6701376
0.3425	158.4016	19800	0.4012	6769520
0.3448	160.0	20000	0.3877	6837904
0.3508	161.6024	20200	0.3783	6905904
0.3433	163.2008	20400	0.3796	6974368
0.3486	164.8032	20600	0.3907	7043152
0.3065	166.4016	20800	0.3945	7112192
0.3435	168.0	21000	0.3880	7179920
0.3113	169.6024	21200	0.4063	7248608
0.3206	171.2008	21400	0.3852	7316928
0.3435	172.8032	21600	0.3895	7385216
0.3453	174.4016	21800	0.3878	7453728
0.343	176.0	22000	0.4076	7521888
0.3385	177.6024	22200	0.4023	7590256
0.3931	179.2008	22400	0.4093	7658736
0.3238	180.8032	22600	0.4106	7727488
0.3377	182.4016	22800	0.4050	7796416
0.3381	184.0	23000	0.3976	7864592
0.2901	185.6024	23200	0.4124	7933232
0.3599	187.2008	23400	0.4046	8001808
0.3262	188.8032	23600	0.3991	8070240
0.3522	190.4016	23800	0.4183	8138688
0.3369	192.0	24000	0.4084	8206576
0.338	193.6024	24200	0.4040	8274800
0.3297	195.2008	24400	0.4067	8342976
0.4162	196.8032	24600	0.4255	8411584
0.3106	198.4016	24800	0.4107	8479856
0.3715	200.0	25000	0.4167	8548304
0.3478	201.6024	25200	0.4055	8617520
0.3255	203.2008	25400	0.4231	8685328
0.3249	204.8032	25600	0.4178	8753696
0.3483	206.4016	25800	0.4118	8821840
0.3787	208.0	26000	0.4074	8889904
0.3291	209.6024	26200	0.4041	8958528
0.3709	211.2008	26400	0.4166	9026416
0.3468	212.8032	26600	0.4104	9094992
0.3566	214.4016	26800	0.4356	9162896
0.3383	216.0	27000	0.4189	9231632
0.3405	217.6024	27200	0.4164	9299920
0.3258	219.2008	27400	0.4156	9368176
0.3361	220.8032	27600	0.4185	9437280
0.2898	222.4016	27800	0.4226	9505712
0.3628	224.0	28000	0.4204	9573776
0.3243	225.6024	28200	0.4185	9641744
0.3167	227.2008	28400	0.4211	9710672
0.3206	228.8032	28600	0.4188	9778976
0.3294	230.4016	28800	0.4227	9846768
0.3084	232.0	29000	0.4223	9915328
0.3243	233.6024	29200	0.4232	9984304
0.3374	235.2008	29400	0.4318	10052656
0.3084	236.8032	29600	0.4261	10121152
0.343	238.4016	29800	0.4245	10188944
0.3477	240.0	30000	0.4352	10257280
0.322	241.6024	30200	0.4333	10326160
0.3433	243.2008	30400	0.4296	10393920
0.3363	244.8032	30600	0.4299	10462528
0.3617	246.4016	30800	0.4291	10530528
0.3522	248.0	31000	0.4298	10599104
0.3835	249.6024	31200	0.4237	10667920
0.3541	251.2008	31400	0.4324	10736624
0.3455	252.8032	31600	0.4334	10804624
0.2929	254.4016	31800	0.4418	10873200
0.3453	256.0	32000	0.4288	10941264
0.3834	257.6024	32200	0.4344	11010000
0.3393	259.2008	32400	0.4357	11077280
0.3256	260.8032	32600	0.4315	11145744
0.3343	262.4016	32800	0.4329	11214112
0.3629	264.0	33000	0.4334	11282096
0.3205	265.6024	33200	0.4353	11350608
0.3315	267.2008	33400	0.4315	11418608
0.3532	268.8032	33600	0.4387	11487936
0.3375	270.4016	33800	0.4322	11556272
0.3382	272.0	34000	0.4305	11624208
0.3449	273.6024	34200	0.4325	11693424
0.3121	275.2008	34400	0.4390	11761200
0.3804	276.8032	34600	0.4334	11830208
0.3658	278.4016	34800	0.4322	11898240
0.3444	280.0	35000	0.4315	11966432
0.3282	281.6024	35200	0.4322	12035232
0.3546	283.2008	35400	0.4355	12103232
0.3597	284.8032	35600	0.4325	12171376
0.325	286.4016	35800	0.4342	12240128
0.3385	288.0	36000	0.4338	12308016
0.3608	289.6024	36200	0.4375	12375936
0.3452	291.2008	36400	0.4358	12444880
0.3183	292.8032	36600	0.4369	12513664
0.3226	294.4016	36800	0.4339	12581616
0.3545	296.0	37000	0.4362	12650688
0.3391	297.6024	37200	0.4362	12718976
0.3545	299.2008	37400	0.4340	12787680
0.3765	300.8032	37600	0.4327	12856448
0.289	302.4016	37800	0.4376	12924128
0.339	304.0	38000	0.4349	12992944
0.3366	305.6024	38200	0.4344	13060928
0.32	307.2008	38400	0.4377	13129472
0.3429	308.8032	38600	0.4317	13198064
0.3499	310.4016	38800	0.4368	13266304
0.3179	312.0	39000	0.4357	13334832
0.3531	313.6024	39200	0.4347	13402912
0.3574	315.2008	39400	0.4346	13470656
0.364	316.8032	39600	0.4393	13539984
0.3461	318.4016	39800	0.4369	13608768
0.319	320.0	40000	0.4327	13676608

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1