train_wsc_1745950306

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3580
• Num Input Tokens Seen: 13676608

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3415	1.6024	200	0.3619	68480
0.3415	3.2008	400	0.3610	137040
0.355	4.8032	600	0.3580	205344
0.3428	6.4016	800	0.3603	273648
0.3268	8.0	1000	0.4596	342192
0.0016	9.6024	1200	0.9259	410624
0.0279	11.2008	1400	1.0216	479392
0.0009	12.8032	1600	1.6723	547360
0.0001	14.4016	1800	1.7013	616128
0.0046	16.0	2000	1.4575	683616
0.0	17.6024	2200	1.8072	751520
0.0	19.2008	2400	2.3819	820000
0.0	20.8032	2600	1.9746	888576
0.0	22.4016	2800	2.1866	956480
0.0	24.0	3000	2.3199	1024784
0.0	25.6024	3200	2.3857	1093536
0.0	27.2008	3400	2.4492	1161248
0.0	28.8032	3600	2.4879	1229760
0.0	30.4016	3800	2.5550	1298112
0.0	32.0	4000	2.5538	1366864
0.0	33.6024	4200	2.6073	1435664
0.0	35.2008	4400	2.6352	1503408
0.0	36.8032	4600	2.6716	1572288
0.0	38.4016	4800	2.6888	1640848
0.0	40.0	5000	2.7171	1708416
0.0	41.6024	5200	2.7452	1776416
0.0	43.2008	5400	2.7619	1845088
0.0	44.8032	5600	2.8002	1913360
0.0	46.4016	5800	2.8253	1981136
0.0	48.0	6000	2.8733	2050304
0.0	49.6024	6200	2.8888	2118640
0.0	51.2008	6400	2.9106	2186992
0.0	52.8032	6600	2.9513	2255392
0.0	54.4016	6800	2.9624	2324240
0.0	56.0	7000	3.0025	2391840
0.0	57.6024	7200	3.0260	2460464
0.0	59.2008	7400	3.0466	2528416
0.0	60.8032	7600	3.0644	2597008
0.0	62.4016	7800	3.0912	2664720
0.0	64.0	8000	3.1174	2733360
0.0	65.6024	8200	3.1414	2801792
0.0	67.2008	8400	3.1659	2870768
0.0	68.8032	8600	3.1882	2939344
0.0	70.4016	8800	3.1810	3007936
0.0	72.0	9000	3.2144	3076384
0.0	73.6024	9200	3.2346	3144624
0.0	75.2008	9400	3.2585	3212896
0.0	76.8032	9600	3.2736	3281408
0.0	78.4016	9800	3.2914	3349872
0.0	80.0	10000	3.3121	3418368
0.0	81.6024	10200	3.3338	3486640
0.0	83.2008	10400	3.3790	3555456
0.0	84.8032	10600	3.3874	3623440
0.0	86.4016	10800	3.4041	3691760
0.0	88.0	11000	3.4354	3760416
0.0	89.6024	11200	3.4421	3829184
0.0	91.2008	11400	3.4572	3897520
0.0	92.8032	11600	3.4706	3965568
0.0	94.4016	11800	3.4683	4033904
0.0	96.0	12000	3.4806	4102480
0.0	97.6024	12200	3.4745	4170912
0.0	99.2008	12400	3.4612	4238208
0.0	100.8032	12600	3.4646	4307408
0.0	102.4016	12800	3.4669	4375136
0.0	104.0	13000	3.4782	4443232
0.0	105.6024	13200	3.4941	4511824
0.0	107.2008	13400	3.5152	4580464
0.0	108.8032	13600	3.5369	4648752
0.0	110.4016	13800	3.5642	4717136
0.0	112.0	14000	3.5913	4785328
0.0	113.6024	14200	3.6293	4853616
0.0	115.2008	14400	3.6603	4922160
0.0	116.8032	14600	3.6906	4990880
0.0	118.4016	14800	3.7268	5059200
0.0	120.0	15000	3.7533	5127856
0.0	121.6024	15200	3.7808	5196320
0.0	123.2008	15400	3.7812	5264752
0.0	124.8032	15600	3.8209	5333360
0.0	126.4016	15800	3.8430	5401648
0.0	128.0	16000	3.8639	5470144
0.0	129.6024	16200	3.9038	5539584
0.0	131.2008	16400	3.9227	5606896
0.0	132.8032	16600	3.9272	5675392
0.0	134.4016	16800	3.9524	5743824
0.0	136.0	17000	3.9851	5812000
0.0	137.6024	17200	3.9894	5880400
0.0	139.2008	17400	4.0143	5949456
0.0	140.8032	17600	4.0252	6017584
0.0	142.4016	17800	3.9903	6086352
0.0	144.0	18000	4.0323	6153776
0.0	145.6024	18200	4.0066	6222672
0.0	147.2008	18400	4.0066	6291168
0.0	148.8032	18600	4.0165	6359136
0.0	150.4016	18800	4.0007	6426976
0.0	152.0	19000	4.0027	6495568
0.0	153.6024	19200	3.9541	6564224
0.0	155.2008	19400	4.0324	6632768
0.0	156.8032	19600	4.0619	6701376
0.0	158.4016	19800	4.0177	6769520
0.0	160.0	20000	3.9997	6837904
0.0	161.6024	20200	3.9923	6905904
0.0	163.2008	20400	4.0133	6974368
0.0	164.8032	20600	4.0236	7043152
0.0	166.4016	20800	4.0731	7112192
0.0	168.0	21000	4.0293	7179920
0.0	169.6024	21200	4.0435	7248608
0.0	171.2008	21400	4.0805	7316928
0.0	172.8032	21600	4.0791	7385216
0.0	174.4016	21800	4.0860	7453728
0.0	176.0	22000	4.0709	7521888
0.0	177.6024	22200	4.0385	7590256
0.0	179.2008	22400	4.0636	7658736
0.0	180.8032	22600	4.0926	7727488
0.0	182.4016	22800	4.1460	7796416
0.0	184.0	23000	4.0785	7864592
0.0	185.6024	23200	4.0887	7933232
0.0	187.2008	23400	4.0638	8001808
0.0	188.8032	23600	4.1313	8070240
0.0	190.4016	23800	4.0751	8138688
0.0	192.0	24000	4.1024	8206576
0.0	193.6024	24200	4.0859	8274800
0.0	195.2008	24400	4.0809	8342976
0.0	196.8032	24600	4.0961	8411584
0.0	198.4016	24800	4.0982	8479856
0.0	200.0	25000	4.0766	8548304
0.0	201.6024	25200	4.1081	8617520
0.0	203.2008	25400	4.1371	8685328
0.0	204.8032	25600	4.1193	8753696
0.0	206.4016	25800	4.1294	8821840
0.0	208.0	26000	4.1679	8889904
0.0	209.6024	26200	4.1413	8958528
0.0	211.2008	26400	4.1673	9026416
0.0	212.8032	26600	4.1709	9094992
0.0	214.4016	26800	4.1801	9162896
0.0	216.0	27000	4.1807	9231632
0.0	217.6024	27200	4.1900	9299920
0.0	219.2008	27400	4.2693	9368176
0.0	220.8032	27600	4.2291	9437280
0.0	222.4016	27800	4.3068	9505712
0.0	224.0	28000	4.2265	9573776
0.0	225.6024	28200	4.2530	9641744
0.0	227.2008	28400	4.2562	9710672
0.0	228.8032	28600	4.2562	9778976
0.0	230.4016	28800	4.2759	9846768
0.0	232.0	29000	4.2658	9915328
0.0	233.6024	29200	4.2759	9984304
0.0	235.2008	29400	4.2222	10052656
0.0	236.8032	29600	4.2791	10121152
0.0	238.4016	29800	4.3058	10188944
0.0	240.0	30000	4.2963	10257280
0.0	241.6024	30200	4.3244	10326160
0.0	243.2008	30400	4.2610	10393920
0.0	244.8032	30600	4.3022	10462528
0.0	246.4016	30800	4.3089	10530528
0.0	248.0	31000	4.3266	10599104
0.0	249.6024	31200	4.3010	10667920
0.0	251.2008	31400	4.3030	10736624
0.0	252.8032	31600	4.2849	10804624
0.0	254.4016	31800	4.2944	10873200
0.0	256.0	32000	4.3089	10941264
0.0	257.6024	32200	4.3110	11010000
0.0	259.2008	32400	4.3047	11077280
0.0	260.8032	32600	4.3071	11145744
0.0	262.4016	32800	4.3129	11214112
0.0	264.0	33000	4.3082	11282096
0.0	265.6024	33200	4.3149	11350608
0.0	267.2008	33400	4.2942	11418608
0.0	268.8032	33600	4.3186	11487936
0.0	270.4016	33800	4.3170	11556272
0.0	272.0	34000	4.3220	11624208
0.0	273.6024	34200	4.3090	11693424
0.0	275.2008	34400	4.3237	11761200
0.0	276.8032	34600	4.3235	11830208
0.0	278.4016	34800	4.3243	11898240
0.0	280.0	35000	4.3129	11966432
0.0	281.6024	35200	4.3173	12035232
0.0	283.2008	35400	4.3101	12103232
0.0	284.8032	35600	4.3159	12171376
0.0	286.4016	35800	4.3255	12240128
0.0	288.0	36000	4.3184	12308016
0.0	289.6024	36200	4.3244	12375936
0.0	291.2008	36400	4.3384	12444880
0.0	292.8032	36600	4.3357	12513664
0.0	294.4016	36800	4.3286	12581616
0.0	296.0	37000	4.3194	12650688
0.0	297.6024	37200	4.3166	12718976
0.0	299.2008	37400	4.3200	12787680
0.0	300.8032	37600	4.3282	12856448
0.0	302.4016	37800	4.3261	12924128
0.0	304.0	38000	4.3148	12992944
0.0	305.6024	38200	4.3438	13060928
0.0	307.2008	38400	4.3106	13129472
0.0	308.8032	38600	4.3406	13198064
0.0	310.4016	38800	4.3171	13266304
0.0	312.0	39000	4.3278	13334832
0.0	313.6024	39200	4.3288	13402912
0.0	315.2008	39400	4.3277	13470656
0.0	316.8032	39600	4.3291	13539984
0.0	318.4016	39800	4.3338	13608768
0.0	320.0	40000	4.3237	13676608

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1