train_cb_1745950319

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.0597
• Num Input Tokens Seen: 23078128

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0002	3.5133	200	0.1592	116248
0.0	7.0177	400	0.0795	232144
0.0	10.5310	600	0.0678	346496
0.0	14.0354	800	0.0721	462696
0.0	17.5487	1000	0.0928	578728
0.0	21.0531	1200	0.0734	692976
0.0	24.5664	1400	0.0864	809080
0.0	28.0708	1600	0.0896	924048
0.0	31.5841	1800	0.0848	1040096
0.0	35.0885	2000	0.0710	1155784
0.0	38.6018	2200	0.0821	1271880
0.0	42.1062	2400	0.0721	1386392
0.0	45.6195	2600	0.0897	1502448
0.0	49.1239	2800	0.0807	1616928
0.0	52.6372	3000	0.0749	1732240
0.0	56.1416	3200	0.0824	1847880
0.0	59.6549	3400	0.0798	1963376
0.0	63.1593	3600	0.0861	2078344
0.0	66.6726	3800	0.0735	2193696
0.0	70.1770	4000	0.0742	2309024
0.0	73.6903	4200	0.0903	2425544
0.0	77.1947	4400	0.0633	2539944
0.0	80.7080	4600	0.0811	2655720
0.0	84.2124	4800	0.0853	2771904
0.0	87.7257	5000	0.0772	2887856
0.0	91.2301	5200	0.0788	3003888
0.0	94.7434	5400	0.0786	3118800
0.0	98.2478	5600	0.0698	3234376
0.0	101.7611	5800	0.0895	3350608
0.0	105.2655	6000	0.0704	3466256
0.0	108.7788	6200	0.0813	3582008
0.0	112.2832	6400	0.0682	3696904
0.0	115.7965	6600	0.0732	3812728
0.0	119.3009	6800	0.0850	3927256
0.0	122.8142	7000	0.0843	4043128
0.0	126.3186	7200	0.0821	4158920
0.0	129.8319	7400	0.0665	4274536
0.0	133.3363	7600	0.0785	4389864
0.0	136.8496	7800	0.0691	4505192
0.0	140.3540	8000	0.0603	4620656
0.0	143.8673	8200	0.0669	4736960
0.0	147.3717	8400	0.0821	4850688
0.0	150.8850	8600	0.0715	4965800
0.0	154.3894	8800	0.0828	5082848
0.0	157.9027	9000	0.0768	5197896
0.0	161.4071	9200	0.0597	5312976
0.0	164.9204	9400	0.0778	5428816
0.0	168.4248	9600	0.0731	5542632
0.0	171.9381	9800	0.0756	5660064
0.0	175.4425	10000	0.1027	5775432
0.0	178.9558	10200	0.0978	5891480
0.0	182.4602	10400	0.1120	6006016
0.0	185.9735	10600	0.0894	6121200
0.0	189.4779	10800	0.1055	6236696
0.0	192.9912	11000	0.0834	6352152
0.0	196.4956	11200	0.1130	6467792
0.0	200.0	11400	0.0989	6581880
0.0	203.5133	11600	0.0918	6697328
0.0	207.0177	11800	0.1132	6811792
0.0	210.5310	12000	0.1098	6928248
0.0	214.0354	12200	0.1551	7043832
0.0	217.5487	12400	0.1159	7157984
0.0	221.0531	12600	0.1273	7274032
0.0	224.5664	12800	0.1509	7390136
0.0	228.0708	13000	0.1490	7505120
0.0	231.5841	13200	0.1334	7619616
0.0	235.0885	13400	0.1268	7736064
0.0	238.6018	13600	0.1315	7850792
0.0	242.1062	13800	0.1406	7965808
0.0	245.6195	14000	0.1521	8081552
0.0	249.1239	14200	0.1561	8197208
0.0	252.6372	14400	0.1383	8312272
0.0	256.1416	14600	0.1530	8426888
0.0	259.6549	14800	0.1685	8542448
0.0	263.1593	15000	0.1516	8658448
0.0	266.6726	15200	0.1675	8773608
0.0	270.1770	15400	0.1873	8887928
0.0	273.6903	15600	0.1526	9004600
0.0	277.1947	15800	0.1623	9119624
0.0	280.7080	16000	0.1726	9233904
0.0	284.2124	16200	0.1667	9351032
0.0	287.7257	16400	0.1740	9465944
0.0	291.2301	16600	0.1786	9581568
0.0	294.7434	16800	0.1864	9696576
0.0	298.2478	17000	0.2003	9811496
0.0	301.7611	17200	0.1928	9926600
0.0	305.2655	17400	0.1818	10042072
0.0	308.7788	17600	0.2123	10156616
0.0	312.2832	17800	0.1878	10272688
0.0	315.7965	18000	0.1764	10386824
0.0	319.3009	18200	0.2033	10502040
0.0	322.8142	18400	0.2033	10617608
0.0	326.3186	18600	0.2334	10731768
0.0	329.8319	18800	0.2246	10848480
0.0	333.3363	19000	0.2037	10963328
0.0	336.8496	19200	0.2281	11078712
0.0	340.3540	19400	0.2352	11193832
0.0	343.8673	19600	0.2309	11309368
0.0	347.3717	19800	0.2452	11424912
0.0	350.8850	20000	0.2584	11539864
0.0	354.3894	20200	0.2509	11654632
0.0	357.9027	20400	0.2576	11771008
0.0	361.4071	20600	0.2603	11886608
0.0	364.9204	20800	0.2550	12002608
0.0	368.4248	21000	0.2712	12117448
0.0	371.9381	21200	0.2755	12233152
0.0	375.4425	21400	0.2934	12346784
0.0	378.9558	21600	0.3063	12463336
0.0	382.4602	21800	0.2790	12578616
0.0	385.9735	22000	0.3174	12693160
0.0	389.4779	22200	0.3153	12808696
0.0	392.9912	22400	0.3176	12924056
0.0	396.4956	22600	0.3333	13039656
0.0	400.0	22800	0.3277	13154552
0.0	403.5133	23000	0.2859	13269320
0.0	407.0177	23200	0.3185	13385512
0.0	410.5310	23400	0.3082	13501208
0.0	414.0354	23600	0.3074	13617048
0.0	417.5487	23800	0.2899	13733448
0.0	421.0531	24000	0.3268	13848288
0.0	424.5664	24200	0.3186	13963536
0.0	428.0708	24400	0.3393	14080024
0.0	431.5841	24600	0.3267	14194520
0.0	435.0885	24800	0.3226	14310080
0.0	438.6018	25000	0.3500	14427448
0.0	442.1062	25200	0.3528	14542448
0.0	445.6195	25400	0.3601	14657640
0.0	449.1239	25600	0.3589	14772328
0.0	452.6372	25800	0.3593	14888712
0.0	456.1416	26000	0.3405	15002944
0.0	459.6549	26200	0.3649	15118544
0.0	463.1593	26400	0.3529	15234184
0.0	466.6726	26600	0.3461	15349544
0.0	470.1770	26800	0.3861	15465448
0.0	473.6903	27000	0.3777	15581752
0.0	477.1947	27200	0.3644	15696720
0.0	480.7080	27400	0.3685	15812864
0.0	484.2124	27600	0.3646	15928512
0.0	487.7257	27800	0.3666	16043264
0.0	491.2301	28000	0.3699	16158992
0.0	494.7434	28200	0.3825	16274040
0.0	498.2478	28400	0.3542	16389944
0.0	501.7611	28600	0.3767	16506208
0.0	505.2655	28800	0.3585	16621272
0.0	508.7788	29000	0.3788	16737072
0.0	512.2832	29200	0.3479	16852312
0.0	515.7965	29400	0.3582	16967744
0.0	519.3009	29600	0.3815	17083368
0.0	522.8142	29800	0.3717	17197984
0.0	526.3186	30000	0.3936	17314032
0.0	529.8319	30200	0.3963	17428904
0.0	533.3363	30400	0.3948	17543048
0.0	536.8496	30600	0.3936	17659880
0.0	540.3540	30800	0.4113	17773728
0.0	543.8673	31000	0.3960	17889344
0.0	547.3717	31200	0.4000	18005392
0.0	550.8850	31400	0.4149	18120296
0.0	554.3894	31600	0.4077	18235552
0.0	557.9027	31800	0.3982	18352024
0.0	561.4071	32000	0.3869	18466080
0.0	564.9204	32200	0.3970	18581584
0.0	568.4248	32400	0.4045	18697408
0.0	571.9381	32600	0.4062	18811608
0.0	575.4425	32800	0.4017	18927640
0.0	578.9558	33000	0.4009	19043672
0.0	582.4602	33200	0.4134	19157776
0.0	585.9735	33400	0.4079	19272744
0.0	589.4779	33600	0.3938	19388520
0.0	592.9912	33800	0.4020	19504472
0.0	596.4956	34000	0.4043	19618408
0.0	600.0	34200	0.4113	19734128
0.0	603.5133	34400	0.4125	19849608
0.0	607.0177	34600	0.4008	19964704
0.0	610.5310	34800	0.4224	20080968
0.0	614.0354	35000	0.4131	20195624
0.0	617.5487	35200	0.3956	20311640
0.0	621.0531	35400	0.4231	20426832
0.0	624.5664	35600	0.3809	20541816
0.0	628.0708	35800	0.4009	20656416
0.0	631.5841	36000	0.4049	20771136
0.0	635.0885	36200	0.4032	20886272
0.0	638.6018	36400	0.3965	21001560
0.0	642.1062	36600	0.3944	21115320
0.0	645.6195	36800	0.4007	21230216
0.0	649.1239	37000	0.3950	21344656
0.0	652.6372	37200	0.4097	21461664
0.0	656.1416	37400	0.4028	21576216
0.0	659.6549	37600	0.4208	21692088
0.0	663.1593	37800	0.4086	21807184
0.0	666.6726	38000	0.4108	21923192
0.0	670.1770	38200	0.4088	22037928
0.0	673.6903	38400	0.3998	22153968
0.0	677.1947	38600	0.3885	22269648
0.0	680.7080	38800	0.4052	22385640
0.0	684.2124	39000	0.3927	22502040
0.0	687.7257	39200	0.4122	22616408
0.0	691.2301	39400	0.3975	22732496
0.0	694.7434	39600	0.4177	22846704
0.0	698.2478	39800	0.3970	22962016
0.0	701.7611	40000	0.4048	23078128

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1