ARC-Challenge_Llama-3.2-1B-2plhzu7d

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.4425
• Model Preparation Time: 0.0059
• Mdl: 2347.7102
• Accumulated Loss: 1627.3087
• Correct Preds: 131.0
• Total Preds: 299.0
• Accuracy: 0.4381
• Correct Gen Preds: 131.0
• Gen Accuracy: 0.4381
• Correct Gen Preds 32: 21.0
• Correct Preds 32: 21.0
• Total Labels 32: 64.0
• Accuracy 32: 0.3281
• Gen Accuracy 32: 0.3281
• Correct Gen Preds 33: 37.0
• Correct Preds 33: 37.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5068
• Gen Accuracy 33: 0.5068
• Correct Gen Preds 34: 37.0
• Correct Preds 34: 37.0
• Total Labels 34: 78.0
• Accuracy 34: 0.4744
• Gen Accuracy 34: 0.4744
• Correct Gen Preds 35: 36.0
• Correct Preds 35: 36.0
• Total Labels 35: 83.0
• Accuracy 35: 0.4337
• Gen Accuracy 35: 0.4337
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_ratio: 0.001
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0059	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.4128	1.0	17	1.3958	0.0059	602.1210	417.3585	70.0	299.0	0.2341	70.0	0.2341	22.0	22.0	64.0	0.3438	0.3438	18.0	18.0	73.0	0.2466	0.2466	6.0	6.0	78.0	0.0769	0.0769	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
1.4079	2.0	34	1.3842	0.0059	597.0913	413.8722	87.0	299.0	0.2910	87.0	0.2910	9.0	9.0	64.0	0.1406	0.1406	44.0	44.0	73.0	0.6027	0.6027	10.0	10.0	78.0	0.1282	0.1282	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
1.2683	3.0	51	1.4562	0.0059	628.1508	435.4010	114.0	299.0	0.3813	114.0	0.3813	9.0	9.0	64.0	0.1406	0.1406	16.0	16.0	73.0	0.2192	0.2192	44.0	44.0	78.0	0.5641	0.5641	45.0	45.0	83.0	0.5422	0.5422	0.0	0.0	1.0	0.0	0.0
0.2495	4.0	68	1.9923	0.0059	859.4051	595.6942	115.0	299.0	0.3846	115.0	0.3846	10.0	10.0	64.0	0.1562	0.1562	20.0	20.0	73.0	0.2740	0.2740	54.0	54.0	78.0	0.6923	0.6923	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0214	5.0	85	3.5196	0.0059	1518.2409	1052.3644	112.0	299.0	0.3746	107.0	0.3579	6.0	6.0	64.0	0.0938	0.0938	31.0	33.0	73.0	0.4521	0.4247	49.0	50.0	78.0	0.6410	0.6282	21.0	23.0	83.0	0.2771	0.2530	0.0	0.0	1.0	0.0	0.0
0.0017	6.0	102	3.2994	0.0059	1423.2282	986.5066	123.0	299.0	0.4114	115.0	0.3846	25.0	27.0	64.0	0.4219	0.3906	35.0	38.0	73.0	0.5205	0.4795	30.0	31.0	78.0	0.3974	0.3846	25.0	27.0	83.0	0.3253	0.3012	0.0	0.0	1.0	0.0	0.0
0.1808	7.0	119	5.4050	0.0059	2331.5234	1616.0889	123.0	299.0	0.4114	115.0	0.3846	20.0	21.0	64.0	0.3281	0.3125	43.0	47.0	73.0	0.6438	0.5890	26.0	28.0	78.0	0.3590	0.3333	26.0	27.0	83.0	0.3253	0.3133	0.0	0.0	1.0	0.0	0.0
0.0001	8.0	136	6.0900	0.0059	2627.0223	1820.9131	114.0	299.0	0.3813	113.0	0.3779	26.0	26.0	64.0	0.4062	0.4062	38.0	39.0	73.0	0.5342	0.5205	26.0	26.0	78.0	0.3333	0.3333	23.0	23.0	83.0	0.2771	0.2771	0.0	0.0	1.0	0.0	0.0
0.0001	9.0	153	5.4425	0.0059	2347.7102	1627.3087	131.0	299.0	0.4381	131.0	0.4381	21.0	21.0	64.0	0.3281	0.3281	37.0	37.0	73.0	0.5068	0.5068	37.0	37.0	78.0	0.4744	0.4744	36.0	36.0	83.0	0.4337	0.4337	0.0	0.0	1.0	0.0	0.0
0.257	10.0	170	5.6481	0.0059	2436.4108	1688.7913	120.0	299.0	0.4013	118.0	0.3946	22.0	22.0	64.0	0.3438	0.3438	39.0	40.0	73.0	0.5479	0.5342	33.0	33.0	78.0	0.4231	0.4231	24.0	25.0	83.0	0.3012	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	11.0	187	6.0737	0.0059	2619.9959	1816.0428	119.0	299.0	0.3980	116.0	0.3880	22.0	23.0	64.0	0.3594	0.3438	37.0	38.0	73.0	0.5205	0.5068	29.0	29.0	78.0	0.3718	0.3718	28.0	29.0	83.0	0.3494	0.3373	0.0	0.0	1.0	0.0	0.0
0.0	12.0	204	6.6299	0.0059	2859.9240	1982.3483	118.0	299.0	0.3946	115.0	0.3846	22.0	22.0	64.0	0.3438	0.3438	45.0	47.0	73.0	0.6438	0.6164	30.0	30.0	78.0	0.3846	0.3846	18.0	19.0	83.0	0.2289	0.2169	0.0	0.0	1.0	0.0	0.0
0.0	13.0	221	6.5693	0.0059	2833.7524	1964.2075	110.0	299.0	0.3679	106.0	0.3545	14.0	14.0	64.0	0.2188	0.2188	34.0	36.0	73.0	0.4932	0.4658	32.0	33.0	78.0	0.4231	0.4103	26.0	27.0	83.0	0.3253	0.3133	0.0	0.0	1.0	0.0	0.0
0.0	14.0	238	6.9098	0.0059	2980.6494	2066.0287	124.0	299.0	0.4147	123.0	0.4114	29.0	29.0	64.0	0.4531	0.4531	44.0	45.0	73.0	0.6164	0.6027	26.0	26.0	78.0	0.3333	0.3333	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	15.0	255	7.0907	0.0059	3058.6812	2120.1163	124.0	299.0	0.4147	123.0	0.4114	28.0	29.0	64.0	0.4531	0.4375	46.0	46.0	73.0	0.6301	0.6301	25.0	25.0	78.0	0.3205	0.3205	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	16.0	272	7.1035	0.0059	3064.2104	2123.9488	124.0	299.0	0.4147	123.0	0.4114	28.0	29.0	64.0	0.4531	0.4375	46.0	46.0	73.0	0.6301	0.6301	25.0	25.0	78.0	0.3205	0.3205	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	17.0	289	7.1234	0.0059	3072.7806	2129.8892	123.0	299.0	0.4114	123.0	0.4114	27.0	27.0	64.0	0.4219	0.4219	46.0	46.0	73.0	0.6301	0.6301	27.0	27.0	78.0	0.3462	0.3462	23.0	23.0	83.0	0.2771	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	18.0	306	7.1214	0.0059	3071.9378	2129.3050	123.0	299.0	0.4114	123.0	0.4114	28.0	28.0	64.0	0.4375	0.4375	46.0	46.0	73.0	0.6301	0.6301	25.0	25.0	78.0	0.3205	0.3205	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	19.0	323	7.0946	0.0059	3060.3876	2121.2990	123.0	299.0	0.4114	123.0	0.4114	28.0	28.0	64.0	0.4375	0.4375	46.0	46.0	73.0	0.6301	0.6301	25.0	25.0	78.0	0.3205	0.3205	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1