ARC-Challenge_Llama-3.2-1B-0gbt3jtk

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1723
• Model Preparation Time: 0.0059
• Mdl: 1368.4051
• Accumulated Loss: 948.5062
• Correct Preds: 100.0
• Total Preds: 299.0
• Accuracy: 0.3344
• Correct Gen Preds: 92.0
• Gen Accuracy: 0.3077
• Correct Gen Preds 32: 19.0
• Correct Preds 32: 21.0
• Total Labels 32: 64.0
• Accuracy 32: 0.3281
• Gen Accuracy 32: 0.2969
• Correct Gen Preds 33: 41.0
• Correct Preds 33: 42.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5753
• Gen Accuracy 33: 0.5616
• Correct Gen Preds 34: 17.0
• Correct Preds 34: 20.0
• Total Labels 34: 78.0
• Accuracy 34: 0.2564
• Gen Accuracy 34: 0.2179
• Correct Gen Preds 35: 15.0
• Correct Preds 35: 17.0
• Total Labels 35: 83.0
• Accuracy 35: 0.2048
• Gen Accuracy 35: 0.1807
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0059	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.5273	1.0	2	1.5168	0.0059	654.3081	453.5318	91.0	299.0	0.3043	91.0	0.3043	0.0	0.0	64.0	0.0	0.0	29.0	29.0	73.0	0.3973	0.3973	61.0	61.0	78.0	0.7821	0.7821	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
1.5762	2.0	4	1.5054	0.0059	649.3764	450.1134	83.0	299.0	0.2776	83.0	0.2776	2.0	2.0	64.0	0.0312	0.0312	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	81.0	81.0	83.0	0.9759	0.9759	0.0	0.0	1.0	0.0	0.0
1.3795	3.0	6	1.4567	0.0059	628.3552	435.5426	64.0	299.0	0.2140	64.0	0.2140	64.0	64.0	64.0	1.0	1.0	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.0852	4.0	8	1.7605	0.0059	759.4333	526.3990	63.0	299.0	0.2107	32.0	0.1070	29.0	58.0	64.0	0.9062	0.4531	1.0	3.0	73.0	0.0411	0.0137	1.0	1.0	78.0	0.0128	0.0128	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.7326	5.0	10	1.5126	0.0059	652.5053	452.2822	93.0	299.0	0.3110	87.0	0.2910	7.0	10.0	64.0	0.1562	0.1094	10.0	10.0	73.0	0.1370	0.1370	43.0	45.0	78.0	0.5769	0.5513	27.0	28.0	83.0	0.3373	0.3253	0.0	0.0	1.0	0.0	0.0
0.5126	6.0	12	2.6698	0.0059	1151.6760	798.2810	78.0	299.0	0.2609	69.0	0.2308	41.0	44.0	64.0	0.6875	0.6406	20.0	21.0	73.0	0.2877	0.2740	7.0	10.0	78.0	0.1282	0.0897	1.0	3.0	83.0	0.0361	0.0120	0.0	0.0	1.0	0.0	0.0
0.095	7.0	14	3.1723	0.0059	1368.4051	948.5062	100.0	299.0	0.3344	92.0	0.3077	19.0	21.0	64.0	0.3281	0.2969	41.0	42.0	73.0	0.5753	0.5616	17.0	20.0	78.0	0.2564	0.2179	15.0	17.0	83.0	0.2048	0.1807	0.0	0.0	1.0	0.0	0.0
0.0111	8.0	16	3.9875	0.0059	1720.0510	1192.2485	94.0	299.0	0.3144	83.0	0.2776	15.0	19.0	64.0	0.2969	0.2344	19.0	21.0	73.0	0.2877	0.2603	13.0	18.0	78.0	0.2308	0.1667	36.0	36.0	83.0	0.4337	0.4337	0.0	0.0	1.0	0.0	0.0
0.0005	9.0	18	5.2320	0.0059	2256.9142	1564.3737	87.0	299.0	0.2910	73.0	0.2441	14.0	19.0	64.0	0.2969	0.2188	19.0	23.0	73.0	0.3151	0.2603	10.0	13.0	78.0	0.1667	0.1282	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	10.0	20	6.3277	0.0059	2729.5669	1891.9916	84.0	299.0	0.2809	73.0	0.2441	17.0	21.0	64.0	0.3281	0.2656	21.0	24.0	73.0	0.3288	0.2877	7.0	9.0	78.0	0.1154	0.0897	28.0	30.0	83.0	0.3614	0.3373	0.0	0.0	1.0	0.0	0.0
0.0	11.0	22	7.0564	0.0059	3043.9051	2109.8742	85.0	299.0	0.2843	74.0	0.2475	17.0	21.0	64.0	0.3281	0.2656	20.0	24.0	73.0	0.3288	0.2740	6.0	8.0	78.0	0.1026	0.0769	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	12.0	24	7.6231	0.0059	3288.3287	2279.2958	83.0	299.0	0.2776	72.0	0.2408	14.0	18.0	64.0	0.2812	0.2188	21.0	25.0	73.0	0.3425	0.2877	7.0	8.0	78.0	0.1026	0.0897	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	13.0	26	8.0091	0.0059	3454.8633	2394.7288	87.0	299.0	0.2910	76.0	0.2542	15.0	18.0	64.0	0.2812	0.2344	23.0	27.0	73.0	0.3699	0.3151	7.0	9.0	78.0	0.1154	0.0897	31.0	33.0	83.0	0.3976	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	14.0	28	8.3140	0.0059	3586.3697	2485.8820	86.0	299.0	0.2876	77.0	0.2575	15.0	18.0	64.0	0.2812	0.2344	22.0	26.0	73.0	0.3562	0.3014	7.0	9.0	78.0	0.1154	0.0897	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	15.0	30	8.5114	0.0059	3671.5155	2544.9006	85.0	299.0	0.2843	77.0	0.2575	15.0	18.0	64.0	0.2812	0.2344	22.0	26.0	73.0	0.3562	0.3014	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	16.0	32	8.6314	0.0059	3723.2851	2580.7845	86.0	299.0	0.2876	76.0	0.2542	15.0	18.0	64.0	0.2812	0.2344	22.0	27.0	73.0	0.3699	0.3014	8.0	9.0	78.0	0.1154	0.1026	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	17.0	34	8.6934	0.0059	3750.0560	2599.3408	87.0	299.0	0.2910	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	24.0	27.0	73.0	0.3699	0.3288	8.0	9.0	78.0	0.1154	0.1026	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	18.0	36	8.7464	0.0059	3772.8926	2615.1699	87.0	299.0	0.2910	77.0	0.2575	15.0	18.0	64.0	0.2812	0.2344	23.0	28.0	73.0	0.3836	0.3151	8.0	9.0	78.0	0.1154	0.1026	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	19.0	38	8.8243	0.0059	3806.5107	2638.4722	88.0	299.0	0.2943	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	20.0	40	8.8775	0.0059	3829.4428	2654.3675	87.0	299.0	0.2910	81.0	0.2709	15.0	18.0	64.0	0.2812	0.2344	26.0	28.0	73.0	0.3836	0.3562	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	21.0	42	8.8562	0.0059	3820.2522	2647.9970	86.0	299.0	0.2876	78.0	0.2609	15.0	18.0	64.0	0.2812	0.2344	24.0	28.0	73.0	0.3836	0.3288	8.0	9.0	78.0	0.1154	0.1026	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	22.0	44	8.8897	0.0059	3834.7007	2658.0120	88.0	299.0	0.2943	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	24.0	28.0	73.0	0.3836	0.3288	8.0	9.0	78.0	0.1154	0.1026	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	23.0	46	8.8760	0.0059	3828.7991	2653.9213	89.0	299.0	0.2977	82.0	0.2742	16.0	18.0	64.0	0.2812	0.25	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	24.0	48	8.8887	0.0059	3834.2910	2657.7280	89.0	299.0	0.2977	81.0	0.2709	15.0	18.0	64.0	0.2812	0.2344	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	25.0	50	8.9167	0.0059	3846.3799	2666.1074	87.0	299.0	0.2910	79.0	0.2642	15.0	18.0	64.0	0.2812	0.2344	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	26.0	52	8.9404	0.0059	3856.5946	2673.1877	87.0	299.0	0.2910	79.0	0.2642	15.0	18.0	64.0	0.2812	0.2344	24.0	28.0	73.0	0.3836	0.3288	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	27.0	54	8.9031	0.0059	3840.5036	2662.0342	90.0	299.0	0.3010	81.0	0.2709	15.0	18.0	64.0	0.2812	0.2344	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	28.0	56	8.9326	0.0059	3853.2189	2670.8478	87.0	299.0	0.2910	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	29.0	58	8.9112	0.0059	3843.9827	2664.4458	89.0	299.0	0.2977	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	30.0	60	8.9122	0.0059	3844.3972	2664.7331	87.0	299.0	0.2910	79.0	0.2642	15.0	18.0	64.0	0.2812	0.2344	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	31.0	62	8.9079	0.0059	3842.5717	2663.4677	87.0	299.0	0.2910	79.0	0.2642	15.0	18.0	64.0	0.2812	0.2344	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	32.0	64	8.8993	0.0059	3838.8672	2660.9000	88.0	299.0	0.2943	80.0	0.2676	16.0	18.0	64.0	0.2812	0.25	24.0	28.0	73.0	0.3836	0.3288	8.0	9.0	78.0	0.1154	0.1026	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	33.0	66	8.9290	0.0059	3851.6474	2669.7585	88.0	299.0	0.2943	81.0	0.2709	15.0	18.0	64.0	0.2812	0.2344	26.0	29.0	73.0	0.3973	0.3562	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	34.0	68	8.9385	0.0059	3855.7508	2672.6028	86.0	299.0	0.2876	80.0	0.2676	16.0	18.0	64.0	0.2812	0.25	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	35.0	70	8.9410	0.0059	3856.8237	2673.3465	88.0	299.0	0.2943	81.0	0.2709	16.0	18.0	64.0	0.2812	0.25	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	36.0	72	8.9406	0.0059	3856.6489	2673.2253	87.0	299.0	0.2910	80.0	0.2676	15.0	18.0	64.0	0.2812	0.2344	25.0	28.0	73.0	0.3836	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	37.0	74	8.9357	0.0059	3854.5444	2671.7666	89.0	299.0	0.2977	81.0	0.2709	16.0	18.0	64.0	0.2812	0.25	25.0	29.0	73.0	0.3973	0.3425	8.0	9.0	78.0	0.1154	0.1026	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1