ARC-Challenge_Llama-3.2-1B-a2pgvai7

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.7404
• Model Preparation Time: 0.0058
• Mdl: 2907.5913
• Accumulated Loss: 2015.3887
• Correct Preds: 115.0
• Total Preds: 299.0
• Accuracy: 0.3846
• Correct Gen Preds: 86.0
• Gen Accuracy: 0.2876
• Correct Gen Preds 32: 23.0
• Correct Preds 32: 27.0
• Total Labels 32: 64.0
• Accuracy 32: 0.4219
• Gen Accuracy 32: 0.3594
• Correct Gen Preds 33: 14.0
• Correct Preds 33: 28.0
• Total Labels 33: 73.0
• Accuracy 33: 0.3836
• Gen Accuracy 33: 0.1918
• Correct Gen Preds 34: 24.0
• Correct Preds 34: 31.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3974
• Gen Accuracy 34: 0.3077
• Correct Gen Preds 35: 24.0
• Correct Preds 35: 28.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3373
• Gen Accuracy 35: 0.2892
• Correct Gen Preds 36: 1.0
• Correct Preds 36: 1.0
• Total Labels 36: 1.0
• Accuracy 36: 1.0
• Gen Accuracy 36: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.5706	1.0	2	1.5442	0.0058	666.1348	461.7295	93.0	299.0	0.3110	93.0	0.3110	0.0	0.0	64.0	0.0	0.0	32.0	32.0	73.0	0.4384	0.4384	59.0	59.0	78.0	0.7564	0.7564	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
1.5503	2.0	4	1.4644	0.0058	631.7094	437.8676	85.0	299.0	0.2843	85.0	0.2843	1.0	1.0	64.0	0.0156	0.0156	1.0	1.0	73.0	0.0137	0.0137	0.0	0.0	78.0	0.0	0.0	83.0	83.0	83.0	1.0	1.0	0.0	0.0	1.0	0.0	0.0
1.3714	3.0	6	1.3927	0.0058	600.7469	416.4060	72.0	299.0	0.2408	72.0	0.2408	12.0	12.0	64.0	0.1875	0.1875	57.0	57.0	73.0	0.7808	0.7808	2.0	2.0	78.0	0.0256	0.0256	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
1.2069	4.0	8	1.5081	0.0058	650.5549	450.9303	77.0	299.0	0.2575	77.0	0.2575	51.0	51.0	64.0	0.7969	0.7969	14.0	14.0	73.0	0.1918	0.1918	7.0	7.0	78.0	0.0897	0.0897	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.4911	5.0	10	2.2426	0.0058	967.3721	670.5312	113.0	299.0	0.3779	111.0	0.3712	14.0	15.0	64.0	0.2344	0.2188	27.0	27.0	73.0	0.3699	0.3699	34.0	35.0	78.0	0.4487	0.4359	36.0	36.0	83.0	0.4337	0.4337	0.0	0.0	1.0	0.0	0.0
0.2163	6.0	12	3.6299	0.0058	1565.8029	1085.3318	98.0	299.0	0.3278	93.0	0.3110	28.0	31.0	64.0	0.4844	0.4375	35.0	35.0	73.0	0.4795	0.4795	24.0	26.0	78.0	0.3333	0.3077	6.0	6.0	83.0	0.0723	0.0723	0.0	0.0	1.0	0.0	0.0
0.0843	7.0	14	3.7551	0.0058	1619.8051	1122.7634	100.0	299.0	0.3344	92.0	0.3077	17.0	23.0	64.0	0.3594	0.2656	36.0	37.0	73.0	0.5068	0.4932	27.0	27.0	78.0	0.3462	0.3462	12.0	13.0	83.0	0.1566	0.1446	0.0	0.0	1.0	0.0	0.0
0.018	8.0	16	4.6761	0.0058	2017.1167	1398.1588	111.0	299.0	0.3712	102.0	0.3411	17.0	21.0	64.0	0.3281	0.2656	33.0	36.0	73.0	0.4932	0.4521	31.0	32.0	78.0	0.4103	0.3974	21.0	22.0	83.0	0.2651	0.2530	0.0	0.0	1.0	0.0	0.0
0.0001	9.0	18	5.6870	0.0058	2453.1608	1700.4015	112.0	299.0	0.3746	100.0	0.3344	22.0	26.0	64.0	0.4062	0.3438	26.0	30.0	73.0	0.4110	0.3562	30.0	32.0	78.0	0.4103	0.3846	22.0	24.0	83.0	0.2892	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	10.0	20	6.3594	0.0058	2743.2220	1901.4566	113.0	299.0	0.3779	88.0	0.2943	23.0	25.0	64.0	0.3906	0.3594	16.0	28.0	73.0	0.3836	0.2192	25.0	33.0	78.0	0.4231	0.3205	23.0	26.0	83.0	0.3133	0.2771	1.0	1.0	1.0	1.0	1.0
0.0	11.0	22	6.7404	0.0058	2907.5913	2015.3887	115.0	299.0	0.3846	86.0	0.2876	23.0	27.0	64.0	0.4219	0.3594	14.0	28.0	73.0	0.3836	0.1918	24.0	31.0	78.0	0.3974	0.3077	24.0	28.0	83.0	0.3373	0.2892	1.0	1.0	1.0	1.0	1.0
0.0	12.0	24	7.0499	0.0058	3041.0683	2107.9079	112.0	299.0	0.3746	81.0	0.2709	24.0	27.0	64.0	0.4219	0.375	12.0	26.0	73.0	0.3562	0.1644	20.0	30.0	78.0	0.3846	0.2564	24.0	28.0	83.0	0.3373	0.2892	1.0	1.0	1.0	1.0	1.0
0.0	13.0	26	7.2388	0.0058	3122.5833	2164.4098	111.0	299.0	0.3712	78.0	0.2609	24.0	26.0	64.0	0.4062	0.375	12.0	25.0	73.0	0.3425	0.1644	18.0	31.0	78.0	0.3974	0.2308	24.0	29.0	83.0	0.3494	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	14.0	28	7.3766	0.0058	3182.0143	2205.6042	111.0	299.0	0.3712	74.0	0.2475	24.0	26.0	64.0	0.4062	0.375	11.0	25.0	73.0	0.3425	0.1507	17.0	32.0	78.0	0.4103	0.2179	22.0	28.0	83.0	0.3373	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	15.0	30	7.4784	0.0058	3225.9151	2236.0339	111.0	299.0	0.3712	74.0	0.2475	24.0	27.0	64.0	0.4219	0.375	11.0	26.0	73.0	0.3562	0.1507	16.0	31.0	78.0	0.3974	0.2051	23.0	27.0	83.0	0.3253	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	16.0	32	7.5458	0.0058	3255.0193	2256.2075	109.0	299.0	0.3645	76.0	0.2542	25.0	28.0	64.0	0.4375	0.3906	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	17.0	34	7.6054	0.0058	3280.7180	2274.0204	109.0	299.0	0.3645	75.0	0.2508	24.0	27.0	64.0	0.4219	0.375	12.0	24.0	73.0	0.3288	0.1644	17.0	31.0	78.0	0.3974	0.2179	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	18.0	36	7.6589	0.0058	3303.7818	2290.0070	109.0	299.0	0.3645	75.0	0.2508	24.0	27.0	64.0	0.4219	0.375	10.0	23.0	73.0	0.3151	0.1370	18.0	31.0	78.0	0.3974	0.2308	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	19.0	38	7.7091	0.0058	3325.4336	2305.0150	105.0	299.0	0.3512	73.0	0.2441	23.0	25.0	64.0	0.3906	0.3594	10.0	22.0	73.0	0.3014	0.1370	18.0	31.0	78.0	0.3974	0.2308	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	20.0	40	7.6806	0.0058	3313.1568	2296.5053	107.0	299.0	0.3579	76.0	0.2542	23.0	25.0	64.0	0.3906	0.3594	12.0	23.0	73.0	0.3151	0.1644	18.0	31.0	78.0	0.3974	0.2308	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	21.0	42	7.7066	0.0058	3324.3498	2304.2637	109.0	299.0	0.3645	76.0	0.2542	24.0	27.0	64.0	0.4219	0.375	11.0	22.0	73.0	0.3014	0.1507	17.0	31.0	78.0	0.3974	0.2179	24.0	29.0	83.0	0.3494	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	22.0	44	7.7066	0.0058	3324.3632	2304.2730	108.0	299.0	0.3612	75.0	0.2508	23.0	25.0	64.0	0.3906	0.3594	12.0	24.0	73.0	0.3288	0.1644	17.0	31.0	78.0	0.3974	0.2179	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	23.0	46	7.7683	0.0058	3350.9932	2322.7315	107.0	299.0	0.3579	71.0	0.2375	23.0	26.0	64.0	0.4062	0.3594	11.0	23.0	73.0	0.3151	0.1507	17.0	31.0	78.0	0.3974	0.2179	20.0	27.0	83.0	0.3253	0.2410	0.0	0.0	1.0	0.0	0.0
0.0	24.0	48	7.7069	0.0058	3324.4804	2304.3542	112.0	299.0	0.3746	76.0	0.2542	25.0	28.0	64.0	0.4375	0.3906	11.0	25.0	73.0	0.3425	0.1507	17.0	31.0	78.0	0.3974	0.2179	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	25.0	50	7.7409	0.0058	3339.1451	2314.5190	106.0	299.0	0.3545	74.0	0.2475	23.0	26.0	64.0	0.4062	0.3594	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	22.0	26.0	83.0	0.3133	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	26.0	52	7.7161	0.0058	3328.4671	2307.1176	109.0	299.0	0.3645	78.0	0.2609	24.0	27.0	64.0	0.4219	0.375	13.0	24.0	73.0	0.3288	0.1781	18.0	31.0	78.0	0.3974	0.2308	23.0	27.0	83.0	0.3253	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	27.0	54	7.7651	0.0058	3349.5949	2321.7623	106.0	299.0	0.3545	75.0	0.2508	23.0	25.0	64.0	0.3906	0.3594	12.0	23.0	73.0	0.3151	0.1644	18.0	31.0	78.0	0.3974	0.2308	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	28.0	56	7.7247	0.0058	3332.1503	2309.6706	111.0	299.0	0.3712	76.0	0.2542	24.0	27.0	64.0	0.4219	0.375	11.0	24.0	73.0	0.3288	0.1507	18.0	31.0	78.0	0.3974	0.2308	23.0	29.0	83.0	0.3494	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	29.0	58	7.7518	0.0058	3343.8563	2317.7845	107.0	299.0	0.3579	75.0	0.2508	23.0	26.0	64.0	0.4062	0.3594	12.0	23.0	73.0	0.3151	0.1644	18.0	31.0	78.0	0.3974	0.2308	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	30.0	60	7.7521	0.0058	3343.9727	2317.8652	112.0	299.0	0.3746	76.0	0.2542	25.0	28.0	64.0	0.4375	0.3906	10.0	24.0	73.0	0.3288	0.1370	18.0	31.0	78.0	0.3974	0.2308	23.0	29.0	83.0	0.3494	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	31.0	62	7.7367	0.0058	3337.3476	2313.2731	108.0	299.0	0.3612	77.0	0.2575	24.0	26.0	64.0	0.4062	0.375	12.0	24.0	73.0	0.3288	0.1644	18.0	31.0	78.0	0.3974	0.2308	23.0	27.0	83.0	0.3253	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	32.0	64	7.7564	0.0058	3345.8568	2319.1712	108.0	299.0	0.3612	75.0	0.2508	24.0	26.0	64.0	0.4062	0.375	12.0	24.0	73.0	0.3288	0.1644	17.0	31.0	78.0	0.3974	0.2179	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	33.0	66	7.7250	0.0058	3332.3099	2309.7812	112.0	299.0	0.3746	78.0	0.2609	25.0	28.0	64.0	0.4375	0.3906	12.0	25.0	73.0	0.3425	0.1644	18.0	31.0	78.0	0.3974	0.2308	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	34.0	68	7.7334	0.0058	3335.9027	2312.2715	108.0	299.0	0.3612	75.0	0.2508	24.0	26.0	64.0	0.4062	0.375	11.0	23.0	73.0	0.3151	0.1507	19.0	32.0	78.0	0.4103	0.2436	21.0	27.0	83.0	0.3253	0.2530	0.0	0.0	1.0	0.0	0.0
0.0	35.0	70	7.7331	0.0058	3335.7975	2312.1986	110.0	299.0	0.3679	77.0	0.2575	24.0	27.0	64.0	0.4219	0.375	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	24.0	29.0	83.0	0.3494	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	36.0	72	7.7199	0.0058	3330.1054	2308.2532	109.0	299.0	0.3645	76.0	0.2542	25.0	27.0	64.0	0.4219	0.3906	12.0	23.0	73.0	0.3151	0.1644	18.0	31.0	78.0	0.3974	0.2308	21.0	28.0	83.0	0.3373	0.2530	0.0	0.0	1.0	0.0	0.0
0.0	37.0	74	7.7475	0.0058	3342.0105	2316.5051	107.0	299.0	0.3579	74.0	0.2475	23.0	26.0	64.0	0.4062	0.3594	12.0	24.0	73.0	0.3288	0.1644	17.0	30.0	78.0	0.3846	0.2179	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	38.0	76	7.7456	0.0058	3341.1845	2315.9326	108.0	299.0	0.3612	75.0	0.2508	25.0	27.0	64.0	0.4219	0.3906	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	21.0	27.0	83.0	0.3253	0.2530	0.0	0.0	1.0	0.0	0.0
0.0	39.0	78	7.7126	0.0058	3326.9537	2306.0686	109.0	299.0	0.3645	76.0	0.2542	24.0	27.0	64.0	0.4219	0.375	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	40.0	80	7.7430	0.0058	3340.0466	2315.1439	110.0	299.0	0.3679	77.0	0.2575	24.0	27.0	64.0	0.4219	0.375	12.0	24.0	73.0	0.3288	0.1644	18.0	31.0	78.0	0.3974	0.2308	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	41.0	82	7.7436	0.0058	3340.3185	2315.3323	108.0	299.0	0.3612	74.0	0.2475	24.0	26.0	64.0	0.4062	0.375	11.0	23.0	73.0	0.3151	0.1507	18.0	31.0	78.0	0.3974	0.2308	21.0	28.0	83.0	0.3373	0.2530	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1