train_cola_42_1763998306

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.2082
• Num Input Tokens Seen: 3668584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2856	0.5	962	0.2924	183584
0.2494	1.0	1924	0.2572	366856
0.2294	1.5	2886	0.2503	550664
0.2223	2.0	3848	0.2383	734320
0.1793	2.5	4810	0.2319	918128
0.1725	3.0	5772	0.2222	1100800
0.2258	3.5	6734	0.2185	1284064
0.2289	4.0	7696	0.2196	1467824
0.1585	4.5	8658	0.2151	1650992
0.1886	5.0	9620	0.2142	1834632
0.1754	5.5	10582	0.2148	2018408
0.0882	6.0	11544	0.2142	2202264
0.1589	6.5	12506	0.2135	2386136
0.3148	7.0	13468	0.2113	2568880
0.2632	7.5	14430	0.2082	2751696
0.1668	8.0	15392	0.2089	2935520
0.303	8.5	16354	0.2105	3119168
0.1812	9.0	17316	0.2088	3302192
0.228	9.5	18278	0.2089	3485264
0.124	10.0	19240	0.2098	3668584

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4