llama-3.1-helpfulness-reg-adapter

This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1070
• Mse: 1.1070
• Rmse: 1.0521
• Mae: 0.8345
• R2: 0.3054
• Rounded Accuracy: 0.3614
• Mae Class 0: 1.7197
• Mse Class 0: 3.7313
• Mae Class 1: 1.3182
• Mse Class 1: 2.3145
• Mae Class 2: 0.8845
• Mse Class 2: 1.0133
• Mae Class 3: 0.4414
• Mse Class 3: 0.3212
• Mae Class 4: 0.8221
• Mse Class 4: 0.9241
• Pred Count 0: 41
• Pred Percent 0: 1.0086
• Pred Count 1: 248
• Pred Percent 1: 6.1009
• Pred Count 2: 662
• Pred Percent 2: 16.2854
• Pred Count 3: 2407
• Pred Percent 3: 59.2128
• Pred Count 4: 707
• Pred Percent 4: 17.3924

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Rmse	Mae	R2	Rounded Accuracy	Mae Class 0	Mse Class 0	Mae Class 1	Mse Class 1	Mae Class 2	Mse Class 2	Mae Class 3	Mse Class 3	Mae Class 4	Mse Class 4	Pred Count 0	Pred Percent 0	Pred Count 1	Pred Percent 1	Pred Count 2	Pred Percent 2	Pred Count 3	Pred Percent 3	Pred Count 4	Pred Percent 4
1.2803	0.2460	500	1.4215	1.4215	1.1923	0.9419	0.1081	0.3183	2.1425	5.1712	1.5197	2.7599	0.8513	1.0542	0.4376	0.3741	0.9689	1.2630	44	1.0824	134	3.2964	828	20.3690	2560	62.9766	499	12.2755
1.2696	0.4920	1000	1.2019	1.2019	1.0963	0.8742	0.2459	0.3191	1.6321	3.4440	1.2188	2.0533	0.8326	0.9035	0.4434	0.3939	0.9707	1.2472	73	1.7958	337	8.2903	713	17.5400	2654	65.2891	288	7.0849
1.2078	0.7381	1500	1.1568	1.1568	1.0756	0.8666	0.2742	0.3092	1.6374	3.4214	1.2276	1.9984	0.7631	0.7646	0.3432	0.2730	1.0407	1.2814	47	1.1562	276	6.7897	732	18.0074	2946	72.4723	64	1.5744
1.3693	0.9841	2000	1.1070	1.1070	1.0521	0.8345	0.3054	0.3614	1.7197	3.7313	1.3182	2.3145	0.8845	1.0133	0.4414	0.3212	0.8221	0.9241	41	1.0086	248	6.1009	662	16.2854	2407	59.2128	707	17.3924

Framework versions

• PEFT 0.13.2
• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1