Regression_albert_8

This model is a fine-tuned version of albert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0710
• Mse: 0.0710
• Mae: 0.1978
• R2: 0.0202
• Accuracy: 0.9259

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Mae	R2	Accuracy
No log	1.0	49	0.0777	0.0777	0.2323	0.2804	0.9464
No log	2.0	98	0.0649	0.0649	0.2176	0.3990	0.9464
No log	3.0	147	0.0885	0.0885	0.2354	0.1799	0.8571
No log	4.0	196	0.0620	0.0620	0.1971	0.4252	0.9643
No log	5.0	245	0.0605	0.0605	0.2071	0.4394	0.9821
No log	6.0	294	0.0523	0.0523	0.1714	0.5155	0.9821
No log	7.0	343	0.1047	0.1047	0.2598	0.0301	0.8393
No log	8.0	392	0.0421	0.0421	0.1543	0.6103	0.9643
No log	9.0	441	0.0445	0.0445	0.1612	0.5875	0.9643
No log	10.0	490	0.0438	0.0438	0.1608	0.5939	0.9821
0.0478	11.0	539	0.0529	0.0529	0.1816	0.5095	0.9464
0.0478	12.0	588	0.0401	0.0401	0.1495	0.6288	0.9643
0.0478	13.0	637	0.0471	0.0471	0.1637	0.5639	0.9643
0.0478	14.0	686	0.0454	0.0454	0.1632	0.5797	0.9643
0.0478	15.0	735	0.0436	0.0436	0.1526	0.5957	0.9643
0.0478	16.0	784	0.0520	0.0520	0.1764	0.5178	0.9643
0.0478	17.0	833	0.0414	0.0414	0.1536	0.6166	0.9821
0.0478	18.0	882	0.0413	0.0413	0.1490	0.6176	0.9643
0.0478	19.0	931	0.0413	0.0413	0.1514	0.6174	0.9821
0.0478	20.0	980	0.0429	0.0429	0.1537	0.6023	0.9821

Framework versions

• Transformers 4.27.3
• Pytorch 1.13.1+cu116
• Datasets 2.10.1
• Tokenizers 0.13.2