pixel-base-finetune-sent

This model is a fine-tuned version of Team-PIXEL/pixel-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.3326
• Accuracy: 0.3892
• Qwk: 0.6388
• Mae: 1.8358

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• training_steps: 50000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Qwk	Mae
1.9432	1.17	1000	1.9813	0.2903	0.6152	1.9717
1.7884	2.33	2000	1.8716	0.3483	0.6381	1.8553
1.6395	3.5	3000	1.7870	0.3800	0.6360	1.8591
1.5441	4.67	4000	1.7913	0.3904	0.6136	1.8680
1.4381	5.83	5000	1.8173	0.4015	0.6175	1.8315
1.3416	7.0	6000	1.8701	0.4048	0.6326	1.8596
1.1117	8.17	7000	2.0323	0.4019	0.6205	1.8443
0.9358	9.33	8000	2.2000	0.3918	0.6131	1.8417
0.8253	10.5	9000	2.3646	0.3874	0.6301	1.8466
0.7368	11.67	10000	2.5149	0.3705	0.6186	1.8927
0.6426	12.84	11000	2.7221	0.3833	0.6394	1.8286
0.5436	14.0	12000	2.9190	0.3732	0.6317	1.8830
0.3693	15.17	13000	3.1721	0.3683	0.6259	1.8774
0.3122	16.34	14000	3.4346	0.3726	0.6375	1.8766
0.2793	17.5	15000	3.5113	0.3800	0.6130	1.8858
0.2391	18.67	16000	3.6291	0.3724	0.5980	1.9398
0.2158	19.84	17000	3.7513	0.3761	0.6269	1.8756
0.19	21.0	18000	3.9027	0.3750	0.6211	1.8698
0.1414	22.17	19000	4.0394	0.3698	0.6385	1.8544
0.1291	23.34	20000	4.0933	0.3750	0.6207	1.8874
0.1224	24.5	21000	4.3359	0.3595	0.6202	1.9335
0.1102	25.67	22000	4.4307	0.3648	0.6204	1.8930
0.1023	26.84	23000	4.4486	0.3866	0.6257	1.8577
0.1066	28.0	24000	4.4646	0.3847	0.6341	1.8550
0.0805	29.17	25000	4.6658	0.3880	0.6249	1.8782
0.08	30.34	26000	4.7634	0.3817	0.6196	1.8784
0.0661	31.51	27000	4.8402	0.3802	0.6244	1.8792
0.0662	32.67	28000	4.9351	0.3787	0.6271	1.8936
0.0601	33.84	29000	5.0376	0.3762	0.6294	1.8736
0.0581	35.01	30000	5.0760	0.3788	0.6269	1.8772
0.0507	36.17	31000	5.3750	0.3761	0.6288	1.8773
0.0485	37.34	32000	5.3407	0.3862	0.6280	1.8542
0.0436	38.51	33000	5.4958	0.3778	0.6356	1.8647
0.0384	39.67	34000	5.5773	0.3859	0.6357	1.8421
0.0357	40.84	35000	5.6658	0.3763	0.6233	1.8824
0.0341	42.01	36000	5.7353	0.3881	0.6377	1.8453
0.0274	43.17	37000	5.9293	0.3752	0.6272	1.8683
0.0324	44.34	38000	5.9421	0.3763	0.6367	1.8514
0.025	45.51	39000	5.9282	0.3770	0.6325	1.8892
0.0207	46.67	40000	6.0769	0.3862	0.6274	1.8492
0.0269	47.84	41000	6.1493	0.3777	0.6328	1.8958
0.0223	49.01	42000	6.1975	0.3724	0.6288	1.8969
0.0176	50.18	43000	6.2215	0.3847	0.6216	1.8710
0.0138	51.34	44000	6.2297	0.3896	0.6426	1.8234
0.0147	52.51	45000	6.2672	0.3886	0.6364	1.8413
0.0148	53.68	46000	6.3318	0.3860	0.6294	1.8544
0.0125	54.84	47000	6.3455	0.3860	0.6329	1.8436
0.0146	56.01	48000	6.3048	0.3904	0.6346	1.8435
0.0142	57.18	49000	6.3378	0.3874	0.6374	1.8393
0.0138	58.34	50000	6.3326	0.3892	0.6388	1.8358

Framework versions

• Transformers 4.17.0
• Pytorch 2.5.1
• Datasets 3.6.0
• Tokenizers 0.21.1