deberta-v3-large-survey-fluency-rater

This model is a fine-tuned version of microsoft/deberta-v3-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3349
• Krippendorff: 0.4934
• Spearman: 0.5298
• Absolute Agreement: 0.4862
• Agreement Within One: 0.8631

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Krippendorff	Spearman	Absolute Agreement	Agreement Within One
No log	1.0	50	1.9418	-0.2790	-0.0817	0.0417	0.9306
No log	2.0	100	1.9563	-0.3968	-0.3236	0.0417	0.8889
No log	3.0	150	1.9360	-0.2790	-0.0817	0.0417	0.9306
No log	4.0	200	1.9826	-0.3968	-0.3236	0.0417	0.8889
No log	5.0	250	1.9908	-0.3665	-0.3871	0.0556	0.7917
No log	6.0	300	1.8144	-0.2888	-0.1075	0.2778	0.7222
No log	7.0	350	1.9314	-0.1378	-0.0729	0.2639	0.75
No log	8.0	400	2.0468	-0.3631	-0.3911	0.1389	0.75
No log	9.0	450	2.0021	-0.1594	-0.1099	0.25	0.7639
1.536	10.0	500	2.0445	-0.2759	-0.1253	0.2639	0.7222
1.536	11.0	550	1.9957	-0.1583	-0.1012	0.2361	0.7917
1.536	12.0	600	2.0293	-0.2232	-0.1460	0.25	0.7639
1.536	13.0	650	2.0920	-0.2296	-0.2063	0.1667	0.8056
1.536	14.0	700	2.2456	-0.2580	-0.3090	0.2083	0.75
1.536	15.0	750	2.2635	-0.1791	-0.1913	0.2778	0.75
1.536	16.0	800	2.4517	-0.1779	-0.2208	0.25	0.7639
1.536	17.0	850	2.5741	-0.2354	-0.3037	0.2361	0.7361
1.536	18.0	900	2.6429	-0.1990	-0.2052	0.2361	0.7639
1.536	19.0	950	2.8684	-0.2073	-0.2513	0.1944	0.7639
0.7381	20.0	1000	2.9184	-0.2585	-0.2424	0.2639	0.7361

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.12.1