spell_correction_M01_LM

This model is a fine-tuned version of facebook/bart-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0170

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	264	0.3516
1.8892	2.0	528	0.0561
1.8892	3.0	792	0.0324
0.0396	4.0	1056	0.0264
0.0396	5.0	1320	0.0238
0.0163	6.0	1584	0.0212
0.0163	7.0	1848	0.0212
0.0081	8.0	2112	0.0194
0.0081	9.0	2376	0.0206
0.0061	10.0	2640	0.0195
0.0061	11.0	2904	0.0182
0.004	12.0	3168	0.0182
0.004	13.0	3432	0.0180
0.0042	14.0	3696	0.0172
0.0042	15.0	3960	0.0183
0.0026	16.0	4224	0.0184
0.0026	17.0	4488	0.0170
0.0034	18.0	4752	0.0169
0.0023	19.0	5016	0.0169
0.0023	20.0	5280	0.0167
0.0029	21.0	5544	0.0182
0.0029	22.0	5808	0.0178
0.0019	23.0	6072	0.0188
0.0019	24.0	6336	0.0179
0.0018	25.0	6600	0.0177
0.0018	26.0	6864	0.0171
0.0023	27.0	7128	0.0172
0.0023	28.0	7392	0.0171
0.0021	29.0	7656	0.0169
0.0021	30.0	7920	0.0170

Framework versions

• Transformers 4.28.0
• Pytorch 1.12.1+cu102
• Datasets 2.13.1
• Tokenizers 0.13.3