vietnamese-correction-finetuning

This model is a fine-tuned version of bmd1905/vietnamese-correction-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0479
• Sacrebleu: 98.4865

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 100000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Sacrebleu
0.0015	19.0840	5000	0.0479	98.4865
0.0009	38.1679	10000	0.0527	98.4236
0.0004	57.2519	15000	0.0565	98.4868
0.0005	76.3359	20000	0.0592	98.4490
0.0001	95.4198	25000	0.0635	98.4469
0.0005	114.5038	30000	0.0618	98.4037
0.0002	133.5878	35000	0.0683	98.2704
0.0001	152.6718	40000	0.0692	98.3530
0.0001	171.7557	45000	0.0645	98.4299
0.0002	190.8397	50000	0.0706	98.3816
0.0002	209.9237	55000	0.0683	98.3817
0.0001	229.0076	60000	0.0692	98.4246
0.0	248.0916	65000	0.0764	98.2300
0.0001	267.1756	70000	0.0706	98.3508
0.0	286.2595	75000	0.0711	98.3797
0.0	305.3435	80000	0.0727	98.4148
0.0	324.4275	85000	0.0744	98.3722
0.0	343.5115	90000	0.0723	98.3660
0.0	362.5954	95000	0.0760	98.3158
0.0	381.6794	100000	0.0763	98.3326

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0+cu128
• Datasets 4.4.2
• Tokenizers 0.22.2