wnkh/byt5-ocr-correction

byt5-medieval-ocr

This model is a fine-tuned version of google/byt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4056
• Cer: 0.2238
• Cer Raw: 0.2395
• Exact Match: 0.1493

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Cer Raw	Exact Match
0.8309	0.1527	500	0.7005	0.3158	0.3339	0.0400
0.6942	0.3055	1000	0.6005	0.2811	0.3007	0.0600
0.6336	0.4582	1500	0.5562	0.2657	0.2847	0.0773
0.5814	0.6110	2000	0.5197	0.2589	0.2779	0.0870
0.5573	0.7637	2500	0.4926	0.2620	0.2792	0.0964
0.5431	0.9165	3000	0.4790	0.2504	0.2691	0.1024
0.4991	1.0690	3500	0.4698	0.2509	0.2684	0.1107
0.4901	1.2218	4000	0.4595	0.2517	0.2685	0.1132
0.4792	1.3745	4500	0.4453	0.2445	0.2611	0.1200
0.463	1.5273	5000	0.4411	0.2366	0.2533	0.1280
0.4682	1.6800	5500	0.4333	0.2364	0.2522	0.1291
0.4451	1.8327	6000	0.4241	0.2312	0.2473	0.1380
0.4526	1.9855	6500	0.4174	0.2363	0.2516	0.1394
0.4055	2.1381	7000	0.4187	0.2290	0.2444	0.1416
0.4046	2.2908	7500	0.4178	0.2252	0.2412	0.1452
0.3895	2.4436	8000	0.4103	0.2280	0.2436	0.1473
0.4058	2.5963	8500	0.4073	0.2263	0.2417	0.1480
0.4	2.7490	9000	0.4056	0.2238	0.2395	0.1493
0.4014	2.9018	9500	0.4034	0.2242	0.2396	0.1528

Framework versions

• Transformers 4.53.2
• Pytorch 2.7.0+cu126
• Datasets 3.0.2
• Tokenizers 0.21.4