variants-ner-modernbert-base

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0705
• Precision: 0.8194
• Recall: 0.8887
• F1: 0.8526
• Accuracy: 0.9901

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Accuracy	F1	Validation Loss	Precision	Recall
0.6763	1.0	757	0.9843	0.7304	0.0400	0.6640	0.8115
0.2144	2.0	1514	0.9882	0.8050	0.0302	0.7767	0.8353
0.1276	3.0	2271	0.9889	0.7944	0.0272	0.7766	0.8131
0.0865	4.0	3028	0.9891	0.8207	0.0309	0.7947	0.8486
0.0529	5.0	3785	0.9882	0.8090	0.0341	0.7666	0.8564
0.0385	6.0	4542	0.9904	0.8383	0.0337	0.8094	0.8694
0.0212	7.0	5299	0.9901	0.8388	0.0384	0.8121	0.8673
0.0132	8.0	6056	0.9896	0.8273	0.0417	0.8002	0.8564
0.0088	9.0	6813	0.9897	0.8355	0.0473	0.8087	0.8641
0.0055	10.0	7570	0.9898	0.8375	0.0498	0.8078	0.8694
0.0346	11.0	8327	0.0458	0.7958	0.8668	0.8298	0.9891
0.0334	12.0	9084	0.0395	0.7841	0.8618	0.8211	0.9893
0.0241	13.0	9841	0.0433	0.8011	0.8544	0.8269	0.9887
0.0114	14.0	10598	0.0488	0.7874	0.8519	0.8184	0.9892
0.0102	15.0	11355	0.0500	0.8068	0.8698	0.8371	0.9897
0.0087	16.0	12112	0.0557	0.8107	0.8703	0.8395	0.9895
0.0070	17.0	12869	0.0620	0.8016	0.8735	0.8360	0.9892
0.0050	18.0	13626	0.0503	0.8074	0.8597	0.8327	0.9893
0.0089	19.0	14383	0.0561	0.8333	0.8592	0.8460	0.9899
0.0058	20.0	15140	0.0569	0.8051	0.8364	0.8205	0.9891
0.0030	21.0	15897	0.0581	0.8112	0.8682	0.8387	0.9900
0.0015	22.0	16654	0.0608	0.8206	0.8689	0.8441	0.9901
0.0004	23.0	17411	0.0633	0.8180	0.8645	0.8406	0.9899
0.0005	24.0	18168	0.0663	0.8195	0.8793	0.8484	0.9901
0.0001	25.0	18925	0.0705	0.8194	0.8887	0.8526	0.9901
0.0002	26.0	19682	0.0687	0.8254	0.8686	0.8464	0.9901
0.0002	27.0	20439	0.0695	0.8220	0.8784	0.8493	0.9901
0.0000	28.0	21196	0.0717	0.8253	0.8728	0.8484	0.9901
0.0001	29.0	21953	0.0735	0.8245	0.8776	0.8502	0.9901
0.0000	30.0	22710	0.0741	0.8254	0.8769	0.8503	0.9901

Framework versions

• Transformers 5.2.0
• Pytorch 2.10.0+cu128
• Datasets 4.6.0
• Tokenizers 0.22.2