---
library_name: transformers
license: apache-2.0
base_model: answerdotai/ModernBERT-base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: modernBert-base_v2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# modernBert-base_v2

This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7185
- Accuracy: 0.9116
- Precision Macro: 0.8041
- Recall Macro: 0.7362
- F1 Macro: 0.7592
- F1 Weighted: 0.9065

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision Macro | Recall Macro | F1 Macro | F1 Weighted |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------------:|:------------:|:--------:|:-----------:|
| 1.2139        | 1.0   | 90   | 0.5024          | 0.8073   | 0.8182          | 0.5993       | 0.6061   | 0.7934      |
| 0.6774        | 2.0   | 180  | 0.2870          | 0.9033   | 0.8421          | 0.7140       | 0.7451   | 0.8960      |
| 0.4571        | 3.0   | 270  | 0.3474          | 0.8920   | 0.8074          | 0.6669       | 0.6824   | 0.8802      |
| 0.2925        | 4.0   | 360  | 0.3089          | 0.9065   | 0.8778          | 0.7074       | 0.7413   | 0.8977      |
| 0.1725        | 5.0   | 450  | 0.3611          | 0.8958   | 0.7729          | 0.7574       | 0.7646   | 0.8946      |
| 0.0977        | 6.0   | 540  | 0.4743          | 0.9090   | 0.8405          | 0.7388       | 0.7695   | 0.9036      |
| 0.0576        | 7.0   | 630  | 0.6044          | 0.8743   | 0.7234          | 0.8019       | 0.7413   | 0.8878      |
| 0.0338        | 8.0   | 720  | 0.6118          | 0.9040   | 0.7756          | 0.7506       | 0.7615   | 0.9019      |
| 0.016         | 9.0   | 810  | 0.6754          | 0.9071   | 0.8334          | 0.7379       | 0.7670   | 0.9019      |
| 0.0113        | 10.0  | 900  | 0.6732          | 0.9065   | 0.7898          | 0.7606       | 0.7733   | 0.9044      |
| 0.0065        | 11.0  | 990  | 0.7871          | 0.9046   | 0.8046          | 0.7277       | 0.7519   | 0.8992      |
| 0.0037        | 12.0  | 1080 | 0.7134          | 0.9109   | 0.7989          | 0.7147       | 0.7386   | 0.9038      |
| 0.0022        | 13.0  | 1170 | 0.7784          | 0.9015   | 0.7765          | 0.7383       | 0.7529   | 0.8982      |
| 0.0013        | 14.0  | 1260 | 0.7176          | 0.9109   | 0.7832          | 0.7486       | 0.7625   | 0.9079      |
| 0.0011        | 15.0  | 1350 | 0.7681          | 0.9059   | 0.7920          | 0.7371       | 0.7565   | 0.9017      |
| 0.0001        | 16.0  | 1440 | 0.7170          | 0.9071   | 0.7833          | 0.7282       | 0.7479   | 0.9024      |
| 0.0007        | 17.0  | 1530 | 0.7219          | 0.9109   | 0.8022          | 0.7442       | 0.7652   | 0.9068      |
| 0.0003        | 18.0  | 1620 | 0.7379          | 0.9103   | 0.7950          | 0.7398       | 0.7596   | 0.9060      |
| 0.0006        | 19.0  | 1710 | 0.7198          | 0.9116   | 0.8074          | 0.7404       | 0.7635   | 0.9068      |
| 0.0004        | 20.0  | 1800 | 0.7185          | 0.9116   | 0.8041          | 0.7362       | 0.7592   | 0.9065      |


### Framework versions

- Transformers 4.55.0
- Pytorch 2.7.0+cu126
- Datasets 4.0.0
- Tokenizers 0.21.4