---
license: mit
base_model: microsoft/deberta-v3-base
tags:
- generated_from_trainer
metrics:
- accuracy
- precision
- recall
- f1
model-index:
- name: DeBERT_50K_steps
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# DeBERT_50K_steps

This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0169
- Accuracy: 0.9941
- Precision: 0.7649
- Recall: 0.5670
- F1: 0.6512
- Hamming: 0.0059

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- training_steps: 50000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision | Recall | F1     | Hamming |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:------:|:-------:|
| 0.2014        | 0.02  | 2500  | 0.0451          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
| 0.0373        | 0.04  | 5000  | 0.0297          | 0.9913   | 0.6879    | 0.2003 | 0.3102 | 0.0087  |
| 0.0286        | 0.06  | 7500  | 0.0250          | 0.9921   | 0.6965    | 0.3329 | 0.4505 | 0.0079  |
| 0.0253        | 0.08  | 10000 | 0.0233          | 0.9925   | 0.7038    | 0.4010 | 0.5109 | 0.0075  |
| 0.0234        | 0.1   | 12500 | 0.0217          | 0.9928   | 0.7085    | 0.4382 | 0.5415 | 0.0072  |
| 0.0223        | 0.12  | 15000 | 0.0208          | 0.9930   | 0.7229    | 0.4559 | 0.5591 | 0.0070  |
| 0.0213        | 0.14  | 17500 | 0.0205          | 0.9931   | 0.7255    | 0.4696 | 0.5701 | 0.0069  |
| 0.0206        | 0.16  | 20000 | 0.0196          | 0.9933   | 0.7325    | 0.4990 | 0.5936 | 0.0067  |
| 0.0203        | 0.18  | 22500 | 0.0191          | 0.9935   | 0.7368    | 0.5125 | 0.6045 | 0.0065  |
| 0.0196        | 0.2   | 25000 | 0.0188          | 0.9935   | 0.7354    | 0.5209 | 0.6098 | 0.0065  |
| 0.0195        | 0.22  | 27500 | 0.0185          | 0.9936   | 0.7415    | 0.5335 | 0.6205 | 0.0064  |
| 0.019         | 0.24  | 30000 | 0.0183          | 0.9936   | 0.7437    | 0.5296 | 0.6186 | 0.0064  |
| 0.0189        | 0.26  | 32500 | 0.0180          | 0.9938   | 0.7585    | 0.5304 | 0.6243 | 0.0062  |
| 0.0187        | 0.28  | 35000 | 0.0178          | 0.9938   | 0.7630    | 0.5342 | 0.6284 | 0.0062  |
| 0.0184        | 0.3   | 37500 | 0.0175          | 0.9939   | 0.7626    | 0.5457 | 0.6362 | 0.0061  |
| 0.0182        | 0.32  | 40000 | 0.0174          | 0.9939   | 0.7621    | 0.5451 | 0.6356 | 0.0061  |
| 0.0179        | 0.34  | 42500 | 0.0172          | 0.9940   | 0.7594    | 0.5563 | 0.6422 | 0.0060  |
| 0.0178        | 0.36  | 45000 | 0.0171          | 0.9940   | 0.7553    | 0.5633 | 0.6453 | 0.0060  |
| 0.0177        | 0.38  | 47500 | 0.0170          | 0.9941   | 0.7623    | 0.5680 | 0.6510 | 0.0059  |
| 0.0175        | 0.4   | 50000 | 0.0169          | 0.9941   | 0.7649    | 0.5670 | 0.6512 | 0.0059  |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.7.1
- Tokenizers 0.14.1