---
library_name: transformers
license: mit
base_model: microsoft/deberta-v3-large
tags:
- single_label_classification
- question-answering
- text-classification
- generated_from_trainer
datasets:
- beavertails
metrics:
- accuracy
model-index:
- name: QA-DeBERTa-IGPooling-binary
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: saiteki-kai/Beavertails-it
      type: beavertails
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8652227434541039
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# QA-DeBERTa-IGPooling-binary

This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the saiteki-kai/Beavertails-it dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4138
- Accuracy: 0.8652
- Unsafe Precision: 0.8925
- Unsafe Recall: 0.8616
- Unsafe F1: 0.8768
- Unsafe Fpr: 0.1302
- Unsafe Aucpr: 0.9567
- Safe Precision: 0.8336
- Safe Recall: 0.8698
- Safe F1: 0.8513
- Safe Fpr: 0.1384
- Safe Aucpr: 0.9230

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 6e-06
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 3

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy | Unsafe Precision | Unsafe Recall | Unsafe F1 | Unsafe Fpr | Unsafe Aucpr | Safe Precision | Safe Recall | Safe F1 | Safe Fpr | Safe Aucpr |
|:-------------:|:------:|:-----:|:---------------:|:--------:|:----------------:|:-------------:|:---------:|:----------:|:------------:|:--------------:|:-----------:|:-------:|:--------:|:----------:|
| 0.4537        | 0.3001 | 5073  | 0.4406          | 0.8504   | 0.8537           | 0.8822        | 0.8678    | 0.1896     | 0.9479       | 0.8458         | 0.8104      | 0.8277  | 0.1178   | 0.9067     |
| 0.4184        | 0.6001 | 10146 | 0.4195          | 0.8592   | 0.8815           | 0.8631        | 0.8722    | 0.1456     | 0.9519       | 0.8326         | 0.8544      | 0.8434  | 0.1369   | 0.9133     |
| 0.4022        | 0.9002 | 15219 | 0.4160          | 0.8613   | 0.8783           | 0.8717        | 0.8749    | 0.1516     | 0.9536       | 0.8405         | 0.8484      | 0.8444  | 0.1283   | 0.9172     |
| 0.3991        | 1.2002 | 20292 | 0.4309          | 0.8618   | 0.8945           | 0.8521        | 0.8728    | 0.1261     | 0.9534       | 0.8248         | 0.8739      | 0.8487  | 0.1479   | 0.9193     |
| 0.4094        | 1.5003 | 25365 | 0.4102          | 0.8634   | 0.8847           | 0.8675        | 0.8760    | 0.1418     | 0.9554       | 0.8377         | 0.8582      | 0.8478  | 0.1325   | 0.9220     |
| 0.3822        | 1.8003 | 30438 | 0.4139          | 0.8652   | 0.8924           | 0.8616        | 0.8767    | 0.1303     | 0.9567       | 0.8336         | 0.8697      | 0.8512  | 0.1384   | 0.9230     |
| 0.3505        | 2.1004 | 35511 | 0.4229          | 0.8639   | 0.8871           | 0.8656        | 0.8762    | 0.1382     | 0.9565       | 0.8363         | 0.8618      | 0.8489  | 0.1344   | 0.9226     |
| 0.4044        | 2.4004 | 40584 | 0.4270          | 0.8634   | 0.8824           | 0.8705        | 0.8764    | 0.1455     | 0.9566       | 0.8403         | 0.8545      | 0.8473  | 0.1295   | 0.9236     |
| 0.3484        | 2.7005 | 45657 | 0.4222          | 0.8638   | 0.8860           | 0.8668        | 0.8763    | 0.1399     | 0.9567       | 0.8373         | 0.8601      | 0.8485  | 0.1332   | 0.9238     |


### Framework versions

- Transformers 4.57.1
- Pytorch 2.9.1+cu128
- Datasets 4.4.1
- Tokenizers 0.22.1