---
library_name: transformers
license: mit
base_model: microsoft/deberta-v3-large
tags:
- single_label_classification
- question-answering
- text-classification
- generated_from_trainer
datasets:
- beavertails
metrics:
- accuracy
model-index:
- name: QA-DeBERTa-v3-large-diff-binary-2
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: saiteki-kai/Beavertails-it
      type: beavertails
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8608643577203313
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# QA-DeBERTa-v3-large-diff-binary-2

This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the saiteki-kai/Beavertails-it dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3409
- Accuracy: 0.8609
- Unsafe Precision: 0.8682
- Unsafe Recall: 0.8842
- Unsafe F1: 0.8761
- Unsafe Fpr: 0.1684
- Unsafe Aucpr: 0.9538
- Safe Precision: 0.8512
- Safe Recall: 0.8316
- Safe F1: 0.8413
- Safe Fpr: 0.1158
- Safe Aucpr: 0.9184

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 6e-06
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 10

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy | Unsafe Precision | Unsafe Recall | Unsafe F1 | Unsafe Fpr | Unsafe Aucpr | Safe Precision | Safe Recall | Safe F1 | Safe Fpr | Safe Aucpr |
|:-------------:|:------:|:-----:|:---------------:|:--------:|:----------------:|:-------------:|:---------:|:----------:|:------------:|:--------------:|:-----------:|:-------:|:--------:|:----------:|
| 0.2998        | 0.2501 | 2114  | 0.3677          | 0.8446   | 0.9027           | 0.8078        | 0.8526    | 0.1093     | 0.9436       | 0.7870         | 0.8907      | 0.8356  | 0.1922   | 0.8961     |
| 0.3262        | 0.5001 | 4228  | 0.3278          | 0.8561   | 0.8786           | 0.8602        | 0.8693    | 0.1491     | 0.9495       | 0.8291         | 0.8509      | 0.8399  | 0.1398   | 0.9087     |
| 0.3019        | 0.7502 | 6342  | 0.3236          | 0.8588   | 0.8972           | 0.8429        | 0.8692    | 0.1211     | 0.9527       | 0.8168         | 0.8789      | 0.8467  | 0.1571   | 0.9155     |
| 0.3479        | 1.0002 | 8456  | 0.3215          | 0.8599   | 0.8690           | 0.8811        | 0.8750    | 0.1666     | 0.9531       | 0.8482         | 0.8334      | 0.8407  | 0.1189   | 0.9175     |
| 0.302         | 1.2503 | 10570 | 0.3221          | 0.8611   | 0.8839           | 0.8639        | 0.8738    | 0.1423     | 0.9536       | 0.8340         | 0.8577      | 0.8457  | 0.1361   | 0.9176     |
| 0.2663        | 1.5004 | 12684 | 0.3409          | 0.8609   | 0.8682           | 0.8842        | 0.8761    | 0.1684     | 0.9538       | 0.8512         | 0.8316      | 0.8413  | 0.1158   | 0.9184     |
| 0.2841        | 1.7504 | 14798 | 0.3223          | 0.8622   | 0.8772           | 0.8748        | 0.8760    | 0.1537     | 0.9551       | 0.8435         | 0.8463      | 0.8449  | 0.1252   | 0.9202     |
| 0.3074        | 2.0005 | 16912 | 0.3244          | 0.8632   | 0.8995           | 0.8490        | 0.8735    | 0.1190     | 0.9553       | 0.8230         | 0.8810      | 0.8510  | 0.1510   | 0.9182     |
| 0.3052        | 2.2505 | 19026 | 0.3200          | 0.8618   | 0.8833           | 0.8660        | 0.8746    | 0.1435     | 0.9546       | 0.8359         | 0.8565      | 0.8461  | 0.1340   | 0.9221     |
| 0.268         | 2.5006 | 21140 | 0.3192          | 0.8627   | 0.8876           | 0.8625        | 0.8748    | 0.1370     | 0.9550       | 0.8334         | 0.8630      | 0.8479  | 0.1375   | 0.9220     |


### Framework versions

- Transformers 4.57.3
- Pytorch 2.7.1+cu118
- Datasets 4.4.1
- Tokenizers 0.22.1