---
license: apache-2.0
base_model: FacebookAI/roberta-large
library_name: transformers
pipeline_tag: text-classification
tags:
- roberta
- safety
- judge
- probguard
- calibeval
---

# CalibEval

CalibEval is a RoBERTa-large sequence-classification safety judge used by ProbGuard to label sampled continuations during calibration and evaluation.

## Usage

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo_id = "hxz-sec/CalibEval"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
```

The model outputs two labels for safety classification. In the EarlyGuard evaluation scripts, the unsafe probability is read from the positive class and thresholded at `0.5` by default.