metadata
license: apache-2.0
base_model: FacebookAI/roberta-large
library_name: transformers
pipeline_tag: text-classification
tags:
- roberta
- safety
- judge
- probguard
- calibeval
CalibEval
CalibEval is a RoBERTa-large sequence-classification safety judge used by ProbGuard to label sampled continuations during calibration and evaluation.
Usage
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo_id = "hxz-sec/CalibEval"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
The model outputs two labels for safety classification. In the EarlyGuard evaluation scripts, the unsafe probability is read from the positive class and thresholded at 0.5 by default.