hxz-sec commited on
Commit
d0cf4f8
·
verified ·
1 Parent(s): e5a4711

Update model card

Browse files
Files changed (1) hide show
  1. README.md +28 -0
README.md ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: FacebookAI/roberta-large
4
+ library_name: transformers
5
+ pipeline_tag: text-classification
6
+ tags:
7
+ - roberta
8
+ - safety
9
+ - judge
10
+ - probguard
11
+ - calibeval
12
+ ---
13
+
14
+ # CalibEval
15
+
16
+ CalibEval is a RoBERTa-large sequence-classification safety judge used by ProbGuard to label sampled continuations during calibration and evaluation.
17
+
18
+ ## Usage
19
+
20
+ ```python
21
+ from transformers import AutoModelForSequenceClassification, AutoTokenizer
22
+
23
+ repo_id = "hxz-sec/CalibEval"
24
+ tokenizer = AutoTokenizer.from_pretrained(repo_id)
25
+ model = AutoModelForSequenceClassification.from_pretrained(repo_id)
26
+ ```
27
+
28
+ The model outputs two labels for safety classification. In the EarlyGuard evaluation scripts, the unsafe probability is read from the positive class and thresholded at `0.5` by default.