hgissbkh commited on
Commit
84cffef
·
verified ·
1 Parent(s): bfb695d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -19,21 +19,21 @@ BERT-as-a-Judge is a family of encoder-based models designed for efficient, refe
19
 
20
  ## Naming Convention Breakdown
21
 
22
- The models are named using the convention: `BERTJudge-<Output_Guidelines>-<Input_Format>-<Additional_Info>`.
23
 
24
  * **Output Guidelines:**
25
- * `Free`: Trained on unconstrained model outputs.
26
- * `Formatted`: Trained on outputs constrained by specific instructions (e.g., "Conclude with Answer: [X]").
27
  * **Input Format:**
28
- * `QCR`: Input contains [Question, Candidate, Reference].
29
- * `CR`: Input contains only [Candidate, Reference].
30
  * **Additional Info:**
31
- * `OOD`: Evaluates Out-of-Distribution performance (certain generative models excluded from training).
32
- * `100k/200k/500k`: Number of training steps (default is 1 Million).
33
 
34
  ## Intended Use
35
 
36
- These models are typically used as sequence classifiers that output a score (0 for incorrect, 1 for correct).
37
 
38
  ```python
39
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
19
 
20
  ## Naming Convention Breakdown
21
 
22
+ Models follow a standardized naming structure: `BERTJudge-<Output_Guidelines>-<Input_Format>-<Additional_Info>`.
23
 
24
  * **Output Guidelines:**
25
+ * `Free`: Trained on unconstrained model generations.
26
+ * `Formatted`: Trained on outputs that adhere to specific structural constraints. For optimized evaluation under the formatted setup, candidate outputs should ideally conclude with `Final answer: <final_answer>` (see the paper for details).
27
  * **Input Format:**
28
+ * `QCR`: The input sequence consists of [Question, Candidate, Reference].
29
+ * `CR`: The input sequence consists only of [Candidate, Reference].
30
  * **Additional Info:**
31
+ * `OOD`: Indicates evaluation of Out-of-Distribution performance (where specific generative models were withheld during training).
32
+ * `100k/200k/500k`: Denotes the total training steps (the standard default is 1 million).
33
 
34
  ## Intended Use
35
 
36
+ These models function as sequence classifiers that generate a binary score (0 for incorrect, 1 for correct). For general applications, we recommend **BERTJudge-Free-QCR**, as it provides the most robust and highest-performing evaluation.
37
 
38
  ```python
39
  from transformers import AutoTokenizer, AutoModelForSequenceClassification