Update README.md
Browse files
README.md
CHANGED
|
@@ -17,13 +17,10 @@ BERT-as-a-Judge is a family of encoder-based models designed for efficient, refe
|
|
| 17 |
- **Model Type:** Encoder-based Judge (EuroBERT-210m backbone)
|
| 18 |
- **Language:** English
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
## Model Variations & Collection Overview
|
| 23 |
|
| 24 |
The models are named using the convention: `BERTJudge-<Output_Guidelines>-<Input_Format>-<Additional_Info>`.
|
| 25 |
|
| 26 |
-
### Naming Convention Breakdown:
|
| 27 |
* **Output Guidelines:** * `Free`: Trained on unconstrained model outputs.
|
| 28 |
* `Formatted`: Trained on outputs constrained by specific instructions (e.g., "Conclude with Answer: [X]").
|
| 29 |
* **Input Format:** * `QCR`: Input contains [Question, Candidate, Reference].
|
|
@@ -31,24 +28,8 @@ The models are named using the convention: `BERTJudge-<Output_Guidelines>-<Input
|
|
| 31 |
* **Additional Info:** * `OOD`: Evaluates Out-of-Distribution performance (certain generative models excluded from training).
|
| 32 |
* `100k/200k/500k`: Number of training steps (Default is 1 Million).
|
| 33 |
|
| 34 |
-
### Model Selection Table
|
| 35 |
-
|
| 36 |
-
| Model Name | Input Format | Guidelines | Training Steps | OOD Tested |
|
| 37 |
-
| :--- | :---: | :---: | :---: | :---: |
|
| 38 |
-
| **BERTJudge-Free-QCR** | QCR | Free | 1M | No |
|
| 39 |
-
| **BERTJudge-Formatted-QCR** | QCR | Formatted | 1M | No |
|
| 40 |
-
| **BERTJudge-Free-CR** | CR | Free | 1M | No |
|
| 41 |
-
| **BERTJudge-Free-QCR-OOD** | QCR | Free | 1M | **Yes** |
|
| 42 |
-
| **BERTJudge-Free-QCR-100k** | QCR | Free | 100k | No |
|
| 43 |
-
| **BERTJudge-Free-QCR-200k** | QCR | Free | 200k | No |
|
| 44 |
-
| **BERTJudge-Free-QCR-500k** | QCR | Free | 500k | No |
|
| 45 |
-
|
| 46 |
-
---
|
| 47 |
-
|
| 48 |
## Intended Use
|
| 49 |
|
| 50 |
-
### How to Use
|
| 51 |
-
|
| 52 |
These models are typically used as sequence classifiers that output a score (0 for incorrect, 1 for correct).
|
| 53 |
|
| 54 |
```python
|
|
|
|
| 17 |
- **Model Type:** Encoder-based Judge (EuroBERT-210m backbone)
|
| 18 |
- **Language:** English
|
| 19 |
|
| 20 |
+
## Naming Convention Breakdown
|
|
|
|
|
|
|
| 21 |
|
| 22 |
The models are named using the convention: `BERTJudge-<Output_Guidelines>-<Input_Format>-<Additional_Info>`.
|
| 23 |
|
|
|
|
| 24 |
* **Output Guidelines:** * `Free`: Trained on unconstrained model outputs.
|
| 25 |
* `Formatted`: Trained on outputs constrained by specific instructions (e.g., "Conclude with Answer: [X]").
|
| 26 |
* **Input Format:** * `QCR`: Input contains [Question, Candidate, Reference].
|
|
|
|
| 28 |
* **Additional Info:** * `OOD`: Evaluates Out-of-Distribution performance (certain generative models excluded from training).
|
| 29 |
* `100k/200k/500k`: Number of training steps (Default is 1 Million).
|
| 30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
## Intended Use
|
| 32 |
|
|
|
|
|
|
|
| 33 |
These models are typically used as sequence classifiers that output a score (0 for incorrect, 1 for correct).
|
| 34 |
|
| 35 |
```python
|