Instructions to use artefactory/BERTJudge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use artefactory/BERTJudge with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="artefactory/BERTJudge", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("artefactory/BERTJudge", trust_remote_code=True) model = AutoModelForSequenceClassification.from_pretrained("artefactory/BERTJudge", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,21 +19,21 @@ BERT-as-a-Judge is a family of encoder-based models designed for efficient, refe
|
|
| 19 |
|
| 20 |
## Naming Convention Breakdown
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
* **Output Guidelines:**
|
| 25 |
-
* `Free`: Trained on unconstrained model
|
| 26 |
-
* `Formatted`: Trained on outputs
|
| 27 |
* **Input Format:**
|
| 28 |
-
* `QCR`:
|
| 29 |
-
* `CR`:
|
| 30 |
* **Additional Info:**
|
| 31 |
-
* `OOD`:
|
| 32 |
-
* `100k/200k/500k`:
|
| 33 |
|
| 34 |
## Intended Use
|
| 35 |
|
| 36 |
-
These models
|
| 37 |
|
| 38 |
```python
|
| 39 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
|
|
|
| 19 |
|
| 20 |
## Naming Convention Breakdown
|
| 21 |
|
| 22 |
+
Models follow a standardized naming structure: `BERTJudge-<Output_Guidelines>-<Input_Format>-<Additional_Info>`.
|
| 23 |
|
| 24 |
* **Output Guidelines:**
|
| 25 |
+
* `Free`: Trained on unconstrained model generations.
|
| 26 |
+
* `Formatted`: Trained on outputs that adhere to specific structural constraints. For optimized evaluation under the formatted setup, candidate outputs should ideally conclude with `Final answer: <final_answer>` (see the paper for details).
|
| 27 |
* **Input Format:**
|
| 28 |
+
* `QCR`: The input sequence consists of [Question, Candidate, Reference].
|
| 29 |
+
* `CR`: The input sequence consists only of [Candidate, Reference].
|
| 30 |
* **Additional Info:**
|
| 31 |
+
* `OOD`: Indicates evaluation of Out-of-Distribution performance (where specific generative models were withheld during training).
|
| 32 |
+
* `100k/200k/500k`: Denotes the total training steps (the standard default is 1 million).
|
| 33 |
|
| 34 |
## Intended Use
|
| 35 |
|
| 36 |
+
These models function as sequence classifiers that generate a binary score (0 for incorrect, 1 for correct). For general applications, we recommend **BERTJudge-Free-QCR**, as it provides the most robust and highest-performing evaluation.
|
| 37 |
|
| 38 |
```python
|
| 39 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|