Spaces:
Sleeping
Sleeping
Adding hyperlinks to the datasets and framework used for evals
Browse files
app.py
CHANGED
|
@@ -47,8 +47,9 @@ with gr.Blocks(title="LLM Propensity Evaluation Leaderboard") as demo:
|
|
| 47 |
* AI regulators and policymakers interested in the alignment and safety aspects of widely used (popular) language models.
|
| 48 |
|
| 49 |
## Evaluation Details:
|
| 50 |
-
- **Instruction Following Score**: Measures a model's tendency to follow instructions accurately. Measured using the IFEval dataset.
|
| 51 |
-
- **Uncommon Facts Hallucination Rate**: Evaluates how often a model hallucinates when questioned on facts. Measured using a subset of the SimpleQA dataset, which explicitly asks uncommon facts. We calculated the rate using this formula : (1 - (correct + not_attempted)), where correct = when the model answered a question correctly and not_attempted = when a model admits to not knowing the answer to a question.*
|
|
|
|
| 52 |
|
| 53 |
## How to Interpret the Scores:
|
| 54 |
* Instruction Following Score: Higher scores indicate better adherence to instructions.
|
|
|
|
| 47 |
* AI regulators and policymakers interested in the alignment and safety aspects of widely used (popular) language models.
|
| 48 |
|
| 49 |
## Evaluation Details:
|
| 50 |
+
- **Instruction Following Score**: Measures a model's tendency to follow instructions accurately. Measured using the **[IFEval](https://arxiv.org/pdf/2311.07911)** dataset.
|
| 51 |
+
- **Uncommon Facts Hallucination Rate**: Evaluates how often a model hallucinates when questioned on facts. Measured using a subset of the **[SimpleQA](https://arxiv.org/abs/2411.04368)** dataset, which explicitly asks uncommon facts. We calculated the rate using this formula : (1 - (correct + not_attempted)), where correct = when the model answered a question correctly and not_attempted = when a model admits to not knowing the answer to a question.*
|
| 52 |
+
- All evals have been run using the **[Inspect](https://github.com/UKGovernmentBEIS/inspect_evals)** framework from UK AISI.
|
| 53 |
|
| 54 |
## How to Interpret the Scores:
|
| 55 |
* Instruction Following Score: Higher scores indicate better adherence to instructions.
|