Spaces:
Running
Running
Docs: Average column is item-weighted mean
Browse files- src/about.py +2 -2
src/about.py
CHANGED
|
@@ -45,7 +45,7 @@ The **Arabic LLM Leaderboard** measures how well large language models understan
|
|
| 45 |
a **synthetic MSA** track and a **webscraped + aptitude** track — graded by strong LLM judges.
|
| 46 |
|
| 47 |
Each submission is scored on **13 capability and domain dimensions** (7 synthetic capability categories and
|
| 48 |
-
6 webscraped domains/groups); the **Average ⬆️** column is the mean across all 13. See the **📝 About** tab for
|
| 49 |
the full dataset composition, scoring methodology, and how to reproduce results, and the **🚀 Submit** tab to add
|
| 50 |
your own model.
|
| 51 |
"""
|
|
@@ -70,7 +70,7 @@ percentages.
|
|
| 70 |
the metric is a single judged-accuracy score per dimension.
|
| 71 |
- The two tracks use different solver/judge pairs (see table above) and their scores are **kept per track**.
|
| 72 |
- The leaderboard's per-model table breaks each model down into **13 dimensions** plus an **Average ⬆️** (the
|
| 73 |
-
mean of all 13). A model must report **all 13** dimensions to appear on the board.
|
| 74 |
|
| 75 |
## 1) By format (combined item counts; scores kept per track)
|
| 76 |
|
|
|
|
| 45 |
a **synthetic MSA** track and a **webscraped + aptitude** track — graded by strong LLM judges.
|
| 46 |
|
| 47 |
Each submission is scored on **13 capability and domain dimensions** (7 synthetic capability categories and
|
| 48 |
+
6 webscraped domains/groups); the **Average ⬆️** column is the **item-weighted** mean across all 13 (each dimension weighted by its number of benchmark items, so every item counts equally). See the **📝 About** tab for
|
| 49 |
the full dataset composition, scoring methodology, and how to reproduce results, and the **🚀 Submit** tab to add
|
| 50 |
your own model.
|
| 51 |
"""
|
|
|
|
| 70 |
the metric is a single judged-accuracy score per dimension.
|
| 71 |
- The two tracks use different solver/judge pairs (see table above) and their scores are **kept per track**.
|
| 72 |
- The leaderboard's per-model table breaks each model down into **13 dimensions** plus an **Average ⬆️** (the
|
| 73 |
+
item-weighted mean of all 13 -- each dimension weighted by its item count). A model must report **all 13** dimensions to appear on the board.
|
| 74 |
|
| 75 |
## 1) By format (combined item counts; scores kept per track)
|
| 76 |
|