Mushari440 commited on
Commit
0038b5b
·
verified ·
1 Parent(s): e33b909

Docs: Average column is item-weighted mean

Browse files
Files changed (1) hide show
  1. src/about.py +2 -2
src/about.py CHANGED
@@ -45,7 +45,7 @@ The **Arabic LLM Leaderboard** measures how well large language models understan
45
  a **synthetic MSA** track and a **webscraped + aptitude** track — graded by strong LLM judges.
46
 
47
  Each submission is scored on **13 capability and domain dimensions** (7 synthetic capability categories and
48
- 6 webscraped domains/groups); the **Average ⬆️** column is the mean across all 13. See the **📝 About** tab for
49
  the full dataset composition, scoring methodology, and how to reproduce results, and the **🚀 Submit** tab to add
50
  your own model.
51
  """
@@ -70,7 +70,7 @@ percentages.
70
  the metric is a single judged-accuracy score per dimension.
71
  - The two tracks use different solver/judge pairs (see table above) and their scores are **kept per track**.
72
  - The leaderboard's per-model table breaks each model down into **13 dimensions** plus an **Average ⬆️** (the
73
- mean of all 13). A model must report **all 13** dimensions to appear on the board.
74
 
75
  ## 1) By format (combined item counts; scores kept per track)
76
 
 
45
  a **synthetic MSA** track and a **webscraped + aptitude** track — graded by strong LLM judges.
46
 
47
  Each submission is scored on **13 capability and domain dimensions** (7 synthetic capability categories and
48
+ 6 webscraped domains/groups); the **Average ⬆️** column is the **item-weighted** mean across all 13 (each dimension weighted by its number of benchmark items, so every item counts equally). See the **📝 About** tab for
49
  the full dataset composition, scoring methodology, and how to reproduce results, and the **🚀 Submit** tab to add
50
  your own model.
51
  """
 
70
  the metric is a single judged-accuracy score per dimension.
71
  - The two tracks use different solver/judge pairs (see table above) and their scores are **kept per track**.
72
  - The leaderboard's per-model table breaks each model down into **13 dimensions** plus an **Average ⬆️** (the
73
+ item-weighted mean of all 13 -- each dimension weighted by its item count). A model must report **all 13** dimensions to appear on the board.
74
 
75
  ## 1) By format (combined item counts; scores kept per track)
76