Mushari440 commited on
Commit
9c7acad
·
verified ·
1 Parent(s): 5458de1

Switch Space SDK gradio -> static

Browse files
Files changed (1) hide show
  1. README.md +9 -44
README.md CHANGED
@@ -2,13 +2,11 @@
2
  title: Arabic LLM Leaderboard
3
  emoji: 🏆
4
  colorFrom: green
5
- colorTo: indigo
6
- sdk: gradio
7
- app_file: app.py
8
  pinned: true
9
  license: apache-2.0
10
  short_description: Benchmarking LLMs on Modern Standard Arabic
11
- sdk_version: 5.43.1
12
  tags:
13
  - leaderboard
14
  - arabic
@@ -16,45 +14,12 @@ tags:
16
  - evaluation
17
  ---
18
 
19
- # Start the configuration
20
 
21
- Most of the variables to change for a default leaderboard are in `src/env.py` (replace the path for your leaderboard) and `src/about.py` (for tasks).
 
 
22
 
23
- Results files should have the following format and be stored as json files. Scores are `0–1` fractions
24
- (rendered as percentages), the metric key is `acc`, and a model must report **all 13** dimensions or it is
25
- skipped:
26
- ```json
27
- {
28
- "config": {
29
- "model_dtype": "torch.float16", # or torch.bfloat16 or 8bit or 4bit
30
- "model_name": "path of the model on the hub: org/model",
31
- "model_sha": "revision on the hub",
32
- },
33
- "results": {
34
- "reasoning_logic": {"acc": 0.559},
35
- "generation_instruction": {"acc": 0.617},
36
- "sentiment_emotion_meaning": {"acc": 0.884},
37
- "safety_trust_bias": {"acc": 0.820},
38
- "arabic_language_understanding": {"acc": 0.638},
39
- "knowledge_stem": {"acc": 0.784},
40
- "multi_turn_dialogue": {"acc": 0.703},
41
- "biology": {"acc": 0.0},
42
- "islam": {"acc": 0.0},
43
- "medicine": {"acc": 0.0},
44
- "completion": {"acc": 0.80},
45
- "contextual": {"acc": 0.74},
46
- "semantic": {"acc": 0.77}
47
- }
48
- }
49
- ```
50
-
51
- Request files are created automatically by this tool.
52
-
53
- If you encounter problem on the space, don't hesitate to restart it to remove the create eval-queue, eval-queue-bk, eval-results and eval-results-bk created folder.
54
-
55
- # Code logic for more complex edits
56
-
57
- You'll find
58
- - the main table' columns names and properties in `src/display/utils.py`
59
- - the logic to read all results and request files, then convert them in dataframe lines, in `src/leaderboard/read_evals.py`, and `src/populate.py`
60
- - the logic to allow or filter submissions in `src/submission/submit.py` and `src/submission/check_validity.py`
 
2
  title: Arabic LLM Leaderboard
3
  emoji: 🏆
4
  colorFrom: green
5
+ colorTo: yellow
6
+ sdk: static
 
7
  pinned: true
8
  license: apache-2.0
9
  short_description: Benchmarking LLMs on Modern Standard Arabic
 
10
  tags:
11
  - leaderboard
12
  - arabic
 
14
  - evaluation
15
  ---
16
 
17
+ # Arabic LLM Leaderboard
18
 
19
+ A static single-page leaderboard for Arabic (Modern Standard Arabic) LLMs. It reads results
20
+ live (client-side) from the **[Mushari440/results](https://huggingface.co/datasets/Mushari440/results)**
21
+ dataset and the evaluation queue from **[Mushari440/requests](https://huggingface.co/datasets/Mushari440/requests)**.
22
 
23
+ Each model is scored on 13 capability & domain dimensions; the headline **Average** is the
24
+ **item-weighted** mean (every one of the 29,417 benchmark items counts equally). No server,
25
+ no framework — just `index.html`, so it never breaks from dependency drift.