---
title: README
emoji: 🏆
colorFrom: green
colorTo: blue
sdk: static
pinned: true
---

# Haakkim — حَكِّم

**An open arena-style human preference evaluation platform for Arabic LLMs.**

🌐 [haakkim.tech](https://haakkim.tech)  ·  🏆 [Live Leaderboard](https://haakkim.tech/#leaderboard)  ·  📦 [Dataset](https://huggingface.co/datasets/Haakkim/Haakkim-1.0v)

---

## What is Haakkim?

Most Arabic LLM benchmarks rely on fixed tasks and MSA-only evaluation. Haakkim takes a different approach: real users compare two models side-by-side on real Arabic prompts, across 11 dialect varieties, and the results are aggregated into a statistically validated leaderboard.

**Key features:**
- **11 Arabic varieties** — MSA, Saudi, Egyptian, Levantine, Tunisian, Iraqi, Moroccan, Algerian, Sudanese, Omani, Libyan
- **3 evaluation modes** — Ranked Arena (official BT leaderboard), Side-by-Side, 10 Questions
- **Principled scoring** — Bradley–Terry with IPW sampling corrections and bootstrap CIs
- **Rankability gate** — BT scores only published when the comparison graph is fully connected and ESS is sufficient
- **Open & reproducible** — full dataset, audit logs, and scoring pipelines are public

---

## Evaluation Modes

| Mode | Description | Used for Ranking |
|---|---|---|
| **Ranked Arena** | Random model pairing, single-turn MSA, matched system instruction | ✅ Official BT leaderboard |
| **Side-by-Side** | User-selected model pair, any dialect | Win-rate only |
| **10 Questions** | Fixed Arabic prompt pool, any dialect | Win-rate only |

---

## Dataset

The first public release of Haakkim battle data is available on Hugging Face:

📦 **[Haakkim/Haakkim-1.0v](https://huggingface.co/datasets/Haakkim/Haakkim-1.0v)**

---

## Team

**College of Computing, Umm Al-Qura University — Mecca, Saudi Arabia**

| Name | Role | Email |
|---|---|---|
| Dr. Mourad Mars | Principal Investigator | msmars@uqu.edu.sa |
| Hassan Barmandah | AI Researcher | hassanhbarmandah@gmail.com |
| Abdulrhman Alassaf | Software Engineer | aaalassaf@outlook.com |

---

## Citation

```bibtex
@misc{mars2026haakkim,
  title        = {Haakkim: An Arena-Style Human Preference Evaluation Platform for Arabic {LLMs}},
  author       = {Mars, Mourad and Barmandah, Hassan and Alassaf, Abdulrhman},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/Haakkim/Haakkim-1.0v}},
  note         = {College of Computing, Umm Al-Qura University, Mecca, Saudi Arabia}
}
```