README.md

metadata

title: README
emoji: 🏆
colorFrom: green
colorTo: blue
sdk: static
pinned: true

Haakkim — حَكِّم

An open arena-style human preference evaluation platform for Arabic LLMs.

🌐 haakkim.tech  ·  🏆 Live Leaderboard  ·  📦 Dataset

---

What is Haakkim?

Most Arabic LLM benchmarks rely on fixed tasks and MSA-only evaluation. Haakkim takes a different approach: real users compare two models side-by-side on real Arabic prompts, across 11 dialect varieties, and the results are aggregated into a statistically validated leaderboard.

Key features:

• 11 Arabic varieties — MSA, Saudi, Egyptian, Levantine, Tunisian, Iraqi, Moroccan, Algerian, Sudanese, Omani, Libyan
• 3 evaluation modes — Ranked Arena (official BT leaderboard), Side-by-Side, 10 Questions
• Principled scoring — Bradley–Terry with IPW sampling corrections and bootstrap CIs
• Rankability gate — BT scores only published when the comparison graph is fully connected and ESS is sufficient
• Open & reproducible — full dataset, audit logs, and scoring pipelines are public

---

Evaluation Modes

Mode	Description	Used for Ranking
Ranked Arena	Random model pairing, single-turn MSA, matched system instruction	✅ Official BT leaderboard
Side-by-Side	User-selected model pair, any dialect	Win-rate only
10 Questions	Fixed Arabic prompt pool, any dialect	Win-rate only

---

Dataset

The first public release of Haakkim battle data is available on Hugging Face:

📦 Haakkim/Haakkim-1.0v

---

Team

College of Computing, Umm Al-Qura University — Mecca, Saudi Arabia

Name	Role	Email
Dr. Mourad Mars	Principal Investigator	msmars@uqu.edu.sa
Hassan Barmandah	AI Researcher	hassanhbarmandah@gmail.com
Abdulrhman Alassaf	Software Engineer	aaalassaf@outlook.com

---

Citation

@misc{mars2026haakkim,
  title        = {Haakkim: An Arena-Style Human Preference Evaluation Platform for Arabic {LLMs}},
  author       = {Mars, Mourad and Barmandah, Hassan and Alassaf, Abdulrhman},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/Haakkim/Haakkim-1.0v}},
  note         = {College of Computing, Umm Al-Qura University, Mecca, Saudi Arabia}
}