README.md

---
title: IDEA Bench Arena
emoji: 📉
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
license: cc-by-4.0
short_description: Official arena of IDEA-Bench.
---

## Installation

- for cuda 11.8
```bash
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
```
- for cuda 12.1
```bash
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip install -r requirements.txt
```

## Start Hugging Face UI
```bash
python app.py
```

## Start Log server
```bash
uvicorn serve.log_server:app --reload --port 22005 --host 0.0.0.0
```

## Update leaderboard
```bash
cd arena_elo && bash update_leaderboard.sh
```

Paper: [https://arxiv.org/abs/2412.11767](https://arxiv.org/abs/2412.11767)