lighteval/pr_1221/en/installation.md

# Installation

Lighteval can be installed from PyPI or from source. This guide covers all installation options and dependencies.

## System Requirements

- **Python**: 3.10 or higher
- **PyTorch**: 2.0 or higher (but less than 3.0)
- **CUDA**: Optional, for GPU acceleration

## From PyPI

The simplest way to install Lighteval is from PyPI:

```bash
pip install lighteval
```

This installs the core package with all essential dependencies for basic evaluation tasks.

## From Source

Source installation is recommended for developers who want to contribute to Lighteval or need the latest features:

```bash
git clone https://github.com/huggingface/lighteval.git
cd lighteval
pip install -e .
```

## Optional Dependencies (Extras)

Lighteval provides several optional dependency groups that you can install based on your needs. Use the format `pip install lighteval[<group>]` or `pip install -e .[<group>]` for source installation.

### Backend Extras

| Extra | Description | Dependencies |
|-------|-------------|--------------|
| `vllm` | Use VLLM as backend for high-performance inference | vllm>=0.10.0, ray, more_itertools |
| `tgi` | Use Text Generation Inference API | text-generation>=0.6.0 |
| `litellm` | Use LiteLLM for unified API access | litellm, diskcache |
| `optimum` | Use Optimum for optimized models | optimum==1.12.0 |
| `quantization` | Evaluate quantized models | bitsandbytes>=0.41.0, auto-gptq>=0.4.2 |
| `adapters` | Evaluate adapter models (PEFT, Delta) | peft==0.3.0 |
| `nanotron` | Evaluate Nanotron models | nanotron, tensorboardX |

### Task and Feature Extras

| Extra | Description | Dependencies |
|-------|-------------|--------------|
| `extended_tasks` | Extended evaluation tasks | langdetect, openai>1.87, tiktoken |
| `multilingual` | Multilingual evaluation support | stanza, spacy[ja,ko,th], jieba, pyvi |
| `math` | Mathematical reasoning tasks | latex2sympy2_extended==1.0.6 |

### Storage and Logging Extras

| Extra | Description | Dependencies |
|-------|-------------|--------------|
| `s3` | Upload results to S3 | s3fs |
| `tensorboardX` | Upload results to TensorBoard | tensorboardX |
| `wandb` | Log results to Weights & Biases | wandb |
| `trackio` | Log results to Trackio | trackio |

### Development Extras

| Extra | Description | Dependencies |
|-------|-------------|--------------|
| `quality` | Code quality tools | ruff>=v0.11.0, pre-commit |
| `tests` | Testing dependencies | pytest>=7.4.0, deepdiff |
| `docs` | Documentation building | hf-doc-builder, watchdog |
| `dev` | All development dependencies | Includes accelerate, quality, tests, multilingual, math, extended_tasks, vllm |