# Using Inference Providers as Backend

Lighteval allows you to use Hugging Face's Inference Providers to evaluate LLMs on supported providers such as Black Forest Labs, Cerebras, Fireworks AI, Nebius, Together AI, and many more.

> [!WARNING]
> Do not forget to set your Hugging Face API key.
> You can set it using the `HF_TOKEN` environment variable or by using the `huggingface-cli` command.

## Basic Usage

```bash
lighteval endpoint inference-providers \
    "model_name=deepseek-ai/DeepSeek-R1,provider=hf-inference" \
    gsm8k
```

## Using a Configuration File

You can use configuration files to define the model and the provider to use.

```bash
lighteval endpoint inference-providers \
    examples/model_configs/inference_providers.yaml \
    gsm8k
```

With the following configuration file:

```yaml
model_parameters:
  model_name: "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B"
  provider: "novita"
  timeout: null
  proxies: null
  parallel_calls_count: 10
  generation_parameters:
    temperature: 0.8
    top_k: 10
    max_new_tokens: 10000
```

By default, inference requests are billed to your personal account.
Optionally, you can charge them to an organization by setting `org_to_bill=""` (requires being a member of that organization).

## Supported Providers

Hugging Face Inference Providers supports a wide range of LLM providers see the [Inference Providers documentation](https://huggingface.co/docs/inference-providers/en/index) for the complete list.

## Billing and Costs

### Personal Account Billing
By default, all inference requests are billed to your personal Hugging Face account. You can monitor your usage in the [Hugging Face billing dashboard](https://huggingface.co/settings/billing).

### Organization Billing
To bill requests to an organization:

1. Ensure you are a member of the organization
2. Add `org_to_bill=""` to your configuration
3. The organization must have sufficient credits

```yaml
model_parameters:
  model_name: "meta-llama/Llama-2-7b-chat-hf"
  provider: "together"
  org_to_bill: "my-organization"
```

For more detailed error handling and provider-specific information, refer to the [Hugging Face Inference Providers documentation](https://huggingface.co/docs/inference-providers/en/index).