README.md

---
title: Llm Proxy
emoji: "🤗"
colorFrom: red
colorTo: indigo
sdk: docker
pinned: false
license: mit
short_description: OpenAI-compatible proxy for Hugging Face Inference Providers
---

# LLM proxy

The service exposes an OpenAI-like `/v1/chat/completions` endpoint and routes calls through Hugging Face Inference Providers.

## Configuration

- `HF_TOKEN`: Hugging Face user token with the **Inference Providers** permission. `HF_API_KEY` remains supported for compatibility.
- `API_KEYS`: comma-separated bearer tokens accepted by this proxy.
- `HF_MODELS`: optional comma-separated `alias=model-id` mappings. Default: `gpt-oss=openai/gpt-oss-120b:cerebras`.
- `HF_TIMEOUT_SECONDS`: optional provider timeout, default `60`.

Do not append a provider suffix unless you intentionally want to pin one. With a plain model ID, the Hugging Face router can select an available provider.

Example `.env`:

```dotenv
HF_TOKEN=hf_your_token
API_KEYS=local-secret
HF_MODELS=gpt-oss=openai/gpt-oss-120b:cerebras
```

Run locally with `start.bat`, then call:

```powershell
$headers = @{ Authorization = "Bearer local-secret" }
$body = @{ model = "gpt-oss"; messages = @(@{ role = "user"; content = "Привет!" }) } | ConvertTo-Json -Depth 4
Invoke-RestMethod http://127.0.0.1:8000/v1/chat/completions -Method Post -Headers $headers -ContentType "application/json" -Body $body
```

Free Hugging Face accounts receive only a small monthly inference credit. Requests stop after the credit is exhausted unless additional credits are purchased.