README.md

---
title: Unified AI Proxy
emoji: 🔀
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
---

# 🔀 Unified AI Proxy

A self-hosted, OpenAI-compatible API gateway that routes requests to any AI provider from a single endpoint. Manage all your provider keys through a clean dashboard and hit one URL for everything.

---

## Features

- **Single endpoint** for OpenAI, Anthropic, Groq, Together, Mistral, Cohere, DeepSeek, OpenRouter, or any OpenAI-compatible API
- **Provider routing** via `providerName/model-name` prefix in the model field
- **Streaming support** — SSE responses forwarded without buffering
- **Unified model list** — `GET /v1/models` aggregates all providers, prefixed by name
- **Dashboard** — add/edit/delete providers, view request logs, manage your master key
- **Persistent SQLite** — data survives Space restarts (mount `/data` as persistent storage)
- **Env var seeding** — pre-seed providers via `PROVIDER_<NAME>_URL` + `PROVIDER_<NAME>_KEY`

---

## Quick Start

### 1. Deploy to Hugging Face Spaces

1. Create a new Space → SDK: **Docker**
2. Upload these files: `Dockerfile`, `main.py`, `requirements.txt`, `README.md`
3. Enable **Persistent Storage** in Space settings → mount at `/data`
4. The Space starts and prints your **Master API Key** to the build logs — copy it

### 2. Open the Dashboard

Navigate to your Space URL. Enter your master key to unlock the dashboard.

### 3. Add Providers

In the **Providers** tab, add any OpenAI-compatible backend:

| Provider | Base URL |
|----------|----------|
| OpenAI | `https://api.openai.com/v1` |
| Anthropic | `https://api.anthropic.com/v1` |
| Groq | `https://api.groq.com/openai/v1` |
| Together | `https://api.together.xyz/v1` |
| Mistral | `https://api.mistral.ai/v1` |
| Cohere | `https://api.cohere.ai/compatibility/v1` |
| DeepSeek | `https://api.deepseek.com/v1` |
| OpenRouter | `https://openrouter.ai/api/v1` |

### 4. Make Requests

Use your Space URL as the base URL in any OpenAI-compatible client:

```bash
# Route to a specific provider using providerName/ prefix
curl https://your-space.hf.space/v1/chat/completions \
  -H "Authorization: Bearer YOUR_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "groq/llama3-8b-8192",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Together AI with a nested model path
curl https://your-space.hf.space/v1/chat/completions \
  -H "Authorization: Bearer YOUR_MASTER_KEY" \
  -d '{
    "model": "together/meta-llama/Llama-3-70b-chat-hf",
    "messages": [{"role": "user", "content": "Hi"}],
    "stream": true
  }'

# Use default provider (no prefix needed)
curl https://your-space.hf.space/v1/chat/completions \
  -H "Authorization: Bearer YOUR_MASTER_KEY" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

# List all models across all providers
curl https://your-space.hf.space/v1/models \
  -H "Authorization: Bearer YOUR_MASTER_KEY"
```

### Use with the OpenAI Python SDK

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://your-space.hf.space/v1",
    api_key="YOUR_MASTER_KEY",
)

# Route to Groq
response = client.chat.completions.create(
    model="groq/llama3-8b-8192",
    messages=[{"role": "user", "content": "Hello!"}],
)

# Route to Anthropic (via OpenAI-compat layer)
response = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

---

## Environment Variable Seeding

Pre-seed providers without touching the dashboard by setting Space secrets:

```
PROVIDER_OPENAI_URL=https://api.openai.com/v1
PROVIDER_OPENAI_KEY=sk-...

PROVIDER_GROQ_URL=https://api.groq.com/openai/v1
PROVIDER_GROQ_KEY=gsk_...

PROVIDER_TOGETHER_URL=https://api.together.xyz/v1
PROVIDER_TOGETHER_KEY=...
```

These are imported on startup if the provider name doesn't already exist in the database.

---

## API Reference

| Route | Auth | Description |
|-------|------|-------------|
| `GET /` | Master key (dashboard login) | Web dashboard |
| `GET /health` | None | Health check + provider count |
| `GET /v1/models` | Master key | Aggregated model list from all providers |
| `POST /v1/chat/completions` | Master key | Proxied chat completions |
| `POST /v1/completions` | Master key | Proxied text completions |
| `POST /v1/embeddings` | Master key | Proxied embeddings |
| `GET /v1/*` | Master key | Any other OpenAI-compatible endpoint |

---

## Model Routing

The proxy determines which provider to use based on the `model` field:

1. **With prefix** — `groq/llama3-8b-8192` → routes to the provider named `Groq`, forwards model as `llama3-8b-8192`
2. **Nested paths** — `together/meta-llama/Llama-3-70b` → routes to `Together`, forwards `meta-llama/Llama-3-70b`
3. **No prefix** — uses the provider marked as **Default** in Settings
4. **No default set** — uses the first enabled provider

Matching is **case-insensitive**: `GROQ/`, `groq/`, and `Groq/` all route to a provider named `Groq`.

---

## Dashboard Tabs

### Providers
Add, edit, enable/disable, and delete providers. Set one as the default. API keys are stored encrypted in SQLite and only the last 4 characters are ever displayed.

### Logs
Auto-refreshing table (every 5s) of the last 200 requests: timestamp, provider, model, endpoint, status code, and latency. Automatically truncated at 500 stored rows.

### Settings
View and copy your master key, regenerate it (invalidates immediately), and set the default provider.

---

## Security Notes

- Provider API keys are **never returned** by any API endpoint — only the last 4 characters are shown in the dashboard
- The master key protects all `/v1/*` endpoints and the dashboard
- CORS is open (`*`) so the proxy can be called from any frontend — restrict this in production if needed
- The dashboard login stores the master key in `sessionStorage` (cleared on tab close)

---

## Local Development

```bash
pip install fastapi uvicorn httpx sqlalchemy
uvicorn main:app --reload --port 7860
# Open http://localhost:7860
# Master key printed to terminal on first run
```

Data is stored in `./db.sqlite` when `/data` doesn't exist.