Deployment guide.md

Easy Guide to Deploying LLM-API-Key-Proxy on Render

This guide walks you through deploying the LLM-API-Key-Proxy as a hosted service on Render.com. The project provides a universal, OpenAI-compatible API endpoint for all your LLM providers (like Gemini or OpenAI), powered by an intelligent key management library. It's perfect for integrating with platforms like JanitorAI, where you can use it as a custom proxy for highly available and resilient chats.

The process is beginner-friendly and takes about 15-30 minutes. We'll use Render's free tier (with limitations like sleep after 15 minutes of inactivity) and upload your .env file as a secret for easy key management—no manual entry of variables required.

Prerequisites

• A free Render.com account (sign up at render.com).
• A GitHub account (for forking the repo).
• Basic terminal access (e.g., Command Prompt, Terminal, or Git Bash).
• API keys from LLM providers (e.g., Gemini, OpenAI—get them from their dashboards). For details on supported providers and how to format their keys (e.g., API key naming conventions), refer to the LiteLLM Providers Documentation.

Note: You don't need Python installed for initial testing—use the pre-compiled Windows EXE from the repo's releases for a quick local trial.

Step 1: Test Locally with the Compiled EXE (No Python Required)

Before deploying, try the proxy locally to ensure your keys work. This uses a pre-built executable that's easy to set up.

1. Go to the repo's GitHub Releases page.
2. Download the latest release ZIP file (e.g., for Windows).
3. Unzip the file.
4. Double-click setup_env.bat. A window will open—follow the prompts to add your PROXY_API_KEY (a strong secret you create) and provider keys. Use the LiteLLM Providers Documentation for guidance on key formats (e.g., GEMINI_API_KEY_1="your-key").
5. Double-click proxy_app.exe to start the proxy. It runs at http://127.0.0.1:8000—visit in a browser to confirm "API Key Proxy is running".
6. Test with curl (replace with your PROXY_API_KEY):

curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'

- Expected: A JSON response with the answer (e.g., "Paris").

If it works, you're ready to deploy. If not, double-check your keys against LiteLLM docs.

Step 2: Fork and Prepare the Repository

1. Go to the original repo: https://github.com/Mirrowel/LLM-API-Key-Proxy.
2. Click Fork in the top-right to create your own copy (this lets you make changes if needed).
3. Clone your forked repo locally:

git clone https://github.com/YOUR-USERNAME/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy

Step 3: Assemble Your .env File

The proxy uses a .env file to store your API keys securely. We'll create this based on the repo's documentation.

1. In your cloned repo, copy the example: copy .env.example .env (Windows) or cp .env.example .env (macOS/Linux).
2. Open .env in a text editor (e.g., Notepad or VS Code).
3. Add your keys following the format from the repo's README and LiteLLM Providers Documentation:
   • PROXY_API_KEY: Create a strong, unique secret (e.g., "my-super-secret-proxy-key"). This authenticates requests to your proxy.
   • Provider Keys: Add keys for your chosen providers. You can add multiple per provider (e.g., _1, _2) for rotation.

Example .env (customize with your real keys):

# Your proxy's authentication key (invent a strong one)
PROXY_API_KEY="my-super-secret-proxy-key"

# Provider API keys (get from provider dashboards; see LiteLLM docs for formats)
GEMINI_API_KEY_1="your-gemini-key-here"
GEMINI_API_KEY_2="another-gemini-key"

OPENROUTER_API_KEY_1="your-openrouter-key"

- Supported providers: Check LiteLLM docs for a full list and specifics (e.g., GEMINI, OPENROUTER, NVIDIA_NIM).
- Tip: Start with 1-2 providers to test. Don't share this file publicly!

Advanced: Stateless Deployment for OAuth Providers (Gemini CLI, Qwen, iFlow)

If you are using providers that require complex OAuth files (like Gemini CLI, Qwen Code, or iFlow), you don't need to upload the JSON files manually. The proxy includes a tool to "export" these credentials into environment variables.

1. Run the credential tool locally: python -m rotator_library.credential_tool
2. Select the "Export ... to .env" option for your provider.
3. The tool will generate a file (e.g., gemini_cli_user_at_gmail.env) containing variables like GEMINI_CLI_ACCESS_TOKEN, GEMINI_CLI_REFRESH_TOKEN, etc.
4. Copy the contents of this file and paste them directly into your .env file or Render's "Environment Variables" section.
5. The proxy will automatically detect and use these variables—no file upload required!

Advanced: Antigravity OAuth Provider

The Antigravity provider requires OAuth2 authentication similar to Gemini CLI. It provides access to:

• Gemini 2.5 models (Pro/Flash)
• Gemini 3 models (Pro/Image-preview) - requires paid-tier Google Cloud project
• Claude Sonnet 4.5 via Google's Antigravity proxy

Setting up Antigravity locally:

1. Run the credential tool: python -m rotator_library.credential_tool
2. Select "Add OAuth Credential" and choose "Antigravity"
3. Complete the OAuth flow in your browser
4. The credential is saved to oauth_creds/antigravity_oauth_1.json

Exporting for stateless deployment:

1. Run: python -m rotator_library.credential_tool
2. Select "Export Antigravity to .env"
3. Copy the generated environment variables to your deployment platform:

   ANTIGRAVITY_ACCESS_TOKEN="..."
   ANTIGRAVITY_REFRESH_TOKEN="..."
   ANTIGRAVITY_EXPIRY_DATE="..."
   ANTIGRAVITY_EMAIL="your-email@gmail.com"
   

Important Notes:

• Antigravity uses Google OAuth with additional scopes for cloud platform access
• Gemini 3 models require a paid-tier Google Cloud project (free tier will fail)
• The provider automatically handles thought signature caching for multi-turn conversations
• Tool hallucination prevention is enabled by default for Gemini 3 models

4. Save the file. (We'll upload it to Render in Step 5.)

Step 4: Create a New Web Service on Render

1. Log in to render.com and go to your Dashboard.
2. Click New > Web Service.
3. Choose Build and deploy from a Git repository > Next.
4. Connect your GitHub account and select your forked repo.
5. In the setup form:
   • Name: Something like "llm-api-key-proxy".
   • Region: Choose one close to you (e.g., Oregon for US West).
   • Branch: "main" (or your default).
   • Runtime: Python 3.
   • Build Command: pip install -r requirements.txt.
   • Start Command: uvicorn src.proxy_app.main:app --host 0.0.0.0 --port $PORT.
   • Instance Type: Free (for testing; upgrade later for always-on service).
6. Click Create Web Service. Render will build and deploy—watch the progress in the Events tab.

Step 5: Upload .env as a Secret File

Render mounts secret files securely at runtime, making your .env available to the app without exposing it.

1. In your new service's Dashboard, go to Environment > Secret Files.
2. Click Add Secret File.
3. File Path: Don't change. Keep it as root directory of the repo.
4. Contents: Upload the .env file you created previously.
5. Save. This injects the file for the app to load via dotenv (already in the code).
6. Trigger a redeploy: Go to Deploy > Manual Deploy > Deploy HEAD (or push a small change to your repo).

Your keys are now loaded automatically!

Step 6: Test Your Deployed Proxy

1. Note your service URL: It's in the Dashboard (e.g., https://llm-api-key-proxy.onrender.com).
2. Test with curl (replace with your PROXY_API_KEY):

curl -X POST https://your-service.onrender.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'

- Expected: A JSON response with the answer (e.g., "Paris").

3. Check logs in Render's Dashboard for startup messages (e.g., "RotatingClient initialized").

Step 7: Integrate with JanitorAI

1. Log in to janitorai.com and go to API settings (usually in a chat or account menu).
2. Select "Proxy" mode.
3. API URL: https://your-service.onrender.com/v1.
4. API Key: Your PROXY_API_KEY (from .env).
5. Model: Format as "provider/model" (e.g., "gemini/gemini-2.5-flash"; check LiteLLM docs for options).
6. Save and test a chat—messages should route through your proxy.

Troubleshooting

• Build Fails: Check Render logs for missing dependencies—ensure requirements.txt is up to date.
• 401 Unauthorized: Verify your PROXY_API_KEY matches exactly (case-sensitive) and includes "Bearer " in requests. Or you have no keys for the provider/model added that you are trying to use.
• 405 on OPTIONS: If CORS issues arise, add the middleware from Step 3 and redeploy.
• Service Sleeps: Free tier sleeps after inactivity—first requests may delay.
• Provider Key Issues: Double-check formats in LiteLLM Providers Documentation.
• More Help: Check Render docs or the repo's README. If stuck, share error logs.

That is it.

---

Appendix: Deploying to a Custom VPS

If you're deploying the proxy to a custom VPS (DigitalOcean, AWS EC2, Linode, etc.) instead of Render.com, you'll encounter special considerations when setting up OAuth providers (Antigravity, Gemini CLI, iFlow). This section covers the professional deployment workflow.

Understanding the OAuth Callback Problem

OAuth providers like Antigravity, Gemini CLI, and iFlow require an interactive authentication flow that:

1. Opens a browser for you to log in
2. Redirects back to a local callback server running on specific ports
3. Receives an authorization code to exchange for tokens

The callback servers bind to localhost on these ports:

Provider	Port	Notes
Antigravity	51121	Google OAuth with extended scopes
Gemini CLI	8085	Google OAuth for Gemini API
iFlow	11451	Authorization Code flow with API key fetch
Qwen Code	N/A	Uses Device Code flow - works on remote VPS ✅

The Issue: When running on a remote VPS, your local browser cannot reach http://localhost:51121 (or other callback ports) on the remote server, causing authentication to fail with a "connection refused" error.

Recommended Deployment Workflow

There are three professional approaches to handle OAuth authentication for VPS deployment, listed from most recommended to least:

---

Option 1: Authenticate Locally, Deploy Credentials (RECOMMENDED)

This is the cleanest and most secure approach. Complete OAuth flows on your local machine, export to environment variables, then deploy.

Step 1: Clone and Set Up Locally

# On your local development machine
git clone https://github.com/YOUR-USERNAME/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy

# Install dependencies
pip install -r requirements.txt

Step 2: Run OAuth Authentication Locally

# Start the credential tool
python -m rotator_library.credential_tool

Select "Add OAuth Credential" and choose your provider:

• Antigravity
• Gemini CLI
• iFlow
• Qwen Code (works directly on VPS, but can authenticate locally too)

The tool will:

1. Open your browser automatically
2. Start a local callback server
3. Complete the OAuth flow
4. Save credentials to oauth_creds/<provider>_oauth_N.json

Step 3: Export Credentials to Environment Variables

Still in the credential tool, select the export option for each provider:

• "Export Antigravity to .env"
• "Export Gemini CLI to .env"
• "Export iFlow to .env"
• "Export Qwen Code to .env"

The tool generates a .env file snippet like:

# Antigravity OAuth Credentials
ANTIGRAVITY_ACCESS_TOKEN="ya29.a0AfB_byD..."
ANTIGRAVITY_REFRESH_TOKEN="1//0gL6dK9..."
ANTIGRAVITY_EXPIRY_DATE="1735901234567"
ANTIGRAVITY_EMAIL="user@gmail.com"
ANTIGRAVITY_CLIENT_ID="1071006060591-..."
ANTIGRAVITY_CLIENT_SECRET="GOCSPX-..."
ANTIGRAVITY_TOKEN_URI="https://oauth2.googleapis.com/token"
ANTIGRAVITY_UNIVERSE_DOMAIN="googleapis.com"

Copy these variables to a file (e.g., oauth_credentials.env).

Step 4: Deploy to VPS

Method A: Using Environment Variables (Recommended)

# On your VPS
cd /path/to/LLM-API-Key-Proxy

# Create or edit .env file
nano .env

# Paste the exported environment variables
# Also add your PROXY_API_KEY and other provider keys

# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000

Method B: Upload Credential Files

# On your local machine - copy credential files to VPS
scp -r oauth_creds/ user@your-vps-ip:/path/to/LLM-API-Key-Proxy/

# On VPS - verify files exist
ls -la oauth_creds/

# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000

Note: Environment variables are preferred for production deployments (more secure, easier to manage, works with container orchestration).

---

Option 2: SSH Port Forwarding (For Direct VPS Authentication)

If you need to authenticate directly on the VPS (e.g., you don't have a local development environment), use SSH port forwarding to create secure tunnels.

How It Works

SSH tunnels forward ports from your local machine to the remote VPS, allowing your local browser to reach the callback servers.

Step-by-Step Process

Step 1: Create SSH Tunnels

From your local machine, open a terminal and run:

# Forward all OAuth callback ports at once
ssh -L 51121:localhost:51121 -L 8085:localhost:8085 -L 11451:localhost:11451 user@your-vps-ip

# Alternative: Forward ports individually as needed
ssh -L 51121:localhost:51121 user@your-vps-ip  # For Antigravity
ssh -L 8085:localhost:8085 user@your-vps-ip    # For Gemini CLI
ssh -L 11451:localhost:11451 user@your-vps-ip  # For iFlow

Keep this SSH session open during the entire authentication process.

Step 2: Run Credential Tool on VPS

In the same SSH terminal (or open a new SSH connection):

cd /path/to/LLM-API-Key-Proxy

# Ensure Python dependencies are installed
pip install -r requirements.txt

# Run the credential tool
python -m rotator_library.credential_tool

Step 3: Complete OAuth Flow

1. Select "Add OAuth Credential" → Choose your provider
2. The tool displays an authorization URL
3. Click the URL in your local browser (works because of the SSH tunnel!)
4. Complete the authentication flow
5. The browser redirects to localhost:<port> - this now routes through the tunnel to your VPS
6. Credentials are saved to oauth_creds/ on the VPS

Step 4: Export to Environment Variables

Still in the credential tool:

1. Select the export option for each provider
2. Copy the generated environment variables
3. Add them to /path/to/LLM-API-Key-Proxy/.env on your VPS

Step 5: Close Tunnels and Deploy

# Exit the SSH session with tunnels (Ctrl+D or type 'exit')
# Tunnels are no longer needed

# Start the proxy on VPS (in a screen/tmux session or as a service)
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000

---

Option 3: Copy Credential Files to VPS

If you've already authenticated locally and have credential files, you can copy them directly.

Copy OAuth Credential Files

# From your local machine
scp -r oauth_creds/ user@your-vps-ip:/path/to/LLM-API-Key-Proxy/

# Verify on VPS
ssh user@your-vps-ip
ls -la /path/to/LLM-API-Key-Proxy/oauth_creds/

Expected files:

• antigravity_oauth_1.json
• gemini_cli_oauth_1.json
• iflow_oauth_1.json
• qwen_code_oauth_1.json

Configure .env to Use Credential Files

On your VPS, edit .env:

# Option A: Use credential files directly (not recommended for production)
# No special configuration needed - the proxy auto-detects oauth_creds/ folder

# Option B: Export to environment variables (recommended)
# Run credential tool and export each provider to .env

---

Environment Variables vs. Credential Files

Aspect	Environment Variables	Credential Files
Security	✅ More secure (no files on disk)	⚠️ Files readable if server compromised
Container-Friendly	✅ Perfect for Docker/K8s	❌ Requires volume mounts
Ease of Rotation	✅ Update .env and restart	⚠️ Need to regenerate JSON files
Backup/Version Control	✅ Easy to manage with secrets managers	❌ Binary files, harder to manage
Auto-Refresh	✅ Uses refresh tokens	✅ Uses refresh tokens
Recommended For	Production deployments	Local development / testing

Best Practice: Always export to environment variables for VPS/cloud deployments.

---

Production Deployment Checklist

Security Best Practices

• Never commit .env or oauth_creds/ to version control
• Use environment variables instead of credential files in production
• Secure your VPS firewall - do not open OAuth callback ports (51121, 8085, 11451) to public internet
• Use SSH port forwarding only during initial authentication
• Rotate credentials regularly using the credential tool's export feature
• Set file permissions on .env: chmod 600 .env

Firewall Configuration

OAuth callback ports should never be publicly exposed:

# ❌ DO NOT DO THIS - keeps ports closed
# sudo ufw allow 51121/tcp
# sudo ufw allow 8085/tcp
# sudo ufw allow 11451/tcp

# ✅ Only open your proxy API port
sudo ufw allow 8000/tcp

# Check firewall status
sudo ufw status

The SSH tunnel method works without opening these ports because traffic routes through the SSH connection (port 22).

Running as a Service

Create a systemd service file on your VPS:

# Create service file
sudo nano /etc/systemd/system/llm-proxy.service

[Unit]
Description=LLM API Key Proxy
After=network.target

[Service]
Type=simple
User=your-username
WorkingDirectory=/path/to/LLM-API-Key-Proxy
Environment="PATH=/path/to/python/bin"
ExecStart=/path/to/python/bin/uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

# Enable and start the service
sudo systemctl daemon-reload
sudo systemctl enable llm-proxy
sudo systemctl start llm-proxy

# Check status
sudo systemctl status llm-proxy

# View logs
sudo journalctl -u llm-proxy -f

---

Troubleshooting VPS Deployment

"localhost:51121 connection refused" Error

Cause: Trying to authenticate directly on VPS without SSH tunnel.

Solution: Use Option 1 (authenticate locally) or Option 2 (SSH port forwarding).

OAuth Credentials Not Loading

# Check if environment variables are set
printenv | grep -E '(ANTIGRAVITY|GEMINI_CLI|IFLOW|QWEN_CODE)'

# Verify .env file exists and is readable
ls -la .env
cat .env | grep -E '(ANTIGRAVITY|GEMINI_CLI|IFLOW|QWEN_CODE)'

# Check credential files if using file-based approach
ls -la oauth_creds/

Token Refresh Failing

The proxy automatically refreshes tokens using refresh tokens. If refresh fails:

1. Re-authenticate: Run credential tool again and export new credentials
2. Check token expiry: Some providers require periodic re-authentication
3. Verify credentials: Ensure REFRESH_TOKEN is present in environment variables

Permission Denied on .env

# Set correct permissions
chmod 600 .env
chown your-username:your-username .env

---

Summary: VPS Deployment Best Practices

1. Authenticate locally on your development machine (easiest, most secure)
2. Export to environment variables using the credential tool's built-in export feature
3. Deploy to VPS by adding environment variables to .env
4. Never open OAuth callback ports to the public internet
5. Use SSH port forwarding only if you must authenticate directly on VPS
6. Run as a systemd service for production reliability
7. Monitor logs for authentication errors and token refresh issues

This approach ensures secure, production-ready deployment while maintaining the convenience of OAuth authentication.