Mirrowel commited on
Commit
fd9c9e4
·
1 Parent(s): 40be1c9

docs: add deployment guide for Render

Browse files
Files changed (1) hide show
  1. Deplyment guide.md +134 -0
Deplyment guide.md ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Easy Guide to Deploying LLM-API-Key-Proxy on Render
2
+
3
+ This guide walks you through deploying the [LLM-API-Key-Proxy](https://github.com/Mirrowel/LLM-API-Key-Proxy) as a hosted service on Render.com. The project provides a universal, OpenAI-compatible API endpoint for all your LLM providers (like Gemini or OpenAI), powered by an intelligent key management library. It's perfect for integrating with platforms like JanitorAI, where you can use it as a custom proxy for highly available and resilient chats.
4
+
5
+ The process is beginner-friendly and takes about 15-30 minutes. We'll use Render's free tier (with limitations like sleep after 15 minutes of inactivity) and upload your `.env` file as a secret for easy key management—no manual entry of variables required.
6
+
7
+ ## Prerequisites
8
+
9
+ - A free Render.com account (sign up at render.com).
10
+ - A GitHub account (for forking the repo).
11
+ - Basic terminal access (e.g., Command Prompt, Terminal, or Git Bash).
12
+ - API keys from LLM providers (e.g., Gemini, OpenAI—get them from their dashboards). For details on supported providers and how to format their keys (e.g., API key naming conventions), refer to the [LiteLLM Providers Documentation](https://docs.litellm.ai/docs/providers).
13
+
14
+ **Note**: You don't need Python installed for initial testing—use the pre-compiled Windows EXE from the repo's releases for a quick local trial.
15
+
16
+ ## Step 1: Test Locally with the Compiled EXE (No Python Required)
17
+
18
+ Before deploying, try the proxy locally to ensure your keys work. This uses a pre-built executable that's easy to set up.
19
+
20
+ 1. Go to the repo's [GitHub Releases page](https://github.com/Mirrowel/LLM-API-Key-Proxy/releases).
21
+ 2. Download the latest release ZIP file (e.g., for Windows).
22
+ 3. Unzip the file.
23
+ 4. Double-click `setup_env.bat`. A window will open—follow the prompts to add your PROXY_API_KEY (a strong secret you create) and provider keys. Use the [LiteLLM Providers Documentation](https://docs.litellm.ai/docs/providers) for guidance on key formats (e.g., `GEMINI_API_KEY_1="your-key"`).
24
+ 5. Double-click `proxy_app.exe` to start the proxy. It runs at `http://127.0.0.1:8000`—visit in a browser to confirm "API Key Proxy is running".
25
+ 6. Test with curl (replace with your PROXY_API_KEY):
26
+
27
+ ```
28
+ curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
29
+ ```
30
+
31
+ - Expected: A JSON response with the answer (e.g., "Paris").
32
+
33
+ If it works, you're ready to deploy. If not, double-check your keys against LiteLLM docs.
34
+
35
+ ## Step 2: Fork and Prepare the Repository
36
+
37
+ 1. Go to the original repo: [https://github.com/Mirrowel/LLM-API-Key-Proxy](https://github.com/Mirrowel/LLM-API-Key-Proxy).
38
+ 2. Click **Fork** in the top-right to create your own copy (this lets you make changes if needed).
39
+ 3. Clone your forked repo locally:
40
+
41
+ ```
42
+ git clone https://github.com/YOUR-USERNAME/LLM-API-Key-Proxy.git
43
+ cd LLM-API-Key-Proxy
44
+ ```
45
+
46
+
47
+ ## Step 3: Assemble Your .env File
48
+
49
+ The proxy uses a `.env` file to store your API keys securely. We'll create this based on the repo's documentation.
50
+
51
+ 1. In your cloned repo, copy the example: `copy .env.example .env` (Windows) or `cp .env.example .env` (macOS/Linux).
52
+ 2. Open `.env` in a text editor (e.g., Notepad or VS Code).
53
+ 3. Add your keys following the format from the repo's README and [LiteLLM Providers Documentation](https://docs.litellm.ai/docs/providers):
54
+ - **PROXY_API_KEY**: Create a strong, unique secret (e.g., "my-super-secret-proxy-key"). This authenticates requests to your proxy.
55
+ - **Provider Keys**: Add keys for your chosen providers. You can add multiple per provider (e.g., _1, _2) for rotation.
56
+
57
+ Example `.env` (customize with your real keys):
58
+
59
+ ```
60
+ # Your proxy's authentication key (invent a strong one)
61
+ PROXY_API_KEY="my-super-secret-proxy-key"
62
+
63
+ # Provider API keys (get from provider dashboards; see LiteLLM docs for formats)
64
+ GEMINI_API_KEY_1="your-gemini-key-here"
65
+ GEMINI_API_KEY_2="another-gemini-key"
66
+
67
+ OPENROUTER_API_KEY_1="your-openrouter-key"
68
+ ```
69
+
70
+ - Supported providers: Check LiteLLM docs for a full list and specifics (e.g., GEMINI, OPENROUTER, NVIDIA_NIM).
71
+ - Tip: Start with 1-2 providers to test. Don't share this file publicly!
72
+ 4. Save the file. (We'll upload it to Render in Step 5.)
73
+
74
+ ## Step 4: Create a New Web Service on Render
75
+
76
+ 1. Log in to render.com and go to your Dashboard.
77
+ 2. Click **New > Web Service**.
78
+ 3. Choose **Build and deploy from a Git repository** > **Next**.
79
+ 4. Connect your GitHub account and select your forked repo.
80
+ 5. In the setup form:
81
+ - **Name**: Something like "llm-api-key-proxy".
82
+ - **Region**: Choose one close to you (e.g., Oregon for US West).
83
+ - **Branch**: "main" (or your default).
84
+ - **Runtime**: Python 3.
85
+ - **Build Command**: `pip install -r requirements.txt`.
86
+ - **Start Command**: `uvicorn src.proxy_app.main:app --host 0.0.0.0 --port $PORT`.
87
+ - **Instance Type**: Free (for testing; upgrade later for always-on service).
88
+ 6. Click **Create Web Service**. Render will build and deploy—watch the progress in the Events tab.
89
+
90
+ ## Step 5: Upload .env as a Secret File
91
+
92
+ Render mounts secret files securely at runtime, making your `.env` available to the app without exposing it.
93
+
94
+ 1. In your new service's Dashboard, go to **Environment > Secret Files**.
95
+ 2. Click **Add Secret File**.
96
+ 3. **File Path**: Don't change. Keep it as root directory of the repo.
97
+ 4. **Contents**: Upload the `.env` file you created previously.
98
+ 5. Save. This injects the file for the app to load via `dotenv` (already in the code).
99
+ 6. Trigger a redeploy: Go to **Deploy > Manual Deploy** > **Deploy HEAD** (or push a small change to your repo).
100
+
101
+ Your keys are now loaded automatically!
102
+
103
+ ## Step 6: Test Your Deployed Proxy
104
+
105
+ 1. Note your service URL: It's in the Dashboard (e.g., https://llm-api-key-proxy.onrender.com).
106
+ 2. Test with curl (replace with your PROXY_API_KEY):
107
+
108
+ ```
109
+ curl -X POST https://your-service.onrender.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
110
+ ```
111
+
112
+ - Expected: A JSON response with the answer (e.g., "Paris").
113
+ 3. Check logs in Render's Dashboard for startup messages (e.g., "RotatingClient initialized").
114
+
115
+ ## Step 7: Integrate with JanitorAI
116
+
117
+ 1. Log in to janitorai.com and go to API settings (usually in a chat or account menu).
118
+ 2. Select "Proxy" mode.
119
+ 3. **API URL**: `https://your-service.onrender.com/v1`.
120
+ 4. **API Key**: Your PROXY_API_KEY (from .env).
121
+ 5. **Model**: Format as "provider/model" (e.g., "gemini/gemini-2.5-flash"; check LiteLLM docs for options).
122
+ 6. Save and test a chat—messages should route through your proxy.
123
+
124
+ ## Troubleshooting
125
+
126
+ - **Build Fails**: Check Render logs for missing dependencies—ensure `requirements.txt` is up to date.
127
+ - **401 Unauthorized**: Verify your PROXY_API_KEY matches exactly (case-sensitive) and includes "Bearer " in requests. Or you have no keys for the provider/model added that you are trying to use.
128
+ - **405 on OPTIONS**: If CORS issues arise, add the middleware from Step 3 and redeploy.
129
+ - **Service Sleeps**: Free tier sleeps after inactivity—first requests may delay.
130
+ - **Provider Key Issues**: Double-check formats in [LiteLLM Providers Documentation](https://docs.litellm.ai/docs/providers).
131
+ - **More Help**: Check Render docs or the repo's README. If stuck, share error logs.
132
+
133
+ That is it.
134
+