feat: Refactor project into reusable API key rotation library and proxy

This commit introduces a significant refactoring of the project structure:

- **Extracted `rotating-api-key-client` library**: The core API key rotation, usage tracking, and error handling logic has been moved into a new, standalone Python library located in `src/rotator_library`. This library is now designed for independent reuse.
- **Updated FastAPI proxy**: The `proxy_app` now consumes the `rotating-api-key-client` library as a local dependency, simplifying its `main.py` and leveraging the new package structure.
- **Enhanced documentation**:
- A new `src/rotator_library/README.md` provides detailed usage instructions for the standalone library.
- The main `README.md` has been extensively updated to reflect the new project architecture, provide clearer setup and running instructions, and include examples for both the proxy and the library.
- **Improved dependency management**: `requirements.txt` now installs the `rotator_library` in editable mode, and `pyproject.toml` has been added to the library for proper package definition.
- **Configurable usage file path**: The `RotatingClient` now allows specifying a custom path for the usage tracking file.

README.md

@@ -1,74 +1,138 @@
-# API Key Proxy and Rotator [![License](https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png)](https://creativecommons.org/licenses/by-nc-sa/4.0/) [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/C0C0UZS4P)
+# API Key Proxy with Rotating Key Library
 
-This project provides a simple and effective solution for managing and rotating API keys for services like Google Gemini, while exposing an OpenAI-compatible endpoint. It's designed to be a lightweight, self-hosted proxy that can help you:
+This project provides two main components:
 
-- **Secure your API keys**: Instead of embedding your keys directly in client-side applications, you can keep them on the server where they are more secure.
-- **Rotate keys to avoid rate limits**: The proxy can automatically rotate through a pool of API keys, reducing the chance of hitting rate limits on any single key.
-- **Monitor key usage**: The system tracks the usage of each key, providing insights into your API consumption.
-- **Provide a unified endpoint**: Exposes an OpenAI-compatible `/v1/chat/completions` endpoint, allowing you to use it with a wide range of existing tools and libraries.
+1.  A reusable Python library (`rotating-api-key-client`) for intelligently rotating API keys.
+2.  A FastAPI proxy application that uses this library to provide an OpenAI-compatible endpoint for various LLM providers.
 
 ## Features
 
-- **OpenAI-Compatible Endpoint**: Drop-in replacement for OpenAI's API.
-- **Smart Key Rotation**: Rotates keys based on usage to minimize errors.
-- **Usage Tracking**: Logs successes and failures for each key.
-- **Streaming and Non-Streaming Support**: Handles both response types seamlessly.
-- **Easy to Deploy**: Can be run with a simple `uvicorn` command.
+-   **Smart Key Rotation**: The library automatically uses the least-used key to distribute load.
+-   **Automatic Retries**: Retries requests on transient server errors.
+-   **Cooldowns**: Puts keys on a temporary cooldown after rate limit or authentication errors.
+-   **Usage Tracking**: Tracks daily and global usage for each key.
+-   **Provider Agnostic**: Works with any provider supported by `litellm`.
+-   **OpenAI-Compatible Proxy**: The proxy provides a familiar API for interacting with different models.
 
-## Getting Started
+## Project Structure
 
-### Prerequisites
-
-- Python 3.8+
-- An `.env` file with your API keys (see `.env.example`).
+```
+.
+├── logs/                     # Logs for failed requests
+├── src/
+│   ├── proxy_app/            # The FastAPI proxy application
+│   │   └── main.py
+│   └── rotator_library/      # The rotating-api-key-client library
+│       ├── __init__.py
+│       ├── client.py
+│       ├── error_handler.py
+│       ├── failure_logger.py
+│       ├── usage_manager.py
+│       ├── pyproject.toml
+│       └── README.md
+├── .env.example
+├── .gitignore
+├── README.md
+└── requirements.txt
+```
 
-### Installation
+## Setup and Installation
 
 1.  **Clone the repository:**
+
     ```bash
-    git clone <your-repo-url>
-    cd <your-repo-name>
+    git clone <repository-url>
+    cd <repository-name>
     ```
 
-2.  **Install dependencies:**
+2.  **Create a virtual environment:**
+
+    ```bash
+    python -m venv venv
+    source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
+    ```
+
+3.  **Install the dependencies:**
+
+    The `requirements.txt` file includes the proxy's dependencies and installs the `rotator_library` in editable mode (`-e`), so you can develop both simultaneously.
+
     ```bash
     pip install -r requirements.txt
     ```
 
-3.  **Configure your API keys:**
+4.  **Configure environment variables:**
+
+    Create a `.env` file by copying the `.env.example`:
+
+    ```bash
+    cp .env.example .env
+    ```
 
-    Create a `.env` file in the project root and add your keys. You'll need a `PROXY_API_KEY` to secure your proxy and at least one `GEMINI_API_KEY`.
+    Edit the `.env` file with your API keys:
 
     ```
-    # .env
+    # A secret key for your proxy to prevent unauthorized access
     PROXY_API_KEY="your-secret-proxy-key"
-    GEMINI_API_KEY_1="your-gemini-key-1"
-    GEMINI_API_KEY_2="your-gemini-key-2"
-    # Add more Gemini keys as needed
+
+    # Add one or more API keys from your chosen provider (e.g., Gemini)
+    # The keys will be tried in order.
+    GEMINI_API_KEY_1="your-gemini-api-key-1"
+    GEMINI_API_KEY_2="your-gemini-api-key-2"
+    # ...and so on
     ```
 
-### Running the Proxy
+## Running the Proxy
 
-You can run the proxy using `uvicorn`:
+To run the proxy application:
 
 ```bash
-uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
+uvicorn src.proxy_app.main:app --reload
 ```
 
-The proxy will now be running and accessible at `http://localhost:8000`.
+The proxy will be available at `http://127.0.0.1:8000`.
 
-## How to Use
+## Using the Proxy
 
-To use the proxy, make a POST request to the `/v1/chat/completions` endpoint, making sure to include your `PROXY_API_KEY` in the `Authorization` header.
+You can make requests to the proxy as if it were the OpenAI API. Make sure to include your `PROXY_API_KEY` in the `Authorization` header.
 
-Here's an example using `curl`:
+### Example with `curl`:
 
 ```bash
-curl -X POST http://localhost:8000/v1/chat/completions \
+curl -X POST http://127.0.0.1:8000/v1/chat/completions \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer your-secret-proxy-key" \
 -d '{
-  "model": "gemini-pro",
-  "messages": [{"role": "user", "content": "Hello, how are you?"}]
+    "model": "gemini/gemini-pro",
+    "messages": [{"role": "user", "content": "What is the capital of France?"}],
+    "stream": false
 }'
-```
+```
+
+### Example with Python `requests`:
+
+```python
+import requests
+import json
+
+proxy_url = "http://127.0.0.1:8000/v1/chat/completions"
+proxy_key = "your-secret-proxy-key"
+
+headers = {
+    "Content-Type": "application/json",
+    "Authorization": f"Bearer {proxy_key}"
+}
+
+data = {
+    "model": "gemini/gemini-pro",
+    "messages": [{"role": "user", "content": "What is the capital of France?"}],
+    "stream": False
+}
+
+response = requests.post(proxy_url, headers=headers, data=json.dumps(data))
+
+print(response.json())
+```
+
+## Using the Library in Other Projects
+
+The `rotating-api-key-client` library is designed to be reusable. You can find more information on how to use it in its own `README.md` file located at `src/rotator_library/README.md`.

requirements.txt

@@ -1,6 +1,4 @@
 fastapi
 uvicorn
 python-dotenv
-litellm
-requests
-filelock
+-e src/rotator_library

src/proxy_app/main.py

@@ -7,10 +7,10 @@ import logging
 from pathlib import Path
 import sys
 
-# This is necessary for the app to find the rotator_library module
-sys.path.append(str(Path(__file__).resolve().parent.parent.parent))
+# Add the 'src' directory to the Python path to allow importing 'rotating_api_key_client'
+sys.path.append(str(Path(__file__).resolve().parent.parent))
 
-from src.rotator_library.client import RotatingClient
+from rotator_library import RotatingClient
 
 # Configure logging
 logging.basicConfig(level=logging.INFO)

src/rotator_library/README.md

@@ -0,0 +1,53 @@
+# Rotating API Key Client
+
+A simple, thread-safe client that intelligently rotates and retries API keys for use with `litellm`.
+
+## Features
+
+-   **Smart Key Rotation**: Automatically uses the least-used key to distribute load.
+-   **Automatic Retries**: Retries requests on transient server errors.
+-   **Cooldowns**: Puts keys on a temporary cooldown after rate limit or authentication errors.
+-   **Usage Tracking**: Tracks daily and global usage for each key.
+-   **Provider Agnostic**: Works with any provider supported by `litellm`.
+
+## Installation
+
+To install the library, you can install it directly from a Git repository or a local path.
+
+### From a local path:
+
+```bash
+pip install -e .
+```
+
+## Usage
+
+Here's a simple example of how to use the `RotatingClient`:
+
+```python
+import asyncio
+from rotating_api_key_client import RotatingClient
+
+async def main():
+    # List of your API keys
+    api_keys = ["key1", "key2", "key3"]
+
+    # Initialize the client
+    client = RotatingClient(api_keys=api_keys)
+
+    # Make a request
+    response = await client.acompletion(
+        model="gemini/gemini-pro",
+        messages=[{"role": "user", "content": "Hello, how are you?"}]
+    )
+
+    print(response)
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+By default, the client will store usage data in a `key_usage.json` file in the current working directory. You can customize this by passing the `usage_file_path` parameter:
+
+```python
+client = RotatingClient(api_keys=api_keys, usage_file_path="/path/to/your/usage.json")

src/rotator_library/__init__.py

@@ -0,0 +1,17 @@
+"""
+Rotating API Key Client
+"""
+from .client import RotatingClient
+from .usage_manager import UsageManager
+from .error_handler import is_authentication_error, is_rate_limit_error, is_server_error, is_unrecoverable_error
+from .failure_logger import log_failure
+
+__all__ = [
+    "RotatingClient",
+    "UsageManager",
+    "is_authentication_error",
+    "is_rate_limit_error",
+    "is_server_error",
+    "is_unrecoverable_error",
+    "log_failure",
+]

src/rotator_library/client.py

@@ -18,12 +18,12 @@ class RotatingClient:
     A client that intelligently rotates and retries API keys using LiteLLM,
     with support for both streaming and non-streaming responses.
     """
-    def __init__(self, api_keys: List[str], max_retries: int = 2):
+    def __init__(self, api_keys: List[str], max_retries: int = 2, usage_file_path: str = "key_usage.json"):
         if not api_keys:
             raise ValueError("API keys list cannot be empty.")
         self.api_keys = api_keys
         self.max_retries = max_retries
-        self.usage_manager = UsageManager()
+        self.usage_manager = UsageManager(file_path=usage_file_path)
 
     async def _streaming_wrapper(self, stream: Any, key: str, model: str) -> AsyncGenerator[Any, None]:
         """

src/rotator_library/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "rotating-api-key-client"
+version = "0.1.0"
+authors = [
+    { name="Mirrowel", email="you@example.com" },
+]
+description = "A client that intelligently rotates and retries API keys using LiteLLM."
+readme = "README.md"
+requires-python = ">=3.7"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+]
+dependencies = [
+    "litellm",
+    "filelock",
+]
+
+[project.urls]
+"Homepage" = "https://github.com/Mirrowel/LLM-API-Key-Proxy"
+"Bug Tracker" = "https://github.com/Mirrowel/LLM-API-Key-Proxy/issues"
+
+[tool.setuptools.packages]
+find = { where = ["."], include = ["rotator_library*"] }