feat: Add OpenRouter and Chutes.ai provider support

This commit introduces support for two new LLM providers: OpenRouter and Chutes.ai.

**Key Changes:**

- **OpenRouter Provider:**
- Added `openrouter_provider.py` to fetch the list of available models from the OpenRouter API.
- Updated `.env.example` to include `OPENROUTER_API_KEYS`.

- **Chutes.ai Provider:**
- Added `chutes_provider.py` to fetch models from the Chutes.ai API.
- Implemented special handling in `client.py` to treat Chutes.ai as a custom OpenAI-compatible endpoint. This involves setting the `api_base` to `https://llm.chutes.ai/v1` and remapping the model name for `litellm`.
- Updated `.env.example` to include `CHUTES_API_KEYS`.

- **Documentation:**
- Updated `DOCUMENTATION.md`, `README.md`, and `src/rotator_library/README.md` to reflect the addition of the new providers.
- Clarified the dynamic provider loading mechanism.
- Added details about the special handling for Chutes.ai.

.env.example

@@ -2,6 +2,10 @@
 # Add more keys by creating GEMINI_API_KEY_2, GEMINI_API_KEY_3, etc.
 GEMINI_API_KEY_1="YOUR_GEMINI_API_KEY_1"
 GEMINI_API_KEY_2="YOUR_GEMINI_API_KEY_2"
+OPENROUTER_API_KEY_1="YOUR_OPENROUTER_API_KEY_1"
+OPENROUTER_API_KEY_2="YOUR_OPENROUTER_API_KEY_2"
+CHUTES_API_KEY_1="YOUR_CHUTES_API_KEY_1"
+CHUTES_API_KEY_2="YOUR_CHUTES_API_KEY_2"
 
 # A secret key for your proxy server to authenticate requests
 PROXY_API_KEY="YOUR_PROXY_API_KEY"

DOCUMENTATION.md

@@ -78,4 +78,13 @@ The provider plugin system allows for easy extension to support model list fetch
 
 -   **`provider_interface.py`**: Defines the abstract base class `ProviderPlugin` with a single abstract method, `get_models`. Any new provider plugin must inherit from this class and implement this method.
 -   **Implementations**: Each provider (e.g., `openai_provider.py`, `gemini_provider.py`) has its own file containing a class that implements the `ProviderPlugin` interface. The `get_models` method contains the specific logic to call the provider's API and return a list of their available models.
--   **`__init__.py`**: This file acts as a registry for the available plugins. The `PROVIDER_PLUGINS` dictionary maps provider names to their corresponding plugin classes. The `RotatingClient` uses this dictionary to instantiate the correct plugin at runtime.
+-   **`__init__.py`**: This file contains a dynamic plugin system that automatically discovers and registers any provider implementation placed in the `providers/` directory.
+
+### Special Provider: `chutes.ai`
+
+The `chutes` provider is handled as a special case within the `RotatingClient`. Since `litellm` does not have native support for `chutes.ai`, the client performs the following modifications at runtime:
+
+1.  **Sets `api_base`**: It sets the `api_base` to `https://llm.chutes.ai/v1`.
+2.  **Remaps the Model**: It changes the model name from `chutes/some-model` to `openai/some-model` before passing the request to `litellm`.
+
+This allows the system to use `chutes.ai` as if it were a custom OpenAI endpoint, while still leveraging the library's key rotation and management features.

README.md

@@ -84,6 +84,11 @@ The FastAPI proxy application exposes this functionality through an API endpoint
     GEMINI_API_KEY_2="your-gemini-api-key-2"
 
     OPENAI_API_KEY_1="your-openai-api-key-1"
+    
+    OPENROUTER_API_KEY_1="your-openrouter-api-key-1"
+
+    # chutes.ai is used as a custom OpenAI endpoint
+    CHUTES_API_KEY_1="your-chutes-api-key-1"
     ```
 
 ## Running the Proxy
@@ -98,7 +103,7 @@ The proxy will be available at `http://127.0.0.1:8000`.
 
 You can make requests to the proxy as if it were the OpenAI API. Remember to include your `PROXY_API_KEY` in the `Authorization` header.
 
-The `model` parameter must be specified in the format `provider/model_name` (e.g., `gemini/gemini-2.5-flash-preview-05-20`, `openai/gpt-4`).
+The `model` parameter must be specified in the format `provider/model_name` (e.g., `gemini/gemini-2.5-flash-preview-05-20`, `openai/gpt-4`, `openrouter/google/gemini-flash-1.5`, `chutes/deepseek-ai/DeepSeek-R1-0528`).
 
 ### Example with `curl` (Non-Streaming):
 ```bash

src/rotator_library/README.md

@@ -46,7 +46,7 @@ client = RotatingClient(
 
 This is the primary method for making API calls. It's a wrapper around `litellm.acompletion` that adds key rotation and retry logic.
 
--   **Parameters**: Accepts the same keyword arguments as `litellm.acompletion` (e.g., `messages`, `stream`). The `model` parameter is required and must be a string in the format `provider/model_name` (e.g., `"gemini/gemini-2.5-flash-preview-05-20"`).
+-   **Parameters**: Accepts the same keyword arguments as `litellm.acompletion` (e.g., `messages`, `stream`). The `model` parameter is required and must be a string in the format `provider/model_name` (e.g., `"gemini/gemini-2.5-flash-preview-05-20"`, `"openrouter/google/gemini-flash-1.5"`, `"chutes/deepseek-ai/DeepSeek-R1-0528"`).
 -   **Returns**:
     -   For non-streaming requests, it returns the `litellm` response object.
     -   For streaming requests, it returns an async generator that yields OpenAI-compatible Server-Sent Events (SSE).
@@ -104,39 +104,29 @@ Cooldowns are managed by the `UsageManager` on a per-model basis, preventing fai
 
 ## Extending with Provider Plugins
 
-You can add support for fetching model lists from new providers by creating a custom provider plugin.
+The library uses a dynamic plugin system. To add support for a new provider, you only need to do two things:
 
-1.  **Create a new provider file** in `src/rotator_library/providers/`, for example, `my_provider.py`.
-2.  **Implement the `ProviderPlugin` interface**:
+1.  **Create a new provider file** in `src/rotator_library/providers/` (e.g., `my_provider.py`). The name of the file (without `_provider.py`) will be used as the provider name (e.g., `my_provider`).
+2.  **Implement the `ProviderInterface`**: Inside your new file, create a class that inherits from `ProviderInterface` and implements the `get_models` method.
 
-    ```python
-    # src/rotator_library/providers/my_provider.py
-    from .provider_interface import ProviderPlugin
-    from typing import List
-
-    class MyProvider(ProviderPlugin):
-        async def get_models(self, api_key: str) -> List[str]:
-            # Logic to fetch and return a list of model names
-            # e.g., ["my-provider/model-1", "my-provider/model-2"]
-            pass
-    ```
-
-3.  **Register the plugin** in `src/rotator_library/providers/__init__.py`:
+```python
+# src/rotator_library/providers/my_provider.py
+from .provider_interface import ProviderInterface
+from typing import List
+
+class MyProvider(ProviderInterface):
+    async def get_models(self, api_key: str) -> List[str]:
+        # Logic to fetch and return a list of model names
+        # The model names should be prefixed with the provider name.
+        # e.g., ["my-provider/model-1", "my-provider/model-2"]
+        pass
+```
 
-    ```python
-    # src/rotator_library/providers/__init__.py
-    from .openai_provider import OpenAIProvider
-    from .gemini_provider import GeminiProvider
-    from .my_provider import MyProvider # Import your new provider
+The system will automatically discover and register your new provider when the library is imported.
 
-    PROVIDER_PLUGINS = {
-        "openai": OpenAIProvider,
-        "gemini": GeminiProvider,
-        "my_provider": MyProvider, # Add it to the dictionary
-    }
-    ```
+### Special Case: `chutes.ai`
 
-The `RotatingClient` will automatically use your new plugin when `get_available_models` is called for `"my_provider"`.
+The `chutes` provider is handled as a special case. Since `litellm` does not support it directly, the `RotatingClient` modifies the request by setting the `api_base` to `https://llm.chutes.ai/v1` and remapping the model from `chutes/model-name` to `openai/model-name`. This allows `chutes.ai` to be used as a custom OpenAI-compatible endpoint.
 
 ## Detailed Documentation
 

src/rotator_library/client.py

@@ -81,7 +81,16 @@ class RotatingClient:
             for attempt in range(self.max_retries):
                 try:
                     print(f"Attempting call with key ...{current_key[-4:]} (Attempt {attempt + 1}/{self.max_retries})")
-                    response = await litellm.acompletion(api_key=current_key, **kwargs)
+                    
+                    # Create a copy of kwargs to modify for the litellm call
+                    litellm_kwargs = kwargs.copy()
+                    
+                    # Handle chutes.ai as a special case
+                    if provider == "chutes":
+                        litellm_kwargs["model"] = f"openai/{model.split('/', 1)[1]}"
+                        litellm_kwargs["api_base"] = "https://llm.chutes.ai/v1"
+
+                    response = await litellm.acompletion(api_key=current_key, **litellm_kwargs)
 
                     if is_streaming:
                         # For streams, we return a wrapper generator that logs usage on completion.

src/rotator_library/providers/chutes_provider.py

@@ -0,0 +1,23 @@
+import requests
+import logging
+from typing import List
+from .provider_interface import ProviderInterface
+
+class ChutesProvider(ProviderInterface):
+    """
+    Provider implementation for the chutes.ai API.
+    """
+    async def get_models(self, api_key: str) -> List[str]:
+        """
+        Fetches the list of available models from the chutes.ai API.
+        """
+        try:
+            response = requests.get(
+                "https://llm.chutes.ai/v1/models",
+                headers={"Authorization": f"Bearer {api_key}"}
+            )
+            response.raise_for_status()
+            return [f"chutes/{model['id']}" for model in response.json().get("data", [])]
+        except requests.RequestException as e:
+            logging.error(f"Failed to fetch chutes.ai models: {e}")
+            return []

src/rotator_library/providers/openrouter_provider.py

@@ -0,0 +1,23 @@
+import requests
+import logging
+from typing import List
+from .provider_interface import ProviderInterface
+
+class OpenRouterProvider(ProviderInterface):
+    """
+    Provider implementation for the OpenRouter API.
+    """
+    async def get_models(self, api_key: str) -> List[str]:
+        """
+        Fetches the list of available models from the OpenRouter API.
+        """
+        try:
+            response = requests.get(
+                "https://openrouter.ai/api/v1/models",
+                headers={"Authorization": f"Bearer {api_key}"}
+            )
+            response.raise_for_status()
+            return [f"openrouter/{model['id']}" for model in response.json().get("data", [])]
+        except requests.RequestException as e:
+            logging.error(f"Failed to fetch OpenRouter models: {e}")
+            return []