final

README.md

@@ -1,527 +1,192 @@
-# Duck.ai OpenAI-Compatible Server
+# DuckAI OpenAI Server
 
-A high-performance HTTP server built with Bun that provides OpenAI-compatible API endpoints using Duck.ai as the backend. This allows you to use any OpenAI SDK or tool with Duck.ai's free AI models.
+A high-performance OpenAI-compatible HTTP server that uses DuckDuckGo's AI backend, providing free access to multiple AI models through the familiar OpenAI API interface.
 
-## 🚀 Features
+## Introduction
 
-- **OpenAI API Compatible**: Drop-in replacement for OpenAI API
-- **Multiple Models**: Support for GPT-4o-mini, Claude-3-Haiku, Llama, Mistral, and more
-- **Function Calling/Tools**: Full support for OpenAI-compatible function calling
-- **Streaming Support**: Real-time streaming responses (including with tools)
-- **Built with Bun**: Ultra-fast TypeScript runtime
-- **CORS Enabled**: Ready for web applications
-- **Comprehensive Testing**: Full test suite ensuring compatibility
+DuckAI OpenAI Server bridges the gap between DuckDuckGo's free AI chat service and the widely-adopted OpenAI API format. This allows you to:
 
-## 📋 Available Models
+- **Use multiple AI models for free** - Access GPT-4o-mini, Claude-3-Haiku, Llama-3.3-70B, and more
+- **Drop-in OpenAI replacement** - Compatible with existing OpenAI client libraries
+- **Tool calling support** - Full function calling capabilities
+- **Streaming responses** - Real-time response streaming
+- **Rate limiting** - Built-in intelligent rate limiting to respect DuckDuckGo's limits
 
-- `gpt-4o-mini`
+### Supported Models
+
+- `gpt-4o-mini` (Default)
 - `o3-mini`
 - `claude-3-haiku-20240307`
 - `meta-llama/Llama-3.3-70B-Instruct-Turbo`
-- `mistralai/Mistral-Small-24B-Instruct-2501`
-
-## 🛠️ Installation
+- `mistralai/Mixtral-8x7B-Instruct-v0.1`
 
-1. **Install Bun** (if not already installed):
-   ```bash
-   curl -fsSL https://bun.sh/install | bash
-   ```
+### Features
 
-2. **Clone and setup the project**:
-   ```bash
-   git clone <your-repo>
-   cd duckai-openai-server
-   bun install
-   ```
+- ✅ Chat completions
+- ✅ Streaming responses
+- ✅ Function/tool calling
+- ✅ Multiple model support
+- ✅ Rate limiting with intelligent backoff
+- ✅ OpenAI-compatible error handling
+- ✅ CORS support
+- ✅ Health check endpoint
 
-3. **Start the server**:
-   ```bash
-   bun run dev
-   ```
+## Usage
 
-The server will start on `http://localhost:3000` by default.
+### Prerequisites
 
-## 🔧 Usage
+- [Bun](https://bun.sh/) runtime (recommended) or Node.js 18+
 
-### Basic cURL Example
+### Installation
 
+1. Clone the repository:
 ```bash
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "gpt-4o-mini",
-    "messages": [
-      {"role": "user", "content": "Hello, how are you?"}
-    ]
-  }'
+git clone git@github.com:amirkabiri/duckai.git
+cd duckai
 ```
 
-### Streaming Example
-
+2. Install dependencies:
 ```bash
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "gpt-4o-mini",
-    "messages": [
-      {"role": "user", "content": "Count from 1 to 10"}
-    ],
-    "stream": true
-  }'
+bun install
 ```
 
-### Using with OpenAI SDK (Python)
-
-```python
-from openai import OpenAI
+3. Start the server:
+```bash
+bun run dev
+```
 
-client = OpenAI(
-    base_url="http://localhost:3000/v1",
-    api_key="dummy-key"  # Not required, but SDK expects it
-)
+The server will start on `http://localhost:3000` by default.
 
-# Non-streaming
-response = client.chat.completions.create(
-    model="gpt-4o-mini",
-    messages=[
-        {"role": "user", "content": "Hello!"}
-    ]
-)
-print(response.choices[0].message.content)
-
-# Streaming
-stream = client.chat.completions.create(
-    model="gpt-4o-mini",
-    messages=[
-        {"role": "user", "content": "Tell me a story"}
-    ],
-    stream=True
-)
-
-for chunk in stream:
-    if chunk.choices[0].delta.content is not None:
-        print(chunk.choices[0].delta.content, end="")
-```
+### Basic Usage
 
-### Using with OpenAI SDK (Node.js)
+#### Using with OpenAI JavaScript Library
 
 ```javascript
-import OpenAI from 'openai';
+import OpenAI from "openai";
 
 const openai = new OpenAI({
-  baseURL: 'http://localhost:3000/v1',
-  apiKey: 'dummy-key', // Not required, but SDK expects it
+  baseURL: "http://localhost:3000/v1",
+  apiKey: "dummy-key", // Any string works
 });
 
-// Non-streaming
+// Basic chat completion
 const completion = await openai.chat.completions.create({
-  model: 'gpt-4o-mini',
+  model: "gpt-4o-mini",
   messages: [
-    { role: 'user', content: 'Hello!' }
+    { role: "user", content: "Hello! How are you?" }
   ],
 });
 
 console.log(completion.choices[0].message.content);
-
-// Streaming
-const stream = await openai.chat.completions.create({
-  model: 'gpt-4o-mini',
-  messages: [
-    { role: 'user', content: 'Tell me a story' }
-  ],
-  stream: true,
-});
-
-for await (const chunk of stream) {
-  process.stdout.write(chunk.choices[0]?.delta?.content || '');
-}
 ```
 
-### Function Calling (Tools)
+#### Tool Calling Example
 
 ```javascript
-import OpenAI from 'openai';
-
-const openai = new OpenAI({
-  baseURL: 'http://localhost:3000/v1',
-  apiKey: 'dummy-key',
-});
+const tools = [
+  {
+    type: "function",
+    function: {
+      name: "calculate",
+      description: "Perform mathematical calculations",
+      parameters: {
+        type: "object",
+        properties: {
+          expression: {
+            type: "string",
+            description: "Mathematical expression to evaluate"
+          }
+        },
+        required: ["expression"]
+      }
+    }
+  }
+];
 
 const completion = await openai.chat.completions.create({
-  model: 'gpt-4o-mini',
+  model: "gpt-4o-mini",
   messages: [
-    { role: 'user', content: 'What time is it?' }
+    { role: "user", content: "What is 15 * 8?" }
   ],
-  tools: [
-    {
-      type: 'function',
-      function: {
-        name: 'get_current_time',
-        description: 'Get the current time'
-      }
-    }
-  ]
+  tools: tools,
+  tool_choice: "auto"
 });
 
-// Handle function calls
-if (completion.choices[0].finish_reason === 'tool_calls') {
-  const toolCalls = completion.choices[0].message.tool_calls;
-  console.log('Function calls:', toolCalls);
-}
-```
-
-## 🌐 API Endpoints
-
-### `GET /health`
-Health check endpoint.
-
-**Response:**
-```json
-{"status": "ok"}
-```
-
-### `GET /v1/models`
-List available models.
-
-**Response:**
-```json
-{
-  "object": "list",
-  "data": [
-    {
-      "id": "gpt-4o-mini",
-      "object": "model",
-      "created": 1640995200,
-      "owned_by": "duckai"
-    }
-  ]
-}
+// The AI will call the calculate function
+console.log(completion.choices[0].message.tool_calls);
 ```
 
-### `POST /v1/chat/completions`
-Create chat completions (OpenAI compatible).
-
-**Request Body:**
-```json
-{
-  "model": "gpt-4o-mini",
-  "messages": [
-    {"role": "user", "content": "Hello!"}
-  ],
-  "stream": false,
-  "temperature": 0.7,
-  "max_tokens": 150,
-  "tools": [
-    {
-      "type": "function",
-      "function": {
-        "name": "get_weather",
-        "description": "Get weather for a location",
-        "parameters": {
-          "type": "object",
-          "properties": {
-            "location": {
-              "type": "string",
-              "description": "City name"
-            }
-          },
-          "required": ["location"]
-        }
-      }
-    }
-  ],
-  "tool_choice": "auto"
-}
-```
+#### Streaming Responses
 
-**Response (Non-streaming):**
-```json
-{
-  "id": "chatcmpl-abc123",
-  "object": "chat.completion",
-  "created": 1640995200,
-  "model": "gpt-4o-mini",
-  "choices": [
-    {
-      "index": 0,
-      "message": {
-        "role": "assistant",
-        "content": "Hello! How can I help you today?"
-      },
-      "finish_reason": "stop"
-    }
+```javascript
+const stream = await openai.chat.completions.create({
+  model: "gpt-4o-mini",
+  messages: [
+    { role: "user", content: "Tell me a story" }
   ],
-  "usage": {
-    "prompt_tokens": 10,
-    "completion_tokens": 20,
-    "total_tokens": 30
-  }
-}
-```
-
-**Response (Streaming):**
-```
-data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk",...}
-
-data: [DONE]
-```
+  stream: true
+});
 
-**Response (Function Call):**
-```json
-{
-  "id": "chatcmpl-abc123",
-  "object": "chat.completion",
-  "created": 1640995200,
-  "model": "gpt-4o-mini",
-  "choices": [
-    {
-      "index": 0,
-      "message": {
-        "role": "assistant",
-        "content": null,
-        "tool_calls": [
-          {
-            "id": "call_1",
-            "type": "function",
-            "function": {
-              "name": "get_weather",
-              "arguments": "{\"location\": \"San Francisco\"}"
-            }
-          }
-        ]
-      },
-      "finish_reason": "tool_calls"
-    }
-  ],
-  "usage": {
-    "prompt_tokens": 25,
-    "completion_tokens": 15,
-    "total_tokens": 40
+for await (const chunk of stream) {
+  const content = chunk.choices[0]?.delta?.content;
+  if (content) {
+    process.stdout.write(content);
   }
 }
 ```
 
-## 🛠️ Function Calling (Tools)
-
-This server implements OpenAI-compatible function calling using a "prompt engineering trick" since Duck.ai doesn't natively support tools. The system works by:
-
-1. **Converting tools to system prompts**: Tool definitions are converted into detailed instructions for the AI
-2. **Parsing AI responses**: The AI is instructed to respond with JSON when it needs to call functions
-3. **Executing functions**: Built-in and custom functions can be executed
-4. **Returning results**: Function results are formatted as OpenAI-compatible responses
-
-### Built-in Functions
-
-The server comes with several built-in functions:
-
-- `get_current_time`: Returns the current timestamp
-- `calculate`: Performs mathematical calculations
-- `get_weather`: Returns mock weather data (for demonstration)
-
-### Custom Functions
-
-You can register custom functions programmatically:
-
-```javascript
-// In your server setup
-import { OpenAIService } from './src/openai-service';
-
-const service = new OpenAIService();
-service.registerFunction('my_function', (args) => {
-  return `Hello ${args.name}!`;
-});
-```
-
-### Tool Choice Options
-
-- `"auto"` (default): AI decides whether to call functions
-- `"none"`: AI will not call any functions
-- `"required"`: AI must call at least one function
-- `{"type": "function", "function": {"name": "specific_function"}}`: AI must call the specified function
-
-### Example: Multi-turn Conversation with Tools
+#### Using with curl
 
 ```bash
 curl -X POST http://localhost:3000/v1/chat/completions \
   -H "Content-Type: application/json" \
+  -H "Authorization: Bearer dummy-key" \
   -d '{
     "model": "gpt-4o-mini",
     "messages": [
-      {"role": "user", "content": "What time is it?"},
-      {
-        "role": "assistant",
-        "content": null,
-        "tool_calls": [
-          {
-            "id": "call_1",
-            "type": "function",
-            "function": {
-              "name": "get_current_time",
-              "arguments": "{}"
-            }
-          }
-        ]
-      },
-      {
-        "role": "tool",
-        "content": "2024-01-15T10:30:00Z",
-        "tool_call_id": "call_1"
-      },
-      {"role": "user", "content": "Thanks! Now calculate 15 + 27"}
-    ],
-    "tools": [
-      {
-        "type": "function",
-        "function": {
-          "name": "get_current_time",
-          "description": "Get the current time"
-        }
-      },
-      {
-        "type": "function",
-        "function": {
-          "name": "calculate",
-          "description": "Perform mathematical calculations",
-          "parameters": {
-            "type": "object",
-            "properties": {
-              "expression": {
-                "type": "string",
-                "description": "Mathematical expression to evaluate"
-              }
-            },
-            "required": ["expression"]
-          }
-        }
-      }
+      {"role": "user", "content": "Hello!"}
     ]
   }'
 ```
 
-## 🧪 Testing
-
-### Run All Tests
-```bash
-bun test
-```
-
-### Run Specific Test Suites
-```bash
-# Run server API tests
-bun test tests/server.test.ts
-
-# Run OpenAI JavaScript library compatibility tests
-bun run test:openai
-
-# Run comprehensive OpenAI library tests (more extensive)
-bun run test:openai-full
-
-# Run all core tests together
-bun run test:all
-```
-
-### Run Manual OpenAI SDK Compatibility Test
-```bash
-# Start the server first
-bun run dev
-
-# In another terminal, run the manual compatibility test
-bun run tests/openai-sdk-test.ts
-```
-
-### Manual Testing with Different Tools
-
-**Test with HTTPie:**
-```bash
-http POST localhost:3000/v1/chat/completions \
-  model=gpt-4o-mini \
-  messages:='[{"role":"user","content":"Hello!"}]'
-```
-
-**Test with Postman:**
-- URL: `http://localhost:3000/v1/chat/completions`
-- Method: POST
-- Headers: `Content-Type: application/json`
-- Body: Raw JSON with the request format above
+### API Endpoints
 
-## 🔧 Configuration
+- `POST /v1/chat/completions` - Chat completions (compatible with OpenAI)
+- `GET /v1/models` - List available models
+- `GET /health` - Health check endpoint
 
 ### Environment Variables
 
-- `PORT`: Server port (default: 3000)
-
-### Custom Model Selection
-
-You can specify any of the available models in your requests:
-
-```json
-{
-  "model": "claude-3-haiku-20240307",
-  "messages": [{"role": "user", "content": "Hello!"}]
-}
-```
-
-## 🏗️ Development
-
-### Project Structure
-```
-src/
-├── types.ts          # TypeScript type definitions
-├── duckai.ts         # Duck.ai integration
-├── openai-service.ts # OpenAI compatibility layer
-└── server.ts         # Main HTTP server
-
-tests/
-├── server.test.ts         # Server API unit tests
-├── openai-simple.test.ts  # OpenAI library compatibility tests
-├── openai-library.test.ts # Comprehensive OpenAI library tests
-└── openai-sdk-test.ts     # Manual SDK compatibility demo
-```
+- `PORT` - Server port (default: 3000)
+- `HOST` - Server host (default: 0.0.0.0)
 
-### Adding New Features
+## Usage with Docker
 
-1. **Add new types** in `src/types.ts`
-2. **Extend Duck.ai integration** in `src/duckai.ts`
-3. **Update OpenAI service** in `src/openai-service.ts`
-4. **Add tests** in `tests/`
-
-### Building for Production
+### Building the Docker Image
 
 ```bash
-bun run build
-bun run start
+docker build -t duckai .
 ```
 
-## 🐛 Troubleshooting
-
-### Common Issues
+### Running with Docker
 
-1. **"fetch is not defined"**: Make sure you're using Bun, not Node.js
-2. **CORS errors**: The server includes CORS headers, but check your client configuration
-3. **Model not found**: Use one of the supported models listed above
-4. **Streaming not working**: Ensure your client properly handles Server-Sent Events
-
-### Debug Mode
-
-Set environment variable for verbose logging:
 ```bash
-DEBUG=1 bun run dev
+docker run -p 3000:3000 duckai
 ```
 
-## 📝 License
-
-MIT License - see LICENSE file for details.
-
-## 🤝 Contributing
+## Contributing
 
 1. Fork the repository
 2. Create a feature branch
-3. Add tests for new functionality
-4. Ensure all tests pass
-5. Submit a pull request
+3. Make your changes
+4. Add tests for new functionality
+5. Run the test suite
+6. Submit a pull request
+
+## License
+
+MIT License - see LICENSE file for details.
 
-## 🙏 Acknowledgments
+## Disclaimer
 
-- Built on top of the reverse-engineered Duck.ai API
-- Compatible with OpenAI API specification
-- Powered by Bun runtime 
+This project is not affiliated with DuckDuckGo or OpenAI. It's an unofficial bridge service for educational and development purposes. Please respect DuckDuckGo's terms of service and rate limits.