src/executable_howto.txt

# FastAPI Server Startup Guide

## Prerequisites

Before starting the server, make sure you have all the required dependencies installed:

```bash
# Install FastAPI and related dependencies
pip install fastapi uvicorn python-dotenv

# Install any other dependencies your connectors need
pip install aiohttp requests  # for API connectors
pip install pandas numpy      # for data processing (if needed)
```

## Project Structure

Your project should look like this:

```
your-osint-project/
├── main.py                    # Your improved main file
├── .env                       # Environment variables (API keys)
├── requirements.txt           # Python dependencies
├── frontend/
│   └── index.html            # The frontend I created
├── connectors/
│   ├── __init__.py
│   ├── intelligencex.py
│   ├── enisa_cve.py
│   ├── virustotal.py
│   └── shodan.py
├── core/
│   ├── __init__.py
│   ├── normalize.py
│   ├── correlate.py
│   └── risk_engine.py
└── llm_agent/
    ├── __init__.py
    ├── llm_chatGPT.py
    └── prompt_templates.py    # The template file I created
```

## Environment Setup

Create a `.env` file in your project root with your API keys:

```env
# API Keys for OSINT sources
INTELLIGENCE_X_API_KEY=your_intelx_key_here
VIRUSTOTAL_API_KEY=your_virustotal_key_here
SHODAN_API_KEY=your_shodan_key_here

# LLM Configuration
OPENAI_API_KEY=your_openai_key_here
# OR if using Google Gemini:
GOOGLE_API_KEY=your_gemini_key_here

# Server Configuration
HOST=0.0.0.0
PORT=8000
DEBUG=True
```

## Method 1: Direct Python Execution

The easiest way to start your server:

```bash
# Navigate to your project directory
cd /path/to/your-osint-project

# Run the server directly
python main.py
```

This works because of the code at the bottom of your `main.py`:

```python
if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000, log_level="info")
```

## Method 2: Using Uvicorn Command

More control over server settings:

```bash
# Basic startup
uvicorn main:app --reload

# With specific host and port
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

# With detailed logging
uvicorn main:app --host 0.0.0.0 --port 8000 --reload --log-level debug
```

### Uvicorn Options Explained:
- `main:app` - Refers to the `app` object in `main.py`
- `--reload` - Automatically restart when code changes (development only)
- `--host 0.0.0.0` - Accept connections from any IP
- `--port 8000` - Server port
- `--log-level debug` - Detailed logging

## Method 3: Production Deployment

For production environments:

```bash
# Install production server
pip install gunicorn

# Run with Gunicorn (more robust for production)
gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000
```

## Verification Steps

Once your server starts, you should see output like:

```
INFO:     Started server process [12345]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
```

### Test Your Server:

1. **Health Check:**
   ```bash
   curl http://localhost:8000/api/health
   ```
   Should return: `{"status": "healthy", "version": "1.0.0"}`

2. **Sources Check:**
   ```bash
   curl http://localhost:8000/api/sources
   ```

3. **Frontend Access:**
   Open your browser and go to: `http://localhost:8000`

4. **API Test:**
   ```bash
   curl "http://localhost:8000/api/osint?entity=8.8.8.8"
   ```

## Common Issues and Solutions

### Issue 1: Frontend Not Loading
```
ERROR: Frontend directory not found
```
**Solution:** Make sure the frontend directory exists and contains `index.html`:
```bash
mkdir -p frontend
# Copy the HTML file I created to frontend/index.html
```

### Issue 2: Import Errors
```
ModuleNotFoundError: No module named 'connectors'
```
**Solution:** Make sure all directories have `__init__.py` files:
```bash
touch connectors/__init__.py
touch core/__init__.py
touch llm_agent/__init__.py
```

### Issue 3: API Key Issues
```
Error: Missing API key
```
**Solution:** Check your `.env` file is in the project root and contains the required keys.

### Issue 4: Port Already in Use
```
OSError: [Errno 98] Address already in use
```
**Solution:** Use a different port:
```bash
uvicorn main:app --port 8001 --reload
```

## Development Workflow

1. **Start the server in development mode:**
   ```bash
   uvicorn main:app --reload --log-level debug
   ```

2. **Make changes to your code** - The server will automatically restart

3. **Test in browser:** Go to `http://localhost:8000`

4. **Check logs** in the terminal for any errors

## Production Deployment

For production, consider:

1. **Use environment variables** for configuration
2. **Set up proper logging**
3. **Use a process manager** like systemd or PM2
4. **Add reverse proxy** (nginx) for better performance
5. **Enable HTTPS** for security

Example systemd service file (`/etc/systemd/system/osint-api.service`):

```ini
[Unit]
Description=OSINT Early Warning API
After=network.target

[Service]
User=your-user
Group=your-group
WorkingDirectory=/path/to/your-osint-project
Environment=PATH=/path/to/your-venv/bin
ExecStart=/path/to/your-venv/bin/gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000
Restart=always

[Install]
WantedBy=multi-user.target
```

## Quick Start Script

Create a `start_server.sh` script:

```bash
#!/bin/bash
echo "Starting OSINT Early Warning System..."

# Check if virtual environment exists
if [ ! -d "venv" ]; then
    echo "Creating virtual environment..."
    python3 -m venv venv
fi

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Start server
echo "Starting FastAPI server..."
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
```

Make it executable:
```bash
chmod +x start_server.sh
./start_server.sh
```

This should get your FastAPI server up and running! Let me know if you encounter any specific issues.