IMPLEMENTATION_GUIDE.md

# Production Features Implementation Guide

This document explains what has been implemented for the Hickey Lab AI Assistant and how to configure and use each feature.

---

## 📦 What Has Been Implemented

All the following features from the production roadmap have been implemented:

### ✅ Phase 1: Foundation - Cost & Security Controls (High Priority 🔴)

#### 1. **Cost Management Module** (`utils/cost_tracker.py`)

Tracks API token usage and costs to prevent budget overruns.



**What it does:**

- Extracts token counts from every Gemini API response

- Calculates costs based on Gemini 2.5 Flash pricing ($0.075 per 1M input tokens, $0.30 per 1M output tokens)

- Logs all usage to `logs/usage.jsonl` with timestamps

- Tracks daily and monthly usage statistics

- Enforces budget caps (blocks service when exceeded)

- Generates usage reports



**How to use it:**

1. Set budget limits in `config.py`:

   - `DAILY_QUERY_LIMIT`: Maximum queries per day (default: 200)

   - `MONTHLY_BUDGET_USD`: Monthly budget cap (default: $50)

   - `DAILY_BUDGET_WARNING`: Warning threshold (default: $5)



2. View usage stats in the sidebar by checking "📊 Show Usage Stats"



3. Generate reports manually:

   ```python

   from utils.cost_tracker import CostTracker
   tracker = CostTracker()
   print(tracker.generate_daily_report())
   print(tracker.generate_monthly_report(2024, 12))
   ```



#### 2. **Rate Limiting System** (`utils/rate_limiter.py`)

Prevents abuse through configurable rate limits.



**What it does:**

- Tracks queries per session using sliding time windows

- Enforces hourly limits (default: 20 queries per hour)

- Enforces daily limits (default: 200 queries per 24 hours)

- Shows warnings when approaching limits (at 80% by default)

- Blocks queries when limits exceeded with friendly messages

- Logs rate limit violations



**How to use it:**

1. Configure limits in `config.py`:

   - `RATE_LIMIT_PER_HOUR`: Queries per hour (default: 20)

   - `RATE_LIMIT_PER_DAY`: Queries per day (default: 200)

   - `RATE_LIMIT_WARNING_THRESHOLD`: When to warn (default: 0.8 = 80%)



2. Users will automatically see warnings like:

   - "⚠️ You have 4 questions remaining this hour"

   - "🕐 Rate limit reached! Please wait 15 minutes..."



#### 3. **Security Module** (`utils/security.py`)

Validates and sanitizes user input to prevent attacks.



**What it does:**

- Checks input length (1-2000 characters by default)

- Detects prompt injection attempts ("ignore previous instructions", etc.)

- Blocks suspicious patterns (script tags, template injection, etc.)

- Detects excessive special characters

- Logs all security violations for review



**How to use it:**

1. Configure limits in `config.py`:

   - `MAX_INPUT_LENGTH`: Maximum characters (default: 2000)

   - `MIN_INPUT_LENGTH`: Minimum characters (default: 1)



2. Security is automatic - invalid inputs are rejected with user-friendly messages



3. Review security logs in `logs/security.jsonl` to monitor threats



#### 4. **Alert System** (`utils/alerts.py`)

Sends push notifications for critical events using ntfy.sh.



**What it does:**

- Sends push notifications to your phone/browser via ntfy.sh (free, no signup)

- Alerts for rate limit violations

- Alerts for cost threshold breaches

- Alerts for suspicious activity

- Alerts for error spikes

- Supports priority levels (min, low, default, high, urgent)



**How to set it up:**



1. **Subscribe to notifications:**

   - Option A (Browser): Go to `https://ntfy.sh/YOUR-TOPIC-NAME` and click "Subscribe"

   - Option B (Mobile App):

     - Install ntfy app (iOS/Android)

     - Add subscription with your topic name



2. **Choose a SECURE topic name:**

   - ⚠️ IMPORTANT: Use a random, hard-to-guess name for security!

   - ✅ Good: `hickeylab-alerts-x9k2m7a4`

   - ❌ Bad: `hickeylab-alerts` (anyone can subscribe)



3. **Configure the topic:**

   - Set in `config.py`: `NTFY_TOPIC = "your-topic-name"`

   - Or set environment variable: `NTFY_TOPIC=your-topic-name`



4. **Test it:**

   ```bash

   python -c "from utils.alerts import AlertSystem; AlertSystem().test_alert()"

   ```
   Or:
   ```bash

   curl -d "Test alert" ntfy.sh/your-topic-name

   ```

**What you'll be notified about:**
- ⚠️ User hits rate limit
- 💰 Daily/monthly cost thresholds (80%, 100%)
- 🔍 Suspicious activity detected
- 🚨 Service paused due to budget limits

---

### ✅ Phase 2: Monitoring & Quality (Medium Priority 🟡)

#### 5. **Enhanced Logging**
All queries are logged with metadata for analysis.

**What's logged:**
- Timestamp
- Session ID (truncated for privacy)
- Question length
- Token counts (prompt, response, total)
- Estimated cost
- Response time
- Success/failure status
- Error messages (if any)

**Log files:**
- `logs/usage.jsonl` - All API usage
- `logs/rate_limits.jsonl` - Rate limit violations
- `logs/security.jsonl` - Security violations

#### 6. **Conversation Context**
Maintains context across multiple messages for better responses.

**What it does:**
- Includes last 5 exchanges in each query (configurable)
- Allows follow-up questions to reference previous messages
- Example:
  - User: "What is CODEX?"
  - Assistant: [explains CODEX]
  - User: "How does it compare to IBEX?"
  - Assistant: [compares CODEX (from context) to IBEX]

**How to configure:**
- Adjust `CONVERSATION_HISTORY_LENGTH` in `config.py` (default: 5)

#### 7. **Enhanced System Prompt**
Improved instructions for better response quality.

**What's improved:**
- Conversation context awareness
- Response structure guidelines (2-4 paragraphs for complex topics)
- Specific citation instructions
- Technical term explanation requirements
- Grounding in knowledge base (no hallucinations)

---

### ✅ Phase 3: User Experience (Low Priority 🟢)

#### 8. **Suggested Questions**
Shows starter questions when chat is empty.

**What it does:**
- Displays 4 suggested questions as clickable buttons
- Questions are configured in `config.py`
- Helps new users get started

**How to customize:**
- Edit `SUGGESTED_QUESTIONS` in `config.py`

#### 9. **Privacy Notice**
Displays privacy and usage information.

**What it shows:**
- Data processing information
- Usage limits
- Privacy policy

**How to customize:**
- Edit `PRIVACY_NOTICE` in `config.py`

#### 10. **Usage Statistics Dashboard**
Shows real-time usage stats in sidebar.

**What it shows:**
- Today's query count and cost
- This month's query count and cost
- Optional display (checkbox in sidebar)

#### 11. **Mobile Responsive Design**
Improved CSS for mobile devices.

**What's improved:**
- Touch-friendly button sizes (44px minimum)
- Appropriate font sizes
- No iOS zoom on input focus
- Responsive layout

---

## 🚀 Deployment Instructions

### For HuggingFace Spaces:

1. **Set up secrets:**
   - Go to Space Settings → Variables and secrets
   - Add `GEMINI_API_KEY` as a Secret
   - (Optional) Add `NTFY_TOPIC` for notifications

2. **Upload files:**
   - Upload the entire `outreach/pipelines/gemini_file_search/` directory
   - Ensure all files are included:
     - `app.py`
     - `config.py`
     - `requirements.txt`
     - `utils/` directory with all modules

3. **The app will automatically:**
   - Install dependencies from `requirements.txt`
   - Start the Streamlit app
   - Create `logs/` directory when first query is made

### Environment Variables:

| Variable | Required | Description |
|----------|----------|-------------|
| `GEMINI_API_KEY` | ✅ Yes | Your Google Gemini API key |
| `NTFY_TOPIC` | ❌ Optional | Your ntfy.sh topic for push notifications |

### First-Time Setup:

1. **Test the app** with a few queries
2. **Subscribe to notifications** if you set up ntfy.sh
3. **Check logs** in `logs/` directory (if accessible)
4. **Adjust limits** in `config.py` if needed

---

## 📊 Monitoring & Maintenance

### Daily Tasks:
- Check usage stats in the sidebar
- Watch for notification alerts on your phone/browser

### Weekly Tasks:
- Review `logs/usage.jsonl` for usage patterns
- Check `logs/security.jsonl` for any threats
- Adjust rate limits if needed

### Monthly Tasks:
- Generate monthly cost report
- Review budget and adjust if needed
- Update system prompt based on user feedback

### Generating Reports:

```python

from utils.cost_tracker import CostTracker



tracker = CostTracker()



# Daily report

print(tracker.generate_daily_report())



# Monthly report

print(tracker.generate_monthly_report(2024, 12))



# Custom date

from datetime import datetime

print(tracker.generate_daily_report(datetime(2024, 12, 15)))

```

---

## ⚙️ Configuration Reference

All configuration is in `config.py`. Key settings:

### Cost Management:
```python

DAILY_QUERY_LIMIT = 200           # Max queries per day

MONTHLY_BUDGET_USD = 50.0         # Hard budget cap

DAILY_BUDGET_WARNING = 5.0        # Alert threshold

```

### Rate Limiting:
```python

RATE_LIMIT_PER_HOUR = 20          # Queries per hour

RATE_LIMIT_PER_DAY = 200          # Queries per 24 hours

RATE_LIMIT_WARNING_THRESHOLD = 0.8  # Warn at 80%

```

### Security:
```python

MAX_INPUT_LENGTH = 2000           # Max characters

MIN_INPUT_LENGTH = 1              # Min characters

```

### Alerts:
```python

NTFY_TOPIC = ""                   # Your ntfy.sh topic

ALERTS_ENABLED = True             # Enable/disable

```

### Response Quality:
```python

CONVERSATION_HISTORY_LENGTH = 5   # Messages of context

ENHANCED_SYSTEM_PROMPT = "..."   # Full prompt in file

```

### UI/UX:
```python

SUGGESTED_QUESTIONS = [...]       # Starter questions

PRIVACY_NOTICE = "..."           # Privacy text

```

---

## 🔧 Troubleshooting

### Logs not being created:
- Check file permissions
- Ensure `logs/` directory is not in `.gitignore` for deployment
- HuggingFace Spaces may not persist logs across restarts

### Notifications not working:
- Verify `NTFY_TOPIC` is set correctly
- Test with: `curl -d "test" ntfy.sh/your-topic`
- Check you're subscribed to the right topic
- Ensure `ALERTS_ENABLED = True` in config

### Rate limits too strict/lenient:
- Adjust `RATE_LIMIT_PER_HOUR` and `RATE_LIMIT_PER_DAY` in `config.py`
- Changes take effect on app restart

### Budget exceeded too quickly:
- Review `logs/usage.jsonl` for unusual activity
- Check if there's an attack (many rapid queries)
- Adjust `MONTHLY_BUDGET_USD` if legitimate traffic

### Conversation context not working:
- Verify `CONVERSATION_HISTORY_LENGTH > 0`
- Check that messages are being stored in `st.session_state.messages`

---

## 📚 Additional Resources

- **Gemini API Pricing**: https://ai.google.dev/pricing
- **ntfy.sh Documentation**: https://ntfy.sh
- **HuggingFace Spaces**: https://huggingface.co/docs/hub/spaces
- **Streamlit Documentation**: https://docs.streamlit.io

---

## 🎯 What You Need to Do

### Required:
1. ✅ Deploy the updated code to HuggingFace Spaces
2. ✅ Set `GEMINI_API_KEY` secret in HuggingFace
3. ✅ Test with a few queries to verify it works

### Optional but Recommended:
1. 📱 Set up ntfy.sh notifications:
   - Pick a random topic name
   - Subscribe on your phone/browser
   - Set `NTFY_TOPIC` in HuggingFace secrets
   - Test it works

2. ⚙️ Adjust configuration in `config.py`:
   - Set appropriate rate limits
   - Set monthly budget
   - Customize suggested questions

3. 📊 Monitor usage:
   - Check sidebar stats regularly
   - Watch for notification alerts
   - Review logs if accessible

---

## 📞 Support

If you encounter any issues:
1. Check the troubleshooting section above
2. Review the logs (if accessible)
3. Check HuggingFace Spaces logs for errors
4. Verify environment variables are set correctly

---

**That's it!** All the production-ready features from the roadmap have been implemented. The system is now protected against cost overruns, abuse, and security threats, with monitoring and alerting in place.