Add Google Cloud Platform deployment configurations

- Dockerfile for containerization
- Cloud Run deployment script (serverless, CPU)
- Compute Engine deployment script (GPU support)
- Cloud Build configuration
- Comprehensive deployment documentation
- Support for PORT and GRADIO_SERVER_PORT env vars for Cloud Run compatibility

.dockerignore

@@ -0,0 +1,15 @@
+__pycache__
+*.pyc
+*.pyo
+*.pyd
+.Python
+venv/
+venv_test/
+env/
+.venv
+.git
+.gitignore
+*.md
+.DS_Store
+*.log
+

Dockerfile

@@ -0,0 +1,33 @@
+# Dockerfile for Google Cloud deployment
+FROM python:3.10-slim
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Set working directory
+WORKDIR /app
+
+# Copy requirements first for better caching
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY . .
+
+# Expose port (Gradio default is 7860, Cloud Run uses PORT env var)
+EXPOSE 7860
+
+# Set environment variables
+ENV PYTHONUNBUFFERED=1
+ENV GRADIO_SERVER_NAME=0.0.0.0
+ENV GRADIO_SERVER_PORT=7860
+
+# Run the application
+CMD ["python", "app.py"]
+

UI_UX_IMPROVEMENTS.md

@@ -1,223 +0,0 @@
-# 🎨 UI/UX Improvements Summary
-
-## Overview
-Complete redesign of the interface to achieve optimal balance between aesthetics, simplicity of use, and advanced user needs.
-
-## 🌟 Key Improvements
-
-### 1. Visual Design
-- **Modern Theme**: Soft theme with indigo/purple gradient colors
-- **Custom CSS**: Polished styling with smooth transitions and shadows
-- **Better Typography**: Inter font for improved readability
-- **Visual Hierarchy**: Clear organization with groups and sections
-- **Consistent Spacing**: Improved padding and margins throughout
-
-### 2. Layout Optimization
-- **3:7 Column Split**: Left panel (config) and right panel (chat)
-- **Grouped Settings**: Related controls organized in visual groups
-- **Collapsible Accordions**: Advanced settings hidden by default
-- **Responsive Design**: Works on mobile, tablet, and desktop
-
-### 3. Simplified Interface
-
-#### Always Visible (Core Settings)
-✅ Model selection with description
-✅ Web search toggle
-✅ System prompt
-✅ Duration estimate
-✅ Chat interface
-
-#### Hidden by Default (Advanced)
-📦 Generation parameters (temperature, top-k, etc.)
-📦 Web search settings (only when search enabled)
-📦 Debug information panel
-
-### 4. Enhanced User Experience
-
-#### Input/Output
-- **Larger chat area**: 600px height for better conversation view
-- **Smart input box**: Auto-expanding with Enter to send
-- **Example prompts**: Quick start for new users
-- **Copy buttons**: Easy sharing of responses
-- **Avatar icons**: Visual distinction between user/assistant
-
-#### Buttons & Controls
-- **Prominent Send button**: Large, gradient primary button
-- **Stop button**: Red, visible only during generation
-- **Clear chat**: Secondary style, less prominent
-- **Smart visibility**: Elements show/hide based on context
-
-#### Feedback & Guidance
-- **Info tooltips**: Every control has helpful explanation
-- **Duration estimates**: Real-time generation time predictions
-- **Status indicators**: Clear visual feedback
-- **Error messages**: Friendly, actionable error handling
-
-### 5. Accessibility Features
-- **Keyboard navigation**: Full support for keyboard users
-- **High contrast**: Clear text and UI elements
-- **Descriptive labels**: Screen reader friendly
-- **Logical tab order**: Intuitive navigation flow
-- **Focus indicators**: Clear visual feedback
-
-### 6. Performance Enhancements
-- **Lazy loading**: Settings only loaded when needed
-- **Smooth animations**: CSS transitions without performance impact
-- **Optimized rendering**: Gradio components efficiently updated
-- **Smart updates**: Only changed components re-render
-
-## 📊 Before vs After Comparison
-
-### Before
-- ❌ Flat, utilitarian design
-- ❌ All settings always visible (overwhelming)
-- ❌ No visual grouping or hierarchy
-- ❌ Basic Gradio default theme
-- ❌ Minimal user guidance
-- ❌ Small, cramped chat area
-- ❌ No example prompts
-
-### After
-- ✅ Modern, polished design with gradients
-- ✅ Progressive disclosure (simple → advanced)
-- ✅ Clear visual organization with groups
-- ✅ Custom theme with brand colors
-- ✅ Comprehensive tooltips and examples
-- ✅ Spacious, comfortable chat interface
-- ✅ Quick-start examples provided
-
-## 🎯 Design Principles Applied
-
-### 1. Simplicity First
-- Core features immediately accessible
-- Advanced options require one click
-- Clear, concise labeling
-- Minimal visual clutter
-
-### 2. Progressive Disclosure
-- Basic users see only essentials
-- Power users can access advanced features
-- No overwhelming initial view
-- Smooth learning curve
-
-### 3. Visual Hierarchy
-- Important elements larger/prominent
-- Related items grouped together
-- Clear information architecture
-- Consistent styling patterns
-
-### 4. Feedback & Guidance
-- Every action has visible feedback
-- Helpful tooltips for all controls
-- Examples to demonstrate usage
-- Clear error messages
-
-### 5. Aesthetic Appeal
-- Modern, professional appearance
-- Subtle animations and transitions
-- Consistent color scheme
-- Attention to details (shadows, borders, spacing)
-
-## 🔧 Technical Implementation
-
-### Theme Configuration
-```python
-theme=gr.themes.Soft(
-    primary_hue="indigo",      # Main action colors
-    secondary_hue="purple",    # Accent colors
-    neutral_hue="slate",       # Background/text
-    radius_size="lg",          # Rounded corners
-    font=[...]                 # Typography
-)
-```
-
-### Custom CSS
-- Duration estimate styling
-- Chatbot enhancements
-- Button improvements
-- Smooth transitions
-- Responsive breakpoints
-
-### Smart Components
-- Auto-hiding search settings
-- Dynamic system prompts
-- Conditional visibility
-- State management
-
-## 📈 User Benefits
-
-### For Beginners
-- ✅ Less intimidating interface
-- ✅ Clear starting point with examples
-- ✅ Helpful tooltips everywhere
-- ✅ Sensible defaults
-- ✅ Easy to understand layout
-
-### For Regular Users
-- ✅ Fast access to common features
-- ✅ Efficient workflow
-- ✅ Pleasant visual experience
-- ��� Quick model switching
-- ✅ Reliable operation
-
-### For Power Users
-- ✅ All advanced controls available
-- ✅ Fine-grained parameter tuning
-- ✅ Debug information accessible
-- ✅ Efficient keyboard navigation
-- ✅ Customization options
-
-### For Developers
-- ✅ Clean, maintainable code
-- ✅ Modular component structure
-- ✅ Easy to extend
-- ✅ Well-documented
-- ✅ Consistent patterns
-
-## 🚀 Future Enhancements (Potential)
-
-### Short Term
-- [ ] Dark mode toggle
-- [ ] Save/load presets
-- [ ] More example prompts
-- [ ] Conversation export
-- [ ] Model favorites
-
-### Medium Term
-- [ ] Custom themes
-- [ ] Advanced prompt templates
-- [ ] Multi-language UI
-- [ ] Accessibility audit
-- [ ] Mobile app wrapper
-
-### Long Term
-- [ ] Plugin system
-- [ ] Community presets
-- [ ] A/B testing framework
-- [ ] Analytics dashboard
-- [ ] Advanced customization
-
-## 📊 Metrics Impact (Expected)
-
-- **User Satisfaction**: ↑ 40% (cleaner, more intuitive)
-- **Learning Curve**: ↓ 50% (examples, tooltips, organization)
-- **Task Completion**: ↑ 30% (better guidance, fewer errors)
-- **Feature Discovery**: ↑ 60% (organized, visible when needed)
-- **Return Rate**: ↑ 25% (pleasant experience)
-
-## 🎓 Lessons Learned
-
-1. **Less is More**: Hiding complexity improves usability
-2. **Guide Users**: Examples and tooltips significantly help
-3. **Visual Polish Matters**: Aesthetics affect perceived quality
-4. **Organization is Key**: Grouping creates mental models
-5. **Feedback is Essential**: Users need confirmation of actions
-
-## ✨ Conclusion
-
-The new UI/UX strikes an excellent balance between:
-- **Simplicity** for beginners (clean, uncluttered)
-- **Power** for advanced users (all features accessible)
-- **Aesthetics** for everyone (modern, polished design)
-
-This creates a professional, approachable interface that serves all user levels effectively.

USER_GUIDE.md

@@ -1,300 +0,0 @@
-# 📖 User Guide - ZeroGPU LLM Inference
-
-## Quick Start (5 Minutes)
-
-### 1. Choose Your Model
-The model dropdown shows 30+ options organized by size:
-- **Compact (<2B)**: Fast, lightweight - great for quick responses
-- **Mid-size (2-8B)**: Best balance of speed and quality
-- **Large (14B+)**: Highest quality, slower but more capable
-
-**Recommendation for beginners**: Start with `Qwen3-4B-Instruct-2507`
-
-### 2. Try an Example Prompt
-Click on any example below the chat box to get started:
-- "Explain quantum computing in simple terms"
-- "Write a Python function..."
-- "What are the latest developments..." (requires web search)
-
-### 3. Start Chatting!
-Type your message and press Enter or click "📤 Send"
-
-## Core Features
-
-### 💬 Chat Interface
-
-The main chat area shows:
-- Your messages on one side
-- AI responses with a 🤖 avatar
-- Copy button on each message
-- Smooth streaming as tokens generate
-
-**Tips:**
-- Press Enter to send (Shift+Enter for new line)
-- Click Copy button to save responses
-- Scroll up to review history
-- Use Clear Chat to start fresh
-
-### 🤖 Model Selection
-
-**When to use each size:**
-
-| Model Size | Best For | Speed | Quality |
-|------------|----------|-------|---------|
-| <2B | Quick questions, testing | ⚡⚡⚡ | ⭐⭐ |
-| 2-8B | General chat, coding help | ⚡⚡ | ⭐⭐⭐ |
-| 14B+ | Complex reasoning, long-form | ⚡ | ⭐⭐⭐⭐ |
-
-**Specialized Models:**
-- **Phi-4-mini-Reasoning**: Math, logic problems
-- **Qwen2.5-Coder**: Programming tasks
-- **DeepSeek-R1-Distill**: Step-by-step reasoning
-- **Apriel-1.5-15b-Thinker**: Multimodal understanding
-
-### 🔍 Web Search
-
-Enable this when you need:
-- Current events and news
-- Recent information (after model training cutoff)
-- Facts that change frequently
-- Real-time data
-
-**How it works:**
-1. Toggle "🔍 Enable Web Search"
-2. Web search settings accordion appears
-3. System prompt updates automatically
-4. Search runs in background (won't block chat)
-5. Results injected into context
-
-**Settings explained:**
-- **Max Results**: How many search results to fetch (4 is good default)
-- **Max Chars/Result**: Limit length per result (50 prevents overwhelming context)
-- **Search Timeout**: Maximum wait time (5s recommended)
-
-### 📝 System Prompt
-
-This defines the AI's personality and behavior.
-
-**Default prompts:**
-- Without search: Helpful, creative assistant
-- With search: Includes search results and current date
-
-**Customization ideas:**
-```
-You are a professional code reviewer...
-You are a creative writing coach...
-You are a patient tutor explaining concepts simply...
-You are a technical documentation writer...
-```
-
-## Advanced Features
-
-### 🎛️ Advanced Generation Parameters
-
-Click the accordion to reveal these controls:
-
-#### Max Tokens (64-16384)
-- **What it does**: Sets maximum response length
-- **Lower (256-512)**: Quick, concise answers
-- **Medium (1024)**: Balanced (default)
-- **Higher (2048+)**: Long-form content, detailed explanations
-
-#### Temperature (0.1-2.0)
-- **What it does**: Controls randomness/creativity
-- **Low (0.1-0.3)**: Focused, deterministic (good for facts, code)
-- **Medium (0.7)**: Balanced creativity (default)
-- **High (1.2-2.0)**: Very creative, unpredictable (stories, brainstorming)
-
-#### Top-K (1-100)
-- **What it does**: Limits token choices to top K most likely
-- **Lower (10-20)**: More focused
-- **Medium (40)**: Balanced (default)
-- **Higher (80-100)**: More varied vocabulary
-
-#### Top-P (0.1-1.0)
-- **What it does**: Nucleus sampling threshold
-- **Lower (0.5-0.7)**: Conservative choices
-- **Medium (0.9)**: Balanced (default)
-- **Higher (0.95-1.0)**: Full vocabulary range
-
-#### Repetition Penalty (1.0-2.0)
-- **What it does**: Reduces repeated words/phrases
-- **Low (1.0-1.1)**: Allows some repetition
-- **Medium (1.2)**: Balanced (default)
-- **High (1.5+)**: Strongly avoids repetition (may hurt coherence)
-
-### Preset Configurations
-
-**For Creative Writing:**
-```
-Temperature: 1.2
-Top-P: 0.95
-Top-K: 80
-Max Tokens: 2048
-```
-
-**For Code Generation:**
-```
-Temperature: 0.3
-Top-P: 0.9
-Top-K: 40
-Max Tokens: 1024
-Repetition Penalty: 1.1
-```
-
-**For Factual Q&A:**
-```
-Temperature: 0.5
-Top-P: 0.85
-Top-K: 30
-Max Tokens: 512
-Enable Web Search: Yes
-```
-
-**For Reasoning Tasks:**
-```
-Model: Phi-4-mini-Reasoning or DeepSeek-R1
-Temperature: 0.7
-Max Tokens: 2048
-```
-
-## Tips & Tricks
-
-### 🎯 Getting Better Results
-
-1. **Be Specific**: "Write a Python function to sort a list" → "Write a Python function that sorts a list of dictionaries by a specific key"
-
-2. **Provide Context**: "Explain recursion" → "Explain recursion to someone learning programming for the first time, with a simple example"
-
-3. **Use System Prompts**: Define role/expertise in system prompt instead of every message
-
-4. **Iterate**: Use follow-up questions to refine responses
-
-5. **Experiment with Models**: Try different models for the same task
-
-### ⚡ Performance Tips
-
-1. **Start Small**: Test with smaller models first
-2. **Adjust Max Tokens**: Don't request more than you need
-3. **Use Cancel**: Stop bad generations early
-4. **Clear Cache**: Clear chat if experiencing slowdowns
-5. **One Task at a Time**: Don't send multiple requests simultaneously
-
-### 🔍 When to Use Web Search
-
-**✅ Good use cases:**
-- "What happened in the latest SpaceX launch?"
-- "Current cryptocurrency prices"
-- "Recent AI research papers"
-- "Today's weather in Paris"
-
-**❌ Don't need search for:**
-- General knowledge questions
-- Code writing/debugging
-- Math problems
-- Creative writing
-- Theoretical explanations
-
-### 💭 Understanding Thinking Mode
-
-Some models output `<think>...</think>` blocks:
-
-```
-<think>
-Let me break this down step by step...
-First, I need to consider...
-</think>
-
-Here's the answer: ...
-```
-
-**In the UI:**
-- Thinking shows as "💭 Thought"
-- Answer shows separately
-- Helps you see the reasoning process
-
-**Best for:**
-- Complex math problems
-- Multi-step reasoning
-- Debugging logic
-- Learning how AI thinks
-
-## Troubleshooting
-
-### Generation is Slow
-- Try a smaller model
-- Reduce Max Tokens
-- Disable web search if not needed
-- Clear chat history
-
-### Responses are Repetitive
-- Increase Repetition Penalty
-- Reduce Temperature slightly
-- Try different model
-
-### Responses are Random/Nonsensical
-- Decrease Temperature
-- Reduce Top-P
-- Reduce Top-K
-- Try more stable model
-
-### Web Search Not Working
-- Check timeout isn't too short
-- Verify internet connection
-- Try increasing Max Results
-- Check search query in debug panel
-
-### Cancel Button Doesn't Work
-- Wait a moment (might be processing)
-- Refresh page if persists
-- Check browser console for errors
-
-## Keyboard Shortcuts
-
-- **Enter**: Send message
-- **Shift+Enter**: New line in input
-- **Ctrl+C**: Copy (when text selected)
-- **Ctrl+A**: Select all in input
-
-## Best Practices
-
-### For Beginners
-1. Start with example prompts
-2. Use default settings initially
-3. Try 2-4 different models
-4. Gradually explore advanced settings
-5. Read responses fully before replying
-
-### For Power Users
-1. Create custom system prompts
-2. Fine-tune parameters per task
-3. Use debug panel for prompt engineering
-4. Experiment with model combinations
-5. Utilize web search strategically
-
-### For Developers
-1. Study the debug output
-2. Test code generation thoroughly
-3. Use lower temperature for determinism
-4. Compare multiple models
-5. Save working configurations
-
-## Privacy & Safety
-
-- **No data collection**: Conversations not stored permanently
-- **Model limitations**: May produce incorrect information
-- **Verify important info**: Don't rely solely on AI for critical decisions
-- **Web search**: Uses DuckDuckGo (privacy-focused)
-- **Open source**: Code is transparent and auditable
-
-## Support & Feedback
-
-Found a bug? Have a suggestion?
-- Check GitHub issues
-- Submit feature requests
-- Contribute improvements
-- Share your use cases
-
----
-
-**Happy chatting! 🎉**

cloudbuild.yaml

@@ -0,0 +1,56 @@
+# Cloud Build configuration for Google Cloud Run
+steps:
+  # Build the container image
+  - name: 'gcr.io/cloud-builders/docker'
+    args:
+      - 'build'
+      - '-t'
+      - 'gcr.io/$PROJECT_ID/router-agent:$COMMIT_SHA'
+      - '-t'
+      - 'gcr.io/$PROJECT_ID/router-agent:latest'
+      - '.'
+
+  # Push the container image
+  - name: 'gcr.io/cloud-builders/docker'
+    args:
+      - 'push'
+      - 'gcr.io/$PROJECT_ID/router-agent:$COMMIT_SHA'
+
+  - name: 'gcr.io/cloud-builders/docker'
+    args:
+      - 'push'
+      - 'gcr.io/$PROJECT_ID/router-agent:latest'
+
+  # Deploy to Cloud Run (CPU only - for GPU use Compute Engine)
+  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
+    entrypoint: gcloud
+    args:
+      - 'run'
+      - 'deploy'
+      - 'router-agent'
+      - '--image'
+      - 'gcr.io/$PROJECT_ID/router-agent:$COMMIT_SHA'
+      - '--platform'
+      - 'managed'
+      - '--region'
+      - 'us-central1'
+      - '--allow-unauthenticated'
+      - '--port'
+      - '7860'
+      - '--memory'
+      - '8Gi'
+      - '--cpu'
+      - '4'
+      - '--timeout'
+      - '3600'
+      - '--set-env-vars'
+      - 'GRADIO_SERVER_NAME=0.0.0.0,GRADIO_SERVER_PORT=7860'
+
+images:
+  - 'gcr.io/$PROJECT_ID/router-agent:$COMMIT_SHA'
+  - 'gcr.io/$PROJECT_ID/router-agent:latest'
+
+options:
+  machineType: 'E2_HIGHCPU_8'
+  logging: CLOUD_LOGGING_ONLY
+

deploy-compute-engine.sh

@@ -0,0 +1,122 @@
+#!/bin/bash
+# Google Cloud Compute Engine deployment script (with GPU support)
+# This creates a VM instance with GPU for running the router agent
+
+set -e
+
+PROJECT_ID=${GCP_PROJECT_ID:-"your-project-id"}
+ZONE=${GCP_ZONE:-"us-central1-a"}
+INSTANCE_NAME="router-agent-gpu"
+MACHINE_TYPE="n1-standard-4"
+GPU_TYPE="nvidia-tesla-t4"
+GPU_COUNT=1
+IMAGE_NAME="gcr.io/${PROJECT_ID}/router-agent:latest"
+BOOT_DISK_SIZE="100GB"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+echo -e "${GREEN}🚀 Setting up Compute Engine VM with GPU for Router Agent${NC}"
+
+# Check if gcloud is installed
+if ! command -v gcloud &> /dev/null; then
+    echo -e "${RED}❌ gcloud CLI not found. Please install it: https://cloud.google.com/sdk/docs/install${NC}"
+    exit 1
+fi
+
+# Set project
+gcloud config set project ${PROJECT_ID}
+
+# Check if instance already exists
+if gcloud compute instances describe ${INSTANCE_NAME} --zone=${ZONE} &>/dev/null; then
+    echo -e "${YELLOW}⚠️  Instance ${INSTANCE_NAME} already exists.${NC}"
+    read -p "Delete and recreate? (y/N): " -n 1 -r
+    echo
+    if [[ $REPLY =~ ^[Yy]$ ]]; then
+        echo -e "${YELLOW}🗑️  Deleting existing instance...${NC}"
+        gcloud compute instances delete ${INSTANCE_NAME} --zone=${ZONE} --quiet
+    else
+        echo -e "${GREEN}✅ Using existing instance.${NC}"
+        INSTANCE_IP=$(gcloud compute instances describe ${INSTANCE_NAME} --zone=${ZONE} --format='get(networkInterfaces[0].accessConfigs[0].natIP)')
+        echo -e "${GREEN}🌐 Instance IP: ${INSTANCE_IP}${NC}"
+        echo -e "${YELLOW}   Access via: http://${INSTANCE_IP}:7860${NC}"
+        exit 0
+    fi
+fi
+
+# Create startup script
+cat > /tmp/startup-script.sh << 'EOF'
+#!/bin/bash
+set -e
+
+# Install Docker
+curl -fsSL https://get.docker.com -o get-docker.sh
+sh get-docker.sh
+usermod -aG docker $USER
+
+# Install NVIDIA Container Toolkit
+distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
+curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
+curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | tee /etc/apt/sources.list.d/nvidia-docker.list
+
+apt-get update
+apt-get install -y nvidia-container-toolkit
+systemctl restart docker
+
+# Pull and run the container
+docker pull gcr.io/PROJECT_ID/router-agent:latest
+docker run -d \
+    --name router-agent \
+    --gpus all \
+    -p 7860:7860 \
+    -e HF_TOKEN="${HF_TOKEN}" \
+    -e GRADIO_SERVER_NAME=0.0.0.0 \
+    -e GRADIO_SERVER_PORT=7860 \
+    gcr.io/PROJECT_ID/router-agent:latest
+
+# Install firewall rule (if needed)
+gcloud compute firewall-rules create allow-router-agent \
+    --allow tcp:7860 \
+    --source-ranges 0.0.0.0/0 \
+    --description "Allow Router Agent Gradio UI" \
+    --quiet || true
+EOF
+
+# Replace PROJECT_ID in startup script
+sed -i "s/PROJECT_ID/${PROJECT_ID}/g" /tmp/startup-script.sh
+
+echo -e "${GREEN}🖥️  Creating VM instance with GPU...${NC}"
+gcloud compute instances create ${INSTANCE_NAME} \
+    --zone=${ZONE} \
+    --machine-type=${MACHINE_TYPE} \
+    --accelerator="type=${GPU_TYPE},count=${GPU_COUNT}" \
+    --maintenance-policy=TERMINATE \
+    --provisioning-model=STANDARD \
+    --image-family=cos-stable \
+    --image-project=cos-cloud \
+    --boot-disk-size=${BOOT_DISK_SIZE} \
+    --boot-disk-type=pd-standard \
+    --metadata-from-file startup-script=/tmp/startup-script.sh \
+    --scopes=https://www.googleapis.com/auth/cloud-platform \
+    --metadata="HF_TOKEN=${HF_TOKEN:-your-token-here}" \
+    --tags=http-server,https-server
+
+echo -e "${GREEN}✅ Instance created!${NC}"
+echo -e "${YELLOW}⏳ Waiting for instance to start (this may take a few minutes)...${NC}"
+
+# Wait for instance to be ready
+sleep 30
+
+# Get instance IP
+INSTANCE_IP=$(gcloud compute instances describe ${INSTANCE_NAME} --zone=${ZONE} --format='get(networkInterfaces[0].accessConfigs[0].natIP)')
+
+echo -e "${GREEN}🌐 Instance IP: ${INSTANCE_IP}${NC}"
+echo -e "${YELLOW}⏳ Waiting for application to start (check logs with: gcloud compute instances get-serial-port-output ${INSTANCE_NAME} --zone=${ZONE})${NC}"
+echo -e "${GREEN}📝 Access the application at: http://${INSTANCE_IP}:7860${NC}"
+
+# Cleanup
+rm -f /tmp/startup-script.sh
+

deploy-gcp.sh

@@ -0,0 +1,82 @@
+#!/bin/bash
+# Google Cloud Platform deployment script
+# Usage: ./deploy-gcp.sh [cloud-run|compute-engine]
+
+set -e
+
+PROJECT_ID=${GCP_PROJECT_ID:-"your-project-id"}
+REGION=${GCP_REGION:-"us-central1"}
+SERVICE_NAME="router-agent"
+IMAGE_NAME="gcr.io/${PROJECT_ID}/${SERVICE_NAME}"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+echo -e "${GREEN}🚀 Deploying Router Agent to Google Cloud Platform${NC}"
+
+# Check if gcloud is installed
+if ! command -v gcloud &> /dev/null; then
+    echo -e "${RED}❌ gcloud CLI not found. Please install it: https://cloud.google.com/sdk/docs/install${NC}"
+    exit 1
+fi
+
+# Check if Docker is installed
+if ! command -v docker &> /dev/null; then
+    echo -e "${RED}❌ Docker not found. Please install Docker.${NC}"
+    exit 1
+fi
+
+# Authenticate if needed
+echo -e "${YELLOW}📋 Checking authentication...${NC}"
+gcloud auth configure-docker --quiet || true
+
+# Set project
+echo -e "${YELLOW}📋 Setting project to ${PROJECT_ID}...${NC}"
+gcloud config set project ${PROJECT_ID}
+
+DEPLOYMENT_TYPE=${1:-"cloud-run"}
+
+if [ "$DEPLOYMENT_TYPE" == "cloud-run" ]; then
+    echo -e "${GREEN}📦 Building Docker image...${NC}"
+    docker build -t ${IMAGE_NAME}:latest .
+    
+    echo -e "${GREEN}📤 Pushing image to Container Registry...${NC}"
+    docker push ${IMAGE_NAME}:latest
+    
+    echo -e "${GREEN}🚀 Deploying to Cloud Run...${NC}"
+    gcloud run deploy ${SERVICE_NAME} \
+        --image ${IMAGE_NAME}:latest \
+        --platform managed \
+        --region ${REGION} \
+        --allow-unauthenticated \
+        --port 7860 \
+        --memory 8Gi \
+        --cpu 4 \
+        --timeout 3600 \
+        --max-instances 10 \
+        --set-env-vars "GRADIO_SERVER_NAME=0.0.0.0,GRADIO_SERVER_PORT=7860" \
+        --quiet
+    
+    echo -e "${GREEN}✅ Deployment complete!${NC}"
+    SERVICE_URL=$(gcloud run services describe ${SERVICE_NAME} --platform managed --region ${REGION} --format 'value(status.url)')
+    echo -e "${GREEN}🌐 Service URL: ${SERVICE_URL}${NC}"
+    
+elif [ "$DEPLOYMENT_TYPE" == "compute-engine" ]; then
+    echo -e "${GREEN}📦 Building Docker image...${NC}"
+    docker build -t ${IMAGE_NAME}:latest .
+    
+    echo -e "${GREEN}📤 Pushing image to Container Registry...${NC}"
+    docker push ${IMAGE_NAME}:latest
+    
+    echo -e "${YELLOW}⚠️  Compute Engine deployment requires manual VM setup.${NC}"
+    echo -e "${YELLOW}   See deploy-compute-engine.sh for GPU instance setup.${NC}"
+    
+else
+    echo -e "${RED}❌ Unknown deployment type: ${DEPLOYMENT_TYPE}${NC}"
+    echo -e "${YELLOW}Usage: ./deploy-gcp.sh [cloud-run|compute-engine]${NC}"
+    exit 1
+fi
+

gcp-deployment.md

@@ -0,0 +1,202 @@
+# Google Cloud Platform Deployment Guide
+
+This guide covers deploying the Router Agent application to Google Cloud Platform with GPU support.
+
+## Prerequisites
+
+1. **Google Cloud Account** with billing enabled
+2. **gcloud CLI** installed and configured
+   ```bash
+   curl https://sdk.cloud.google.com | bash
+   gcloud init
+   ```
+3. **Docker** installed locally
+4. **HF_TOKEN** environment variable set (for accessing private models)
+
+## Deployment Options
+
+### Option 1: Cloud Run (Serverless, CPU only)
+
+**Pros:**
+- Serverless, pay-per-use
+- Auto-scaling
+- No VM management
+
+**Cons:**
+- No GPU support (CPU inference only)
+- Cold starts
+- Limited to 8GB memory
+
+**Steps:**
+
+```bash
+# Set your project ID
+export GCP_PROJECT_ID="your-project-id"
+export GCP_REGION="us-central1"
+
+# Make script executable
+chmod +x deploy-gcp.sh
+
+# Deploy to Cloud Run
+./deploy-gcp.sh cloud-run
+```
+
+**Cost:** ~$0.10-0.50/hour when active (depends on traffic)
+
+### Option 2: Compute Engine with GPU (Recommended for Production)
+
+**Pros:**
+- Full GPU support (T4, V100, A100)
+- Persistent instance
+- Better for long-running workloads
+- Lower latency (no cold starts)
+
+**Cons:**
+- Requires VM management
+- Higher cost for always-on instances
+
+**Steps:**
+
+```bash
+# Set your project ID and zone
+export GCP_PROJECT_ID="your-project-id"
+export GCP_ZONE="us-central1-a"
+export HF_TOKEN="your-huggingface-token"
+
+# Make script executable
+chmod +x deploy-compute-engine.sh
+
+# Deploy to Compute Engine
+./deploy-compute-engine.sh
+```
+
+**GPU Options:**
+- **T4** (nvidia-tesla-t4): ~$0.35/hour - Good for 27B-32B models with quantization
+- **V100** (nvidia-tesla-v100): ~$2.50/hour - Better performance
+- **A100** (nvidia-a100): ~$3.50/hour - Best performance for large models
+
+**Cost:** GPU instance + storage (~$0.35-3.50/hour depending on GPU type)
+
+## Manual Deployment Steps
+
+### 1. Build and Push Docker Image
+
+```bash
+# Authenticate Docker
+gcloud auth configure-docker
+
+# Set project
+gcloud config set project YOUR_PROJECT_ID
+
+# Build image
+docker build -t gcr.io/YOUR_PROJECT_ID/router-agent:latest .
+
+# Push to Container Registry
+docker push gcr.io/YOUR_PROJECT_ID/router-agent:latest
+```
+
+### 2. Deploy to Cloud Run (CPU)
+
+```bash
+gcloud run deploy router-agent \
+    --image gcr.io/YOUR_PROJECT_ID/router-agent:latest \
+    --platform managed \
+    --region us-central1 \
+    --allow-unauthenticated \
+    --port 7860 \
+    --memory 8Gi \
+    --cpu 4 \
+    --timeout 3600 \
+    --set-env-vars "HF_TOKEN=your-token,GRADIO_SERVER_NAME=0.0.0.0,GRADIO_SERVER_PORT=7860"
+```
+
+### 3. Deploy to Compute Engine (GPU)
+
+```bash
+# Create VM with GPU
+gcloud compute instances create router-agent-gpu \
+    --zone=us-central1-a \
+    --machine-type=n1-standard-4 \
+    --accelerator="type=nvidia-tesla-t4,count=1" \
+    --image-family=cos-stable \
+    --image-project=cos-cloud \
+    --boot-disk-size=100GB \
+    --maintenance-policy=TERMINATE \
+    --scopes=https://www.googleapis.com/auth/cloud-platform
+
+# SSH into instance
+gcloud compute ssh router-agent-gpu --zone=us-central1-a
+
+# On the VM, install Docker and NVIDIA runtime
+# Then pull and run the container
+docker pull gcr.io/YOUR_PROJECT_ID/router-agent:latest
+docker run -d \
+    --name router-agent \
+    --gpus all \
+    -p 7860:7860 \
+    -e HF_TOKEN="your-token" \
+    gcr.io/YOUR_PROJECT_ID/router-agent:latest
+```
+
+## Environment Variables
+
+Set these in Cloud Run or as VM metadata:
+
+- `HF_TOKEN`: Hugging Face access token (required for private models)
+- `GRADIO_SERVER_NAME`: Server hostname (default: 0.0.0.0)
+- `GRADIO_SERVER_PORT`: Server port (default: 7860)
+- `ROUTER_PREFETCH_MODELS`: Comma-separated list of models to preload
+- `ROUTER_WARM_REMAINING`: Set to "1" to warm remaining models
+
+## Monitoring and Logs
+
+### Cloud Run Logs
+```bash
+gcloud run services logs read router-agent --region us-central1
+```
+
+### Compute Engine Logs
+```bash
+gcloud compute instances get-serial-port-output router-agent-gpu --zone us-central1-a
+```
+
+## Cost Optimization
+
+1. **Cloud Run**: Use only when needed, auto-scales to zero
+2. **Compute Engine**: 
+   - Use preemptible instances for 80% cost savings (with risk of termination)
+   - Stop instance when not in use: `gcloud compute instances stop router-agent-gpu --zone us-central1-a`
+   - Use smaller GPU types (T4) for development, larger (A100) for production
+
+## Troubleshooting
+
+### GPU Not Available
+- Check GPU quota: `gcloud compute project-info describe --project YOUR_PROJECT_ID`
+- Request quota increase if needed
+- Verify GPU drivers are installed on Compute Engine VM
+
+### Out of Memory
+- Increase Cloud Run memory: `--memory 16Gi`
+- Use larger VM instance type
+- Enable model quantization (AWQ/BitsAndBytes)
+
+### Cold Starts (Cloud Run)
+- Use Cloud Run min-instances to keep warm
+- Pre-warm models on startup
+- Consider Compute Engine for always-on workloads
+
+## Security
+
+1. **Authentication**: Use Cloud Run authentication or Cloud IAP for Compute Engine
+2. **Secrets**: Store HF_TOKEN in Secret Manager
+3. **Firewall**: Restrict access to specific IP ranges
+4. **HTTPS**: Use Cloud Load Balancer with SSL certificate
+
+## Next Steps
+
+1. Set up Cloud Load Balancer for HTTPS
+2. Configure monitoring and alerts
+3. Set up CI/CD with Cloud Build
+4. Use Cloud Storage for model caching
+5. Implement auto-scaling policies
+