Refactor project for Lung Cancer AI Advisor: update app and API descriptions, modify .gitignore to exclude Jupyter notebooks, and remove outdated deployment documentation. Delete unused files and enhance logging for better traceability.

.gitignore

@@ -207,6 +207,6 @@ marimo/_static/
 marimo/_lsp/
 __marimo__/
 
-
+*.ipynb
 Lung Cancer Guidelines/
-#frontend/frontend/ 
+

0.0.18

@@ -1,2 +0,0 @@
-Defaulting to user installation because normal site-packages is not writeable
-Requirement already satisfied: python-multipart in c:\users\moaze\appdata\roaming\python\python313\site-packages (0.0.9)

DEPLOYMENT.md

@@ -1,206 +0,0 @@
-# Hugging Face Deployment Guide
-
-## Overview
-This guide explains how to deploy the Lung Cancer Clinical Decision Support System to Hugging Face Spaces.
-
-## Prerequisites
-- Hugging Face account
-- Git installed locally
-- OpenAI API key (for the agent)
-- GitHub Personal Access Token (for side effects storage)
-
-## Deployment Steps
-
-### 1. Create a New Hugging Face Space
-
-1. Go to [Hugging Face Spaces](https://huggingface.co/spaces)
-2. Click "Create new Space"
-3. Configure:
-   - **Space name**: `moazx-api` (or your preferred name)
-   - **License**: Choose appropriate license
-   - **SDK**: Docker
-   - **Hardware**: CPU Basic (or upgrade as needed)
-
-### 2. Configure Environment Variables
-
-In your Hugging Face Space settings, add these secrets:
-
-```bash
-OPENAI_API_KEY=your_openai_api_key_here
-GITHUB_TOKEN=your_github_token_here
-GITHUB_REPO=your_username/your_repo_name
-GITHUB_BRANCH=main
-PORT=7860
-```
-
-### 3. Deploy the Application
-
-#### Option A: Direct Push to Hugging Face
-
-```bash
-# Clone your Hugging Face Space repository
-git clone https://huggingface.co/spaces/YOUR_USERNAME/moazx-api
-cd moazx-api
-
-# Copy all backend files
-cp -r /path/to/backend/* .
-
-# Add and commit
-git add .
-git commit -m "Initial deployment"
-git push
-```
-
-#### Option B: Using Hugging Face CLI
-
-```bash
-# Install Hugging Face CLI
-pip install huggingface_hub
-
-# Login
-huggingface-cli login
-
-# Push to Space
-huggingface-cli upload YOUR_USERNAME/moazx-api . --repo-type=space
-```
-
-### 4. Verify Deployment
-
-1. Wait for the Space to build (check the logs)
-2. Once running, test the API:
-   - Visit: `https://YOUR_USERNAME-moazx-api.hf.space`
-   - Check health: `https://YOUR_USERNAME-moazx-api.hf.space/health`
-   - View docs: `https://YOUR_USERNAME-moazx-api.hf.space/docs`
-
-### 5. Deploy Frontend
-
-The frontend is configured to use the API at `https://moazx-api.hf.space`.
-
-#### Option A: Serve from the same Space
-The frontend files are already in the `/frontend` directory and will be served automatically.
-
-#### Option B: Deploy to separate hosting
-Deploy the frontend folder to:
-- Netlify
-- Vercel
-- GitHub Pages
-- Any static hosting service
-
-## API Endpoints
-
-Once deployed, your API will be available at:
-
-```
-Base URL: https://moazx-api.hf.space
-
-Endpoints:
-- GET  /                          - API information
-- GET  /health                    - Health check
-- GET  /health/initialization     - Initialization status
-- POST /auth/login                - User login
-- POST /auth/logout               - User logout
-- GET  /auth/status               - Authentication status
-- GET  /ask                       - Ask a question (non-streaming)
-- GET  /ask/stream                - Ask a question (streaming)
-- GET  /export/{format}           - Export conversation
-```
-
-## Frontend Configuration
-
-The frontend is already configured to use the Hugging Face API:
-
-```javascript
-// In frontend/script.js
-this.apiBase = 'https://moazx-api.hf.space';
-```
-
-## Authentication
-
-The system uses session-based authentication:
-
-1. Default credentials (change in production):
-   - Username: `admin`
-   - Password: `admin123`
-
-2. To change credentials, update `api/routers/auth.py`
-
-## Monitoring
-
-Monitor your deployment:
-
-1. **Hugging Face Space Logs**: Check the logs tab in your Space
-2. **API Health**: Monitor `/health` endpoint
-3. **Initialization Status**: Check `/health/initialization`
-
-## Troubleshooting
-
-### Issue: Space fails to build
-- Check Dockerfile syntax
-- Verify all dependencies in requirements.txt
-- Check Space logs for specific errors
-
-### Issue: API returns 500 errors
-- Verify environment variables are set correctly
-- Check that OPENAI_API_KEY is valid
-- Review application logs
-
-### Issue: CORS errors in frontend
-- Verify CORS middleware configuration in `api/middleware.py`
-- Ensure frontend URL is in allowed origins
-
-### Issue: Slow initialization
-- The system loads models in the background
-- Check `/health/initialization` for status
-- Consider upgrading to better hardware tier
-
-## Performance Optimization
-
-### For Better Performance:
-1. Upgrade to GPU hardware tier (for faster embeddings)
-2. Use persistent storage for cached data
-3. Enable CDN for frontend assets
-
-### Memory Management:
-- Current setup uses CPU-optimized models
-- Faiss-cpu for vector search
-- Sentence-transformers for embeddings
-
-## Security Considerations
-
-1. **Change default credentials** in production
-2. **Rotate API keys** regularly
-3. **Enable rate limiting** (already configured)
-4. **Use HTTPS** (automatic on Hugging Face)
-5. **Review CORS settings** for production
-
-## Updating the Deployment
-
-To update your deployment:
-
-```bash
-# Make changes locally
-git add .
-git commit -m "Update description"
-git push
-
-# Hugging Face will automatically rebuild
-```
-
-## Cost Considerations
-
-- **Free tier**: CPU Basic (limited resources)
-- **Paid tiers**: Better performance and reliability
-- **API costs**: OpenAI API usage (pay per token)
-
-## Support
-
-For issues:
-1. Check Hugging Face Space logs
-2. Review application logs at `/logs/app.log`
-3. Test endpoints using `/docs` (Swagger UI)
-
-## Additional Resources
-
-- [Hugging Face Spaces Documentation](https://huggingface.co/docs/hub/spaces)
-- [FastAPI Documentation](https://fastapi.tiangolo.com/)
-- [Docker Documentation](https://docs.docker.com/)

DEPLOYMENT_SUMMARY.md

@@ -1,296 +0,0 @@
-# Deployment Summary - Hugging Face Integration
-
-## Changes Made for Hugging Face Deployment
-
-### 1. Frontend Configuration (`frontend/script.js`)
-**Changed:**
-- Updated API base URL from `http://127.0.0.1:8000` to `https://moazx-api.hf.space`
-
-**Impact:**
-- Frontend now connects to the deployed Hugging Face Space API
-- Works seamlessly with the production backend
-
-### 2. Backend Configuration (`app.py`)
-**Changed:**
-- Updated host from `127.0.0.1` to `0.0.0.0` (bind to all interfaces)
-- Updated port to use environment variable `PORT` (default: 7860)
-- Disabled reload for production
-- Configured for single worker deployment
-
-**Impact:**
-- Backend now accepts connections from external sources
-- Compatible with Hugging Face Spaces port configuration
-- Optimized for production deployment
-
-### 3. CORS Middleware (`api/middleware.py`)
-**Already Configured:**
-- CORS middleware already includes `https://moazx-api.hf.space`
-- Supports multiple origins for development and production
-- Allows credentials for authentication
-
-**No changes needed** - already production-ready!
-
-### 4. Docker Configuration (`Dockerfile`)
-**Already Configured:**
-- Multi-stage build for optimized image size
-- Exposes port 7860 (Hugging Face standard)
-- Runs as non-root user for security
-- Uses Python 3.11-slim for minimal footprint
-
-**No changes needed** - already production-ready!
-
-### 5. Environment Variables (`.env.example`)
-**Updated:**
-- Added comprehensive documentation for all environment variables
-- Included GitHub storage configuration
-- Added server configuration (PORT, HOST)
-- Added CORS configuration
-- Documented authentication credentials
-
-**Action Required:**
-- Copy `.env.example` to `.env` and fill in your actual values
-- Set these as secrets in Hugging Face Space settings
-
-### 6. Documentation
-**Created/Updated:**
-- `DEPLOYMENT.md` - Comprehensive deployment guide
-- `README.md` - Updated with full feature list and usage instructions
-- `.env.example` - Complete environment variable documentation
-
-## Deployment Checklist
-
-### ✅ Code Changes Complete
-- [x] Frontend API endpoint updated
-- [x] Backend configured for production
-- [x] CORS properly configured
-- [x] Docker configuration verified
-- [x] Environment variables documented
-
-### 📋 Next Steps for Deployment
-
-1. **Prepare Hugging Face Space**
-   ```bash
-   # Create a new Space on Hugging Face
-   # Name: moazx-api
-   # SDK: Docker
-   # Hardware: CPU Basic (or better)
-   ```
-
-2. **Set Environment Variables in Hugging Face**
-   Go to Space Settings → Variables and Secrets:
-   ```
-   OPENAI_API_KEY=your_actual_key
-   GITHUB_TOKEN=your_github_token
-   GITHUB_REPO=username/repo
-   GITHUB_BRANCH=main
-   PORT=7860
-   ```
-
-3. **Deploy Code to Hugging Face**
-   ```bash
-   # Clone your HF Space
-   git clone https://huggingface.co/spaces/YOUR_USERNAME/moazx-api
-   cd moazx-api
-   
-   # Copy all backend files
-   cp -r /path/to/backend/* .
-   
-   # Commit and push
-   git add .
-   git commit -m "Initial deployment"
-   git push
-   ```
-
-4. **Verify Deployment**
-   - Wait for build to complete (check logs)
-   - Test health endpoint: `https://moazx-api.hf.space/health`
-   - Test API docs: `https://moazx-api.hf.space/docs`
-   - Test frontend by opening `frontend/index.html`
-
-5. **Test Functionality**
-   - Login with credentials (admin/admin123)
-   - Ask a test question
-   - Verify citations are working
-   - Test export functionality
-   - Check streaming responses
-
-## File Structure for Deployment
-
-```
-backend/
-├── api/
-│   ├── __init__.py
-│   ├── app.py                 # Main FastAPI application
-│   ├── middleware.py          # CORS, auth, rate limiting
-│   ├── exceptions.py
-│   ├── models.py
-│   └── routers/
-│       ├── medical.py         # Medical query endpoints
-│       ├── health.py          # Health check endpoints
-│       ├── export.py          # Export endpoints
-│       └── auth.py            # Authentication endpoints
-├── core/
-│   ├── agent.py              # LangChain agent configuration ⭐
-│   ├── tools.py              # Agent tools
-│   ├── retrievers.py         # Hybrid search
-│   ├── context_enrichment.py # Context page enrichment
-│   ├── vector_store.py       # FAISS vector store
-│   └── ...
-├── frontend/
-│   ├── index.html            # Main UI
-│   ├── script.js             # Frontend logic ⭐ (updated)
-│   ├── styles.css            # Styling
-│   └── login.html            # Login page
-├── data/
-│   ├── chunks.pkl            # Preprocessed document chunks
-│   └── medical_terms_cache.json
-├── Dockerfile                # Docker configuration
-├── requirements.txt          # Python dependencies
-├── app.py                    # Entry point ⭐ (updated)
-├── README.md                 # Documentation ⭐ (updated)
-├── DEPLOYMENT.md             # Deployment guide ⭐ (new)
-├── .env.example              # Environment variables ⭐ (updated)
-└── .gitignore
-
-⭐ = Files modified/created for deployment
-```
-
-## Configuration Summary
-
-### API Endpoint
-- **Production**: `https://moazx-api.hf.space`
-- **Local Dev**: `http://localhost:7860`
-
-### Authentication
-- **Default Username**: `admin`
-- **Default Password**: `admin123`
-- **⚠️ Change in production!**
-
-### Required Environment Variables
-```bash
-OPENAI_API_KEY=required
-GITHUB_TOKEN=optional (for side effects)
-GITHUB_REPO=optional
-PORT=7860
-```
-
-### Optional Environment Variables
-```bash
-LANGSMITH_API_KEY=optional (for tracing)
-ALLOWED_ORIGINS=optional (auto-configured)
-AUTH_USERNAME=optional (defaults to admin)
-AUTH_PASSWORD=optional (defaults to admin123)
-```
-
-## Testing the Deployment
-
-### 1. Health Check
-```bash
-curl https://moazx-api.hf.space/health
-```
-
-Expected response:
-```json
-{
-  "status": "healthy",
-  "timestamp": "2025-01-22T...",
-  "version": "1.0.0"
-}
-```
-
-### 2. API Documentation
-Visit: `https://moazx-api.hf.space/docs`
-
-### 3. Test Query (with authentication)
-```bash
-# Login first
-curl -X POST https://moazx-api.hf.space/auth/login \
-  -H "Content-Type: application/json" \
-  -d '{"username":"admin","password":"admin123"}' \
-  -c cookies.txt
-
-# Ask a question
-curl -X GET "https://moazx-api.hf.space/ask?query=What%20is%20EGFR%20mutation&session_id=test123" \
-  -b cookies.txt
-```
-
-## Troubleshooting
-
-### Issue: Build fails on Hugging Face
-- Check Dockerfile syntax
-- Verify requirements.txt has all dependencies
-- Check Space logs for specific errors
-
-### Issue: API returns 500 errors
-- Verify OPENAI_API_KEY is set correctly
-- Check application logs in Space
-- Verify data files (chunks.pkl) are present
-
-### Issue: Frontend can't connect
-- Verify CORS settings in middleware.py
-- Check that frontend is using correct API URL
-- Test API endpoint directly first
-
-### Issue: Authentication fails
-- Verify credentials in auth.py
-- Check cookie settings
-- Ensure HTTPS is being used
-
-## Performance Considerations
-
-### Current Setup
-- **CPU-optimized**: Uses faiss-cpu and CPU-only PyTorch
-- **Memory**: ~2-4GB RAM usage
-- **Startup time**: 30-60 seconds (background initialization)
-
-### Optimization Options
-1. **Upgrade to GPU tier** - Faster embeddings and inference
-2. **Enable caching** - Cache frequently accessed documents
-3. **Optimize chunk size** - Reduce memory footprint
-4. **Use persistent storage** - Store vector index on disk
-
-## Security Checklist
-
-- [x] HTTPS enabled (automatic on Hugging Face)
-- [x] Session-based authentication implemented
-- [x] Rate limiting configured (100 req/min)
-- [x] CORS properly configured
-- [x] Input validation in place
-- [ ] Change default credentials (TODO in production)
-- [ ] Rotate API keys regularly (TODO)
-- [ ] Enable monitoring/logging (TODO)
-
-## Monitoring
-
-### Key Metrics to Monitor
-1. **API Response Time**: Check X-Process-Time header
-2. **Error Rate**: Monitor 500 errors in logs
-3. **Initialization Status**: `/health/initialization` endpoint
-4. **OpenAI API Usage**: Monitor token consumption
-
-### Logs Location
-- Hugging Face Space logs tab
-- Application logs: `/logs/app.log`
-
-## Next Steps After Deployment
-
-1. **Test thoroughly** with real clinical questions
-2. **Monitor performance** and optimize as needed
-3. **Update documentation** with actual deployment URL
-4. **Set up monitoring** and alerts
-5. **Plan for scaling** if usage increases
-6. **Regular updates** to medical guidelines
-7. **Security audit** and credential rotation
-
-## Support Resources
-
-- **Deployment Guide**: See `DEPLOYMENT.md`
-- **API Documentation**: Visit `/docs` on deployed Space
-- **Hugging Face Docs**: https://huggingface.co/docs/hub/spaces
-- **FastAPI Docs**: https://fastapi.tiangolo.com/
-
----
-
-**Deployment Status**: ✅ Ready for Deployment
-
-All code changes are complete. Follow the deployment checklist to deploy to Hugging Face Spaces.

QUICK_DEPLOY.md

@@ -1,100 +0,0 @@
-# Quick Deployment Guide - Hugging Face
-
-## 🚀 Deploy in 5 Steps
-
-### Step 1: Create Hugging Face Space
-1. Go to https://huggingface.co/spaces
-2. Click "Create new Space"
-3. Settings:
-   - Name: `moazx-api`
-   - SDK: **Docker**
-   - Hardware: CPU Basic (minimum)
-
-### Step 2: Set Environment Variables
-In Space Settings → Secrets, add:
-```
-OPENAI_API_KEY=sk-...your-key...
-GITHUB_TOKEN=ghp_...your-token...
-GITHUB_REPO=username/repo-name
-GITHUB_BRANCH=main
-PORT=7860
-```
-
-### Step 3: Push Code
-```bash
-# Clone your Space
-git clone https://huggingface.co/spaces/YOUR_USERNAME/moazx-api
-cd moazx-api
-
-# Copy all files from backend folder
-cp -r /path/to/backend/* .
-
-# Commit and push
-git add .
-git commit -m "Deploy Lung Cancer Clinical Decision Support System"
-git push
-```
-
-### Step 4: Wait for Build
-- Watch the build logs in your Space
-- Wait for "Running" status (30-60 seconds)
-
-### Step 5: Test
-```bash
-# Test health endpoint
-curl https://YOUR_USERNAME-moazx-api.hf.space/health
-
-# Visit API docs
-open https://YOUR_USERNAME-moazx-api.hf.space/docs
-```
-
-## ✅ Verification Checklist
-
-- [ ] Space is running (green status)
-- [ ] `/health` returns `{"status": "healthy"}`
-- [ ] `/docs` shows API documentation
-- [ ] Can login with admin/admin123
-- [ ] Can ask a test question
-- [ ] Streaming responses work
-- [ ] Citations appear in answers
-
-## 🔧 Quick Fixes
-
-### Build Failed?
-- Check Dockerfile syntax
-- Verify all files are committed
-- Check Space logs for errors
-
-### API Not Responding?
-- Verify OPENAI_API_KEY is set
-- Check Space logs
-- Restart the Space
-
-### Frontend Can't Connect?
-- Update `frontend/script.js` with your Space URL:
-  ```javascript
-  this.apiBase = 'https://YOUR_USERNAME-moazx-api.hf.space';
-  ```
-
-## 📱 Access Your Deployment
-
-- **API**: `https://YOUR_USERNAME-moazx-api.hf.space`
-- **Docs**: `https://YOUR_USERNAME-moazx-api.hf.space/docs`
-- **Health**: `https://YOUR_USERNAME-moazx-api.hf.space/health`
-
-## 🔐 Default Credentials
-
-- Username: `admin`
-- Password: `admin123`
-
-**⚠️ Change these in production!**
-
-## 📚 Full Documentation
-
-- Detailed guide: `DEPLOYMENT.md`
-- Complete summary: `DEPLOYMENT_SUMMARY.md`
-- README: `README.md`
-
----
-
-**Need Help?** Check the full deployment guide in `DEPLOYMENT.md`

README.md

@@ -33,22 +33,16 @@ A specialized AI-powered clinical decision support system for thoracic oncologis
 ## 🚀 Deployment
 
 ### Live API
-The API is deployed at: **https://moazx-api.hf.space**
+The API is deployed at: **https://moazx-lung-cancer-ai-advisor.hf.space**
 
 ### Quick Start
 
 1. **Access the API**:
-   - API Docs: https://moazx-api.hf.space/docs
-   - Health Check: https://moazx-api.hf.space/health
+   - API Docs: https://moazx-lung-cancer-ai-advisor.hf.space/docs
+   - Health Check: https://moazx-lung-cancer-ai-advisor.hf.space/health
 
-2. **Use the Frontend**:
-   - Open `frontend/index.html` in a browser
-   - Login with credentials (default: admin/admin123)
-   - Start asking clinical questions
 
-### Deploy Your Own Instance
 
-See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed deployment instructions.
 
 ## 📚 API Endpoints
 
@@ -61,10 +55,22 @@ See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed deployment instructions.
 - `POST /auth/login` - User login
 - `POST /auth/logout` - User logout
 - `GET /auth/status` - Check authentication status
-
+up
 ### Medical Queries
-- `GET /ask?query={question}&session_id={id}` - Ask a question (non-streaming)
-- `GET /ask/stream?query={question}&session_id={id}` - Ask a question (streaming)
+- `POST /ask` - Ask a question (complete response)
+  ```json
+  {
+    "query": "What are the early symptoms of lung cancer?",
+    "session_id": "user_123_session_1699612345"
+  }
+  ```
+- `POST /ask/stream` - Ask a question (streaming response)
+  ```json
+  {
+    "query": "What are the treatment options?",
+    "session_id": "user_123_session_1699612345"
+  }
+  ```
 
 ### Export
 - `GET /export/{format}?session_id={id}` - Export conversation (format: pdf, docx, txt)
@@ -116,46 +122,6 @@ See `.env.example` for all configuration options:
 - `PORT`: Server port (default: 7860)
 - `ALLOWED_ORIGINS`: CORS allowed origins
 
-### Authentication
-
-Default credentials (change in production):
-- Username: `admin`
-- Password: `admin123`
-
-Update in `api/routers/auth.py` or via environment variables.
-
-## 📖 Usage Examples
-
-### Using the API
-
-```python
-import requests
-
-# Login
-response = requests.post(
-    "https://moazx-api.hf.space/auth/login",
-    json={"username": "admin", "password": "admin123"}
-)
-cookies = response.cookies
-
-# Ask a question
-response = requests.get(
-    "https://moazx-api.hf.space/ask",
-    params={
-        "query": "What is the first-line treatment for EGFR-mutated NSCLC?",
-        "session_id": "my-session-123"
-    },
-    cookies=cookies
-)
-print(response.json()["response"])
-```
-
-### Using the Frontend
-
-1. Open `frontend/index.html`
-2. Login with credentials
-3. Type your clinical question
-4. Receive evidence-based answers with citations
 
 ## 🏗️ Architecture
 
@@ -178,10 +144,11 @@ print(response.json()["response"])
 ## 📊 Response Format
 
 The agent provides:
-- **Concise, targeted answers** for busy clinicians
+- **Concise, evidence-based answers** for busy clinicians
 - **Inline citations** after each statement
 - **Comprehensive reference list** at the end
-- **Structured formatting** for easy scanning
+- **Structured markdown formatting** for easy scanning
+- **Real-time streaming** for immediate feedback
 
 Example:
 ```
@@ -194,35 +161,3 @@ Example:
 **References:**
 (Source: NCCN.pdf, Pages: 45, 46, Provider: NCCN, Location: NSCLC Treatment Algorithm)
 ```
-
-## 🔒 Security
-
-- Session-based authentication
-- Rate limiting (100 requests/minute)
-- CORS protection
-- Input validation
-- Secure cookie handling
-
-## 📝 License
-
-[Add your license here]
-
-## 🤝 Contributing
-
-Contributions are welcome! Please read the contributing guidelines first.
-
-## 📧 Support
-
-For issues or questions:
-- Check the [DEPLOYMENT.md](DEPLOYMENT.md) guide
-- Review API docs at `/docs`
-- Open an issue on GitHub
-
-## 🙏 Acknowledgments
-
-Built with:
-- FastAPI
-- LangChain
-- OpenAI
-- FAISS
-- Sentence Transformers

api/app.py

@@ -31,7 +31,7 @@ logger = logging.getLogger(__name__)
 async def lifespan(app: FastAPI):
     """Application lifespan management with background initialization"""
     # Startup
-    logger.info("Starting Medical RAG AI Advisor API...")
+    logger.info("Starting Lung Cancer AI Advisor API...")
     
     # Start background initialization of heavy components
     try:
@@ -46,13 +46,29 @@ async def lifespan(app: FastAPI):
     yield
     
     # Shutdown
-    logger.info("Shutting down Medical RAG AI Advisor API...")
+    logger.info("Shutting down Lung Cancer AI Advisor API...")
 
 
 # Create FastAPI application
 app = FastAPI(
-    title="Medical RAG AI Advisor API",
-    description="Professional API for medical information retrieval and advisory services",
+    title="Lung Cancer AI Advisor API",
+    description="""AI-Powered Lung Cancer Information & Support API
+
+This API provides intelligent responses to lung cancer-related queries using advanced AI and medical knowledge retrieval.
+
+**Key Features:**
+- Intelligent Query Processing: AI agent automatically selects appropriate tools and data sources
+- Session Management: Maintains conversation context across multiple queries
+- Streaming Support: Real-time response streaming for better UX
+- Medical Knowledge Base: Access to comprehensive lung cancer information
+
+**Main Endpoints:**
+- POST /ask - Get complete AI response for a query
+- POST /ask/stream - Stream AI response in real-time (recommended for better UX)
+- GET /health - Check API health and initialization status
+- POST /export/{format} - Export conversation history
+
+""",
     version="1.0.0",
     docs_url="/docs",
     redoc_url="/redoc",
@@ -83,9 +99,9 @@ app.include_router(export.router)
 async def root():
     """Root endpoint with API information"""
     return {
-        "name": "Medical RAG AI Advisor API",
+        "name": "Lung Cancer AI Advisor API",
         "version": "1.0.0",
-        "description": "Professional API for medical information retrieval and advisory services",
+        "description": "AI-powered advisor for lung cancer information and support",
         "docs": "/docs",
         "health": "/health",
         "endpoints": {

api/exceptions.py

@@ -1,5 +1,5 @@
 """
-Exception handlers for Medical RAG AI Advisor API
+Exception handlers for Lung Cancer AI Advisor API
 """
 import logging
 from datetime import datetime

api/middleware.py

@@ -1,5 +1,5 @@
 """
-Middleware for Medical RAG AI Advisor API
+Middleware for Lung Cancer AI Advisor API
 """
 import time
 import logging
@@ -147,7 +147,8 @@ def get_cors_middleware_config():
         # Default to allowing Hugging Face Space and localhost
         # Include null for file:// protocol and common local development origins
         allowed_origins = [
-            "https://moazx-api.hf.space",
+            "http://127.0.0.1:7860",
+            "https://huggingface.co",
             "http://localhost:8000",
             "http://127.0.0.1:8000",
             "http://localhost:5500",  # Live Server default port
@@ -160,7 +161,7 @@ def get_cors_middleware_config():
     return {
         "allow_origins": allowed_origins,
         "allow_credentials": True,
-        "allow_methods": ["*"],
-        "allow_headers": ["*"],
-        "expose_headers": ["*"],
+        "allow_methods": ["GET", "POST", "PUT", "DELETE", "OPTIONS"],
+        "allow_headers": ["Content-Type", "Authorization", "Accept", "Origin", "X-Requested-With", "Cookie"],
+        "expose_headers": ["Set-Cookie"],
     }

api/models.py

@@ -1,5 +1,5 @@
 """
-API Models and Schemas for Medical RAG AI Advisor
+API Models and Schemas for Lung Cancer AI Advisor
 """
 from pydantic import BaseModel, Field
 from typing import Optional, List, Dict, Any

api/routers/auth.py

@@ -100,6 +100,9 @@ async def login(
     """
     Login endpoint - validates credentials and creates session
     """
+    # Log login attempt
+    logger.info(f"Login attempt for username: {username}, Origin: {request.headers.get('origin')}")
+    
     # Verify credentials
     if not verify_credentials(username, password):
         logger.warning(f"Failed login attempt for username: {username}")
@@ -107,18 +110,22 @@ async def login(
     
     # Create session
     token = create_session(username)
+    logger.info(f"Session created for user: {username}")
     
     # Set secure cookie
-    # In development (HTTP), use lax samesite and secure=False
-    # In production (HTTPS), use none samesite and secure=True
-    is_production = os.getenv("ENVIRONMENT", "development") == "production"
+    # Detect if we're running on HTTPS (Hugging Face Spaces use HTTPS)
+    is_https = request.url.scheme == "https" or request.headers.get("x-forwarded-proto") == "https"
+    
+    # For HTTPS (production/HF Spaces), use SameSite=None with Secure=True for cross-origin
+    # For HTTP (local dev), use SameSite=Lax with Secure=False
+    if is_https:
+        samesite = "none"
+        secure = True
+    else:
+        samesite = "lax"
+        secure = False
     
-    origin = request.headers.get("origin")
-    parsed_origin = urlparse(origin) if origin else None
-    is_cross_site = bool(parsed_origin and parsed_origin.hostname and parsed_origin.hostname != request.url.hostname)
-    is_https = request.url.scheme == "https"
-    samesite = "none" if (is_https and (is_production or is_cross_site)) else "lax"
-    secure = True if samesite == "none" else is_production
+    logger.info(f"Setting cookie with samesite={samesite}, secure={secure}, is_https={is_https}")
 
     response.set_cookie(
         key="session_token",
@@ -126,7 +133,8 @@ async def login(
         httponly=True,
         max_age=SESSION_MAX_AGE,
         samesite=samesite,
-        secure=secure
+        secure=secure,
+        path="/"
     )
     
     logger.info(f"Successful login for user: {username}")
@@ -175,12 +183,18 @@ async def verify(session_token: Optional[str] = Cookie(None)):
 
 
 @router.get("/status")
-async def status(session_token: Optional[str] = Cookie(None)):
+async def status(request: Request, session_token: Optional[str] = Cookie(None)):
     """
     Check authentication status without raising exception
     """
+    logger.info(f"Status check - Cookie present: {session_token is not None}, Origin: {request.headers.get('origin')}")
     session_data = verify_session(session_token)
     
+    if session_data:
+        logger.info(f"Status check - Authenticated as: {session_data.get('username')}")
+    else:
+        logger.info("Status check - Not authenticated")
+    
     return {
         "authenticated": session_data is not None,
         "username": session_data.get("username") if session_data else None

api/routers/health.py

@@ -3,11 +3,6 @@ Health Check and System Status Router
 """
 from datetime import datetime
 from fastapi import APIRouter
-import sys
-import os
-
-# Add src to path for imports
-sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
 
 from api.models import HealthStatus, InitializationStatus
 
@@ -23,28 +18,58 @@ async def health_check():
     
     # Check agent availability
     try:
-        from agent import safe_run_agent
+        from core.agent import safe_run_agent
         components["agent"] = "healthy"
     except Exception:
         components["agent"] = "unhealthy"
     
     # Check vector store
     try:
-        from vector_store import VectorStore
-        components["vector_store"] = "healthy"
-    except Exception:
+        from core.vector_store import get_vector_store, load_vector_store
+        from core.background_init import is_initialization_complete
+        
+        # If initialization is complete, check the vector store directly
+        if is_initialization_complete():
+            vector_store = get_vector_store()
+            if vector_store is not None:
+                components["vector_store"] = "healthy"
+            else:
+                # If vector store is None, try to load it
+                try:
+                    loaded_store = load_vector_store()
+                    if loaded_store is not None:
+                        components["vector_store"] = "healthy"
+                    else:
+                        components["vector_store"] = "unhealthy"
+                except Exception as e:
+                    logger.error(f"Failed to load vector store: {e}")
+                    components["vector_store"] = "unhealthy"
+        else:
+            # If initialization is not complete, the vector store might not be ready yet
+            # but we'll check if it exists and can be loaded
+            try:
+                loaded_store = load_vector_store()
+                if loaded_store is not None:
+                    components["vector_store"] = "healthy"
+                else:
+                    components["vector_store"] = "initializing"
+            except Exception as e:
+                logger.warning(f"Vector store not ready yet: {e}")
+                components["vector_store"] = "initializing"
+    except Exception as e:
+        logger.error(f"Error checking vector store health: {e}")
         components["vector_store"] = "unhealthy"
     
     # Check data loaders
     try:
-        from data_loaders import load_pdf_documents
+        from core.data_loaders import load_pdf_documents
         components["data_loaders"] = "healthy"
     except Exception:
         components["data_loaders"] = "unhealthy"
     
     # Check tools
     try:
-        from tools import medical_guidelines_knowledge_tool
+        from core.tools import medical_guidelines_knowledge_tool
         components["tools"] = "healthy"
     except Exception:
         components["tools"] = "unhealthy"
@@ -52,7 +77,7 @@ async def health_check():
     # Check initialization status
     initialization_status = None
     try:
-        from background_init import (
+        from core.background_init import (
             is_initialization_complete, 
             get_initialization_status, 
             is_initialization_successful,
@@ -130,8 +155,8 @@ async def get_version():
     """
     return {
         "version": "1.0.0",
-        "name": "Medical RAG AI Advisor API",
-        "description": "Professional API for medical information retrieval and advisory services",
+        "name": "Lung Cancer AI Advisor API",
+        "description": "AI-powered advisor for lung cancer information and support",
         "build_date": "2024-01-01"
     }
 

api/routers/medical.py

@@ -1,9 +1,11 @@
 """
-Medical Query Router for RAG AI Advisor
+Medical Query Router for Lung Cancer AI Advisor
 """
 import asyncio
+import inspect
 from fastapi import APIRouter, HTTPException
 from fastapi.responses import StreamingResponse
+from pydantic import BaseModel, Field
 import sys
 import os
 
@@ -15,18 +17,78 @@ from core.agent import safe_run_agent, safe_run_agent_streaming
 router = APIRouter(tags=["medical"])
 
 
-@router.get("/ask")
-async def ask(query: str, session_id: str = "default"):
+class QueryRequest(BaseModel):
     """
-    Process a medical query - agent decides which tools to use
+    Request model for medical queries
     
-    Args:
-        query: The medical question or query
-        session_id: Optional session identifier for conversation continuity (default: "default")
+    Example:
+        {
+            "query": "What are the early symptoms of lung cancer?",
+            "session_id": "user_123_session_456"
+        }
+    """
+    query: str = Field(
+        ..., 
+        description="The medical question or query about lung cancer",
+        example="Give me the options for first line treatment for NSCLC?"
+    )
+    session_id: str = Field(
+        ..., 
+        description="Unique session identifier for conversation continuity. Use the same `session_id` to maintain context across multiple queries. Format: `user_{user_id}_session_{timestamp}`",
+        example="user_123_session_1699612345"
+    )
+
+
+@router.post(
+    "/ask",
+    summary="Ask a lung cancer question",
+)
+async def ask(request: QueryRequest):
     """
+    Process a lung cancer-related medical query and return a complete response.
+    
+The AI agent intelligently selects appropriate tools and data sources to provide
+accurate, evidence-based information about lung cancer.
+
+Request Body:
+- `query` (required): Your medical question about lung cancer
+- `session_id` (required): Unique identifier to maintain conversation context
+
+Response:
+- `response`: Complete AI-generated answer in markdown format
+- `session_id`: Echo of the session identifier used
+
+Example Request:
+    {
+        "query": "What are the early symptoms of lung cancer?",
+        "session_id": "user_123_session_1699612345"
+    }
+
+Example Response:
+    {
+        "response": "Early symptoms of lung cancer may include...\n\n**Common Early Signs:**\n- Persistent cough...",
+        "session_id": "user_123_session_1699612345"
+    }
+
+Frontend Integration Tips:
+- Use the same `session_id` for follow-up questions to maintain context
+- Display response in markdown renderer for better formatting
+- Show loading state while waiting for response
+- Handle 500 errors gracefully with user-friendly messages
+
+Args:
+    request: QueryRequest containing query and session_id
+    
+Returns:
+    Dictionary with response text and session_id
+    
+Raises:
+    HTTPException: 500 if query processing fails
+"""
+
     try:
-        response = await safe_run_agent(user_input=query, session_id=session_id)
-        return {"response": response, "session_id": session_id}
+        response = await safe_run_agent(user_input=request.query, session_id=request.session_id)
+        return {"response": response, "session_id": request.session_id}
         
     except Exception as e:
         raise HTTPException(
@@ -34,20 +96,60 @@ async def ask(query: str, session_id: str = "default"):
             detail=f"Error processing medical query: {str(e)}"
         )
 
+# Dedent the docstring so OpenAPI/Redoc renderers don't treat the
+# indented lines as a markdown code block (leading 4-space indentation).
+ask.__doc__ = inspect.cleandoc(ask.__doc__ or "")
 
-@router.get("/ask/stream")
-async def ask_stream(query: str, session_id: str = "default"):
+
+@router.post(
+    "/ask/stream",
+    summary="Ask a lung cancer question with streaming response",
+    
+)
+async def ask_stream(request: QueryRequest):
     """
-    Process a medical query with streaming response - agent decides which tools to use
+    Process a lung cancer-related medical query with real-time streaming response.
+    
+    Recommended for frontend use - Provides better user experience by streaming
+    the response as it's generated, similar to ChatGPT.
+    
+    Request Body:
+    - `query` (required): Your medical question about lung cancer
+    - `session_id` (required): Unique identifier to maintain conversation context
+    
+    Response:
+    - Streaming text/markdown content
+    - Response is sent in chunks as it's generated
+    - Connection stays open until response is complete
     
+    
+    Example Request:
+        {
+            "query": "Explain the difference between small cell and non-small cell lung cancer",
+            "session_id": "user_123_session_1699612345"
+        }
+    
+    Frontend Integration Tips:
+    - Use the same `session_id` for follow-up questions to maintain context
+    - Display response in markdown renderer for better formatting
+    - Show loading state while waiting for response
+    - Render markdown progressively as chunks arrive
+    - Show typing indicator while streaming
+    - Handle 500 errors gracefully with user-friendly messages
+
     Args:
-        query: The medical question or query
-        session_id: Optional session identifier for conversation continuity (default: "default")
+        request: QueryRequest containing query and session_id
+        
+    Returns:
+        StreamingResponse with text/markdown content
+    
+    Raises:
+        HTTPException: 500 if query processing fails
     """
     async def event_stream():
         try:
             chunk_buffer = ""
-            async for chunk in safe_run_agent_streaming(user_input=query, session_id=session_id):
+            async for chunk in safe_run_agent_streaming(user_input=request.query, session_id=request.session_id):
                 chunk_buffer += chunk
                 
                 # Send chunks in reasonable sizes for smoother streaming
@@ -64,3 +166,6 @@ async def ask_stream(query: str, session_id: str = "default"):
             yield f"Error: {str(e)}"
     
     return StreamingResponse(event_stream(), media_type="text/markdown")
+
+# Dedent streaming endpoint docstring for proper Markdown rendering in docs
+ask_stream.__doc__ = inspect.cleandoc(ask_stream.__doc__ or "")

app.py

@@ -1,5 +1,5 @@
 """
-Startup script for Medical RAG AI Advisor API
+Startup script for Lung Cancer AI Advisor API
 """
 import sys
 import os

backup/backup_20251022_110950/chunks.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b038845d797cac35024d39df8c7a861d741a1f7c2edc1a54286e17de1806b38e
-size 3878660

backup/backup_20251022_110950/vector_store/index.faiss

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:824156db2ada7613098cc7c9a8c27d66b33553885146fb6f66ab450ddc5d95cb
-size 8248365

backup/backup_20251022_110950/vector_store/index.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2655709226a3c13f4dae2efc131aaee81f68ac696a9b9a7aa8daeabc026d40d4
-size 4020637

backup/backup_20251022_111044/chunks.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b038845d797cac35024d39df8c7a861d741a1f7c2edc1a54286e17de1806b38e
-size 3878660

backup/backup_20251022_111044/vector_store/index.faiss

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:824156db2ada7613098cc7c9a8c27d66b33553885146fb6f66ab450ddc5d95cb
-size 8248365

backup/backup_20251022_111044/vector_store/index.pkl

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2655709226a3c13f4dae2efc131aaee81f68ac696a9b9a7aa8daeabc026d40d4
-size 4020637

core/agent.py

@@ -4,6 +4,7 @@ from typing import Any, AsyncGenerator
 import asyncio
 import requests
 import os
+import httpx
 from langchain.agents import create_openai_tools_agent, AgentExecutor
 from langchain.memory import ConversationBufferWindowMemory
 from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
@@ -101,22 +102,30 @@ Your primary purpose is to provide evidence-based clinical guidance on lung canc
 
 **AUDIENCE**: Your responses are for thoracic oncologists, pulmonologists, and medical experts managing lung cancer. Use appropriate medical terminology, clinical precision, and expert-level detail specific to lung cancer management.
 
-**RESPONSE STYLE - CRITICAL: CONCISE, PRECISE, DOCTOR-SPECIFIC ANSWERS**:
+**RESPONSE STYLE - CRITICAL: CONCISE, ACCURATE, MEDIUM-LENGTH ANSWERS FOR MEDICAL EXPERTS**:
 - **IMMEDIATE DIRECT ANSWERS**: Start immediately with the answer - NO introductory phrases like "I will retrieve...", "Let me search...", "Please hold on...", or status updates
 - **NO PREAMBLES**: Never announce what you're about to do - just do it and present the results directly
 - **ZERO PROCEDURAL STATEMENTS**: Do NOT write "I will retrieve", "I will search", "I will gather", "Please wait", "Hold on", or any similar phrases - START DIRECTLY WITH THE CLINICAL ANSWER
 - **FIRST WORD RULE**: Your response must begin with the actual answer content (e.g., a heading, clinical information, or direct statement) - never with a procedural announcement
-- **CONCISE & TARGETED**: Provide focused, actionable answers directly addressing the clinical question
-- **PRECISION OVER VOLUME**: Include only the most clinically relevant information - avoid unnecessary elaboration
+
+**RESPONSE LENGTH - MEDIUM AND BALANCED**:
+- **NOT TOO LONG**: Avoid excessive detail, lengthy explanations, or exhaustive lists that overwhelm busy clinicians
+- **NOT TOO SHORT**: Provide sufficient clinical context, key recommendations, and essential details for informed decision-making
+- **MEDIUM LENGTH TARGET**: Aim for 200-400 words for standard queries; 400-600 words for complex multi-part questions
+- **QUALITY OVER QUANTITY**: Every sentence must add clinical value - eliminate redundancy and filler content
+- **CONCISE & COMPLETE**: Cover all essential aspects of the query without unnecessary elaboration
+
+**EXPERT-TAILORED CONTENT**:
+- **PRECISION & ACCURACY**: Provide exact, evidence-based information from guidelines - no speculation or general knowledge
 - **CLINICAL EFFICIENCY**: Respect physicians' time by delivering key information first, then supporting details
-- **STRUCTURED BREVITY**: Use clear hierarchical formatting (headers, bullet points) to enable rapid information scanning
-- **ESSENTIAL DETAILS ONLY**: Include specific clinical parameters, dosing, biomarkers, and monitoring when directly relevant to the query
+- **STRUCTURED CLARITY**: Use clear hierarchical formatting (headers, bullet points) to enable rapid information scanning
+- **ESSENTIAL DETAILS**: Include specific clinical parameters, dosing, biomarkers, and monitoring when directly relevant to the query
 - **PRIORITIZED INFORMATION**: Lead with the most critical clinical decision points, contraindications, and evidence-based recommendations
 - **LUNG CANCER FOCUS**: Prioritize lung cancer-specific information including histology, molecular markers, staging, and treatment selection
-- Use precise medical terminology without oversimplification
-- Reference specific guideline sources (tables, figures, algorithms) with concise citations
-- Highlight critical nuances, contraindications, and special populations only when clinically significant
-- When multiple approaches exist, prioritize by evidence level and clinical context
+- **MEDICAL TERMINOLOGY**: Use precise medical terminology appropriate for thoracic oncologists and pulmonologists
+- **CONCISE CITATIONS**: Reference specific guideline sources (tables, figures, algorithms) with brief inline citations
+- **CLINICALLY SIGNIFICANT NUANCES**: Highlight critical nuances, contraindications, and special populations only when clinically significant
+- **EVIDENCE-BASED PRIORITIZATION**: When multiple approaches exist, prioritize by evidence level and clinical context
 - **CONTEXT AWARENESS**: Use context pages to ensure accuracy, but synthesize information concisely
 - **DIRECT ANSWERS**: Answer the specific question asked without providing tangential information
 
@@ -129,6 +138,13 @@ Your primary purpose is to provide evidence-based clinical guidance on lung canc
 - Even for basic lung cancer concepts (e.g., "what is EGFR mutation", "ALK rearrangement", "PD-L1 expression"), you MUST retrieve information from the guidelines first
 - Only after retrieving guideline information should you formulate your answer based on what was retrieved
 
+**STRICT QUERY ADHERENCE - ALL PROVIDERS REQUIREMENT:**
+- When the user explicitly requests information from "all guidelines", "all providers", "according to all guidelines", or similar phrasing, you MUST retrieve and present information from ALL available guideline providers (NCCN, ASCO, ESMO, NICE, Manus)
+- Do NOT selectively omit providers - if the user asks for "all", you must query each provider separately and include ALL results
+- Call the medical_guidelines_knowledge_tool multiple times (once per provider) when "all providers" is requested
+- Present a comprehensive answer that includes recommendations from every available provider
+- If a specific provider has no information on the topic, explicitly state that in your response
+
 **TOOL USAGE REQUIREMENTS:**
 1. **MEDICAL QUESTIONS** (definitions, treatments, guidelines, etc.): 
    - MANDATORY: Use "medical_guidelines_knowledge_tool" FIRST
@@ -250,16 +266,19 @@ class SessionMemoryManager:
     
     def __init__(self):
         self._sessions = {}
-        self._default_window_size = 10
+        self._default_window_size = 20  # Increased from 10 to maintain better context
     
     def get_memory(self, session_id: str = "default") -> ConversationBufferWindowMemory:
         """Get or create memory for a specific session."""
         if session_id not in self._sessions:
-            self._sessions[session_id] = ConversationBufferWindowMemory(
-                memory_key="chat_history",
-                return_messages=True,
-                max_window_size=self._default_window_size
-            )
+            import warnings
+            with warnings.catch_warnings():
+                warnings.filterwarnings("ignore", category=DeprecationWarning)
+                self._sessions[session_id] = ConversationBufferWindowMemory(
+                    memory_key="chat_history",
+                    return_messages=True,
+                    max_window_size=self._default_window_size
+                )
         return self._sessions[session_id]
     
     def clear_session(self, session_id: str) -> bool:
@@ -359,7 +378,7 @@ def _perform_automatic_validation(user_input: str, response: str) -> None:
     """
     try:
         # Import here to avoid circular imports
-        from .tools import _last_question, _last_documents, _last_user_question
+        from .tools import _last_question, _last_documents
         
         # Check if we have the necessary context for validation
         if not _last_question or not _last_documents:
@@ -414,10 +433,6 @@ async def run_agent_streaming(user_input: str, session_id: str = "default", max_
         yield "Sorry, I didn't receive any questions. Please enter your question or request."
         return
     
-    # Store the original user question for validation
-    from .tools import store_user_question
-    store_user_question(user_input.strip())
-    
     retry_count = 0
     last_error = None
     current_run_id = None
@@ -545,13 +560,16 @@ async def run_agent_streaming(user_input: str, session_id: str = "default", max_
                 yield "Sorry, the system is currently busy. Please try again in a little while."
                 return
                 
-        except APIError as e:
+        except (APIError, httpx.RemoteProtocolError, httpx.ReadError, httpx.ConnectError) as e:
             retry_count += 1
             last_error = e
-            logger.error(f"OpenAI API error: {str(e)}")
+            error_type = type(e).__name__
+            logger.error(f"OpenAI API/Connection error ({error_type}): {str(e)}")
             
             if retry_count <= max_retries:
-                await asyncio.sleep(2)
+                wait_time = min(2 ** retry_count, 10)  # Exponential backoff, max 10 seconds
+                logger.info(f"Retrying after {wait_time} seconds... (Attempt {retry_count}/{max_retries})")
+                await asyncio.sleep(wait_time)
                 continue
             else:
                 yield "Sorry, there was an error connecting to the service. Please try again later."

core/context_enrichment.py

@@ -25,6 +25,7 @@ class ContextEnricher:
         """
         self._document_cache: Dict[str, List[Document]] = {}
         self._cache_size = cache_size
+        self._all_chunks_cache: Optional[List[Document]] = None  # Cache all chunks to avoid reloading
     
     def enrich_documents(
         self,
@@ -187,8 +188,13 @@ class ContextEnricher:
         try:
             from . import utils
             
-            # Load all chunks
-            all_chunks = utils.load_chunks()
+            # Load all chunks (use cached version to avoid redundant loading)
+            if self._all_chunks_cache is None:
+                self._all_chunks_cache = utils.load_chunks()
+                if self._all_chunks_cache:
+                    logger.debug(f"Loaded {len(self._all_chunks_cache)} chunks into enricher cache")
+            
+            all_chunks = self._all_chunks_cache
             if not all_chunks:
                 logger.debug(f"No chunks available for enrichment")
                 return None