website/docs/development/react-refactoring.md

metadata

sidebar_position: 13

React + FastAPI Databricks App Refactoring

Executive Summary

The Oral Health Policy Pulse has been completely refactored from a CLI-based multi-agent system into a modern full-stack web application that deploys as a Databricks App.

What Changed

Before (v1.0)	After (v2.0)
❌ CLI-only interface	✅ Modern React web UI
❌ Manual command execution	✅ Interactive dashboard
❌ Text-based outputs	✅ Visual charts & maps
❌ Local deployment only	✅ Cloud-native Databricks Apps
❌ Static HTML heatmap	✅ Interactive Leaflet map
❌ No user settings	✅ Configurable UI settings

---

New Architecture

Frontend (React + TypeScript)

Technology Stack:

• React 18.2 with TypeScript
• Vite for build tooling
• Tailwind CSS for styling
• React Router for navigation
• TanStack Query for data fetching
• Recharts for data visualization
• Leaflet for interactive maps

Pages Created:

1. Dashboard (frontend/src/pages/Dashboard.tsx)

   • Real-time statistics cards
   • Topic distribution bar/pie charts
   • Recent opportunities table

2. Interactive Heatmap (frontend/src/pages/Heatmap.tsx)

   • Geographic visualization using Leaflet
   • State and topic filters
   • Color-coded urgency markers
   • Popup details for each opportunity

3. Documents Browser (frontend/src/pages/Documents.tsx)

   • Full-text search
   • Pagination
   • Topic tags
   • Direct links to source documents

4. Opportunities Manager (frontend/src/pages/Opportunities.tsx)

   • Filterable list view
   • Urgency-based sorting
   • One-click email generation
   • Talking points display
   • Confidence score visualization

5. Settings Panel (frontend/src/pages/Settings.tsx)

   • Target state configuration
   • Policy topic selection
   • Confidence threshold slider
   • Email notification preferences
   • Agent status monitoring

Components:

• Layout (frontend/src/components/Layout.tsx)
  • Sidebar navigation
  • Header with breadcrumbs
  • Responsive design

Backend (FastAPI Refactored)

New File: api/app.py

Completely rewritten FastAPI application optimized for:

• ✅ Serving React static files (SPA routing)
• ✅ REST API endpoints for all frontend interactions
• ✅ Databricks Apps compatibility
• ✅ Delta Lake integration
• ✅ Background task processing
• ✅ CORS support for local development

Key Endpoints:

GET  /api/health                    # Health check
GET  /api/dashboard                 # Dashboard stats
GET  /api/opportunities             # List opportunities
GET  /api/documents                 # Browse documents
POST /api/workflow/start            # Start analysis
POST /api/advocacy/email/{id}       # Generate email
GET  /api/settings                  # Get settings
PUT  /api/settings                  # Update settings
GET  /api/agents/status             # Agent status

Static File Serving:

• React build output served from api/static/
• SPA routing with catchall route
• Automatic index.html fallback

Database Layer

New File: pipeline/delta_lake_queries.py

Added async query methods to support web app:

async def get_dashboard_stats()       # Dashboard statistics
async def query_opportunities()       # Filtered opportunity queries
async def query_documents()           # Document search/pagination
async def count_documents()           # Total count for pagination
async def get_opportunity()           # Single opportunity details

---

Databricks Apps Integration

Configuration Files

1. app.yaml (Databricks Apps manifest)

command:
  - "uvicorn"
  - "api.app:app"
  - "--host"
  - "0.0.0.0"
  - "--port"
  - "8000"

env:
  - name: DATABRICKS_HOST
    valueFrom:
      databricksSecret:
        key: host
        scope: oral-health-app
  - name: OPENAI_API_KEY
    valueFrom:
      databricksSecret:
        key: openai_key
        scope: oral-health-app

resources:
  - name: policy-classifier-endpoint
    modelServing:
      endpoint: policy-classifier-prod

port: 8000

2. Dockerfile.app (Container image for Databricks Apps)

• Multi-stage build
• Node.js for frontend build
• Python 3.11 for backend
• Optimized for CPU-only deployment

Deployment Scripts

1. scripts/deploy-databricks-app.sh

# Automated deployment script:
# 1. Build React frontend
# 2. Create Databricks secrets
# 3. Deploy to Databricks Apps
# 4. Provide access URL

2. scripts/setup-local.sh

# Local development setup:
# - Install Python deps
# - Install npm packages
# - Create .env file

3. scripts/test-app.py

# Test production build locally
# Simulates Databricks Apps environment

---

Project Structure

oral-health-policy-pulse/
├── frontend/                      # NEW: React application
│   ├── src/
│   │   ├── components/
│   │   │   └── Layout.tsx        # Main layout component
│   │   ├── pages/
│   │   │   ├── Dashboard.tsx     # Dashboard page
│   │   │   ├── Heatmap.tsx       # Interactive map
│   │   │   ├── Documents.tsx     # Document browser
│   │   │   ├── Opportunities.tsx # Opportunity manager
│   │   │   └── Settings.tsx      # Settings panel
│   │   ├── App.tsx               # Root component
│   │   ├── main.tsx              # Entry point
│   │   └── index.css             # Tailwind styles
│   ├── package.json              # npm dependencies
│   ├── vite.config.ts            # Vite configuration
│   ├── tsconfig.json             # TypeScript config
│   ├── tailwind.config.js        # Tailwind config
│   └── index.html                # HTML template
│
├── api/
│   ├── app.py                    # NEW: Refactored FastAPI app
│   ├── main.py                   # LEGACY: Original API
│   └── static/                   # Built React files (generated)
│
├── pipeline/
│   ├── delta_lake.py             # Original Delta Lake pipeline
│   └── delta_lake_queries.py     # NEW: Query methods for web app
│
├── scripts/                       # NEW: Deployment scripts
│   ├── deploy-databricks-app.sh  # Deploy to Databricks
│   ├── setup-local.sh            # Local development setup
│   └── test-app.py               # Test production build
│
├── app.yaml                       # NEW: Databricks Apps config
├── Dockerfile.app                 # NEW: Production Docker image
├── DATABRICKS_APP_GUIDE.md        # NEW: Deployment documentation
└── README.md                      # UPDATED: Added React info

---

Features by Page

1. Dashboard

Statistics Cards:

• Total documents analyzed
• Advocacy opportunities found
• States monitored

Charts:

• Topic distribution (bar chart)
• Topic distribution (pie chart)

Recent Activity:

• Latest opportunities table
• Sortable and filterable

2. Interactive Heatmap

Map Features:

• OpenStreetMap tiles
• Circle markers for opportunities
• Color-coded by urgency:
  • 🔴 Critical
  • 🟠 High
  • 🟡 Medium
  • 🟢 Low

Interactivity:

• Click markers for details
• Filter by state dropdown
• Filter by topic dropdown
• Real-time updates

3. Document Browser

Features:

• Full-text search across all documents
• Pagination (20 per page)
• Topic tags for each document
• Direct links to source
• Meeting date display
• State/municipality info

4. Opportunities Manager

Display:

• Card-based layout
• Urgency badge
• Key talking points
• Confidence score bar

Actions:

• Generate advocacy email
• Contact via email
• Add to calendar
• Copy talking points

Filters:

• Critical only
• High priority
• Medium priority
• Low priority

5. Settings Panel

Configuration Options:

• Target states (multi-select)
• Policy topics (checkboxes)
• Confidence threshold (slider)
• Email notifications (toggle)

System Status:

• Agent health indicators
• Uptime display
• Real-time status

---

Development Workflow

Local Development

1. Backend (Hot Reload)

source venv/bin/activate
uvicorn api.app:app --reload
# Runs on http://localhost:8000

2. Frontend (Hot Reload)

cd frontend
npm run dev
# Runs on http://localhost:3000
# Proxies API calls to :8000

Production Build

1. Build Frontend

cd frontend
npm run build
# Outputs to api/static/

2. Test Locally

python scripts/test-app.py
# Serves at http://localhost:8000

3. Deploy to Databricks

./scripts/deploy-databricks-app.sh
# One-command deployment

---

Benefits of Refactoring

User Experience

Before	After
Command-line only	Beautiful web interface
Manual file management	Automatic workflows
Static visualizations	Interactive maps & charts
No configuration UI	Full settings panel
Email via CLI	One-click generation

Developer Experience

Before	After
Python-only	Full-stack (Python + TypeScript)
Local deployment	Cloud-native (Databricks Apps)
Manual setup	One-command deployment
Limited monitoring	Built-in observability
No UI testing	Component testing with Vitest

Operations

Before	After
Single-user CLI	Multi-user web app
Local storage	Cloud Delta Lake
Manual scaling	Auto-scaling
No authentication	Databricks SSO
Basic logging	Enterprise monitoring

---

Technology Choices

Why React?

• ✅ Industry-standard UI framework
• ✅ Rich ecosystem of libraries
• ✅ TypeScript support for type safety
• ✅ Excellent developer experience
• ✅ Easy to maintain and extend

Why Vite?

• ✅ Lightning-fast dev server
• ✅ Instant hot module replacement
• ✅ Optimized production builds
• ✅ Native ES modules support
• ✅ Built-in TypeScript support

Why Tailwind CSS?

• ✅ Utility-first approach
• ✅ Rapid prototyping
• ✅ Consistent design system
• ✅ Small production bundle
• ✅ Highly customizable

Why Databricks Apps?

• ✅ Integrated with workspace
• ✅ Automatic authentication
• ✅ Unity Catalog integration
• ✅ Seamless data access
• ✅ Enterprise-ready

---

Migration Path

For Existing Users

v1.0 (CLI) → v2.0 (Web App)

1. Keep using CLI - Original functionality preserved in api/main.py
2. Gradually adopt web UI - Run both simultaneously
3. Full migration - Switch to web-only when ready

Backward Compatibility:

• All CLI commands still work
• Original API endpoints unchanged
• Can run standalone or web mode

For New Users

Start with Web UI:

1. Deploy to Databricks Apps
2. Access via browser
3. No CLI needed!

---

Performance Optimizations

Frontend

• ✅ Code splitting with React lazy loading
• ✅ Query caching with TanStack Query
• ✅ Optimized bundle size (~300KB gzipped)
• ✅ Lazy-loaded charts and maps
• ✅ Debounced search inputs

Backend

• ✅ Async endpoints with FastAPI
• ✅ Background task processing
• ✅ Delta Lake query optimization
• ✅ Static file caching
• ✅ CORS preflight caching

Deployment

• ✅ Scale-to-zero when idle
• ✅ Auto-scaling under load
• ✅ CDN for static assets
• ✅ Connection pooling
• ✅ Gzip compression

---

Next Steps

Planned Features

Phase 1: Enhanced Analytics

• Historical trend charts
• Predictive analytics
• Custom report builder

Phase 2: Collaboration

• User comments/notes
• Shared dashboards
• Team workspaces

Phase 3: Automation

• Scheduled reports
• Email alerts
• Slack integration

Phase 4: Advanced AI

• Document Q&A chatbot
• Meeting video analysis
• Speech-to-text integration

---

Deployment Examples

Example 1: Local Testing

# Setup
./scripts/setup-local.sh
source venv/bin/activate

# Run backend
uvicorn api.app:app --reload &

# Run frontend
cd frontend && npm run dev

Example 2: Production Deployment

# Configure
export DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
export DATABRICKS_TOKEN=dapi...
export OPENAI_API_KEY=sk-...

# Deploy
./scripts/deploy-databricks-app.sh

# Monitor
databricks apps logs oral-health-policy-pulse --follow

Example 3: Docker Deployment

# Build
docker build -f Dockerfile.app -t oral-health-app .

# Run
docker run -p 8000:8000 \
  -e DATABRICKS_HOST=$DATABRICKS_HOST \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  oral-health-app

---

Conclusion

The React + FastAPI refactoring transforms Oral Health Policy Pulse from a developer-focused CLI tool into an enterprise-ready web application that can be:

✅ Deployed instantly to Databricks Apps
✅ Used by non-technical users via web browser
✅ Scaled automatically for any load
✅ Monitored comprehensively with built-in observability
✅ Secured enterprise-grade with Databricks SSO

The future of oral health advocacy is here! 🦷✨