PLAN.md

Implementation Plan: React + FastAPI Architecture

Overview

Convert the Streamlit-based UAP Data Analysis Tool to a modern React frontend with FastAPI backend, maintaining all existing functionality.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     React Frontend                          │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌───────┐ │
│  │ Parsing │ │Analysis │ │ Search  │ │Magnetic │ │  Map  │ │
│  └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └───┬───┘ │
└───────┼──────────┼──────────┼──────────┼──────────┼───────┘
        │          │          │          │          │
        ▼          ▼          ▼          ▼          ▼
┌─────────────────────────────────────────────────────────────┐
│                    FastAPI Backend                          │
│  /api/parse  /api/analyze  /api/search  /api/magnetic /api/map │
└─────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────┐
│              Existing Python Services                       │
│  UAPParser  UAPAnalyzer  UAPVisualizer  Cohere  InterMagnet │
└─────────────────────────────────────────────────────────────┘

File Structure

UAP-Data-Analysis-Tool/
├── api/                          # FastAPI backend
│   ├── __init__.py
│   ├── main.py                   # FastAPI app entry point
│   ├── config.py                 # Settings and API keys
│   ├── routes/
│   │   ├── __init__.py
│   │   ├── upload.py             # File upload endpoints
│   │   ├── parse.py              # OpenAI parsing endpoints
│   │   ├── analyze.py            # UMAP/HDBSCAN/XGBoost endpoints
│   │   ├── search.py             # Cohere rerank endpoints
│   │   ├── magnetic.py           # InterMagnet correlation endpoints
│   │   └── map.py                # Geospatial data endpoints
│   ├── services/
│   │   ├── __init__.py
│   │   ├── parser_service.py     # Wraps UAPParser
│   │   ├── analyzer_service.py   # Wraps UAPAnalyzer
│   │   ├── visualizer_service.py # Wraps UAPVisualizer
│   │   ├── search_service.py     # Cohere rerank logic
│   │   ├── magnetic_service.py   # InterMagnet API + DTW
│   │   └── map_service.py        # Kepler.gl data prep
│   ├── models/
│   │   ├── __init__.py
│   │   ├── schemas.py            # Pydantic request/response models
│   │   └── jobs.py               # Background job tracking
│   └── utils/
│       ├── __init__.py
│       ├── file_handler.py       # CSV/Excel/HDF5 handling
│       └── serialization.py      # NumPy/DataFrame serialization
│
├── frontend/                     # React frontend
│   ├── package.json
│   ├── tailwind.config.js
│   ├── vite.config.js
│   ├── index.html
│   ├── src/
│   │   ├── main.jsx
│   │   ├── App.jsx
│   │   ├── api/
│   │   │   └── client.js         # Axios/fetch wrapper
│   │   ├── components/
│   │   │   ├── layout/
│   │   │   │   ├── Navbar.jsx
│   │   │   │   ├── Sidebar.jsx
│   │   │   │   └── Layout.jsx
│   │   │   ├── common/
│   │   │   │   ├── FileUpload.jsx
│   │   │   │   ├── DataTable.jsx
│   │   │   │   ├── LoadingSpinner.jsx
│   │   │   │   └── ErrorBoundary.jsx
│   │   │   ├── charts/
│   │   │   │   ├── Treemap.jsx
│   │   │   │   ├── Histogram.jsx
│   │   │   │   ├── ScatterPlot.jsx
│   │   │   │   ├── Heatmap.jsx
│   │   │   │   └── ConfusionMatrix.jsx
│   │   │   └── map/
│   │   │       └── KeplerMap.jsx
│   │   ├── pages/
│   │   │   ├── Home.jsx
│   │   │   ├── Parsing.jsx
│   │   │   ├── Analysis.jsx
│   │   │   ├── Search.jsx
│   │   │   ├── Magnetic.jsx
│   │   │   └── Map.jsx
│   │   ├── hooks/
│   │   │   ├── useFileUpload.js
│   │   │   ├── useAnalysis.js
│   │   │   └── useWebSocket.js
│   │   ├── store/
│   │   │   └── index.js          # Zustand or React Context
│   │   └── styles/
│   │       └── globals.css
│   └── public/
│       └── assets/

Implementation Steps

Phase 1: FastAPI Backend Setup

Step 1.1: Create API structure and main entry point

• Create api/ directory structure
• Set up FastAPI app with CORS middleware
• Configure settings from environment/secrets
• Add health check endpoint

Step 1.2: Create Pydantic schemas

• Define request models for each endpoint
• Define response models with proper typing
• Create job status models for async operations

Step 1.3: Implement file upload endpoint

• POST /api/upload - Accept CSV/Excel files
• Store uploaded files temporarily
• Return file ID and column list
• Support chunked uploads for large files

Step 1.4: Implement parsing endpoints

• POST /api/parse/start - Start async parsing job
• GET /api/parse/status/{job_id} - Check job status
• GET /api/parse/result/{job_id} - Get parsed results
• Wrap existing UAPParser with proper error handling

Step 1.5: Implement analysis endpoints

• POST /api/analyze/start - Start clustering analysis
• GET /api/analyze/status/{job_id} - Check status
• GET /api/analyze/clusters/{job_id} - Get cluster data
• GET /api/analyze/embeddings/{job_id} - Get 2D embeddings for visualization
• GET /api/analyze/predictions/{job_id} - Get XGBoost results

Step 1.6: Implement search endpoints

• POST /api/search/rerank - Cohere rerank search
• Return ranked results with relevance scores

Step 1.7: Implement magnetic endpoints

• GET /api/magnetic/stations - List InterMagnet stations
• POST /api/magnetic/correlate - Run DTW correlation
• Return correlation results and time series data

Step 1.8: Implement map endpoints

• GET /api/map/sightings - Get sighting GeoJSON
• GET /api/map/bases - Get military bases data
• GET /api/map/plants - Get nuclear facilities data
• GET /api/map/config - Get Kepler.gl config

Phase 2: React Frontend Setup

Step 2.1: Initialize React project

• Create Vite + React project in frontend/
• Install dependencies: react-router, axios, recharts/plotly, kepler.gl
• Configure Tailwind CSS
• Set up project structure

Step 2.2: Create layout components

• Navbar with navigation links
• Sidebar for feature options
• Main layout wrapper
• Responsive design

Step 2.3: Create common components

• FileUpload with drag-and-drop
• DataTable with sorting/filtering
• LoadingSpinner and progress indicators
• Error boundary and toast notifications

Step 2.4: Create chart components

• Treemap using Plotly.js
• Histogram using Recharts
• ScatterPlot for embeddings visualization
• Heatmap for Cramer's V and confusion matrix
• Feature importance bar chart

Step 2.5: Create Kepler.gl map component

• Integrate kepler.gl React component
• Handle data layers dynamically
• Support filtering by attributes

Phase 3: Feature Pages

Step 3.1: Parsing page

• File upload interface
• Column selector
• Custom JSON schema editor (optional)
• Progress indicator for parsing
• Results table with download option

Step 3.2: Analysis page

• Dataset loader (upload or use parsed data)
• Column multi-selector for analysis
• Visualization tabs: Embeddings, Clusters, Predictions, Correlations
• Interactive charts with tooltips

Step 3.3: Search page

• Dataset display
• Query input
• Column selector for search scope
• Ranked results with relevance scores
• Click-to-expand details

Step 3.4: Magnetic page

• Date range selector
• Location input (lat/lon or from dataset)
• Station selector
• Correlation results with time series chart

Step 3.5: Map page

• Full-screen Kepler.gl map
• Layer toggles (sightings, bases, plants)
• Filter controls
• Export functionality

Phase 4: Integration and Polish

Step 4.1: State management

• Set up Zustand store for global state
• Persist uploaded data across pages
• Handle authentication state (API keys)

Step 4.2: WebSocket for long-running tasks

• Add WebSocket endpoint for job progress
• Real-time updates during parsing/analysis

Step 4.3: Error handling

• Consistent error responses from API
• User-friendly error messages in frontend
• Retry logic for failed requests

Step 4.4: Testing

• API endpoint tests with pytest
• Component tests with React Testing Library

Key API Endpoints Summary

Method	Endpoint	Description
POST	/api/upload	Upload CSV/Excel file
POST	/api/parse/start	Start parsing job
GET	/api/parse/status/{job_id}	Get parsing status
GET	/api/parse/result/{job_id}	Get parsed data
POST	/api/analyze/start	Start analysis job
GET	/api/analyze/clusters/{job_id}	Get cluster results
GET	/api/analyze/embeddings/{job_id}	Get 2D embeddings
POST	/api/search/rerank	Semantic search
GET	/api/magnetic/stations	List stations
POST	/api/magnetic/correlate	Run correlation
GET	/api/map/sightings	Get sighting GeoJSON
GET	/api/map/bases	Get bases GeoJSON

Dependencies to Add

Backend (add to pyproject.toml)

fastapi
uvicorn[standard]
python-multipart
aiofiles
websockets

Frontend (package.json)

{
  "dependencies": {
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-router-dom": "^6.x",
    "axios": "^1.x",
    "plotly.js": "^2.x",
    "react-plotly.js": "^2.x",
    "kepler.gl": "^3.x",
    "react-dropzone": "^14.x",
    "@tanstack/react-table": "^8.x",
    "zustand": "^4.x",
    "react-hot-toast": "^2.x"
  },
  "devDependencies": {
    "vite": "^5.x",
    "tailwindcss": "^3.x",
    "autoprefixer": "^10.x",
    "postcss": "^8.x"
  }
}

Running the Application

# Terminal 1: Start FastAPI backend
cd api && uvicorn main:app --reload --port 8000

# Terminal 2: Start React frontend
cd frontend && npm run dev

Notes

• Long-running tasks (parsing, analysis) use background jobs with polling or WebSocket updates
• Embeddings are stored server-side and referenced by job_id to avoid large payloads
• Visualizations are generated as Plotly JSON for interactive frontend rendering
• The existing uap_analyzer.py and utils/ modules are reused as services