codex_app_analysis.md

# UAP Analysis App: Improvement and Debugging Notes

## Scope Reviewed
- Backend API: `backend/main.py`
- Frontend app/store/pages: `frontend/src/**`
- Project positioning/features: `README.md`

## High-Priority Debugging and Reliability Issues

### 1) Global in-memory backend state is shared across all users/sessions
- File: `backend/main.py`
- Current behavior: `state` is a single process-level dictionary storing dataset, filtered data, analysis results, and query context.
- Risk:
  - Cross-user data leakage.
  - Race conditions (one user can overwrite another user’s data mid-session).
  - Non-deterministic behavior in multi-worker deployment.
- Improvement:
  - Introduce per-session/project IDs and isolate state in Redis or a DB cache keyed by session.
  - Add dataset ownership + TTL cleanup.

### 2) Dashboard “Analysis Runs” counter is effectively broken
- File: `backend/main.py`
- Current behavior: `/api/dashboard/summary` reports `analyzed_columns` from `state["col_names"]`, but `col_names` is never populated in `run_analysis`.
- Impact: dashboard can show incorrect/always-zero run stats.
- Improvement:
  - Set `state["col_names"] = req.columns` (validated columns only) during analysis.
  - Consider storing analysis timestamp and run count.

### 3) Numeric filtering ignores partial ranges
- Files: `backend/main.py`, `frontend/src/components/data/FilterPanel.tsx`
- Current behavior: numeric filter only applies when both `min_val` and `max_val` are provided.
- Impact: user-provided minimum-only or maximum-only constraints do nothing.
- Improvement:
  - Support all combinations: min-only, max-only, and bounded range.

### 4) Filtering error handling is silent on frontend
- File: `frontend/src/components/data/DataExplorer.tsx`
- Current behavior: `handleFilter` catches errors and suppresses them.
- Impact: user cannot tell whether filtering failed or returned no matches.
- Improvement:
  - Surface backend error in UI (same pattern used by upload/load errors).

### 5) Backend query endpoint has expensive prompt construction and token-risk
- File: `backend/main.py`
- Current behavior: concatenates up to 500 rows of full text into one prompt string.
- Risks:
  - Large latency/cost spikes.
  - Prompt truncation or model failure on long text columns.
- Improvement:
  - Add token-aware chunking/sampling and optional map-reduce summarization.
  - Enforce max characters/tokens per request with clear user feedback.

### 6) CORS config is overly permissive and potentially invalid for credentials
- File: `backend/main.py`
- Current behavior: `allow_origins=["*"]` with `allow_credentials=True`.
- Risk: browser credential behavior can be inconsistent; security posture is weak for production.
- Improvement:
  - Use explicit origin allowlist per environment.
  - Keep credentials disabled unless required.

## Data-Analysis Quality Gaps (Core Product)

### 7) Analysis pipeline is currently mock/simulated, not aligned with README claims
- File: `backend/main.py`
- Current behavior: analysis uses value counts + random 2D points + correlation proxy for “XGBoost-like” output.
- Impact: users may interpret synthetic outputs as real model outputs.
- Improvement:
  - Label this mode explicitly as `demo/mock` in API/UI.
  - Add a production pipeline path using real embeddings + UMAP/HDBSCAN + trained model artifacts.

### 8) Cluster assignment is too coarse for high-cardinality text fields
- File: `backend/main.py`
- Current behavior: top 32 frequent values become “clusters”; all others mapped to `Other`.
- Impact: weak signal extraction for long-tail UAP narratives.
- Improvement:
  - Use embedding-based similarity clustering with min-cluster-size tuning.
  - Keep top terms as labels only, not as cluster definitions.

### 9) “XGBoost” results are correlation-based placeholders
- File: `backend/main.py`
- Current behavior: feature importance derived from absolute correlation among category codes, with random “accuracy”.
- Impact: misleading ML interpretation.
- Improvement:
  - Either rename section (`Association Importance`) or run real train/validation with metrics and confidence intervals.

### 10) Cramer’s V stability safeguards are minimal
- File: `backend/main.py`
- Current behavior: exceptions are swallowed to `0.0` values.
- Impact: matrix can hide data-quality problems.
- Improvement:
  - Return diagnostics (insufficient contingency size, sparse table warning, low sample counts).

## UX and Feature Improvements for Analysis Workflows

### 11) Add reproducibility controls
- Current gap: random projections are generated without surfaced seed controls; pipeline details are hidden.
- Improvement:
  - UI inputs for random seed and analysis config profile.
  - Persist configuration alongside results.

### 12) Add time/location-first analysis modules
- Context: UAP datasets are usually spatiotemporal.
- Improvement ideas:
  - Temporal anomaly detection (daily/weekly trend breaks).
  - Geo heatmaps + hotspot evolution over time.
  - Co-occurrence matrices for shape/light/motion features.

### 13) Add model/result provenance panel
- Improvement:
  - Track dataset hash, row count after filters, analysis timestamp, selected columns, pipeline version.
  - Show this metadata in Analysis and export payloads.

### 14) Improve filter capabilities for real EDA
- Current gap: categorical filter relies on top-values list and cannot easily search rare categories.
- Improvement:
  - Add searchable categorical picker and “include nulls/exclude nulls”.
  - Add reusable saved filter presets.

### 15) Add export/reporting features
- Improvement:
  - Export filtered dataset, correlation matrix, and feature-importance JSON/CSV.
  - One-click markdown/PDF report with charts and configuration metadata.

## Engineering Quality and Maintainability

### 16) Add automated tests for core API behaviors
- Suggested minimal suite:
  - `/api/data/load`, `/api/data/filter`, `/api/analyze/run`, `/api/dashboard/summary`, `/api/query/gemini` failure paths.
  - Numeric filter edge cases (min-only/max-only).
  - State isolation once sessionized.

### 17) Add request/analysis observability
- Improvement:
  - Structured logging + request IDs.
  - Timing metrics per stage (load/filter/analyze/query).
  - Distinguish user errors (4xx) from pipeline errors (5xx).

### 18) Clarify mode separation: demo vs production
- Improvement:
  - Feature flags/environment variable to select mock vs full analysis backend.
  - UI badges and warning copy to prevent scientific misinterpretation.

## Suggested Implementation Order
1. Fix state isolation and dashboard run-count correctness.
2. Fix filtering behavior + frontend error surfacing.
3. Mark current analysis mode as mock and rename misleading outputs.
4. Add reproducibility/provenance metadata and exports.
5. Introduce real embedding + clustering + model pipeline behind a feature flag.