claude.md

# NBA Sage - Complete Codebase Context for AI Assistants

> **Purpose**: This document provides comprehensive context for AI assistants to understand the NBA Sage prediction system. Read this entire document before making any code modifications.

---

## 🏗️ Project Architecture Overview

```
NBA ML/
├── server.py              # Main production server (Flask + React)
├── api/api.py             # Development API server
│
├── src/                   # Core Python modules
│   ├── prediction_pipeline.py   # Main prediction orchestrator
│   ├── feature_engineering.py   # ELO + feature generation
│   ├── data_collector.py        # Historical NBA API data
│   ├── live_data_collector.py   # Real-time game data
│   ├── injury_collector.py      # Player injury tracking
│   ├── prediction_tracker.py    # ChromaDB prediction storage
│   ├── auto_trainer.py          # Automated training scheduler
│   ├── continuous_learner.py    # Incremental model updates
│   ├── preprocessing.py         # Data preprocessing
│   ├── config.py                # Global configuration
│   └── models/
│       ├── game_predictor.py    # XGBoost+LightGBM ensemble
│       ├── mvp_predictor.py     # MVP prediction model
│       └── championship_predictor.py
│
├── web/                   # React Frontend
│   └── src/
│       ├── App.jsx        # Main app with sidebar navigation
│       ├── pages/         # LiveGames, Predictions, MVP, etc.
│       ├── api.js         # API client
│       └── index.css      # Comprehensive CSS design system
│
├── data/
│   ├── api_data/          # Cached NBA API responses (parquet)
│   ├── processed/         # Processed datasets (joblib)
│   └── raw/               # Raw game data
│
└── models/
    └── game_predictor.joblib  # Trained ML model
```

---

## 🔄 Data Flow

```
NBA API ──► Data Collectors ──► Feature Engineering ──► ML Training
                                       │
                                       ▼
Live API ──► Live Collector ──► Prediction Pipeline ──► Flask API ──► React UI
                                       │
                                       ▼
                              Prediction Tracker (ChromaDB)
```

---

## 📦 Core Components Deep Dive

### 1. `server.py` - Production Server (39KB, 929 lines)

**Critical for Hugging Face deployment. Combines Flask API + React static serving.**

**Key Sections:**
- **Cache Configuration (lines 30-40)**: In-memory caching for rosters, predictions, live games
- **Startup Cache Warming (lines 140-225)**: `warm_starter_cache()` fetches all 30 team rosters on startup
- **Background Scheduler (lines 340-370)**: APScheduler jobs for ELO updates, retraining, prediction sync
- **API Endpoints (lines 400-860)**: All REST endpoints for frontend

**Important Functions:**
```python
warm_starter_cache()      # Fetches real NBA API data for all teams
startup_cache_warming()   # Runs synchronously on server start
auto_retrain_model()      # Smart retraining after all daily games complete
sync_prediction_results() # Updates prediction correctness from final scores
update_elo_ratings()      # Daily ELO recalculation
```

**Endpoints:**
- `GET /api/live-games` - Today's games with predictions
- `GET /api/roster/<team>` - Team's projected starting 5
- `GET /api/accuracy` - Model accuracy statistics
- `GET /api/mvp` - MVP race standings
- `GET /api/championship` - Championship odds

---

### 2. `prediction_pipeline.py` - Prediction Orchestrator (41KB, 765 lines)

**The heart of the system. Orchestrates all predictions.**

**Key Properties:**
```python
self.live_collector      # LiveDataCollector instance
self.injury_collector    # InjuryCollector instance
self.feature_gen         # FeatureGenerator instance
self.tracker             # PredictionTracker (ChromaDB)
self._game_model         # Lazy-loaded GamePredictor
```

**Important Methods:**

| Method | Purpose |
|--------|---------|
| `predict_game(home, away)` | Generate single game prediction |
| `get_upcoming_games(days)` | Fetch future NBA schedule |
| `get_mvp_race()` | Calculate MVP standings from live stats |
| `get_championship_odds()` | Calculate championship probabilities |
| `get_team_roster(team)` | Fast fallback roster data |

**⚠️ CRITICAL: Prediction Algorithm (lines 349-504)**

The `predict_game()` method uses a **formula-based approach**, NOT the trained ML model:

```python
# Weights in predict_game():
home_talent = (
    0.40 * home_win_pct +      # Current season record
    0.30 * home_form +          # Last 10 games
    0.20 * home_elo_strength +  # Historical ELO
    0.10 * 0.5                  # Baseline
)
# Plus: +3.5% home court, -2% per injury point
# Uses Log5 formula for head-to-head probability
```

The trained `GamePredictor` model exists but is NOT called for live predictions.

---

### 3. `feature_engineering.py` - Feature Generation (29KB, 696 lines)

**Contains ELO system and all feature generation logic.**

**Classes:**

| Class | Purpose | Key Methods |
|-------|---------|-------------|
| `ELOCalculator` | ELO rating system | `update_ratings()`, `calculate_game_features()` |
| `EraNormalizer` | Z-score normalization across seasons | `fit_season()`, `transform()` |
| `StatLoader` | Load all stat types | `get_team_season_stats()`, `get_team_top_players_stats()` |
| `FeatureGenerator` | Main feature orchestrator | `generate_game_features()`, `generate_features_for_dataset()` |

**ELO Configuration:**
```python
initial_rating = 1500
k_factor = 20
home_advantage = 100  # ELO points for home court
regression_factor = 0.25  # Season regression to mean
```

**Feature Types Generated:**
- ELO features (team_elo, opponent_elo, elo_diff, elo_expected_win)
- Rolling averages (5, 10, 20 game windows)
- Rest days, back-to-back detection
- Season record features
- Head-to-head history

---

### 4. `data_collector.py` - Historical Data (27KB, 650 lines)

**Collects comprehensive NBA data from official API.**

**Classes:**
| Class | Data Collected |
|-------|---------------|
| `GameDataCollector` | Game results per season |
| `TeamDataCollector` | Team stats (basic, advanced, clutch, hustle, defense) |
| `PlayerDataCollector` | Player stats |
| `CacheManager` | Parquet file caching |

**Key Features:**
- Exponential backoff retry for rate limiting
- Per-season parquet caching
- Checkpoint system for resumable collection

---

### 5. `live_data_collector.py` - Real-Time Data (9KB, 236 lines)

**Uses `nba_api.live` endpoints for real-time game data.**

**Key Methods:**
```python
get_live_scoreboard()  # Today's games with live scores
get_game_boxscore(id)  # Detailed box score
get_games_by_status() # Filter: NOT_STARTED, IN_PROGRESS, FINAL
```

**Data Fields Returned:**
- game_id, game_code
- home_team, away_team (tricodes)
- home_score, away_score
- period, clock
- status

---

### 6. `prediction_tracker.py` - Persistence (20KB, 508 lines)

**Stores predictions and tracks accuracy using ChromaDB Cloud.**

**Features:**
- ChromaDB Cloud integration (with local JSON fallback)
- Prediction storage before games start
- Result updating after games complete
- Comprehensive accuracy statistics

**Key Methods:**
```python
save_prediction(game_id, prediction)  # Store pre-game prediction
update_result(game_id, winner, scores)  # Update with final result
get_accuracy_stats()  # Overall, by confidence, by team
get_pending_predictions()  # Awaiting results
```

---

### 7. `models/game_predictor.py` - ML Model (12KB, 332 lines)

**XGBoost + LightGBM ensemble classifier.**

**Architecture:**
```
Input Features ──┬──► XGBoost ──┐
                 │              │──► Weighted Average ──► Win Probability
                 └──► LightGBM ─┘
                      (50/50 weight)
```

**Key Methods:**
```python
train(X_train, y_train, X_val, y_val)  # Train both models
predict_proba(X)  # Get [loss_prob, win_prob]
predict_with_confidence(X)  # Detailed prediction info
explain_prediction(X)  # Feature importance for prediction
save() / load()  # Persist to models/game_predictor.joblib
```

**⚠️ NOTE: Model exists but `predict_game()` doesn't use it!**

---

### 8. `auto_trainer.py` & `continuous_learner.py` - Auto Training

**AutoTrainer** (Singleton scheduler):
- Runs background loop checking for tasks
- Ingests completed games every hour
- Smart retraining: only after ALL daily games complete
- If new accuracy < old accuracy, reverts model

**ContinuousLearner** (Update workflow):
```
ingest_completed_games() ──► update_features() ──► retrain_model()
```

---

## 🗄️ Database & Storage

### ChromaDB Cloud
- **Purpose**: Persistent prediction storage
- **Credentials**: Set via environment variables (`CHROMA_TENANT`, `CHROMA_DATABASE`, `CHROMA_API_KEY`)
- **Fallback**: `data/processed/predictions_local.json`

### Parquet Files
- `data/api_data/*.parquet` - Cached API responses
- `data/api_data/all_games_summary.parquet` - Consolidated game history (41K+ games)

### Joblib Files
- `models/game_predictor.joblib` - Trained ML model
- `data/processed/game_dataset.joblib` - Processed training data

---

## 🌐 Frontend Architecture

**React + Vite with custom CSS design system.**

**Pages:**
| Page | File | Purpose |
|------|------|---------|
| Live Games | `LiveGames.jsx` | Today's games, live scores, predictions |
| Predictions | `Predictions.jsx` | Upcoming games with predictions |
| Head to Head | `HeadToHead.jsx` | Compare two teams |
| Accuracy | `Accuracy.jsx` | Model performance stats |
| MVP Race | `MvpRace.jsx` | Current MVP standings |
| Championship | `Championship.jsx` | Championship odds |

**Key Frontend Components:**
- `TeamLogo.jsx` - Official NBA team logos
- `api.js` - API client with base URL handling
- `index.css` - Complete design system (27KB)

---

## 🔧 Configuration (`src/config.py`)

**Critical Settings:**
```python
# NBA Teams mapping (team_id -> tricode)
NBA_TEAMS = {1610612737: "ATL", 1610612738: "BOS", ...}

# Data paths
API_CACHE_DIR = Path("data/api_data")
PROCESSED_DATA_DIR = Path("data/processed")
MODELS_DIR = Path("models")

# Feature engineering
FEATURE_CONFIG = {
    "rolling_windows": [5, 10, 20],
    "min_games_for_features": 5
}

# ELO system
ELO_CONFIG = {
    "initial_rating": 1500,
    "k_factor": 20,
    "home_advantage": 100
}
```

---

## ⚠️ Known Issues & Technical Debt

1. **ML Model Not Used**: `predict_game()` uses formula, not trained `GamePredictor`
2. **Season Hardcoding**: Some places use `2025-26` explicitly
3. **Fallback Data**: Pipeline has hardcoded rosters as backup
4. **Function Order**: `warm_starter_cache()` must be defined before scheduler calls it

---

## 🚀 Deployment Notes

**Hugging Face Spaces:**
- Uses persistent `/data` directory for storage
- Dockerfile copies `models/` and `data/api_data/` 
- Git LFS for large files (`.joblib`, `.parquet`)
- Port 7860 for HF Spaces

**Environment Variables:**
```
CHROMA_TENANT, CHROMA_DATABASE, CHROMA_API_KEY  # ChromaDB
NBA_ML_DATA_DIR, NBA_ML_MODELS_DIR  # Override paths
```

---

## 📋 Quick Reference: Common Tasks

**Add new API endpoint:**
1. Add route in `server.py` (production) AND `api/api.py` (development)
2. Add frontend call in `web/src/api.js`
3. Create/update page component in `web/src/pages/`

**Modify prediction algorithm:**
1. Edit `PredictionPipeline.predict_game()` in `prediction_pipeline.py`
2. Consider blending with `GamePredictor` model

**Update ML model:**
1. Retrain via `ContinuousLearner.retrain_model()`
2. Or trigger via `POST /api/admin/retrain`

**Add new feature:**
1. Add to `FeatureGenerator` in `feature_engineering.py`
2. Update preprocessing pipeline
3. Retrain model

---

*Last updated: January 2026*