docs/code_explanation.md

# Codebase Explanation & Walkthrough

This document provides a detailed technical explanation of the "AI Stock Prediction System" (Stockker). Use this to understand the underlying logic and architecture.

## 1. System Architecture (The Big Picture)
The system operates as a **Microservices-based Pipeline** with four distinct stages:
1.  **Ingestion Layer**: Fetches raw market data (Open, High, Low, Close, Volume) from Alpha Vantage.
2.  **Processing Layer**: Transforms raw data into technical indicators (Features).
3.  **Training Orchestration**: A Prefect pipeline that trains, evaluates, and saves models.
4.  **Inference API**: A FastAPI server that loads these models to provide real-time predictions.

## 2. Key Components Explained

### A. Data Ingestion & Processing
**File:** `src/processing/features.py`
Feature engineering is critical for financial ML. We treat the market data as a time-series problem.
-   **SMA (Simple Moving Average)**: Calculates the trend over 20 and 50 days.
-   **RSI (Relative Strength Index)**: A momentum oscillator (0-100) to identify overbought/oversold conditions.
-   **MACD (Moving Average Convergence Divergence)**: Tracks momentum changes.
-   **Target Variables**:
    -   `target_price`: Next day's closing price (Regression).
    -   `target_direction`: 1 (Up) or 0 (Down) for next day (Classification).

### B. Machine Learning Models (Ensemble Approach)
**File:** `src/models/train.py`
Instead of relying on a single algorithm, we use **Ensemble Learning** for robustness.

#### 1. Regression (Predicting Price)
We use a **Voting Regressor**, which averages predictions from three models:
-   **Linear Regression**: Captures simple linear trends.
-   **Random Forest Regressor**: Captures complex, non-linear patterns (100 Decision Trees).
-   **SVR (Support Vector Regressor)**: Uses an RBF kernel to find the optimal hyperplane in high-dimensional space.
*Why?* Combining these reduces the variance and error of any single model.

#### 2. Classification (Predicting Direction)
We use a **Voting Classifier** (Soft Voting) combining:
-   **Random Forest Classifier**: Robust against overfitting.
-   **SVC (Support Vector Classifier)**: Good at separating classes with clear margins.
*Soft Voting* means we average the *probabilities* of each model class, not just their final votes, leading to more nuanced predictions.

#### 3. Unsupervised Learning (Market Analysis)
-   **K-Means Clustering**: Groups market days into 3 clusters (e.g., Low Volatility, High Volatility) based on volatility and RSI.
-   **PCA**: Reduces our 4 dimensions (SMA20, SMA50, RSI, MACD) into 2 principal components for 2D visualization.

### C. Orchestration (The Pipeline)
**File:** `src/orchestration/flows.py`
We use **Prefect** to manage the workflow.
-   **`main_pipeline`**: Loops through our stock list (`AAPL`, `GOOGL`, `MSFT`, `AMZN`, `TSLA`, `NVDA`).
-   For each stock, it sequentially runs: Fetch -> Process -> Train -> Evaluate -> Notify Discord.
-   **Error Handling**: If one stock fails, the pipeline logs the error (via Discord) and continues to the next, ensuring resilience.

### D. The API (Model Serving)
**File:** `src/api/main.py`
-   **Dynamic Loading**: On startup (`@app.on_event("startup")`), the API scans the `models/` directory. It dynamically loads whatever models it finds (e.g., `models/NVDA/regression_model.pkl`), making the system easily extensible to new stocks without changing code.
-   **Enpoints**: Exposes REST endpoints (`/predict/price`, `/predict/direction`) that the frontend consumes.

## 3. Infrastructure & DevOps

### Docker
**File:** `Dockerfile` & `docker-compose.yml`
-   We containerize the application to ensure it runs identically on your laptop and the cloud.
-   The `docker-compose` setup includes a Postgres service, which is used **exclusively by Prefect** to store flow run history. The main app uses a **file-based system** (CSVs/PKLs) for simplicity and portability.

### CI/CD (GitHub Actions)
**File:** `.github/workflows/deploy_to_hf.yml`
-   On every push to `main`, GitHub Actions automatically:
    1.  Runs `pytest` to verify code correctness.
    2.  Pushes the code to the Hugging Face Space, triggering a new deployment.

## 4. Why This Architecture?
-   **Modularity**: Separation of concerns (Ingestion vs Training vs Serving) makes debugging easy.
-   **Scalability**: Adding a new stock (like NVDA) only required adding a string to the list; the pipeline handled the rest.
-   **Reliability**: Ensembles prevent "putting all eggs in one basket" model-wise.