Architecture.md

# Traffic Control Reinforcement Learning Architecture

This document provides a comprehensive A-Z breakdown of the Reinforcement Learning (RL) architecture implemented in the Traffic Signal Control project. 

The system replaces traditional fixed-timing traffic lights with intelligent, adaptive agents that learn optimal signal switching policies. **It features a full-stack React and FastAPI Web Dashboard** to visualize the intersection in real-time.

---

## 1. High-Level System Flow

The architecture follows the standard Reinforcement Learning MDP (Markov Decision Process) loop:

1. **Observe**: The Agent receives the current **State** (queue lengths for 8 lanes, current light phase) from the Environment.
2. **Decide**: The Agent selects an **Action** (keep the current phase OR switch to the next phase).
3. **Act**: The Environment executes the action, simulating traffic flow for a specific duration.
4. **Learn**: The Environment returns a **Reward** (penalty based on total waiting traffic). The Agent uses this to update its neural network.

---

## 2. The Environment (`TrafficEnvironment`)

The environment is the simulated 8-lane intersection built using the `Gymnasium` API.

### 2.1 State Space (Observation)
We use a continuous **9-dimensional state vector** to perfectly track Straight/Right (SR) and Left-Turn (L) queues:
*   `[N_SR, N_L, E_SR, E_L, S_SR, S_L, W_SR, W_L, current_phase]`
*   **Absolute Normalization**: Queue lengths are normalized linearly by dividing by `20.0` and clipping to `[0, 1]`. This ensures the neural network correctly perceives absolute traffic volume.
*   **Phase Representation**: The current phase (0 to 3) is normalized to `phase / 3.0`.

### 2.2 Action Space & 4-Phase Cycle
The agent has a discrete action space of size 2 (Keep or Switch). It cycles through **4 Directional Phases** to completely eliminate turning collisions:
*   `Phase 0`: North Green (Straight + Left)
*   `Phase 1`: East Green (Straight + Left)
*   `Phase 2`: South Green (Straight + Left)
*   `Phase 3`: West Green (Straight + Left)

### 2.3 Reward Function
*   **Calculation**: `reward = -(Total Queue Length) / 20.0`
*   **Clipping**: The reward is hard-clipped to `[-1.0, 1.0]`. 
*   **Sensitivity**: Dividing by 20.0 ensures the agent receives strong gradient signals even in low-density traffic conditions, forcing it to actively clear small queues.

---

## 3. Traffic Generation (`TrafficGenerator`)

*   **Realistic Volume**: The traffic density is tuned to produce roughly 2,000–4,000 total arrivals per episode.
*   **Left-Turn Probabilities**: Roughly 20% of generated traffic is routed into the dedicated Left-Turn queues.
*   **Stochastic Bursts**: There is a 15% probability that a sudden "burst" of traffic will arrive in a random lane, testing the agent's adaptability.

---

## 4. The Deep Q-Network Agent (`DQNAgent`)

A modern Deep RL approach utilizing PyTorch.
*   **Neural Network Architecture**: 
    `Input Layer (9)` -> `Hidden Layer (256, ReLU)` -> `Hidden Layer (256, ReLU)` -> `Output Layer (2, Keep/Switch)`
*   **Experience Replay Buffer**: Transitions `(s, a, r, s', done)` are stored in a circular buffer (size: 50,000). The network trains by sampling random mini-batches (size 256).
*   **Target Network**: Uses an Online and Target network synced every 10 episodes to stabilize training.
*   **Hardware Acceleration**: Automatically utilizes CUDA (NVIDIA GPUs) for accelerated tensor operations.

---

## 5. Web Dashboard Architecture

The project features a beautiful full-stack simulation dashboard.

### 5.1 Backend (`backend/app.py`)
*   Built with **FastAPI**.
*   Loads the fully trained `dqn_best.pth` model into memory.
*   Exposes `/api/reset` and `/api/step` endpoints.
*   When the frontend calls `/api/step`, the backend asks the DQN agent for an action, steps the Python Gymnasium environment, and returns the 9D State and reward back to the UI.

### 5.2 Frontend (`frontend/src/App.jsx`)
*   Built with **React and Vite**.
*   **Ultra-Premium Aesthetics**: Features a dark glassmorphism UI, a glowing neon background, and etched asphalt road textures.
*   **Live Telemetry**: Tracks total throughput and RL reward signals in real-time.
*   **CSS Animations**: Cars are dynamically rendered in all 8 lanes and visually animate driving straight or turning left (-90deg rotation) when their specific directional light turns green.