pages/12_Logistic_Regression.py

import streamlit as st

st.set_page_config(page_title="Logistic Regression", page_icon="📌", layout="wide")

st.sidebar.title("🤖 Logistic Regression")
st.sidebar.markdown("Explore the theory behind Logistic Regression step-by-step.")
st.sidebar.markdown("---")

st.markdown("<h1 style='text-align: center;'>📈 Logistic Regression (Theory)</h1>", unsafe_allow_html=True)

section = st.radio(
    "📚 Choose a Topic",
    [
        "Introduction",
        "Step vs Sigmoid",
        "Loss Function",
        "Gradient Descent",
        "Multiclass Logistic Regression",
        "Regularization",
        "Multicollinearity",
        "Hyperparameters",
        "Colab Notebook"
    ],
    horizontal=True
)

# Section Content
if section == "Introduction":
    st.header("📘 What is Logistic Regression?")
    st.write("""
    Logistic Regression is a **supervised learning algorithm** used for **classification tasks**.
    - Mostly used for **binary classification**
    - Can be extended to **multi-class** using techniques like **Softmax** or **One-vs-Rest (OvR)**

    🔍 The goal is to find a **hyperplane** that linearly separates the classes.
    ⚠️ Assumption: The data should be **linearly separable** or nearly so.
    """)

elif section == "Step vs Sigmoid":
    st.header("⚙️ Step vs Sigmoid Function")
    st.write("""
    Initially, people used **step function** for classification:

    ```text
    Step Function Output:
    if z >= 0 → Class 1
    if z <  0 → Class 0
    ```

    ❌ But it's not differentiable → can't optimize via gradient descent.  
    ✅ Instead, we use the **Sigmoid Function**:
    $$
    \\sigma(z) = \\frac{1}{1 + e^{-z}} \\quad \\text{where } z = w^T x + b
    $$
    Properties:
    - \\( z >> 0 \\) → output ≈ 1  
    - \\( z << 0 \\) → output ≈ 0  
    - \\( z = 0 \\) → output = 0.5
    """)

elif section == "Loss Function":
    st.header("🧮 Loss Function in Logistic Regression")
    st.write("""
    Logistic Regression uses **Log Loss** (or Binary Cross Entropy) for optimization:
    $$
    \\mathcal{L} = - \\left[y \\log(\\hat{y}) + (1 - y) \\log(1 - \\hat{y}) \\right]
    $$
    - \\( y \\) is the actual class (0 or 1)  
    - \\( \\hat{y} \\) is the predicted probability from sigmoid  
    - Minimize this loss using **Gradient Descent**
    """)

elif section == "Gradient Descent":
    st.header("🔁 Gradient Descent & Learning Rate")
    st.write("""
    Gradient Descent updates weights to minimize loss:
    $$
    w = w - \\alpha \\cdot \\frac{\\partial \\mathcal{L}}{\\partial w}
    $$

    - \\( \\alpha \\): Learning rate  
    ⚖️ Choosing Learning Rate:
    - Too high → overshoots  
    - Too low → slow convergence  
    - Recommended: 0.01 or 0.1 for starters

    📦 Types:
    - **Batch**: Uses full dataset each update  
    - **Stochastic (SGD)**: Updates per data point  
    - **Mini-Batch**: Updates per small batch → widely used
    """)

elif section == "Multiclass Logistic Regression":
    st.header("🔢 Multiclass Logistic Regression")

    st.subheader("1️⃣ Softmax Regression")
    st.write("""
    Generalizes binary logistic regression to multi-class using **Softmax Function**:
    $$
    \\text{Softmax}(z_i) = \\frac{e^{z_i}}{\\sum_j e^{z_j}}
    $$
    Each class gets a probability, and we pick the class with the highest.  
    Example: If predicted probabilities are:

    ```text
    [0.02, 0.05, 0.07, 0.10, 0.08, 0.12, 0.10, 0.30, 0.05, 0.11]
    ```

    → Predict class **7** (highest probability)
    """)

    st.subheader("2️⃣ One-vs-Rest (OvR) Classification")
    st.write("""
    OvR builds one binary classifier **per class**:
    - Apple vs Not-Apple  
    - Banana vs Not-Banana  
    - Orange vs Not-Orange  

    Then pick the class with the highest confidence score.
    """)

elif section == "Regularization":
    st.header("🧲 Regularization in Logistic Regression")
    st.write("""
    Regularization helps reduce **overfitting** by penalizing large weights.
    - **L1 (Lasso)**: \\( \\lambda \\sum |w| \\) → Feature selection  
    - **L2 (Ridge)**: \\( \\lambda \\sum w^2 \\) → Smooths weights  
    - **ElasticNet**: Combines L1 + L2

    ⚖️ Helps balance **bias vs variance**
    """)

elif section == "Multicollinearity":
    st.header("🔍 Detecting Multicollinearity")
    st.write("""
    Multicollinearity = when predictors are highly correlated.  
    🎯 Check using **VIF (Variance Inflation Factor)**:
    - \\( \\text{VIF} > 10 \\) → High multicollinearity

    ✅ Reduce it by:
    - Removing correlated features  
    - Applying regularization
    """)

elif section == "Hyperparameters":
    st.header("⚙️ Hyperparameters in Logistic Regression")
    st.table([
        ["penalty", "Regularization type ('l1', 'l2', 'elasticnet', None)"],
        ["dual", "Use dual formulation (for liblinear only)"],
        ["tol", "Tolerance for stopping criteria"],
        ["C", "Inverse of regularization strength"],
        ["fit_intercept", "Whether to include the bias term"],
        ["solver", "Optimization algorithm ('lbfgs', 'saga', etc.)"],
        ["multi_class", "'ovr' or 'multinomial' for multi-class"],
        ["max_iter", "Max iterations for convergence"],
        ["class_weight", "'balanced' or manual dictionary"],
        ["n_jobs", "Number of parallel jobs"],
        ["l1_ratio", "Used with ElasticNet"],
    ])

elif section == "Colab Notebook":
    st.header("📓 Google Colab Notebook")
    st.markdown("""
    <a href='https://colab.research.google.com/drive/1IiZmfkCcMltcE5-r5PbQqRpj_maR_M-f?usp=sharing' target='_blank'>
    🔗 Open Logistic Regression Notebook
    </a>
    """, unsafe_allow_html=True)

st.markdown("---")
st.success("Logistic Regression is simple yet powerful. Master it before diving into complex models!")