Spaces:

AashishAIHub
/

DataScience

Running

App Files Files Community

AashishAIHub commited on 23 days ago

Commit

c2b5d4f

1 Parent(s): 939a3b2

feat: Enhanced Feature Engineering module with MathJax and Python code

Browse files

Files changed (1) hide show

feature-engineering/index.html +501 -38

feature-engineering/index.html CHANGED Viewed

@@ -1,11 +1,25 @@
 <!DOCTYPE html>
 <html lang="en">
 <head>
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <title>Feature Engineering Explorer</title>
   <link rel="stylesheet" href="style.css" />
 </head>
 <body>
   <div class="app flex">
     <!-- Sidebar Navigation -->
@@ -33,26 +47,49 @@
       <!-- ============================ 1. INTRO ============================ -->
       <section id="intro" class="topic-section">
         <h2>Introduction to Feature Engineering</h2>
-        <p>Feature Engineering is the process of transforming raw data into meaningful inputs that boost machine-learning model performance. A well-crafted feature set can improve accuracy by 10-30% without changing the underlying algorithm.</p>
         <div class="info-card">
-          <strong>Key Idea:</strong> 💡 Thoughtful features provide the model with clearer patterns, like lenses sharpening a blurry picture.
         </div>
         <!-- Canvas Visual -->
         <div class="canvas-wrapper">
           <canvas id="canvas-intro" width="600" height="280"></canvas>
         </div>
       </section>
       <!-- ====================== 2. HANDLING MISSING DATA ================== -->
       <section id="missing-data" class="topic-section">
         <h2>Handling Missing Data</h2>
-        <p>Missing values come in three flavors: MCAR (Missing Completely At Random), MAR (Missing At Random), and MNAR (Missing Not At Random). Each demands different treatment to avoid bias.</p>
         <!-- Real-world Example -->
         <div class="info-card">
-          <strong>Real Example:</strong> A hospital's patient records often have absent <em>cholesterol</em> values because certain tests were not ordered for healthy young adults.
         </div>
         <!-- Controls -->
@@ -70,13 +107,49 @@
         <!-- Callouts -->
         <div class="callout callout--insight">💡 Mean/Median work best when data is MCAR or MAR.</div>
         <div class="callout callout--mistake">⚠️ Using mean imputation on skewed data can distort distributions.</div>
-        <div class="callout callout--tip">✅ Always impute <strong>after</strong> splitting into train and test to avoid leakage.</div>
       </section>
       <!-- ======================= 3. HANDLING OUTLIERS ===================== -->
       <section id="outliers" class="topic-section">
         <h2>Handling Outliers</h2>
-        <p>Outliers are data points that deviate markedly from others. Detecting and treating them prevents skewed models.</p>
         <div class="form-group">
           <button id="btn-detect-iqr" class="btn btn--primary">IQR Method</button>
@@ -90,6 +163,40 @@
         <div class="callout callout--insight">💡 The IQR method is robust to non-normal data.</div>
         <div class="callout callout--mistake">⚠️ Removing legitimate extreme values can erase important signals.</div>
       </section>
       <!-- ========================== 4. SCALING ============================ -->
@@ -106,6 +213,43 @@
         <div class="canvas-wrapper">
           <canvas id="canvas-scaling" width="600" height="300"></canvas>
         </div>
       </section>
       <!-- ========================== 5. ENCODING =========================== -->
@@ -122,6 +266,44 @@
         <div class="canvas-wrapper">
           <canvas id="canvas-encoding" width="600" height="300"></canvas>
         </div>
       </section>
       <!-- ===================== 6. FEATURE SELECTION ======================= -->
@@ -138,6 +320,47 @@
         <div class="canvas-wrapper">
           <canvas id="canvas-selection" width="600" height="300"></canvas>
         </div>
       </section>
       <!-- =================== 7. IMBALANCED DATA =========================== -->
@@ -154,12 +377,56 @@
         <div class="canvas-wrapper">
           <canvas id="canvas-imbalanced" width="600" height="300"></canvas>
         </div>
       </section>
       <!-- ========================== 8. EDA ================================ -->
       <section id="eda" class="topic-section">
         <h2>Exploratory Data Analysis (EDA)</h2>
-        <p><strong>Exploratory Data Analysis (EDA)</strong> is a critical step in the machine learning pipeline that comes BEFORE feature engineering. EDA helps you understand your data, discover patterns, identify anomalies, detect outliers, test hypotheses, and check assumptions through summary statistics and graphical representations.</p>
         <div class="info-card">
           <strong>Key Questions EDA Answers:</strong>
@@ -175,7 +442,8 @@
         </div>
         <div class="info-card">
-          <strong>Real-World Example:</strong> Imagine you're analyzing customer data for a bank to predict loan defaults. EDA helps you understand:
           <ul>
             <li>Age distribution of customers (histogram)</li>
             <li>Income levels (box plot for outliers)</li>
@@ -217,7 +485,8 @@
         <h4>2. Inferential Statistics</h4>
         <p><strong>Purpose:</strong> Make inferences or generalizations about the population from the sample</p>
-        <p><strong>Key Question:</strong> Can we claim this effect exists in the larger population, or is it just by chance?</p>
         <div class="info-card">
           <strong>A. Hypothesis Testing:</strong><br>
@@ -250,7 +519,8 @@
           <strong>3. Analyze Distributions:</strong> Histograms, count plots, box plots<br>
           <strong>4. Check for Imbalance:</strong> Count target classes, plot distribution<br>
           <strong>5. Correlation Analysis:</strong> Correlation matrix, heatmap, identify multicollinearity<br>
-          <strong>6. Statistical Testing:</strong> Compare groups (t-test, ANOVA), test assumptions, calculate effect sizes
         </div>
         <h3>Interactive EDA Dashboard</h3>
@@ -264,7 +534,8 @@
         </div>
         <div class="form-group">
-          <label for="confidenceLevel" class="form-label">Confidence Level: <span id="confidenceValue">95</span>%</label>
           <input type="range" id="confidenceLevel" min="90" max="99" step="1" value="95" class="form-control" />
         </div>
@@ -278,9 +549,57 @@
           <canvas id="canvas-eda" width="800" height="500"></canvas>
         </div>
-        <div class="callout callout--insight">💡 EDA typically takes 30-40% of total project time. Good EDA reveals which features to engineer.</div>
-        <div class="callout callout--mistake">⚠️ Common Mistakes: Skipping EDA, not checking outliers before scaling, ignoring missing value patterns, overlooking class imbalance, ignoring multicollinearity.</div>
-        <div class="callout callout--tip">✅ Best Practices: ALWAYS start with EDA, visualize EVERY feature, check correlations with target, document insights, use both descriptive and inferential statistics.</div>
         <h3>Use Cases and Applications</h3>
         <ul>
@@ -292,19 +611,25 @@
         </ul>
         <h3>Summary &amp; Key Takeaways</h3>
-        <p>Exploratory Data Analysis is the foundation of any successful machine learning project. It combines <strong>descriptive statistics</strong> (mean, median, variance, correlation) with <strong>inferential statistics</strong> (hypothesis testing, confidence intervals) to understand data deeply.</p>
         <p><strong>Descriptive EDA</strong> answers: "What is happening in the dataset?"<br>
-        <strong>Inferential EDA</strong> answers: "Can we claim this effect exists in the larger population?"</p>
         <p>Remember: <strong>Data → EDA → Feature Engineering → ML → Deployment</strong></p>
       </section>
       <!-- =================== 9. FEATURE TRANSFORMATION ==================== -->
       <section id="feature-transformation" class="topic-section">
         <h2>Feature Transformation</h2>
-        <p>Feature transformation creates new representations of data to capture non-linear patterns. Techniques like polynomial features, binning, and mathematical transformations unlock hidden relationships.</p>
         <div class="info-card">
-          <strong>Real Example:</strong> Predicting house prices with polynomial features (adding x² terms) improves model fit for non-linear relationships between square footage and price.
         </div>
         <h3>Mathematical Foundations</h3>
@@ -332,9 +657,50 @@
           <canvas id="canvas-transformation" width="700" height="350"></canvas>
         </div>
-        <div class="callout callout--insight">💡 Polynomial features capture curve fitting, but degree=3 on 10 features creates 286 features!</div>
-        <div class="callout callout--mistake">⚠️ Always scale features after polynomial transformation to prevent magnitude issues.</div>
-        <div class="callout callout--tip">✅ Start with degree=2 and visualize distributions before/after transformation.</div>
         <h3>Use Cases</h3>
         <ul>
@@ -347,10 +713,12 @@
       <!-- =================== 10. FEATURE CREATION ========================= -->
       <section id="feature-creation" class="topic-section">
         <h2>Feature Creation</h2>
-        <p>Creating new features from existing ones based on domain knowledge. Interaction terms, ratios, and domain-specific calculations enhance model performance.</p>
         <div class="info-card">
-          <strong>Real Example:</strong> E-commerce revenue = price × quantity. Profit margin = (selling_price - cost_price) / cost_price. These derived features often have stronger predictive power than raw features.
         </div>
         <h3>Mathematical Foundations</h3>
@@ -379,9 +747,49 @@
           <canvas id="canvas-creation" width="700" height="350"></canvas>
         </div>
-        <div class="callout callout--insight">💡 Interaction terms are especially powerful in linear models - neural networks learn them automatically.</div>
-        <div class="callout callout--mistake">⚠️ Creating features without domain knowledge leads to meaningless combinations.</div>
-        <div class="callout callout--tip">✅ Always check correlation between new and existing features to avoid redundancy.</div>
         <h3>Use Cases</h3>
         <ul>
@@ -394,22 +802,24 @@
       <!-- ================ 11. DIMENSIONALITY REDUCTION ==================== -->
       <section id="dimensionality-reduction" class="topic-section">
         <h2>Dimensionality Reduction</h2>
-        <p>Reducing the number of features while preserving information. PCA (Principal Component Analysis) projects high-dimensional data onto lower dimensions by finding directions of maximum variance.</p>
         <div class="info-card">
-          <strong>Real Example:</strong> Image compression and genome analysis with thousands of genes benefit from PCA. First 2-3 principal components often capture 80%+ of variance.
         </div>
         <h3>PCA Mathematical Foundations</h3>
         <div class="info-card">
           <strong>Algorithm Steps:</strong><br>
-          1. Standardize data: X_scaled = (X - μ) / σ<br>
-          2. Compute covariance matrix: Cov = (1/n) X^T X<br>
-          3. Calculate eigenvalues and eigenvectors<br>
           4. Sort eigenvectors by eigenvalues (descending)<br>
-          5. Select top k eigenvectors (principal components)<br>
-          6. Transform: X_new = X × PC_matrix<br><br>
-          <strong>Explained Variance:</strong> λᵢ / Σλⱼ<br>
           <strong>Cumulative Variance:</strong> Shows total information preserved<br><br>
           <strong>Why PCA Works:</strong><br>
           • Removes correlated features<br>
@@ -428,9 +838,61 @@
           <canvas id="canvas-pca" width="700" height="400"></canvas>
         </div>
-        <div class="callout callout--insight">💡 PCA is unsupervised - it doesn't use the target variable. First PC always captures most variance.</div>
-        <div class="callout callout--mistake">⚠️ Not standardizing before PCA is a critical error - features with large scales will dominate.</div>
-        <div class="callout callout--tip">✅ Aim for 95% cumulative explained variance when choosing number of components.</div>
         <h3>Use Cases</h3>
         <ul>
@@ -452,4 +914,5 @@
   <script src="app.js" defer></script>
 </body>
-</html>

 <!DOCTYPE html>
 <html lang="en">
 <head>
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <title>Feature Engineering Explorer</title>
+  <!-- MathJax for rendering LaTeX formulas -->
+  <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
+  <script>
+    MathJax = {
+      tex: {
+        inlineMath: [['$', '$'], ['\\(', '\\)']]
+      }
+    };
+  </script>
+  <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
   <link rel="stylesheet" href="style.css" />
 </head>
 <body>
   <div class="app flex">
     <!-- Sidebar Navigation -->
       <!-- ============================ 1. INTRO ============================ -->
       <section id="intro" class="topic-section">
         <h2>Introduction to Feature Engineering</h2>
+        <p>Feature Engineering is the process of transforming raw data into meaningful inputs that boost
+          machine-learning model performance. A well-crafted feature set can improve accuracy by 10-30% without changing
+          the underlying algorithm.</p>
         <div class="info-card">
+          <strong>Key Idea:</strong> 💡 Thoughtful features provide the model with clearer patterns, like lenses
+          sharpening a blurry picture.
         </div>
         <!-- Canvas Visual -->
         <div class="canvas-wrapper">
           <canvas id="canvas-intro" width="600" height="280"></canvas>
         </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>setup.py - Pandas Basics</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>import pandas as pd
+import numpy as np
+# Load the dataset
+df = pd.read_csv('housing_data.csv')
+# Inspect raw data types and missing values
+df.info()
+# View summary statistics
+display(df.describe())</code></pre>
+        </div>
       </section>
       <!-- ====================== 2. HANDLING MISSING DATA ================== -->
       <section id="missing-data" class="topic-section">
         <h2>Handling Missing Data</h2>
+        <p>Missing values come in three flavors: MCAR (Missing Completely At Random), MAR (Missing At Random), and MNAR
+          (Missing Not At Random). Each demands different treatment to avoid bias.</p>
         <!-- Real-world Example -->
         <div class="info-card">
+          <strong>Real Example:</strong> A hospital's patient records often have absent <em>cholesterol</em> values
+          because certain tests were not ordered for healthy young adults.
         </div>
         <!-- Controls -->
         <!-- Callouts -->
         <div class="callout callout--insight">💡 Mean/Median work best when data is MCAR or MAR.</div>
         <div class="callout callout--mistake">⚠️ Using mean imputation on skewed data can distort distributions.</div>
+        <div class="callout callout--tip">✅ Always impute <strong>after</strong> splitting into train and test to avoid
+          leakage.</div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Imputation Math</h3>
+          <p><strong>KNN Imputation</strong> predicts missing values by finding the $k$ closest neighbors using a
+            distance metric like Euclidean distance. For two samples $x$ and $y$ with $n$ features, ignoring missing
+            dimensions:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ d(x, y) = \sqrt{\sum_{i=1}^{n} w_i (x_i - y_i)^2} $$
+          </div>
+          <p style="margin-bottom: 0;">Once the $k$ neighbors are found, their values are averaged (or weighted by
+            distance) to fill the missing slot. This preserves local cluster distributions better than global mean
+            imputation.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>missing_data.py - Scikit-Learn Imputers</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from sklearn.impute import SimpleImputer, KNNImputer
+# 1. Simple Imputation (Mean/Median/Most Frequent)
+# Good for MCAR (Missing Completely At Random)
+mean_imputer = SimpleImputer(strategy='mean')
+df['age_imputed'] = mean_imputer.fit_transform(df[['age']])
+# 2. KNN Imputation (Distance-based)
+# Good for MAR (Missing At Random) when variables are correlated
+knn_imputer = KNNImputer(n_neighbors=5, weights='distance')
+df_imputed = knn_imputer.fit_transform(df)
+# Note: Tree-based models like XGBoost can handle NaNs natively!</code></pre>
+        </div>
       </section>
       <!-- ======================= 3. HANDLING OUTLIERS ===================== -->
       <section id="outliers" class="topic-section">
         <h2>Handling Outliers</h2>
+        <p>Outliers are data points that deviate markedly from others. Detecting and treating them prevents skewed
+          models.</p>
         <div class="form-group">
           <button id="btn-detect-iqr" class="btn btn--primary">IQR Method</button>
         <div class="callout callout--insight">💡 The IQR method is robust to non-normal data.</div>
         <div class="callout callout--mistake">⚠️ Removing legitimate extreme values can erase important signals.</div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Outlier Math</h3>
+          <p><strong>Z-Score</strong> measures how many standard deviations $\sigma$ a point is from the mean $\mu$. It
+            assumes the data is normally distributed:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ z = \frac{x - \mu}{\sigma} \quad \text{(Threshold: } |z| > 3 \text{)} $$
+          </div>
+          <p style="margin-bottom: 0;"><strong>IQR (Interquartile Range)</strong> is non-parametric. It defines fences
+            based on the 25th ($Q1$) and 75th ($Q3$) percentiles: $[Q1 - 1.5 \times \text{IQR},\ Q3 + 1.5 \times
+            \text{IQR}]$. <em>Winsorization</em> caps values at these percentiles instead of dropping them.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>outliers.py - Z-Score and Winsorization</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>import numpy as np
+from scipy import stats
+# 1. Z-Score Method (Dropping Outliers)
+z_scores = np.abs(stats.zscore(df['income']))
+# Keep only rows where z-score is less than 3
+df_clean = df[z_scores < 3]
+# 2. IQR Method (Winsorization / Capping)
+# Capping at 5th and 95th percentiles to retain data points
+lower_limit = df['income'].quantile(0.05)
+upper_limit = df['income'].quantile(0.95)
+df['income_capped'] = np.clip(df['income'], lower_limit, upper_limit)</code></pre>
+        </div>
       </section>
       <!-- ========================== 4. SCALING ============================ -->
         <div class="canvas-wrapper">
           <canvas id="canvas-scaling" width="600" height="300"></canvas>
         </div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Scaling Math</h3>
+          <p><strong>Min-Max Scaling (Normalization)</strong> scales data to a fixed range, usually $[0, 1]$:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ X_{norm} = \frac{X - X_{min}}{X_{max} - X_{min}} $$
+          </div>
+          <p><strong>Standardization (Z-Score Scaling)</strong> centers the data around a mean of 0 with a standard
+            deviation of 1. It does not bound data to a specific range, handling outliers better than Min-Max:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ X_{std} = \frac{X - \mu}{\sigma} $$
+          </div>
+          <p style="margin-bottom: 0;"><strong>Robust Scaling</strong> uses statistics that are robust to outliers, like
+            the median and Interquartile Range (IQR): $X_{robust} = \frac{X - \text{median}}{Q3 - Q1}$.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>scaling.py - Scikit-Learn Scalers</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
+# 1. Min-Max Scaler (Best for Neural Networks/Images)
+minmax = MinMaxScaler()
+df[['age_minmax', 'income_minmax']] = minmax.fit_transform(df[['age', 'income']])
+# 2. Standard Scaler (Best for PCA, SVM, Logistic Regression)
+standard = StandardScaler()
+df_scaled = standard.fit_transform(df)
+# 3. Robust Scaler (Best when dataset has many outliers)
+robust = RobustScaler()
+df_robust = robust.fit_transform(df)</code></pre>
+        </div>
       </section>
       <!-- ========================== 5. ENCODING =========================== -->
         <div class="canvas-wrapper">
           <canvas id="canvas-encoding" width="600" height="300"></canvas>
         </div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Target Encoding Math</h3>
+          <p><strong>One-Hot Encoding</strong> creates $N$ sparse binary columns for $N$ categories, which can cause the
+            "Curse of Dimensionality" for high-cardinality features.</p>
+          <p><strong>Target Encoding</strong> replaces a categorical value with the average target value for that
+            category. To prevent overfitting (especially on rare categories), a <em>Bayesian Smoothing</em> average is
+            applied:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ S = \lambda \cdot \bar{y}_{cat} + (1 - \lambda) \cdot \bar{y}_{global} $$
+          </div>
+          <p style="margin-bottom: 0;">Where $\bar{y}_{cat}$ is the mean of the target for the specific category,
+            $\bar{y}_{global}$ is the global target mean, and $\lambda$ is a weight between 0 and 1 determined by the
+            category's frequency.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>encoding.py - Category Encoders</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>import pandas as pd
+from sklearn.preprocessing import OneHotEncoder
+from category_encoders import TargetEncoder
+# 1. One-Hot Encoding (Best for nominal variables with few categories)
+ohe = OneHotEncoder(sparse_output=False, drop='first') # drop='first' avoids multicollinearity
+color_encoded = ohe.fit_transform(df[['color']])
+# Pandas alternative (easy but not ideal for pipelines):
+# pd.get_dummies(df, columns=['color'], drop_first=True)
+# 2. Target Encoding (Best for high-cardinality nominal variables like zipcodes)
+# Requires 'category_encoders' library
+te = TargetEncoder(smoothing=10) # Higher smoothing pulls estimates closer to global mean
+df['zipcode_encoded'] = te.fit_transform(df['zipcode'], df['target'])</code></pre>
+        </div>
       </section>
       <!-- ===================== 6. FEATURE SELECTION ======================= -->
         <div class="canvas-wrapper">
           <canvas id="canvas-selection" width="600" height="300"></canvas>
         </div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Selection Math</h3>
+          <p>Feature selection can be filter-based, wrapper-based, or intrinsic.</p>
+          <p><strong>Filter Method (ANOVA F-Value):</strong> Scikit-Learn's `f_classif` computes the ANOVA F-value
+            between numerical features and a categorical target. The F-statistic measures the ratio of variance
+            <em>between</em> groups to the variance <em>within</em> groups:
+          </p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ F = \frac{\text{Between-group variability}}{\text{Within-group variability}} $$
+          </div>
+          <p style="margin-bottom: 0;"><strong>Wrapper Method (RFE):</strong> Recursive Feature Elimination fits a model
+            (e.g., Logistic Regression or Random Forest), ranks features by importance coefficients, drops the weakest
+            feature, and repeats until the desired $N$ features remain.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>selection.py - Feature Selection</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from sklearn.feature_selection import SelectKBest, f_classif, RFE
+from sklearn.linear_model import LogisticRegression
+X = df.drop('target', axis=1)
+y = df['target']
+# 1. Filter Method: SelectKBest (ANOVA F-value)
+# Keeps the 5 features with the highest ANOVA F-scores
+selector = SelectKBest(score_func=f_classif, k=5)
+X_top_5 = selector.fit_transform(X, y)
+selected_columns = X.columns[selector.get_support()]
+# 2. Wrapper Method: Recursive Feature Elimination (RFE)
+# Uses a model's intrinsic feature importance assigning to prune
+estimator = LogisticRegression()
+rfe = RFE(estimator, n_features_to_select=5, step=1)
+X_rfe = rfe.fit_transform(X, y)
+rfe_columns = X.columns[rfe.support_]</code></pre>
+        </div>
       </section>
       <!-- =================== 7. IMBALANCED DATA =========================== -->
         <div class="canvas-wrapper">
           <canvas id="canvas-imbalanced" width="600" height="300"></canvas>
         </div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: SMOTE Math</h3>
+          <p><strong>SMOTE (Synthetic Minority Over-sampling Technique)</strong> doesn't just duplicate data (like
+            Random Over-Sampling). It creates novel synthetic examples by interpolating between existing minority
+            instances.</p>
+          <p>For a minority class point $x_i$, SMOTE finds its $k$-nearest minority neighbors. It picks one neighbor
+            $x_{zi}$ and generates a synthetic point $x_{new}$ along the line segment joining them:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ x_{new} = x_i + \lambda \times (x_{zi} - x_i) $$
+          </div>
+          <p style="margin-bottom: 0;">Where $\lambda$ is a random number between 0 and 1. This creates a denser, more
+            generalized decision region for the minority class.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>imbalanced.py - Imblearn Resampling</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from imblearn.over_sampling import SMOTE
+from imblearn.under_sampling import RandomUnderSampler
+from imblearn.pipeline import Pipeline
+# 1. SMOTE (Over-sampling the minority class)
+smote = SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=42)
+X_smote, y_smote = smote.fit_resample(X, y)
+# 2. Random Under-Sampling (Reducing the majority class)
+rus = RandomUnderSampler(sampling_strategy='auto', random_state=42)
+X_rus, y_rus = rus.fit_resample(X, y)
+# 3. Best Practice Pipeline: Under-sample majority THEN SMOTE minority
+# Prevents creating too many synthetic points if the imbalance is extreme
+resample_pipe = Pipeline([
+    ('rus', RandomUnderSampler(sampling_strategy=0.1)), # Reduce majority until minority is 10%
+    ('smote', SMOTE(sampling_strategy=0.5))             # SMOTE minority until it's 50%
+])
+X_resampled, y_resampled = resample_pipe.fit_resample(X, y)</code></pre>
+        </div>
       </section>
       <!-- ========================== 8. EDA ================================ -->
       <section id="eda" class="topic-section">
         <h2>Exploratory Data Analysis (EDA)</h2>
+        <p><strong>Exploratory Data Analysis (EDA)</strong> is a critical step in the machine learning pipeline that
+          comes BEFORE feature engineering. EDA helps you understand your data, discover patterns, identify anomalies,
+          detect outliers, test hypotheses, and check assumptions through summary statistics and graphical
+          representations.</p>
         <div class="info-card">
           <strong>Key Questions EDA Answers:</strong>
         </div>
         <div class="info-card">
+          <strong>Real-World Example:</strong> Imagine you're analyzing customer data for a bank to predict loan
+          defaults. EDA helps you understand:
           <ul>
             <li>Age distribution of customers (histogram)</li>
             <li>Income levels (box plot for outliers)</li>
         <h4>2. Inferential Statistics</h4>
         <p><strong>Purpose:</strong> Make inferences or generalizations about the population from the sample</p>
+        <p><strong>Key Question:</strong> Can we claim this effect exists in the larger population, or is it just by
+          chance?</p>
         <div class="info-card">
           <strong>A. Hypothesis Testing:</strong><br>
           <strong>3. Analyze Distributions:</strong> Histograms, count plots, box plots<br>
           <strong>4. Check for Imbalance:</strong> Count target classes, plot distribution<br>
           <strong>5. Correlation Analysis:</strong> Correlation matrix, heatmap, identify multicollinearity<br>
+          <strong>6. Statistical Testing:</strong> Compare groups (t-test, ANOVA), test assumptions, calculate effect
+          sizes
         </div>
         <h3>Interactive EDA Dashboard</h3>
         </div>
         <div class="form-group">
+          <label for="confidenceLevel" class="form-label">Confidence Level: <span
+              id="confidenceValue">95</span>%</label>
           <input type="range" id="confidenceLevel" min="90" max="99" step="1" value="95" class="form-control" />
         </div>
           <canvas id="canvas-eda" width="800" height="500"></canvas>
         </div>
+        <div class="callout callout--insight">💡 EDA typically takes 30-40% of total project time. Good EDA reveals
+          which features to engineer.</div>
+        <div class="callout callout--mistake">⚠️ Common Mistakes: Skipping EDA, not checking outliers before scaling,
+          ignoring missing value patterns, overlooking class imbalance, ignoring multicollinearity.</div>
+        <div class="callout callout--tip">✅ Best Practices: ALWAYS start with EDA, visualize EVERY feature, check
+          correlations with target, document insights, use both descriptive and inferential statistics.</div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Skewness & Kurtosis</h3>
+          <p>Beyond mean and variance, we examine the geometric shape of our distributions using the 3rd and 4th
+            statistical moments.</p>
+          <p><strong>Skewness ($s$)</strong> measures asymmetry. Positive means right-tailed, negative means
+            left-tailed:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ s = \frac{\frac{1}{n} \sum_{i=1}^{n} (x_i - \bar{x})^3}{\sigma^3} $$
+          </div>
+          <p><strong>Kurtosis ($k$)</strong> measures "tailedness" (presence of outliers). A normal distribution has a
+            kurtosis of 3. High kurtosis means heavy tails:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ k = \frac{\frac{1}{n} \sum_{i=1}^{n} (x_i - \bar{x})^4}{\sigma^4} $$
+          </div>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>eda.py - Automated & Visual EDA</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>import pandas as pd
+import seaborn as sns
+import matplotlib.pyplot as plt
+# 1. Deep Descriptive Stats (includes skewness)
+display(df.describe().T)
+print("Skewness:\n", df.skew())
+print("\nMissing Values:\n", df.isnull().sum())
+# 2. Visual Distributions (Pairplot)
+# Plots histograms on the diagonal and scatter plots for every relationship
+sns.pairplot(df, hue='target_class', diag_kind='kde', corner=True)
+plt.show()
+# 3. Correlation Heatmap
+plt.figure(figsize=(10, 8))
+corr_matrix = df.corr(method='spearman') # Spearman is robust to non-linear relationships
+sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', vmin=-1, vmax=1)
+plt.title("Spearman Correlation Heatmap")
+plt.show()</code></pre>
+        </div>
         <h3>Use Cases and Applications</h3>
         <ul>
         </ul>
         <h3>Summary &amp; Key Takeaways</h3>
+        <p>Exploratory Data Analysis is the foundation of any successful machine learning project. It combines
+          <strong>descriptive statistics</strong> (mean, median, variance, correlation) with <strong>inferential
+            statistics</strong> (hypothesis testing, confidence intervals) to understand data deeply.
+        </p>
         <p><strong>Descriptive EDA</strong> answers: "What is happening in the dataset?"<br>
+          <strong>Inferential EDA</strong> answers: "Can we claim this effect exists in the larger population?"
+        </p>
         <p>Remember: <strong>Data → EDA → Feature Engineering → ML → Deployment</strong></p>
       </section>
       <!-- =================== 9. FEATURE TRANSFORMATION ==================== -->
       <section id="feature-transformation" class="topic-section">
         <h2>Feature Transformation</h2>
+        <p>Feature transformation creates new representations of data to capture non-linear patterns. Techniques like
+          polynomial features, binning, and mathematical transformations unlock hidden relationships.</p>
         <div class="info-card">
+          <strong>Real Example:</strong> Predicting house prices with polynomial features (adding x² terms) improves
+          model fit for non-linear relationships between square footage and price.
         </div>
         <h3>Mathematical Foundations</h3>
           <canvas id="canvas-transformation" width="700" height="350"></canvas>
         </div>
+        <div class="callout callout--insight">💡 Polynomial features capture curve fitting, but degree=3 on 10 features
+          creates 286 features!</div>
+        <div class="callout callout--mistake">⚠️ Always scale features after polynomial transformation to prevent
+          magnitude issues.</div>
+        <div class="callout callout--tip">✅ Start with degree=2 and visualize distributions before/after transformation.
+        </div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Power Transforms</h3>
+          <p>When log transformations $\ln(1+x)$ aren't enough to fix severe skewness, we use parametric Power
+            Transformations like <strong>Box-Cox</strong> (requires $x > 0$) or <strong>Yeo-Johnson</strong> (supports
+            negative values). They automatically find the optimal $\lambda$ parameter using Maximum Likelihood
+            Estimation.</p>
+          <p><strong>Box-Cox Transformation Formula:</strong></p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ x^{(\lambda)} = \begin{cases} \frac{x^\lambda - 1}{\lambda} & \text{if } \lambda \neq 0 \\ \ln(x) &
+            \text{if } \lambda = 0 \end{cases} $$
+          </div>
+          <p style="margin-bottom: 0;">These transforms stretch and compress the variable to map it as closely to a
+            Gaussian (Normal) distribution as mathematically possible.</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>transformation.py - Power Transforms & Binning</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>import numpy as np
+from sklearn.preprocessing import PowerTransformer, KBinsDiscretizer
+# 1. Power Transformation (Yeo-Johnson)
+# Attempts to map skewed feature to a Gaussian distribution
+pt = PowerTransformer(method='yeo-johnson', standardize=True)
+df['income_gaussian'] = pt.fit_transform(df[['income']])
+# 2. Log Transformation (np.log1p handles zeros safely by doing log(1+x))
+df['revenue_log'] = np.log1p(df['revenue'])
+# 3. Discretization / Binning
+# Converts continuous age into 5 categorical bins (quantiles ensures equal frequency per bin)
+binner = KBinsDiscretizer(n_bins=5, encode='ordinal', strategy='quantile')
+df['age_group'] = binner.fit_transform(df[['age']])</code></pre>
+        </div>
         <h3>Use Cases</h3>
         <ul>
       <!-- =================== 10. FEATURE CREATION ========================= -->
       <section id="feature-creation" class="topic-section">
         <h2>Feature Creation</h2>
+        <p>Creating new features from existing ones based on domain knowledge. Interaction terms, ratios, and
+          domain-specific calculations enhance model performance.</p>
         <div class="info-card">
+          <strong>Real Example:</strong> E-commerce revenue = price × quantity. Profit margin = (selling_price -
+          cost_price) / cost_price. These derived features often have stronger predictive power than raw features.
         </div>
         <h3>Mathematical Foundations</h3>
           <canvas id="canvas-creation" width="700" height="350"></canvas>
         </div>
+        <div class="callout callout--insight">💡 Interaction terms are especially powerful in linear models - neural
+          networks learn them automatically.</div>
+        <div class="callout callout--mistake">⚠️ Creating features without domain knowledge leads to meaningless
+          combinations.</div>
+        <div class="callout callout--tip">✅ Always check correlation between new and existing features to avoid
+          redundancy.</div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: Polynomial Combinations</h3>
+          <p>Scikit-Learn's `PolynomialFeatures` generates a new feature matrix consisting of all polynomial
+            combinations of the features with degree less than or equal to the specified degree.</p>
+          <p>For two features $X = [x_1, x_2]$ and a degree of 2, the expanded polynomial vector is:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ [1,\; x_1,\; x_2,\; x_1^2,\; x_1 \cdot x_2,\; x_2^2] $$
+          </div>
+          <p style="margin-bottom: 0;">Notice the $x_1 \cdot x_2$ term. This is an <strong>interaction term</strong>,
+            which lets a linear model learn conditional relationships (e.g., "if $x_1$ is high, the effect of $x_2$
+            changes").</p>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>creation.py - Automated Polynomial Features</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from sklearn.preprocessing import PolynomialFeatures
+import pandas as pd
+# Assume df has two features: 'length' and 'width'
+X = df[['length', 'width']]
+# Create polynomial and interaction features up to degree 2
+# include_bias=False prevents adding a column of 1s (intercept)
+poly = PolynomialFeatures(degree=2, include_bias=False)
+X_poly = poly.fit_transform(X)
+# Get the names of the new features (e.g., 'length^2', 'length width')
+feature_names = poly.get_feature_names_out(['length', 'width'])
+df_poly = pd.DataFrame(X_poly, columns=feature_names)
+print(df_poly.head())</code></pre>
+        </div>
         <h3>Use Cases</h3>
         <ul>
       <!-- ================ 11. DIMENSIONALITY REDUCTION ==================== -->
       <section id="dimensionality-reduction" class="topic-section">
         <h2>Dimensionality Reduction</h2>
+        <p>Reducing the number of features while preserving information. PCA (Principal Component Analysis) projects
+          high-dimensional data onto lower dimensions by finding directions of maximum variance.</p>
         <div class="info-card">
+          <strong>Real Example:</strong> Image compression and genome analysis with thousands of genes benefit from PCA.
+          First 2-3 principal components often capture 80%+ of variance.
         </div>
         <h3>PCA Mathematical Foundations</h3>
         <div class="info-card">
           <strong>Algorithm Steps:</strong><br>
+          1. Standardize data: $X_{scaled} = \frac{X - \mu}{\sigma}$<br>
+          2. Compute covariance matrix: $\Sigma = \frac{1}{n-1} X^T X$<br>
+          3. Calculate eigenvalues $\lambda$ and eigenvectors $v$<br>
           4. Sort eigenvectors by eigenvalues (descending)<br>
+          5. Select top $k$ eigenvectors (principal components)<br>
+          6. Transform: $X_{new} = X \times v_k$<br><br>
+          <strong>Explained Variance:</strong> $\frac{\lambda_i}{\sum \lambda_j}$<br>
           <strong>Cumulative Variance:</strong> Shows total information preserved<br><br>
           <strong>Why PCA Works:</strong><br>
           • Removes correlated features<br>
           <canvas id="canvas-pca" width="700" height="400"></canvas>
         </div>
+        <div class="callout callout--insight">💡 PCA is unsupervised - it doesn't use the target variable. First PC
+          always captures most variance.</div>
+        <div class="callout callout--mistake">⚠️ Not standardizing before PCA is a critical error - features with large
+          scales will dominate.</div>
+        <div class="callout callout--tip">✅ Aim for 95% cumulative explained variance when choosing number of
+          components.</div>
+        <div class="info-card" style="margin-top: 20px; border-left-color: #9900ff;">
+          <h3 style="margin-top: 0; color: #9900ff;">🧠 Under the Hood: PCA Math</h3>
+          <p>PCA finds the directions (Principal Components) that maximize the variance of the data. Mathematically, it
+            works by computing the covariance matrix of the standardized dataset $X$:</p>
+          <div
+            style="background: rgba(0,0,0,0.2); padding: 15px; border-radius: 8px; text-align: center; margin: 15px 0; font-size: 1.1em; color: #e4e6eb;">
+            $$ \Sigma = \frac{1}{n-1} X^T X $$
+          </div>
+          <p>Then, we solve for the eigenvectors $V$ and eigenvalues $\lambda$ solving $\Sigma V = \lambda V$.</p>
+          <ul style="margin-top: 10px; margin-bottom: 0;">
+            <li><strong>Eigenvectors</strong> ($v_i$) are the axes of the new feature space (the directions).</li>
+            <li><strong>Eigenvalues</strong> ($\lambda_i$) represent the magnitude of variance captured by each vector.
+            </li>
+          </ul>
+        </div>
+        <div class="code-block" style="margin-top: 20px;">
+          <div class="code-header">
+            <span>pca.py - Principal Component Analysis</span>
+            <button class="copy-btn" onclick="copyCode(this)">Copy</button>
+          </div>
+          <pre><code>from sklearn.decomposition import PCA
+from sklearn.preprocessing import StandardScaler
+import matplotlib.pyplot as plt
+import numpy as np
+# 1. ALWAYS scale data before PCA
+scaler = StandardScaler()
+X_scaled = scaler.fit_transform(X)
+# 2. Fit PCA without specifying components to see all variance
+pca_full = PCA()
+pca_full.fit(X_scaled)
+# 3. Plot Cumulative Explained Variance
+cumulative_variance = np.cumsum(pca_full.explained_variance_ratio_)
+plt.plot(cumulative_variance, marker='o')
+plt.axhline(y=0.95, color='r', linestyle='--') # 95% threshold
+plt.xlabel('Number of Components')
+plt.ylabel('Cumulative Explained Variance')
+plt.show()
+# 4. Apply PCA retaining 95% variance
+# Float between 0 and 1 selects components covering that % of variance
+pca = PCA(n_components=0.95)
+X_pca = pca.fit_transform(X_scaled)
+print(f"Reduced from {X.shape[1]} to {X_pca.shape[1]} features.")</code></pre>
+        </div>
         <h3>Use Cases</h3>
         <ul>
   <script src="app.js" defer></script>
 </body>
+</html>