Spaces:
Build error
Build error
File size: 19,984 Bytes
f7c7e26 c61ce8c f7c7e26 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 |
{% extends "layout.html" %}
{% block content %}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Study Guide: Principal Component Analysis (PCA)</title>
<!-- MathJax for rendering mathematical formulas -->
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
<script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<style>
/* General Body Styles */
body {
background-color: #ffffff; /* White background */
color: #000000; /* Black text */
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
font-weight: normal;
line-height: 1.8;
margin: 0;
padding: 20px;
}
/* Container for centering content */
.container {
max-width: 800px;
margin: 0 auto;
padding: 20px;
}
/* Headings */
h1, h2, h3 {
color: #000000;
border: none;
font-weight: bold;
}
h1 {
text-align: center;
border-bottom: 3px solid #000;
padding-bottom: 10px;
margin-bottom: 30px;
font-size: 2.5em;
}
h2 {
font-size: 1.8em;
margin-top: 40px;
border-bottom: 1px solid #ddd;
padding-bottom: 8px;
}
h3 {
font-size: 1.3em;
margin-top: 25px;
}
/* Main words are even bolder */
strong {
font-weight: 900;
}
/* Paragraphs and List Items with a line below */
p, li {
font-size: 1.1em;
border-bottom: 1px solid #e0e0e0; /* Light gray line below each item */
padding-bottom: 10px; /* Space between text and the line */
margin-bottom: 10px; /* Space below the line */
}
/* Remove bottom border from the last item in a list for cleaner look */
li:last-child {
border-bottom: none;
}
/* Ordered lists */
ol {
list-style-type: decimal;
padding-left: 20px;
}
ol li {
padding-left: 10px;
}
/* Unordered Lists */
ul {
list-style-type: none;
padding-left: 0;
}
ul li::before {
content: "•";
color: #000;
font-weight: bold;
display: inline-block;
width: 1em;
margin-left: 0;
}
/* Code block styling */
pre {
background-color: #f4f4f4;
border: 1px solid #ddd;
border-radius: 5px;
padding: 15px;
white-space: pre-wrap;
word-wrap: break-word;
font-family: "Courier New", Courier, monospace;
font-size: 0.95em;
font-weight: normal;
color: #333;
border-bottom: none;
}
/* PCA Specific Styling */
.story-pca {
background-color: #fff4e6;
border-left: 4px solid #fd7e14; /* Orange accent for PCA */
margin: 15px 0;
padding: 10px 15px;
font-style: italic;
color: #555;
font-weight: normal;
border-bottom: none;
}
.story-pca p, .story-pca li {
border-bottom: none;
}
.example-pca {
background-color: #fff9f0;
padding: 15px;
margin: 15px 0;
border-radius: 5px;
border-left: 4px solid #ff9a3c; /* Lighter Orange accent for PCA */
}
.example-pca p, .example-pca li {
border-bottom: none !important;
}
/* Table Styling */
table {
width: 100%;
border-collapse: collapse;
margin: 25px 0;
}
th, td {
border: 1px solid #ddd;
padding: 12px;
text-align: left;
}
th {
background-color: #f2f2f2;
font-weight: bold;
}
/* --- Mobile Responsive Styles --- */
@media (max-width: 768px) {
body, .container {
padding: 10px;
}
h1 { font-size: 2em; }
h2 { font-size: 1.5em; }
h3 { font-size: 1.2em; }
p, li { font-size: 1em; }
pre { font-size: 0.85em; }
table, th, td { font-size: 0.9em; }
}
</style>
</head>
<body>
<div class="container">
<h1>📉 Study Guide: Principal Component Analysis (PCA)</h1>
<!-- button -->
<div>
<!-- Audio Element -->
<!-- Note: Browsers may block audio autoplay if the user hasn't interacted with the document first,
but since this is triggered by a click, it should work fine. -->
<a
href="/pca-three"
target="_blank"
onclick="playSound()"
class="
cursor-pointer
inline-block
relative
bg-blue-500
text-white
font-bold
py-4 px-8
rounded-xl
text-2xl
transition-all
duration-150
/* 3D Effect (Hard Shadow) */
shadow-[0_8px_0_rgb(29,78,216)]
/* Pressed State (Move down & remove shadow) */
active:shadow-none
active:translate-y-[8px]
">
Tap Me!
</a>
</div>
<script>
function playSound() {
const audio = document.getElementById("clickSound");
if (audio) {
audio.currentTime = 0;
audio.play().catch(e => console.log("Audio play failed:", e));
}
}
</script>
<!-- button -->
<h2>🔹 Core Concepts</h2>
<div class="story-pca">
<p><strong>Story-style intuition: The Shadow Puppet Master</strong></p>
<p>Imagine you have a complex 3D object, like a toy airplane. If you shine a light on it, you create a 2D shadow. From one angle, the shadow might look like a simple line. But if you rotate the airplane and find the perfect angle, the shadow will capture its main shape—the wings and body. <strong>PCA</strong> is like a mathematical shadow puppet master for your data. It takes high-dimensional data (the 3D airplane) and finds the best "angles" to project it onto a lower-dimensional surface (the 2D shadow), making sure the shadow preserves as much of the original shape (the <strong>variance</strong>) as possible.</p>
</div>
<p><strong>Principal Component Analysis (PCA)</strong> is a dimensionality reduction technique. Its main goal is to reduce the number of features in a dataset while keeping as much important information as possible. It doesn't just pick features; it creates new, powerful features called <strong>principal components</strong>, which are combinations of the original ones.</p>
<div class="example-pca">
<p><strong>Example:</strong> A dataset about houses has 10 features: square footage, number of rooms, number of bathrooms, lot size, etc. Many of these features are correlated and essentially measure the same thing: the "size" of the house. PCA can combine them into a single new feature like "Overall House Size," reducing 10 features to 1 without losing much information.</p>
</div>
<h2>🔹 Mathematical Foundation</h2>
<div class="story-pca">
<p><strong>Story: The "Data Squishing" Machine</strong></p>
<p>PCA is a five-step machine that intelligently squishes your data:</p>
<ol>
<li><strong>Step 1: Put everything on the same scale.</strong> (Standardize Data).</li>
<li><strong>Step 2: Figure out which features move together.</strong> (Compute Covariance Matrix).</li>
<li><strong>Step 3: Find the main directions of "stretch" in the data.</strong> (Find Eigenvectors and Eigenvalues).</li>
<li><strong>Step 4: Rank these directions from most to least important.</strong> (Sort Eigenvalues).</li>
<li><strong>Step 5: Keep the top few important directions and discard the rest.</strong> (Select top k components).</li>
</ol>
</div>
<p>The core of PCA relies on linear algebra to find the principal components. The process is:</p>
<ol>
<li><strong>Standardize the data:</strong> Rescale features to have a mean of 0 and a variance of 1. This is crucial!</li>
<li><strong>Compute the Covariance Matrix:</strong> This matrix shows how every feature relates to every other feature.</li>
<li><strong>Find Eigenvectors and Eigenvalues:</strong> These are calculated from the covariance matrix. The <strong>eigenvectors</strong> are the new axes (the principal components), and the <strong>eigenvalues</strong> tell you how much information (variance) each eigenvector holds.</li>
<li><strong>Sort Eigenvalues:</strong> Rank them from highest to lowest. The eigenvector with the highest eigenvalue is the first principal component (PC1).</li>
<li><strong>Select Top k Components:</strong> Choose the top `k` eigenvectors to form your new, smaller feature set.</li>
</ol>
<h2>🔹 Geometric Interpretation</h2>
<div class="story-pca">
<p><strong>Story: Finding the Best Camera Angle</strong></p>
<p>Imagine your data is a cloud of points in 3D space. PCA is like finding the best camera angle to take a 2D picture of this cloud.
<br>• The <strong>First Principal Component (PC1)</strong> is the direction (or camera angle) that shows the biggest spread of data. It's the longest axis of the data cloud.
<br>• The <strong>Second Principal Component (PC2)</strong> is the direction that shows the next biggest spread, but it must be at a 90-degree angle (<strong>orthogonal</strong>) to PC1.
<br>By projecting the 3D cloud onto a 2D plane defined by these two new axes, you get the most informative and representative 2D picture of your data.</p>
</div>
<h2>🔹 Variance Explained</h2>
<p>Each principal component captures a certain amount of the total variance (information) from the original dataset. The "explained variance ratio" tells you the percentage of the total information that each component holds.</p>
<div class="example-pca">
<p><strong>Example:</strong> After running PCA, you might find:</p>
<ul>
<li>PC1 explains 75% of the variance.</li>
<li>PC2 explains 20% of the variance.</li>
<li>PC3 explains 3% of the variance.</li>
<li>...and so on.</li>
</ul>
<p>In this case, the first two components alone capture 95% of the total information. This means you can likely discard all other components and just use PC1 and PC2, reducing your data's complexity while retaining almost all of its structure. This is often visualized using a <strong>scree plot</strong>.</p>
</div>
<h2>🔹 Comparison</h2>
<table>
<thead>
<tr>
<th>Comparison</th>
<th>PCA (Principal Component Analysis)</th>
<th>Alternative Method</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>vs. Feature Selection</strong></td>
<td><strong>Creates new features</strong> by combining old ones. (Making a smoothie from different fruits).</td>
<td><strong>Selects a subset</strong> of the original features. (Picking the best fruits for a fruit basket).</td>
</tr>
<tr>
<td><strong>vs. Autoencoders</strong></td>
<td>A <strong>linear</strong> method. Can't capture complex, curved patterns in data. (Taking a simple photo).</td>
<td>Can learn complex, <strong>nonlinear</strong> patterns. (Drawing a detailed, artistic sketch).</td>
</tr>
</tbody>
</table>
<h2>🔹 Strengths & Weaknesses</h2>
<h3>Advantages:</h3>
<ul>
<li>✅ <strong>Reduces Dimensionality:</strong> Makes models train faster and require less memory. <strong>Example:</strong> A model might train in 1 minute on 5 principal components vs. 10 minutes on 100 original features.</li>
<li>✅ <strong>Removes Multicollinearity:</strong> It gets rid of redundant, correlated features, which can improve the performance of some models like Linear Regression.</li>
<li>✅ <strong>Helps with Visualization:</strong> Allows you to plot high-dimensional data in 2D or 3D to see patterns.</li>
</ul>
<h3>Disadvantages:</h3>
<ul>
<li>❌ <strong>Features are Hard to Interpret:</strong> The new principal components are mathematical combinations (e.g., `0.7*age - 0.3*income + 0.1*education`). It's hard to explain what "PC1" means in a business context.</li>
<li>❌ <strong>It's a Linear Method:</strong> PCA might miss important patterns in data that aren't linear (e.g., a spiral or circular pattern).</li>
<li>❌ <strong>Sensitive to Scaling:</strong> If you don't scale your data first, features with large values (like income) will dominate the PCA process, leading to poor results.</li>
</ul>
<h2>🔹 When to Use PCA</h2>
<ul>
<li><strong>High-Dimensional Data:</strong> When you have datasets with dozens or hundreds of features, especially if many are correlated. <strong>Example:</strong> Analyzing gene expression data with thousands of genes.</li>
<li><strong>Visualization:</strong> When you need to plot and explore a dataset with more than 3 features.</li>
<li><strong>Preprocessing:</strong> As a step before feeding data into another machine learning model to improve its speed and sometimes its performance.</li>
<li><strong>Noise Reduction:</strong> By keeping only the components with the most variance, you can sometimes filter out noise in your data.</li>
</ul>
<h2>🔹 Python Implementation (Beginner Example with Iris Dataset)</h2>
<div class="story-pca">
<p>In this example, we take the famous Iris dataset, which has 4 features, and use PCA to squish it down to just 2 features (principal components). This allows us to create a 2D scatter plot that effectively visualizes the separation between the different flower species.</p>
</div>
<pre><code>
import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris
# --- 1. Load and Scale the Data ---
# The Iris dataset has 4 features for 3 species of iris flowers.
iris = load_iris()
X = iris.data
# Scaling is CRITICAL for PCA!
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# --- 2. Create and Apply PCA ---
# We'll reduce the 4 features down to 2 principal components.
pca = PCA(n_components=2)
# Fit PCA to the scaled data and transform it.
X_pca = pca.fit_transform(X_scaled)
# --- 3. Check the Explained Variance ---
# Let's see how much information our 2 new components hold.
explained_variance = pca.explained_variance_ratio_
print(f"Explained variance by component 1: {explained_variance[0]:.2%}")
print(f"Explained variance by component 2: {explained_variance[1]:.2%}")
print(f"Total variance explained by 2 components: {np.sum(explained_variance):.2%}")
# --- 4. Visualize the Results ---
# We can now plot our 4D dataset in 2D.
plt.figure(figsize=(8, 6))
plt.scatter(X_pca[:, 0], X_pca[:, 1], c=iris.target, cmap='viridis')
plt.title('PCA of Iris Dataset (4D -> 2D)')
plt.xlabel('First Principal Component')
plt.ylabel('Second Principal Component')
plt.grid(True)
plt.show()
</code></pre>
<h2>🔹 Best Practices</h2>
<ul>
<li><strong>Always Scale Your Data:</strong> This is the most important rule. Use `StandardScaler` before applying PCA.</li>
<li><strong>Choose `n_components` Wisely:</strong> Use a scree plot or the explained variance ratio to decide how many components to keep. A common rule of thumb is to keep enough components to explain 90-99% of the variance.</li>
<li><strong>Consider Interpretability:</strong> If you absolutely must be able to explain what each feature means, PCA might not be the right choice. Simple feature selection could be better.</li>
</ul>
<h2>🔹 Key Terminology Explained (PCA)</h2>
<div class="story-pca">
<p><strong>The Story: Decoding the Shadow Master's Toolkit</strong></p>
<p>Let's clarify the key terms the PCA shadow master uses.</p>
</div>
<ul>
<li>
<strong>Dimensionality Reduction:</strong>
<br>
<strong>What it is:</strong> The process of reducing the number of features (dimensions) in a dataset.
<br>
<strong>Story Example:</strong> This is like summarizing a 500-page book into a 1-page summary. You lose some detail, but you keep the main plot points. <strong>Dimensionality reduction</strong> creates a simpler version of your data.
</li>
<li>
<strong>Covariance Matrix:</strong>
<br>
<strong>What it is:</strong> A square table that shows how each pair of features in your data moves together.
<br>
<strong>Story Example:</strong> Imagine you're tracking a group of dancers. The <strong>covariance matrix</strong> is your notebook where you write down which pairs of dancers tend to move in the same direction at the same time.
</li>
<li>
<strong>Eigenvectors & Eigenvalues:</strong>
<br>
<strong>What they are:</strong> A pair of mathematical concepts. The eigenvector is a direction, and the eigenvalue is a number telling you how important that direction is.
<br>
<strong>Story Example:</strong> Imagine stretching a rubber sheet with a picture on it. The <strong>eigenvectors</strong> are the directions of stretch where the picture only gets scaled, not rotated. The <strong>eigenvalues</strong> tell you *how much* it stretched in those directions. PCA finds the directions of greatest "stretch" in your data.
</li>
<li>
<strong>Orthogonal:</strong>
<br>
<strong>What it is:</strong> A mathematical term that simply means "at a right angle (90°) to each other."
<br>
<strong>Story Example:</strong> The corner of a square or the intersection of the x-axis and y-axis on a graph are <strong>orthogonal</strong>. The principal components PCA finds are all orthogonal to each other.
</li>
</ul>
</div>
</body>
</html>
{% endblock %}
|