A rigorous treatment of probabilistic models, optimization theory, and algorithmic decision-making — from convex loss surfaces to ensemble methods and deep architectures.
Machine Learning allows systems to learn autonomously from data — identifying patterns, constructing models, and making decisions without being explicitly programmed for each task.
Just as humans build knowledge from experience, ML algorithms are exposed to data from which they extract features, relationships, and patterns. This acquired structure enables accurate predictions on new, unseen data — and continuous improvement as more data becomes available.
Human analysts face hard limits on the data they can process manually. ML systems operate at arbitrary scale, uncovering structure that would otherwise remain hidden. Applications span self-driving vehicles, fraud detection, medical diagnosis, and recommendation systems at Netflix, Amazon, and YouTube.
Trained on labeled data — each input is paired with the correct output. The model learns a mapping and generalizes to new examples. Covers both classification (discrete targets) and regression (continuous targets).
Works on unlabeled data, discovering hidden structures autonomously. No human labeling required, enabling the algorithm to scale to terabytes of raw data. Representations adapt dynamically post-deployment.
An agent interacts with an environment via a reward signal. Favorable actions are reinforced; unfavorable ones penalized. The algorithm iterates via trial-and-error until it discovers the optimal policy.
Select any model to explore its theory, interactive visualizations, and real-world applications.
Estimates a continuous output by fitting the best-fit hyperplane, minimizing MSE. Fully interpretable — each β quantifies the exact linear contribution of a feature.
Probabilistic binary classifier using the sigmoid function to map linear combinations to [0,1]. Decision boundary is a hyperplane; trained via maximum likelihood.
Non-parametric instance-based learner. Classifies by majority vote among k nearest neighbors. No training phase — all computation deferred to query time.
Finds the maximum-margin hyperplane separating classes. Kernel trick extends to non-linear boundaries. Dual formulation leads to sparse support vector solutions.
Recursive axis-aligned partitioning of feature space. Splits chosen to maximize information gain (ID3) or minimize Gini impurity (CART). Interpretable; prone to overfitting.
Bagging ensemble of decision trees with random feature subsampling. Reduces variance via averaging. Provides feature importance scores. Robust to outliers and high-dimensional data.
Gradient-boosted trees with L1/L2 regularization, column subsampling, and second-order Taylor expansion of the loss. Industry-standard for tabular data competitions.
Probabilistic classifier based on Bayes' theorem with conditional independence assumption. Extremely fast; works well on text and high-dimensional sparse data.
Layered composition of parameterized non-linear transformations. Trained via backpropagation and gradient descent. Universal approximator; foundation of deep learning.
Iterative algorithm partitioning data into k clusters by alternating assignment and centroid update steps. Minimizes within-cluster sum of squared distances (inertia).
Linear orthogonal projection that maximizes variance. Computes eigenvectors of the covariance matrix. Foundation of dimensionality reduction, noise filtering, and feature extraction.
Three perspectives on all 11 algorithms — their computational and explanatory footprint, how they cluster into families, and how much labeled data each one needs to learn effectively.
Core concepts from optimization, probability, and statistical learning theory that underpin every algorithm in the course.
Gradient descent, SGD, momentum, and Adam. Convexity conditions, Lagrangian relaxation, and KKT optimality conditions for constrained problems.
Maximum Likelihood Estimation, MAP inference, Bayesian reasoning, conjugate priors, and posterior predictive distributions.
VC dimension, PAC learning framework, L1/L2 regularization, cross-validation strategies, and the bias-variance decomposition of generalization error.
The kernel trick, Reproducing Kernel Hilbert Spaces, Mercer's theorem, and standard kernels: RBF, polynomial, and sigmoid.
Bagging and boosting theory: bias-variance reduction, AdaBoost convergence, gradient boosting as functional gradient descent.
Eigendecomposition, SVD, and their application to dimensionality reduction, pseudo-inverses, and covariance geometry.
Total expected error decomposes as Bias² + Variance + Irreducible Noise. Simple models underfit (high bias); complex models overfit (high variance). Optimal complexity sits at the minimum of total error.
Textbooks, lecture notes, useful links, and the full semester syllabus.