22 lessons across 4 weeks, 57 verified past-year questions. Free preview below — sign in for the full interactive lessons, AI tutor, and mock exams.
Week 1
- Introduction to Machine Learning — Provides a high-level overview of what machine learning is, why it matters, its major application areas, and how it differs from procedural programming and memorization.
- Three Major Paradigms of Machine Learning — Describes the three broad learning paradigms—supervised, unsupervised, and sequential learning—their subcategories, and how real-world problems map to each.
- Representation Learning and Compression — Introduces unsupervised representation learning through the lens of compression, showing how projecting data onto a representative line reduces storage while preserving structure.
- Principal Component Analysis — Develops the PCA algorithm by connecting iterative error-minimizing line-finding to eigenvectors of the covariance matrix, and explains eigenvalues as explained variance and principal components as de-correlated directions.
Week 2
- Eigenfaces Application — PCA applied to face images, where each image is treated as a high-dimensional vector and principal components (eigenfaces) enable compressed yet recognizable face representations.
- Issues with PCA — Two key limitations of standard PCA: cubic time complexity in feature dimension d, and inability to capture nonlinear relationships among features.
- Time-Complexity Issue with PCA — When d >> n, PCA's O(d³) cost can be reduced to O(n³) by solving an equivalent eigenequation on the n×n Gram matrix K = X^T X instead of the d×d covariance matrix.
- Feature Transformation and the Curse of Dimensionality — Nonlinear relationships among features can be linearized by mapping data to a higher-dimensional space via a feature map φ, but the dimension of this map grows exponentially with degree and number of features.
- Kernel Functions — Kernel functions implicitly compute dot products in a high-dimensional feature space without explicitly computing the mapping φ, with validity characterized by Mercer's theorem requiring symmetry and positive semi-definiteness.
- Kernel PCA — Kernel PCA combines the Gram-matrix reformulation of PCA with kernel functions to perform nonlinear dimensionality reduction, representing each data point by its projections onto the top eigenvectors of the centered kernel matrix.
Week 3
- Introduction to Clustering — Clustering is an unsupervised learning paradigm that partitions data into K groups by minimizing the sum of squared distances of points to their cluster means.
- Lloyd's Algorithm (K-Means) — Lloyd's algorithm iteratively reassigns each data point to its nearest cluster mean and recomputes means until no point wants to switch clusters.
- Convergence of K-Means Algorithm — Lloyd's algorithm is guaranteed to converge because each reassignment strictly decreases the objective function, and the number of possible partitions is finite.
- Nature of Clusters Produced by K-Means — K-means produces clusters that are Voronoi regions—convex regions separated by perpendicular bisectors between means—which limits its ability to find non-convex clusters.
- Initialization of Centroids, K-Means++ — K-Means++ initializes cluster centers by sequentially sampling data points with probability proportional to their squared distance from already-chosen centers, yielding a provable approximation guarantee.
- Choice of K — The number of clusters K is chosen by balancing the clustering objective value against a penalty for model complexity, using criteria such as AIC or BIC.
Week 4
- Introduction to Estimation — Estimation is the process of assuming a probabilistic model for data and using observations to infer unknown model parameters, illustrated through the coin-toss example with i.i.d. assumptions.
- Maximum Likelihood Estimation — The principle of maximum likelihood selects the parameter value that maximizes the probability (or density) of observing the given data, yielding intuitive estimators such as the sample mean for Gaussian data and the fraction of ones for Bernoulli data.
- Bayesian Estimation — Bayesian estimation encodes prior beliefs about parameters as a probability distribution and uses Bayes' theorem to update that belief into a posterior distribution after observing data, with MAP and posterior mean as point estimates.
- Gaussian Mixture Models — A Gaussian Mixture Model is a latent-variable generative model that explains multi-modal data as a probabilistic mixture of K Gaussians, where a hidden cluster indicator selects which Gaussian generates each observation.
- Convex Functions and Jensen's Inequality — Convex and concave functions satisfy a key inequality between function values and linear interpolations; Jensen's inequality generalises this to weighted combinations and is the key tool for handling the log-of-sum in the GMM likelihood.
- EM Algorithm — The Expectation-Maximization algorithm iteratively alternates between computing soft cluster responsibilities (E-step) and updating GMM parameters in closed form (M-step), provably converging to a local maximum of the log-likelihood.
Unofficial, student-built, not affiliated with IIT Madras.