lectures/Machine_Learning_MOC.md

metadata

title: Machine Learning Map of Content (MOC)
course: '[[Machine Learning]]'
type: moc
updated: '2026-08-20'
tags:
  - moc
  - course/MachineLearning

🗺️ Machine Learning: Map of Content (MOC)

Master Course Knowledge Hub This hub connects all lecture notes, derivations, and exam warnings for Machine Learning.

---

📅 1. Chronological Lecture Syllabus

Date	Topic	Note Link
2026-08-19	Curl Test	[[2026-08-19_Machine_Learning_Curl_Test|Curl Test Note]]
2026-08-19	Quick Test	[[2026-08-19_Machine_Learning_Quick_Test|Quick Test Note]]
2026-10-15	Backpropagation	[[2026-10-15_Machine_Learning_Backpropagation|Backpropagation Note]]

---

📐 2. Key Derivations & Theorems Index

• [[2026-10-15_Machine_Learning_Backpropagation#Forward Pass Equations|Forward Pass Equations]] (Topic: Backpropagation, Date: 2026-10-15)
• [[2026-10-15_Machine_Learning_Backpropagation#Backward Pass & Gradient Equations|Backward Pass & Gradient Equations]] (Topic: Backpropagation, Date: 2026-10-15)
• [[2026-10-15_Machine_Learning_Backpropagation#Section 2: Mathematical Formulation|Section 2: Mathematical Formulation]] (Topic: Backpropagation, Date: 2026-10-15)

---

⚠️ 3. High-Yield Exam Pitfalls Aggregator

From [[2026-08-19_Machine_Learning_Curl_Test|Curl Test (2026-08-19)]]:

• Highlight direct warnings, potential exam questions, and common conceptual traps.

From [[2026-08-19_Machine_Learning_Quick_Test|Quick Test (2026-08-19)]]:

• Dimension Mismatch: Ensure weight matrices and input vectors satisfy inner-dimension equality ($A \in \mathbb{R}^{n \times d}, x \in \mathbb{R}^{d \times 1}$).
• Vanishing Gradients: Pay attention to non-linear activations in deep networks; professors often test on the Sigmoid/ReLU saturation regions.
• Notation Trap: Always define whether your vectors are column vectors or row vectors at the start of the exam paper.

From [[2026-10-15_Machine_Learning_Backpropagation|Backpropagation (2026-10-15)]]:

• Matrix Dimension Verification: Dimension checks on parameter gradients are guaranteed exam questions. Always explicitly verify that $\operatorname{dim}\left(\frac{\partial L}{\partial W^{(l)}}\right) = \operatorname{dim}\left(W^{(l)}\right) = (n_l \times n_{l-1})$.
• Outer Product Order: A common mistake is swapping the order in the weight gradient formula. Remember: $\frac{\partial L}{\partial W^{(l)}} = \delta^{(l)} (A^{(l-1)})^T$, NOT $A^{(l-1)} (\delta^{(l)})^T$.
• Vanishing Gradient Pathology: Occurs when pre-activations $Z^{(l)}$ enter the saturated regime of activation functions like Sigmoid ($\sigma'(Z^{(l)}) \approx 0$). Multiplying by near-zero terms recursively drives $\delta^{(l)} \to 0$ for early layers.
• Memory Caching Necessity: Intermediate activations $A^{(l-1)}$ MUST be stored in RAM during the forward pass; without caching, computing $\frac{\partial L}{\partial W^{(l)}}$ requires redundant re-computation of the forward pass.