Eric-Yyvuw commited on
Commit
8a751f3
Β·
verified Β·
1 Parent(s): 989d20b

Upload notebooks/Foundation.ipynb with huggingface_hub

Browse files
Files changed (1) hide show
  1. notebooks/Foundation.ipynb +334 -0
notebooks/Foundation.ipynb ADDED
@@ -0,0 +1,334 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {},
6
+ "source": [
7
+ "# CS401 β€” ML & Neural Networks: Foundations\n",
8
+ "\n",
9
+ "**Lecture 1 β€” 2026-09-21**\n",
10
+ "\n",
11
+ "---\n",
12
+ "\n",
13
+ "## What is Machine Learning?\n",
14
+ "\n",
15
+ "- **Machine Learning** is a subset of **AI** (Artificial Intelligence)\n",
16
+ "- Hierarchy: **AI β†’ ML β†’ Neural Networks**\n",
17
+ "- ML is a branch of AI that learns patterns from data\n",
18
+ "\n",
19
+ "```mermaid\n",
20
+ "graph TD\n",
21
+ " A[AI] --> B[ML]\n",
22
+ " B --> C[Neural Networks]\n",
23
+ "```\n",
24
+ "\n",
25
+ "### Core Pipeline\n",
26
+ "\n",
27
+ "```\n",
28
+ "β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”\n",
29
+ "β”‚ DATA β”‚ ───▢ β”‚ ML Algorithm β”‚ ───▢ β”‚ Prediction Modelβ”‚\n",
30
+ "β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n",
31
+ "```"
32
+ ]
33
+ },
34
+ {
35
+ "cell_type": "markdown",
36
+ "metadata": {},
37
+ "source": [
38
+ "### Key References\n",
39
+ "\n",
40
+ "| Reference | Contribution |\n",
41
+ "|---|---|\n",
42
+ "| **Andrew Ng** | Prominent ML educator |\n",
43
+ "| **Tom Mitchell** | *Machine Learning* (textbook) |\n",
44
+ "| **Jugal Kalita** | *ML Learning Theory and Practice* |\n",
45
+ "| **Donald Hebb (1949)** | Hebbian learning β€” foundational to neural networks |"
46
+ ]
47
+ },
48
+ {
49
+ "cell_type": "markdown",
50
+ "metadata": {},
51
+ "source": [
52
+ "---\n",
53
+ "\n",
54
+ "## Learning Types\n",
55
+ "\n",
56
+ "### 1. Supervised Learning\n",
57
+ "\n",
58
+ "- Uses **labelled** data: features (x) β†’ target (y)\n",
59
+ "- Tasks: classification, regression\n",
60
+ "- Example: predicting outcomes from known input-output pairs\n",
61
+ "\n",
62
+ "### 2. Unsupervised Learning\n",
63
+ "\n",
64
+ "- Uses **unlabelled** data\n",
65
+ "- Tasks: clustering, dimensionality reduction\n",
66
+ "- Example: **customer segmentation** (grouping similar customers)\n",
67
+ "\n",
68
+ "### 3. Reinforcement Learning (RL)\n",
69
+ "\n",
70
+ "- Agent takes actions in an environment\n",
71
+ "- Receives **reward** signals\n",
72
+ "- Learns a policy to maximise cumulative reward"
73
+ ]
74
+ },
75
+ {
76
+ "cell_type": "markdown",
77
+ "metadata": {},
78
+ "source": [
79
+ "---\n",
80
+ "\n",
81
+ "## Interactive Demo: Supervised vs Unsupervised Learning\n",
82
+ "\n",
83
+ "Let's visualize the difference between supervised and unsupervised learning."
84
+ ]
85
+ },
86
+ {
87
+ "cell_type": "code",
88
+ "execution_count": null,
89
+ "metadata": {},
90
+ "outputs": [],
91
+ "source": [
92
+ "import numpy as np\n",
93
+ "import matplotlib.pyplot as plt\n",
94
+ "\n",
95
+ "# Set random seed for reproducibility\n",
96
+ "np.random.seed(42)\n",
97
+ "\n",
98
+ "fig, axes = plt.subplots(1, 3, figsize=(15, 4))\n",
99
+ "\n",
100
+ "# --- Supervised Learning (Regression) ---\n",
101
+ "# Known input-output pairs\n",
102
+ "X_sup = np.array([50, 60, 70, 80, 90, 100, 110, 120, 130, 140])\n",
103
+ "Y_sup = np.array([150, 180, 210, 240, 270, 300, 330, 360, 390, 420])\n",
104
+ "\n",
105
+ "axes[0].scatter(X_sup, Y_sup, c='blue', edgecolors='black', zorder=5)\n",
106
+ "# Fit a simple line\n",
107
+ "z = np.polyfit(X_sup, Y_sup, 1)\n",
108
+ "p = np.poly1d(z)\n",
109
+ "axes[0].plot(X_sup, p(X_sup), 'r--', label=f'y = {z[0]:.1f}x + {z[1]:.0f}')\n",
110
+ "axes[0].set_xlabel('Area (mΒ²)')\n",
111
+ "axes[0].set_ylabel('Price (€k)')\n",
112
+ "axes[0].set_title('Supervised: Regression\\n(known labels guide the model)')\n",
113
+ "axes[0].legend()\n",
114
+ "axes[0].grid(True, alpha=0.3)\n",
115
+ "\n",
116
+ "# --- Supervised Learning (Classification) ---\n",
117
+ "# Two classes with labels\n",
118
+ "np.random.seed(0)\n",
119
+ "class0 = np.random.randn(20, 2) + [2, 2]\n",
120
+ "class1 = np.random.randn(20, 2) + [6, 6]\n",
121
+ "\n",
122
+ "axes[1].scatter(class0[:, 0], class0[:, 1], c='blue', label='Class 0', edgecolors='black')\n",
123
+ "axes[1].scatter(class1[:, 0], class1[:, 1], c='red', label='Class 1', edgecolors='black')\n",
124
+ "axes[1].set_xlabel('Feature 1')\n",
125
+ "axes[1].set_ylabel('Feature 2')\n",
126
+ "axes[1].set_title('Supervised: Classification\\n(colours = known labels)')\n",
127
+ "axes[1].legend()\n",
128
+ "axes[1].grid(True, alpha=0.3)\n",
129
+ "\n",
130
+ "# --- Unsupervised Learning (Clustering) ---\n",
131
+ "# Same data but NO labels\n",
132
+ "all_points = np.vstack([class0, class1])\n",
133
+ "axes[2].scatter(all_points[:, 0], all_points[:, 1], c='gray', edgecolors='black')\n",
134
+ "axes[2].set_xlabel('Feature 1')\n",
135
+ "axes[2].set_ylabel('Feature 2')\n",
136
+ "axes[2].set_title('Unsupervised: Clustering\\n(no labels β€” algorithm finds groups)')\n",
137
+ "axes[2].grid(True, alpha=0.3)\n",
138
+ "\n",
139
+ "plt.tight_layout()\n",
140
+ "plt.show()"
141
+ ]
142
+ },
143
+ {
144
+ "cell_type": "markdown",
145
+ "metadata": {},
146
+ "source": [
147
+ "---\n",
148
+ "\n",
149
+ "## Reinforcement Learning: Agent-Environment Loop\n",
150
+ "\n",
151
+ "```\n",
152
+ " β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” Action (A₁) β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”\n",
153
+ " β”‚ Agent β”‚ ───────────────▢ β”‚ Environment β”‚\n",
154
+ " β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n",
155
+ " β–² β”‚\n",
156
+ " β”‚ State (S₁) β”‚\n",
157
+ " β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n",
158
+ " β–²\n",
159
+ " β”‚ Reward (R)\n",
160
+ "```\n",
161
+ "\n",
162
+ "Let's simulate a simple RL agent learning to navigate a grid."
163
+ ]
164
+ },
165
+ {
166
+ "cell_type": "code",
167
+ "execution_count": null,
168
+ "metadata": {},
169
+ "outputs": [],
170
+ "source": [
171
+ "# Simple RL simulation: Agent learning to reach a goal\n",
172
+ "\n",
173
+ "class SimpleGridEnv:\n",
174
+ " def __init__(self, size=5):\n",
175
+ " self.size = size\n",
176
+ " self.goal = (size - 1, size - 1)\n",
177
+ " self.reset()\n",
178
+ " \n",
179
+ " def reset(self):\n",
180
+ " self.agent_pos = (0, 0)\n",
181
+ " return self.agent_pos\n",
182
+ " \n",
183
+ " def step(self, action):\n",
184
+ " \"\"\"actions: 0=up, 1=down, 2=left, 3=right\"\"\"\n",
185
+ " x, y = self.agent_pos\n",
186
+ " if action == 0: x = max(0, x - 1) # up\n",
187
+ " elif action == 1: x = min(self.size - 1, x + 1) # down\n",
188
+ " elif action == 2: y = max(0, y - 1) # left\n",
189
+ " elif action == 3: y = min(self.size - 1, y + 1) # right\n",
190
+ " \n",
191
+ " self.agent_pos = (x, y)\n",
192
+ " \n",
193
+ " if self.agent_pos == self.goal:\n",
194
+ " return self.agent_pos, 10.0, True # reward, done\n",
195
+ " else:\n",
196
+ " return self.agent_pos, -0.1, False # small penalty for each step\n",
197
+ "\n",
198
+ "# Q-Learning\n",
199
+ "env = SimpleGridEnv(size=5)\n",
200
+ "q_table = np.zeros((25, 4)) # 25 states (5x5 grid), 4 actions\n",
201
+ "\n",
202
+ "learning_rate = 0.1\n",
203
+ "discount_factor = 0.95\n",
204
+ "episodes = 500\n",
205
+ "\n",
206
+ "rewards_per_episode = []\n",
207
+ "\n",
208
+ "for episode in range(episodes):\n",
209
+ " state = env.reset()\n",
210
+ " state_idx = state[0] * 5 + state[1]\n",
211
+ " total_reward = 0\n",
212
+ " \n",
213
+ " for step in range(100):\n",
214
+ " # Epsilon-greedy action selection\n",
215
+ " if np.random.random() < 0.1:\n",
216
+ " action = np.random.randint(4)\n",
217
+ " else:\n",
218
+ " action = np.argmax(q_table[state_idx])\n",
219
+ " \n",
220
+ " next_state, reward, done = env.step(action)\n",
221
+ " next_idx = next_state[0] * 5 + next_state[1]\n",
222
+ " \n",
223
+ " # Q-Learning update\n",
224
+ " best_next = np.max(q_table[next_idx])\n",
225
+ " q_table[state_idx, action] += learning_rate * (\n",
226
+ " reward + discount_factor * best_next - q_table[state_idx, action]\n",
227
+ " )\n",
228
+ " \n",
229
+ " state_idx = next_idx\n",
230
+ " total_reward += reward\n",
231
+ " \n",
232
+ " if done:\n",
233
+ " break\n",
234
+ " \n",
235
+ " rewards_per_episode.append(total_reward)\n",
236
+ "\n",
237
+ "# Plot learning curve\n",
238
+ "plt.figure(figsize=(10, 4))\n",
239
+ "plt.plot(rewards_per_episode, alpha=0.5, label='Episode reward')\n",
240
+ "# Moving average\n",
241
+ "window = 20\n",
242
+ "if len(rewards_per_episode) >= window:\n",
243
+ " moving_avg = np.convolve(rewards_per_episode, np.ones(window)/window, mode='valid')\n",
244
+ " plt.plot(range(window-1, len(rewards_per_episode)), moving_avg, 'r-', linewidth=2, label=f'{window}-episode moving avg')\n",
245
+ "plt.xlabel('Episode')\n",
246
+ "plt.ylabel('Total Reward')\n",
247
+ "plt.title('RL Agent Learning Curve\\n(Agent learns to reach the goal)')\n",
248
+ "plt.legend()\n",
249
+ "plt.grid(True, alpha=0.3)\n",
250
+ "plt.show()\n",
251
+ "\n",
252
+ "print(f\"Average reward (last 50 episodes): {np.mean(rewards_per_episode[-50:]):.2f}\")\n",
253
+ "print(f\"Average reward (first 50 episodes): {np.mean(rewards_per_episode[:50]):.2f}\")"
254
+ ]
255
+ },
256
+ {
257
+ "cell_type": "markdown",
258
+ "metadata": {},
259
+ "source": [
260
+ "---\n",
261
+ "\n",
262
+ "## Related Fields\n",
263
+ "\n",
264
+ "| Field | Relevance to ML |\n",
265
+ "|---|---|\n",
266
+ "| **Statistics** | Probability, distributions, inference |\n",
267
+ "| **Dynamic Programming** | Optimisation, sequence problems |\n",
268
+ "| **State Transmission** | Modelling state changes |\n",
269
+ "| **Data Mining** | Pattern discovery in large datasets |"
270
+ ]
271
+ },
272
+ {
273
+ "cell_type": "markdown",
274
+ "metadata": {},
275
+ "source": [
276
+ "---\n",
277
+ "\n",
278
+ "## Summary\n",
279
+ "\n",
280
+ "| Concept | Key Takeaway |\n",
281
+ "|---|---|\n",
282
+ "| AI β†’ ML β†’ NN | Neural networks are a subset of ML |\n",
283
+ "| Supervised | Labelled data, classification & regression |\n",
284
+ "| Unsupervised | Unlabelled data, clustering & reduction |\n",
285
+ "| Reinforcement | Reward-based, agent-environment loop |\n",
286
+ "| Pipeline | DATA β†’ Algorithm β†’ Prediction Model |"
287
+ ]
288
+ },
289
+ {
290
+ "cell_type": "code",
291
+ "execution_count": null,
292
+ "metadata": {},
293
+ "outputs": [],
294
+ "source": [
295
+ "# Summary statistics from the RL experiment\n",
296
+ "print(\"=\" * 50)\n",
297
+ "print(\"CS401 Foundations β€” Notebook Complete\")\n",
298
+ "print(\"=\" * 50)\n",
299
+ "print(f\"Supervised learning demo: {len(X_sup)} labelled data points\")\n",
300
+ "print(f\"Unsupervised learning demo: {len(all_points)} unlabelled points\")\n",
301
+ "print(f\"RL experiment: {episodes} episodes, {len(q_table)} states\")\n",
302
+ "print(f\"\"\"\n",
303
+ "Key formulas covered in this course:\n",
304
+ " β€’ Hypothesis: h(x) = ΞΈβ‚€ + θ₁x\n",
305
+ " β€’ MSE Loss: J(ΞΈ) = (1/n) Ξ£(h(xα΅’) - yα΅’)Β²\n",
306
+ " β€’ GD Update: ΞΈβ±Ό := ΞΈβ±Ό - Ξ± Β· βˆ‚J/βˆ‚ΞΈβ±Ό\n",
307
+ " β€’ Normal Eq: ΞΈ = (Xα΅€X)⁻¹Xα΅€y\n",
308
+ " β€’ Sigmoid: Οƒ(z) = 1/(1 + e⁻ᢻ)\n",
309
+ "\"\"\")"
310
+ ]
311
+ }
312
+ ],
313
+ "metadata": {
314
+ "kernelspec": {
315
+ "display_name": "Python 3",
316
+ "language": "python",
317
+ "name": "python3"
318
+ },
319
+ "language_info": {
320
+ "codemirror_mode": {
321
+ "name": "ipython",
322
+ "version": 3
323
+ },
324
+ "file_extension": ".py",
325
+ "mimetype": "text/x-python",
326
+ "name": "python",
327
+ "nbconvert_exporter": "python",
328
+ "pygments_lexer": "ipython3",
329
+ "version": "3.11.0"
330
+ }
331
+ },
332
+ "nbformat": 4,
333
+ "nbformat_minor": 4
334
+ }