File size: 2,257 Bytes
4f1d200
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
---
license: mit
tags:
  - image-classification
  - mnist
  - numpy
  - from-scratch
  - educational
datasets:
  - mnist
metrics:
  - accuracy
---

# Four-Layer Neural Network from Scratch (NumPy)

A fully-connected network built with **only NumPy**: hand-derived backpropagation and mini-batch SGD, no deep-learning framework. It reaches **97.83% test accuracy** on MNIST and trains in about 88 seconds on a Colab CPU.

## Model Details

- **Author:** Vishwajeet Adkine
- **Architecture:** `784 → 128 → 64 → 32 → 10` (four weight layers, 111,146 parameters)
- **Activations / loss:** ReLU hidden layers, softmax output, cross-entropy with optional L2
- **Initialisation:** He (normal, variance 2 / fan-in)
- **Implementation:** NumPy `NeuralNetwork` class with `forward`, `backward`, `step` and `fit`; backprop verified against numerical gradients before training

## Intended Use

Learning and demonstration: seeing exactly how forward propagation, backpropagation and SGD work, without framework abstractions. **Not intended** for production use.

## Training

| | |
|---|---|
| Data | MNIST (90% train / 10% validation split of the training set, official test set held out) |
| Preprocessing | pixels scaled to [0, 1], flattened to 784 |
| Optimiser | mini-batch SGD, batch size 128 |
| Learning rate | 0.1 with ×0.95 decay per epoch |
| Regularisation | L2, λ = 1e-4 |
| Epochs | 30 (default notebook config) |
| Hardware | Colab CPU |

## Results

| Metric | Value |
|--------|-------|
| Test accuracy | **97.83%** |
| Test loss | 0.0824 |
| Training time | 88.2 s (Colab CPU) |

## Limitations

- Fully-connected layers ignore image structure, so accuracy is below convolutional models (the CIFAR-10 CNN project in this series covers that step).
- Plain SGD with a fixed decay: no momentum, Adam, dropout or batch normalisation.
- Evaluated on MNIST only, which is an easy, clean benchmark. Do not expect this accuracy on other digit images (different handwriting, backgrounds or scanning).
- No robustness or calibration analysis.

## How to Use

```python
net = load_weights("nn_weights.npz")     # helper defined in the notebook
pred = net.predict(X_test[:5])
```

## Licence

Code and weights: MIT. MNIST is by LeCun, Cortes and Burges.