---
license: mit
datasets:
- MuzzammilShah/people-names
language:
- en
model_name: Batch Normalization for Neural Networks
library_name: pytorch
tags:
- makemore
- batch-normalization
- neural-networks
- andrej-karpathy
---

# Batch Normalization for Neural Networks: Makemore (Part 3)

In this repository, I implemented **Batch Normalization** within a neural network framework to enhance training stability and performance, following Andrej Karpathy's approach in the **Makemore - Part 3** video.

## Overview
This implementation focuses on:
- **Normalizing activations and gradients**.
- Addressing initialization issues.
- Utilizing Kaiming initialization to prevent saturation of activation functions.

Additionally, **visualization graphs** were created at the end to analyze the effects of these techniques on the training process and model performance.

## Documentation
For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part3/)**.

## Acknowledgments
Notes and implementations inspired by the **Makemore - Part 3** video by [Andrej Karpathy](https://karpathy.ai/).

For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).