README.md

metadata

title: LLM From Scratch
emoji: 🧠
colorFrom: green
colorTo: red
sdk: streamlit
sdk_version: 1.30.0
app_file: app.py
pinned: false

LLM From Scratch

Ibrahim Khalid

The hosted project is available on HuggingFace

The purpose of this project is to build a simple large language model from scratch.

This repo is following the guide from https://www.youtube.com/watch?v=UU1WVnMk4E8

In this repo:

• ./shakespeare.txt - This is a sample text used for training a smaller scale model
• ./bigram/bigram_testing.sync.ipynb - This notebook is where I test a basic BiGram model
• ./bigram/bigram_testing.sync.py - Python script for BiGram testing
• ./simple_gpt/gpt_shakespeare.sync.ipynb - Notebook implementing simple GPT model using entire works of shakespeare
• ./simple_gpt/gpt_shakespeare.sync.py - Python script for GPT shakespeare model

Prepare environment

pip install -r ./pre-requirements.txt
pip install -r ./requirements.txt

---

On GitHub

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference