{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 🎓 Lab 00: The Simplest Starting Point\n",
    "\n",
    "Welcome! Before we spin up massive cloud GPUs or dive into complex AI loops, let's start with the simplest, most fundamental step of training any AI model: **The Data and The Tokenizer**.\n",
    "\n",
    "We can run this entire notebook locally on your laptop's CPU. No cloud required!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Where is our Data?\n",
    "For our autonomous research agent, we are using the **TinyStories** dataset. These are short stories generated by GPT-4 designed to teach small models english grammar and reasoning without needing billions of parameters.\n",
    "\n",
    "Let's look at the data prep script we already ran."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "data_dir = os.path.expanduser(\"~/.cache/autoresearch/data\")\n",
    "print(f\"Checking {data_dir}...\")\n",
    "\n",
    "# List the downloaded parquet files\n",
    "files = sorted(os.listdir(data_dir))\n",
    "for f in files[:5]:\n",
    "    print(f\"Found shard: {f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Read a Tiny Story\n",
    "Let's actually open one of these data shards and read what a \"Tiny Story\" looks like. This is exactly what the AI will read over and over again to learn how to speak."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read the first shard using pandas (pyarrow required)\n",
    "df = pd.read_parquet(os.path.join(data_dir, files[0]))\n",
    "\n",
    "# Print the very first story!\n",
    "story = df.iloc[0]['text']\n",
    "print(\"--- STORY ---\")\n",
    "print(story)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: The Tokenizer (How the AI Reads)\n",
    "AI doesn't read letters. It reads \"Tokens\" (groups of characters). Earlier, we trained a custom **Byte-Pair Encoding (BPE)** tokenizer specifically on this dataset.\n",
    "\n",
    "Let's load the tokenizer and see how it converts the story above into numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"autoresearch\")\n",
    "from autoresearch.prepare import Tokenizer\n",
    "\n",
    "# Load our trained tokenizer\n",
    "tokenizer = Tokenizer.from_directory()\n",
    "print(f\"Our AI has a vocabulary of {tokenizer.get_vocab_size()} unique tokens.\")\n",
    "\n",
    "# Let's translate a sentence into tokens!\n",
    "sentence = \"Once upon a time, there was a little dog.\"\n",
    "tokens = tokenizer.encode(sentence)\n",
    "\n",
    "print(f\"\\nSentence: {sentence}\")\n",
    "print(f\"Tokens: {tokens}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: The Mojo Compiler (Our Secret Weapon)\n",
    "Instead of writing the neural network in Python (which is slow), we pivoted to writing it in **Mojo**.\n",
    "Let's execute the Mojo code we just wrote right here in the notebook using the shell!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "cd autoresearch/mojo && ~/.pixi/bin/pixi run mojo train.mojo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🎉 Congratulations!\n",
    "You just:\n",
    "1. Loaded raw AI training data.\n",
    "2. Tokenized text like a neural network does.\n",
    "3. Compiled a blazing-fast Mojo file.\n",
    "\n",
    "You are now officially building an AI lab from the ground up."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}