README.md

metadata

title: README
emoji: 🌖
colorFrom: gray
colorTo: yellow
sdk: static
pinned: false

Welcome to the nanochat students organization! This is a community organization for students following Andrej Karpathy's nanochat course. We are learning to build a full-stack LLM implementation from tokenization to web serving, all for under $100.

Right Now!

Day 1 of Nano Chat

<div style="padding: 20px;">
    <div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 12px;">
        <div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">1. Environment Setup</div>
        <div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
            Support on your Python environment using uv, create a virtual environment, and install all necessary dependencies for the nanochat project.
        </div>
        <a href="https://huggingface.co/spaces/nanochat-students/README/discussions/6" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View setup instructions →</a>
    </div>
    
    <div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 12px;">
        <div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">2. Tokenizer Training</div>
        <div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
            Train a custom BPE tokenizer using Rust bindings.
        </div>
        <a href="https://huggingface.co/spaces/nanochat-students/README/discussions/3" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View tokenizer guide →</a>
    </div>
    
    <div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 0;">
        <div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">3. Pre-training</div>
        <div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
            Base training across 8 GPUs using torchrun, with metrics tracked in a shared trackio space below.
        </div>
        <a href="https://huggingface.co/spaces/nanochat-students/README/discussions/2" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View pre-training steps →</a>
    </div>
</div>

What is nanochat?

nanochat is a complete implementation of an LLM like ChatGPT in a minimal, hackable codebase. It's designed as the capstone project for the LLM101n course by Eureka Labs, teaching you to build and train your own ChatGPT clone end-to-end.

What You'll Find Here

This organization hosts community-contributed resources to help you learn and succeed with nanochat. You'll find:

• notebooks that break down the implementation.
• spaces that demo or illustrate the concepts we’re learning.
• trained models and checkpoints from the community
• relevant curated datasets.

Getting Help and Sharing Ideas

The Discussions section is where you can ask questions, share your training results and report cards, discuss optimization techniques, and collaborate on experiments.

Contributing

We welcome contributions from all students or experts. Here's how you can help: notebooks, demos, models, and articles:

• Join the org, we'll give you write access.
• If you make anything, share it in this discussion thread
• If you can, help answer questions in discussions

Let's make this a fun, supportive, and efficient community of learners.

Resources

• nanochat repo - karpathy/nanochat
• Introduction post: "Introducing nanochat: The best ChatGPT that $100 can buy"