Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,40 +11,6 @@ pinned: false
|
|
| 11 |
|
| 12 |
Welcome to the **nanochat students** organization\! This is a community organization for students following Andrej Karpathy's [nanochat course](https://github.com/karpathy/nanochat). We are learning to build a full-stack LLM implementation from tokenization to web serving, all for under $100.
|
| 13 |
|
| 14 |
-
## Right Now!
|
| 15 |
-
|
| 16 |
-
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #f5f5f5; border: 2px solid #333; border-radius: 8px; max-width: 600px; margin: 20px auto; padding: 0;">
|
| 17 |
-
<div style="background: #333; color: white; padding: 16px 20px; border-bottom: 2px solid #333;">
|
| 18 |
-
<h2 style="margin: 0; font-size: 20px; font-weight: 600;">Day 1 of Nano Chat</h2>
|
| 19 |
-
</div>
|
| 20 |
-
|
| 21 |
-
<div style="padding: 20px;">
|
| 22 |
-
<div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 12px;">
|
| 23 |
-
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">1. Environment Setup</div>
|
| 24 |
-
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 25 |
-
Support on your Python environment using uv, create a virtual environment, and install all necessary dependencies for the nanochat project.
|
| 26 |
-
</div>
|
| 27 |
-
<a href="https://huggingface.co/spaces/nanochat-students/README/discussions/6" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View setup instructions →</a>
|
| 28 |
-
</div>
|
| 29 |
-
|
| 30 |
-
<div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 12px;">
|
| 31 |
-
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">2. Tokenizer Training</div>
|
| 32 |
-
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 33 |
-
Train a custom BPE tokenizer using Rust bindings.
|
| 34 |
-
</div>
|
| 35 |
-
<a href="https://huggingface.co/spaces/nanochat-students/README/discussions/3" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View tokenizer guide →</a>
|
| 36 |
-
</div>
|
| 37 |
-
|
| 38 |
-
<div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 0;">
|
| 39 |
-
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;">3. Pre-training</div>
|
| 40 |
-
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 41 |
-
Base training across 8 GPUs using torchrun, with metrics tracked in a shared trackio space below.
|
| 42 |
-
</div>
|
| 43 |
-
<a href="https://huggingface.co/spaces/nanochat-students/README/discussions/2" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">View pre-training steps →</a>
|
| 44 |
-
</div>
|
| 45 |
-
</div>
|
| 46 |
-
</div>
|
| 47 |
-
|
| 48 |
## What is nanochat?
|
| 49 |
|
| 50 |
nanochat is a complete implementation of an LLM like ChatGPT in a minimal, hackable codebase. It's designed as the capstone project for the LLM101n course by Eureka Labs, teaching you to build and train your own ChatGPT clone end-to-end.
|
|
@@ -82,4 +48,31 @@ Let's make this a fun, supportive, and efficient community of learners.
|
|
| 82 |
## **Resources**
|
| 83 |
|
| 84 |
- nanochat repo - [karpathy/nanochat](https://github.com/karpathy/nanochat)
|
| 85 |
-
- Introduction post: ["Introducing nanochat: The best ChatGPT that $100 can buy"](https://github.com/karpathy/nanochat/discussions/1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
Welcome to the **nanochat students** organization\! This is a community organization for students following Andrej Karpathy's [nanochat course](https://github.com/karpathy/nanochat). We are learning to build a full-stack LLM implementation from tokenization to web serving, all for under $100.
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
## What is nanochat?
|
| 15 |
|
| 16 |
nanochat is a complete implementation of an LLM like ChatGPT in a minimal, hackable codebase. It's designed as the capstone project for the LLM101n course by Eureka Labs, teaching you to build and train your own ChatGPT clone end-to-end.
|
|
|
|
| 48 |
## **Resources**
|
| 49 |
|
| 50 |
- nanochat repo - [karpathy/nanochat](https://github.com/karpathy/nanochat)
|
| 51 |
+
- Introduction post: ["Introducing nanochat: The best ChatGPT that $100 can buy"](https://github.com/karpathy/nanochat/discussions/1)
|
| 52 |
+
|
| 53 |
+
---
|
| 54 |
+
|
| 55 |
+
## Journal!
|
| 56 |
+
|
| 57 |
+
Check out these steps to join in or get help:
|
| 58 |
+
|
| 59 |
+
### Day 1
|
| 60 |
+
|
| 61 |
+
<div style="background: white; border: 1px solid #ddd; border-radius: 4px; padding: 16px; margin-bottom: 12px;">
|
| 62 |
+
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;"><a href="https://huggingface.co/spaces/nanochat-students/README/discussions/6" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">1. Environment Setup →</a></div>
|
| 63 |
+
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 64 |
+
Support on your Python environment using uv, create a virtual environment, and install all necessary dependencies for the nanochat project.
|
| 65 |
+
</div>
|
| 66 |
+
|
| 67 |
+
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;"><a href="https://huggingface.co/spaces/nanochat-students/README/discussions/3" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">2. Tokenizer Training→</a>
|
| 68 |
+
</div>
|
| 69 |
+
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 70 |
+
Train a custom BPE tokenizer using Rust bindings.
|
| 71 |
+
</div>
|
| 72 |
+
|
| 73 |
+
<div style="font-weight: 600; font-size: 16px; margin-bottom: 8px; color: #333;"><a href="https://huggingface.co/spaces/nanochat-students/README/discussions/2" style="color: #0066cc; text-decoration: none; font-size: 14px;" target="_blank">3. Pre-training →</a></div>
|
| 74 |
+
<div style="color: #666; font-size: 14px; line-height: 1.5; margin-bottom: 8px;">
|
| 75 |
+
Base training across 8 GPUs using torchrun, with metrics tracked in a shared trackio space below.
|
| 76 |
+
</div>
|
| 77 |
+
|
| 78 |
+
</div>
|