# TinyStories demo slice (~700 KB train + 70 KB valid)

This is a small slice of the **TinyStories** dataset by Eldan & Li (2023):
- Source: https://huggingface.co/datasets/roneneldan/TinyStories
- Original file: `TinyStoriesV2-GPT4-valid.txt` (~22 MB)
- This slice: the first ~700 KB of stories from that file, packed as raw uint8 bytes
- License: the upstream dataset is CC-BY-4.0; this redistribution preserves that license

## Files

| File | Size | Purpose |
|---|---|---|
| `train.bin` | ~700 KB | training shard (uint8 byte sequence) |
| `valid.bin` | ~70 KB | held-out validation shard |

## Why this slice and not the full thing

The full TinyStories train file is ~2 GB. We didn't want every kit user to download
2 GB just to do their first smoke-training run. 700 KB is enough to:

- Run 50–500 training steps in a few minutes on CPU and see loss fall
- Verify your install end-to-end
- Get a feel for how the trainer behaves before committing to a real run

For a real ~10M-param training run you want millions of bytes minimum; download
the full dataset from the source URL above and point `--data-dir` at it.

## Format

Files are flat sequences of `uint8` bytes — no headers, no separators between
stories beyond the natural `<|endoftext|>` strings inside the text. The trainer
memmaps these and samples random windows of `seq_len` bytes. Each byte IS a
token (vocabulary size = 256).