Upload folder using huggingface_hub

README.md

@@ -0,0 +1,18 @@
+---
+license: mit
+language: en
+datasets: [roneneldan/TinyStories]
+tags: [tiny, from-scratch, pytorch, educational, text-generation]
+---
+# Tiny Storyteller
+
+A 5.9M-parameter GPT-style language model trained **from scratch** (random init, no pretrained weights)
+on 20,000 TinyStories, as a learning project. Use case: a bedtime story starter for young kids.
+
+- Architecture: 6 transformer blocks, 4 heads, 256-dim embeddings, 256-token context
+- Tokenizer: byte-level BPE, 4,096 tokens, trained on the same data
+- Training: 1500 steps, batch 32, AdamW, cosine schedule
+- Final validation loss 2.185 (perplexity 8.9)
+
+The full build, explained step by step, is in `tiny_llm_from_scratch.ipynb` in this repo.
+Load with plain PyTorch: `GPT(cfg).load_state_dict(torch.load("model.pt"))`.

config.json

@@ -0,0 +1,8 @@
+{
+  "vocab_size": 4096,
+  "block_size": 256,
+  "n_embd": 256,
+  "n_head": 4,
+  "n_layer": 6,
+  "dropout": 0.0
+}

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:514b81b866785efaf42512b1ab7e368384987b90ec4eb83351c59d8a96a5f532
+size 23434075

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}