singhpratech/emc2-lesson-03-four-sizes

e=mc² Lesson 03: Why Momo Recites

Four GPT-style models (0.95M, 3.06M, 5.85M and 12.3M parameters) trained from random weights with the same recipe, data, tokenizer and 1,500 steps as Lesson 01 (Tiny Storyteller), to see where verbatim recitation of training stories begins.

Lesson: https://theaivibe.org/emc2/why-small-llms-memorize-train-four-sizes

• lesson_03.ipynb runs everything (free Colab T4 or a Mac). Run all cells: the clickable toy and the figure appear when the cells execute.
• RESULTS.md holds every number, the leakage check and the caveats.
• quiz_loss_vs_params.png plots loss against parameters.
• tiny-storyteller-1m|3m|6m|12m/ hold model.pt and config.json. The tokenizer is Lesson 01's (singhpratech/tiny-storyteller); the notebook loads it from there. The model class is in the notebook.

Load one: GPT(Config(n_layer, n_embd)).load_state_dict(torch.load("model.pt")) with the sizes in config.json.