Upload README.md with huggingface_hub

README.md

@@ -1,52 +1,39 @@
 ---
-language: en
-license: mit
-library_name: custom
+language:
+- en
+library_name: julia
+pipeline_tag: text-generation
 tags:
-  - gpt
-  - character-level
-  - transformer
-  - from-scratch
-  - ancient-scripts
-  - classical-texts
+- character-level
+- philosophy
+- mathematics
+- julia
+- scalar-autograd
+- pure-julia
+- scriptio-continua
+- reduced-vocab
 datasets:
-  - custom
-pipeline_tag: text-generation
+- LisaMegaWatts/juliagpt-data
 ---
 
 # JuliaGPT
 
-An optimized character-level GPT in Julia for training on ancient scripts and classical texts. Evolution of [MicroJulia](https://github.com/DavinciDreams/micro-julia).
-
-[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DavinciDreams/JuliaGPT/blob/main/juliagpt.ipynb)
-
-## Roadmap
-
-Starting from MicroJulia's minimal scalar-autograd GPT, optimizing toward:
-
-- Array-based autograd for 100-1000x speedup
-- Multi-layer transformers with GELU activations
-- Learnable RMSNorm, gradient clipping, cosine LR schedule
-- Ancient script support (Greek, Latin, Cuneiform, etc.)
-- Flexible vocabulary configuration per script
-- Batched training and proper attention masking
-
-## Current Architecture
-
-- Custom autograd engine in pure Julia
-- Transformer with multi-head attention
-- Character-level tokenization
-- Adam optimizer with LR decay
-- W&B logging + HuggingFace Hub integration
+An experimental character-level GPT in pure Julia exploring minimal vocabularies inspired by ancient Greek *scriptio continua*. Built with scalar autograd, no external ML dependencies.
 
-## Quick Start
+## Architecture
+- 1 transformer layer, 4 attention heads
+- n_embd=16, block_size=256
+- RMSNorm, ReLU, KV cache for causal masking
+- Adam optimizer with linear LR decay
+- ~5K parameters
 
-1. Click "Open in Colab" above
-2. Add Colab secrets: `HF_TOKEN`, `WANDB_KEY`, `HF_REPO`
-3. Run Python login cell, install Julia, switch to Julia 1.10
-4. Run all cells
+## Vocabulary
+28 characters (a-z + space + period) + BOS = 29 vocab. Numerals converted to words, all punctuation removed except period.
 
-## Related
+## Training
+- **Dataset:** Aristotle's Rhetoric + Euclid's Elements (8,461 chunks)
+- **Current checkpoint:** step 650, val_loss=2.3414
 
-- [micro-julia](https://github.com/DavinciDreams/micro-julia) - Original minimal implementation
-- [text-pipeline](https://github.com/DavinciDreams/text-pipeline) - Text processing pipeline for training data
+## Links
+- [Training data](https://huggingface.co/datasets/LisaMegaWatts/juliagpt-data)
+- [Source code](https://github.com/DavinciDreams/JuliaGPT)