Add zero-shot benchmark results (ARC, HellaSwag, SciQ, PIQA) and full-split val ppl

README.md

@@ -16,6 +16,7 @@ datasets:
   - roneneldan/TinyStories
 metrics:
   - perplexity
+  - accuracy
 ---
 
 # TinyStories-24m
@@ -37,13 +38,30 @@ narrative flow.
 
 ## Quality
 
-- **Val perplexity:** 8.76 (2.1618 nats/token on the 2M held-out val set).
+- **Val perplexity:** 8.76 (2.1618 nats/token, training-time fixed-window score).
+  Full-split repro (100 random 512-token windows over the 4.5M-token val split):
+  **12.39** (2.5172 nats/token). The gap is methodology (fixed window vs random
+  windows), not a card error.
 - **Generation:** coherent. Sampled 9/9 seeded generations (3 seeds × 3 prompts)
   produce proper dialogue, character names (Ben, Lily, Mom, Tom, Sarah, Max),
   punctuation and narrative flow. This model is a story generator for its
   training domain — it is **not** a general-purpose assistant and will not
   answer questions it was not trained on.
 
+### Benchmark results (zero-shot loglikelihood, 400 examples per task)
+
+| Task | Accuracy | Chance | Notes |
+|------|----------|--------|-------|
+| ARC-Easy | 13.3% | 25% | below chance |
+| ARC-Challenge | 12.5% | 25% | below chance |
+| HellaSwag | 25.0% | 25% | at chance |
+| SciQ | 25.0% | 25% | at chance |
+| PIQA | 50.0% | 50% | at chance |
+
+All results are at or below chance — expected for a 24M model trained exclusively
+on simple children's stories. The model has learned the distribution of story
+text but has no general reasoning, commonsense, or science knowledge.
+
 ## Honest caveats
 
 - **Divergence:** the full 13,600-step run diverged to NaN at step 9,350 (LR 6e-4
@@ -85,4 +103,4 @@ print(tok.decode(ids, skip_special_tokens=True))
 | `model.safetensors` | 98,349,056 | 75 tensors, float32 |
 | `config.json` | — | architecture + training metadata |
 | `modeling.py` | — | the `TinyStoriesGPT` class (load with `from_pretrained`) |
-| `tokenizer.json` | 560,804 | BPE-8k tokenizer (HF `tokenizers` format) |
+| `tokenizer.json` | 560,804 | BPE-8k tokenizer (HF `tokenizers` format) |