Update README.md

README.md

@@ -1,10 +1,95 @@
+# LittleLearner
+
+Research checkpoints for **LittleLearner: Language Models Under Pedagogically
+Controlled Knowledge Exposure** — a study of how filtering the
+pretraining corpus to K–5 (kindergarten through grade 5) educational content
+affects a language model's downstream capabilities.
+
+This organisation hosts the paired **LittleLearner** (K–5-filtered) and
+**Unfiltered** (baseline general corpus) checkpoints at three sizes, plus
+matched post-trained variants. All checkpoints share the same architecture
+and tokenizer, so a LittleLearner *X* B and Unfiltered *X* B differ only in
+what data they saw during pretraining — the controlled comparison the paper
+is built around.
+
+📄 Paper: **<link>** · ✉️ Contact: **fanfei.li@tuebingen.mpg.de**
+
 ---
-title: README
-emoji: 🏆
-colorFrom: blue
-colorTo: green
-sdk: static
-pinned: false
+
+## Naming conventions
+
+Two model families:
+
+- **LittleLearner** — pretrained on the K–5-filtered corpus.
+- **Unfiltered** — pretrained on the same corpus without the K–5 filter (baseline general corpus).
+
+Post-training data uses the same two names:
+
+- **K-5** — the K–5-filtered instruction/SFT corpus (matched to LittleLearner pretraining).
+- **Unfiltered** — the unfiltered instruction/SFT corpus.
+
 ---
 
-Edit this `README.md` markdown file to author your organization card.
+## Released checkpoints
+
+### Base — pretraining only, no chat template
+
+| Size | LittleLearner (K-5-filtered) | Unfiltered |
+|---|---|---|
+| 0.6 B | `littlelearner/<...>-0.6b-bounded` | `littlelearner/<...>-0.6b-unbounded` |
+| 1.3 B | `littlelearner/<...>-1.3b-bounded` | `littlelearner/<...>-1.3b-unbounded` |
+| 5 B   | `littlelearner/<...>-5b-bounded`   | `littlelearner/<...>-5b-unbounded`   |
+
+*Use these in **completion** mode — do not apply a chat template.*
+
+### Post-trained — SFT + GRPO, chat template **required**
+
+Encoded as **(pretrain → post-training)**:
+
+| Pretrain → Post-training | Description | Repo |
+|---|---|---|
+| LittleLearner → K-5           | Matched-pipeline LL post-training | `littlelearner/littlelearner-5b-bounded-sft-chatty` |
+| LittleLearner → Unfiltered    | Hybrid: LL pretrain, Unfiltered post-training data | `<...>` |
+| Unfiltered → Unfiltered       | Matched-pipeline Unfiltered post-training | `littlelearner/littlelearner-5b-unbounded-sft-chatty` |
+
+
+---
+
+## Which checkpoint should I use?
+
+- **Studying the effect of the K-5 filter on pretraining** → paired base checkpoints at matched sizes (`…-bounded` vs `…-unbounded`).
+- **K-5 math / educational assistant** → LittleLearner → K-5 SFT+GRPO chatty.
+- **General-purpose small assistant** → Unfiltered → Unfiltered SFT+GRPO chatty.
+- **Effect of post-training data on a filtered pretrain** → compare the two `LittleLearner → …` SFT+GRPO variants.
+
+---
+
+## Important inference notes
+
+All chatty checkpoints share two non-standard defaults. Please read the
+individual model cards for the exact runnable snippet, but at a glance:
+
+1. **No system prompt.** These models were SFT'd without a system message;
+   leaving transformers' default system prompt on, or adding your own,
+   measurably degrades on-task accuracy. Build the conversation as
+   `messages = [{"role": "user", "content": ...}]` only — no system role.
+2. **Custom stop tokens.** vLLM's default stop-token list derived from the
+   tokenizer is **not** sufficient — generation can run past the assistant
+   turn. Pass the `stop_token_ids=[…]` documented on each model card.
+
+Base checkpoints do not carry a chat template and should be used in
+completion mode.
+
+---
+
+## Citation
+
+```bibtex
+@article{littlelearner2026,
+  title   = {LittleLearner: Language Models Under Pedagogically-Controlled Knowledge Exposure},
+  author  = {Fanfei Li and Jana Zeller and Manuel Prada-Corral and Thadd{\"a}us Wiedemer
+             and Prasanna Mayilvahanan and Ryan Cotterell and Wieland Brendel},
+  journal = {arXiv preprint arXiv:26xx.xxxxx},
+  year    = {2026}
+}
+```