Update README.md

README.md

@@ -1,3 +1,129 @@
 ---
+language: en
 license: apache-2.0
+tags:
+- text-generation
+- small-language-model
+- slm
+pipeline_tag: text-generation
+inference: true
+widget:
+- text: "What is the capital of France?"
+  example_title: "Geography"
+- text: "7 * 8 ="
+  example_title: "Arithmetic"
 ---
+
+# Haiku H2
+
+Haiku H2 is the canonical successor to Haiku Mini — a 217M-parameter small language model from **Rootcomputer**. It features a modernized architecture, a complete SFT + DPO post-training pipeline, and a published benchmark profile that wins its weight class.
+
+H2 is available for public testing in open beta via [chathaiku.com](https://chathaiku.com/). It is an experimental research model, designed to make small-model behavior studyable end-to-end, validate training pipelines, and publish reproducible benchmarks against comparable open small models.
+
+---
+
+## Model Specifications
+
+H2 is a decoder-only transformer architecture optimized to keep parameter counts low without sacrificing capacity.
+
+| Specification | Value |
+| :--- | :--- |
+| **Parameters** | 217M |
+| **Transformer Layers** | 16 |
+| **Embedding Dimension** | 1,024 |
+| **Attention Mechanism** | Grouped-Query Attention (GQA) with 4 KV heads |
+| **Positional Base** | 50k RoPE (Rotary Positional Embeddings) |
+| **Activation Function** | SwiGLU |
+| **Embeddings** | Tied Input/Output Embeddings |
+| **Precision & Scaling** | bf16 (Flash Attention 2 supported) |
+
+---
+
+## Training Pipeline
+
+H2 is trained across three distinct stages. Each stage is validated independently against a stable upstream checkpoint to allow cheap, rapid iteration on commodity hardware.
+
+1. **Pretrain:** Base language modeling executed on the Rootcomputer multi-source corpus, comprising web text, books, news, scientific papers, and biomedical sources.
+2. **SFT (Supervised Fine-Tuning):** Conversational tuning that teaches the model the shape of instructions, responses, refusals, and proper output formatting.
+3. **DPO (Direct Preference Optimization):** Preference alignment on accumulated chosen/rejected pairs to sharpen identity consistency, formatting adherence, false-premise handling, and response quality. *Note: DPO is always retrained from the SFT base, never stacked.*
+
+---
+
+## Evaluation: Featherweight v1 Benchmark
+
+H2 is evaluated on **Featherweight v1**, a 100-question multiple-choice benchmark designed for the sub-1B parameter class. The benchmark spans 10 categories (10 questions each) with balanced multiple-choice keys (25% each for A/B/C/D) to eliminate position bias. 
+
+Testing uses deterministic greedy decoding (`temperature=0`, `top_k=1`, `max_new_tokens=20`) across two formats:
+* **Plain Prompt:** Matches deployed evaluators, ending with *"Answer with just the letter of the correct choice."*
+* **Completion Prompt:** An MMLU-style format ending with a bare *"Answer:"* (fairer to non-instruction-tuned base models).
+
+### Key Takeaways
+* **38–39% Accuracy:** Solid performance across both prompt formats (random chance is 25%).
+* **1st in Weight Class:** Beats every tested model $\le$ 217M parameters by 7–8 points.
+* **Highly Robust:** Only a 1 percentage point swing between plain and completion prompts, making it the most prompt-robust model tested.
+
+### Comparison Leaderboard
+
+| Model | Params | Plain | Completion | $\Delta$ |
+| :--- | :---: | :---: | :---: | :---: |
+| distilgpt2 | 82M | 27% | 24% | −3 |
+| gpt2 | 124M | 23% | 26% | +3 |
+| opt-125m | 125M | 28% | 23% | −5 |
+| smollm-135m | 135M | 26% | 19% | −7 |
+| smollm-135m-instruct | 135M | 30% | 32% | +2 |
+| pythia-160m | 162M | 25% | 28% | +3 |
+| **Haiku H2** | **217M** | **38%** | **39%** | **+1** |
+| gpt2-medium | 355M | 24% | 23% | −1 |
+| smollm-360m | 362M | 20% | 29% | +9 |
+| pythia-410m | 405M | 25% | 32% | +7 |
+| qwen2.5-0.5b | 494M | 40% | 70% | +30 |
+| tinyllama-1.1b-chat | 1.1B | 45% | 60% | +15 |
+
+*$\Delta$ = completion − plain. With n = 100, the 95% confidence interval on individual scores is roughly $\pm$9 percentage points.*
+
+---
+
+## Capability Profile
+
+On the completion prompt, H2's category-specific breakdown shows an uneven but stable profile that remains strictly above the 25% random chance floor everywhere:
+
+* **Strongest Categories:** Geography (60%) and Arithmetic (50%).
+* **Weakest Categories:** Categorization (20%) and Logic (30%).
+
+### Size and Recency Gap
+When plotted against larger systems like **Qwen 2.5 (494M)** and **TinyLlama 1.1B Chat**, H2's performance curve sits nested uniformly inside theirs. Larger, more recent 2024-vintage systems outperform H2 on knowledge-heavy fields (e.g., Qwen reaches 100% on categorization). H2's primary claim is that it represents the strongest option within its explicit weight class and against older architectures up through ~410M parameters.
+
+---
+
+## Key Model Claims
+
+* **Wins its weight class:** Outperforms all tested models $\le$ 217M parameters on both prompt settings.
+* **Beats larger base models:** Comfortably clears larger pre-pretrained models like `gpt2-medium` (355M) and `pythia-410m` (405M) by 7–18 points, demonstrating that SFT+DPO alignment optimizes small weights far better than raw scale alone.
+* **Unmatched prompt robustness:** Shifts by only 1% between testing formats, while competitors swing by up to 15% (TinyLlama) or 30% (Qwen 2.5).
+
+---
+
+## Intended Use & Limitations
+
+### Best For
+* **Research baseline:** A fully-documented, reproducible small architecture to compare against, fork, or iterate on.
+* **Small-model behavior studies:** Investigating how SFT and DPO alignment mechanisms behave at the 200M parameter scale.
+* **Pipeline validation:** Testing architecture changes and data mixtures quickly and cheaply before scaling to larger models.
+
+### Out of Scope & Limitations
+* **Multi-step reasoning:** Logical reasoning is near chance (20–30%). Avoid chains of inference or syllogisms.
+* **Unreliable arithmetic:** Arithmetic beyond single-digit operations is unstable and prone to hallucinations (e.g., outputs like `7 × 8 = 57`).
+* **Corpus bounds:** Factual knowledge is strictly limited to the training corpus. Factual outputs should always be independently verified.
+* **Safety critical zones:** Not evaluated or certified for medical, legal, financial, or safety-critical deployment. 
+
+---
+
+## Reproducibility
+
+The validation framework is entirely open. Anyone using a standard CUDA GPU environment with `pip install transformers torch` can verify these benchmarks. 
+
+* `featherweight.jsonl`: Contains the 100 benchmark questions balanced evenly across choice keys.
+* `run_featherweight.py`: Standalone evaluation script supporting local Hugging Face pipelines, HTTP requests, and OpenAI-compatible backends using deterministic greedy decoding.
+
+---
+*Model card generated based on Featherweight v1 evaluation metadata, 2026-06-13.*