Update README.md

README.md

@@ -1,3 +1,143 @@
 ---
+language:
+- en
+tags:
+- text-generation
+- causal-lm
+- edge-ai
+- lightweight
 license: apache-2.0
+pipeline_tag: text-generation
 ---
+
+# Monad-1
+
+Monad-1 is an ultra-compact, **7.3-million-parameter** causal language model built from scratch to explore
+parameter efficiency, lightweight architectural dynamics, and foundational syntax acquisition. It is trained
+entirely from the ground up — no distillation, no pretrained backbone — to test how much linguistic structure
+a minimal decoder-only Transformer can internalize.
+
+Despite its minimal footprint, Monad-1 successfully learns core English structural mechanics, grammar, and
+token co-occurrence patterns, performing over **2,500× better than random chance** on standard next-token
+evaluation tasks.
+
+**Disclaimer:** Monad-1 is an experimental, research-scale model. It is not intended to compete with
+large-scale foundation models on knowledge-intensive or reasoning-heavy tasks — see
+[Intended Use & Limitations](#intended-use--limitations) below.
+
+## Model Highlights
+
+- **Ultra-lightweight footprint** — at ~7.3M parameters, Monad-1 runs effortlessly on virtually any device,
+  including low-power edge hardware and CPU-only environments.
+- **Modern Transformer foundations** — built on standard GPT-2 decoder-only principles using scaled
+  dot-product attention, LayerNorm, and zero-bias linear layers.
+- **Standard BPE tokenization** — uses the standard 50,257-token vocabulary, compatible with `gpt2` /
+  `tiktoken` encodings, so it drops into existing tooling with no custom tokenizer required.
+- **Safe & fast weight serialization** — distributed natively in `.safetensors` format for zero-copy loading
+  and safe deserialization.
+
+## Model Details
+
+Monad-1 is a causal language model with its own custom architecture, registered under the model type
+`monad` with its own `MonadForCausalLM` class and `config.json` — it is not a repurposed GPT-2 checkpoint,
+though it follows similar decoder-only design principles. The full model totals roughly 7.3 million
+parameters, arranged across 4 transformer layers with a hidden dimension of 128 and 4 attention heads per
+layer. It operates over a context window of 512 tokens and shares the standard 50,257-token vocabulary used
+by GPT-2 / tiktoken encodings, which keeps it compatible with widely available tokenizers. Weights are
+distributed in both float32 and float16 precision, packaged as SafeTensors (`model.safetensors`) for safe,
+zero-copy loading.
+
+## Usage
+
+Monad-1 is compatible with the Hugging Face 🤗 Transformers library via `trust_remote_code`.
+
+### 1. Requirements
+
+```bash
+pip install torch safetensors transformers
+```
+
+### 2. Loading the model
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load tokenizer (standard GPT-2 BPE vocabulary)
+tokenizer = AutoTokenizer.from_pretrained("gpt2")
+
+# Load model weights from local repository or Hugging Face directory
+model = AutoModelForCausalLM.from_pretrained(
+    "./monad-1",
+    trust_remote_code=True,
+    torch_dtype=torch.float32,
+)
+```
+
+### 3. Generating text
+
+```python
+prompt = "The journey of learning begins with"
+inputs = tokenizer(prompt, return_tensors="pt")
+
+output = model.generate(
+    **inputs,
+    max_new_tokens=30,
+    temperature=0.8,
+    do_sample=True,
+)
+
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+## Benchmark Performance
+
+Monad-1 was evaluated on the LAMBADA benchmark suite using zero-shot sampling (T = 1.2):
+
+| Metric | Monad-1 Result | Random Chance Baseline |
+|---|---|---|
+| Accuracy (exact word completion) | 5.01% | ~0.002% |
+| Perplexity | 11,600 | ~50,257 |
+
+**Key insight:** for a 7.3M-parameter model with a 128-dimensional hidden state, scoring 5.01% exact-match
+accuracy demonstrates that the model has successfully internalized core language syntax, vocabulary bounds,
+and phrase construction rules — well beyond what would be expected from chance alone.
+
+## Intended Use & Limitations
+
+### Intended use
+
+Monad-1 is intended for:
+
+- Educational research into parameter-efficient language modeling
+- Studying architectural dynamics and syntax acquisition at small scale
+- Lightweight text completion on resource-constrained or edge devices
+- Prototyping and iterating on custom Transformer architectures before scaling up
+
+### Limitations
+
+Due to its 7.3M-parameter size and 128-dimensional representation layer, Monad-1 has significant limitations:
+
+- **World knowledge:** minimal capacity for factual recall or broad world-knowledge retrieval
+- **Reasoning:** not capable of complex multi-step or chain-of-thought reasoning
+- **Long-form coherence:** limited long-form context memory beyond its 512-token window; may lose
+  coherence over longer generations
+- **Hallucination:** like larger language models, Monad-1 can generate fluent but factually incorrect or
+  nonsensical text, and this effect is more pronounced given its limited capacity
+
+We recommend against deploying Monad-1 in any high-stakes, safety-critical, or decision-making context. It
+is best suited to research, experimentation, and lightweight on-device demos rather than production
+applications requiring factual accuracy or robust reasoning.
+
+## Citation
+
+If you use Monad-1 in your research, please cite:
+
+```bibtex
+@misc{monad1,
+  title  = {Monad-1: An Ultra-Compact 7.3M-Parameter Causal Language Model},
+  author = {MonadAi},
+  year   = {2026},
+  url    = {https://huggingface.co/MonadAi/monad-1}
+}
+```