Upload folder using huggingface_hub

README.md

@@ -0,0 +1,45 @@
+---
+language: en
+tags:
+  - causal-lm
+  - chat
+  - reasoning
+  - momo
+license: apache-2.0
+---
+
+# 🌸 {MOMO_VERSION}
+
+Momo is a friendly 336M parameter language model trained from scratch,
+designed to feel like chatting with a warm, knowledgeable friend.
+
+## Model Details
+- **Parameters:** ~336M
+- **Architecture:** Transformer (RoPE + RMSNorm + GQA + SwiGLU)
+- **Trained on:** WikiText-103 + Alpaca + Custom reasoning data
+- **Context length:** {MAX_SEQ_LEN} tokens
+- **Vocabulary:** {VOCAB_FINAL} tokens
+## Capabilities
+- 💬 Friendly, casual chat
+- 🧠 Reasoning with `<think>` tags
+- ❓ Question answering
+- 🤗 Emotional support
+
+## Quick Start
+```python
+# Load and chat with Momo
+model = MomoForCausalLM.from_pretrained('path/to/Momo-336M')
+tokenizer = AutoTokenizer.from_pretrained('path/to/Momo-336M')
+
+messages = [{{'role': 'user', 'content': 'Hey Momo! How are you?'}}]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(prompt, return_tensors='pt')
+output = model.generate(**inputs, max_new_tokens=200, temperature=0.75)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+## Training Setup
+- GPU: 2× NVIDIA T4 (Kaggle)
+- Precision: float16 AMP
+- Gradient checkpointing: enabled
+- Training stages: Pretrain → SFT → Reasoning