Create README.md

README.md

@@ -0,0 +1,70 @@
+---
+datasets:
+- HuggingFaceFW/fineweb-edu
+---
+
+
+# RSCaLM-138M-LLaMA
+
+**RSCaLM** (Research Scale Causal Language Model) is an experimental 138M-parameter LLaMA-architecture model trained for **20,000 steps**.
+This run was conducted purely for **experimental and benchmarking purposes** — **no high expectations** for downstream task quality.
+
+---
+
+## 📌 Experiment Summary
+
+* **Architecture:** LLaMA-style causal decoder
+* **Parameter Count:** \~138M
+* **Training Steps:** 20,000
+* **Purpose:** Early-stage test run for verifying training pipeline & scaling behavior
+* **Tokenizer:** LLaMA tokenizer
+* **Framework:** PyTorch + Hugging Face Transformers
+
+---
+
+## 📉 Validation Loss Progress
+
+| Step  | Val Loss |
+| ----- | -------- |
+| 1000  | 5.5968   |
+| 2000  | 4.8513   |
+| 5000  | 4.2105   |
+| 10000 | 3.9603   |
+| 15000 | 3.8497   |
+| 20000 | 3.7891   |
+
+Loss shows steady improvement over the limited training period.
+
+---
+
+## ⚠️ Notes
+
+* This is an **early prototype** — not tuned for production use.
+* Training stopped after \~32% of planned total steps.
+* Possible repetition loops observed in generation — expected for low-step runs.
+* Intended for research reference, not for deployment in critical tasks.
+
+---
+
+## 🔧 Example Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "yasserrmd/RSCaLM-138M-LLaMA"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+prompt = "The sun is"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+## 📜 License
+
+
+