Upload folder using huggingface_hub

README.md

@@ -0,0 +1,45 @@
+---
+tags:
+- transformer
+- language-model
+- educational
+license: mit
+---
+
+# ScholarSage - Tiny Transformer LM
+
+A tiny transformer language model built from scratch for educational purposes.
+
+## Model Details
+
+- **Architecture**: Decoder-only transformer (GPT-style)
+- **Parameters**: 
+  - Vocabulary: 50,257 tokens (GPT-2 tokenizer)
+  - Embedding dimension: 256
+  - Layers: 4
+  - Attention heads: 4
+  - FFN dimension: 1024
+  - Max sequence length: 512
+
+## Training
+
+- **Dataset**: WikiText-2
+- **Optimizer**: AdamW
+- **Learning rate**: 3e-4
+
+## Usage
+
+```python
+import torch
+from transformers import AutoTokenizer
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("TheCodeKat/scholar-sage")
+
+# Load model (you'll need to load the architecture separately)
+# This is a custom model, not a standard transformers model
+```
+
+## Purpose
+
+This model is built for educational purposes to understand transformer architecture from scratch.

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da2a0209b5b6df63bd29f71d28a73dcb36a13a41f625274459e472430e9087f4
+size 116089486

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,20 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}