Upload Meet25M base model as safetensors

README.md

@@ -0,0 +1,54 @@
+---
+language:
+- en
+license: other
+library_name: pytorch
+tags:
+- causal-lm
+- from-scratch
+- gpt
+- safetensors
+- small-language-model
+- meet25m
+---
+
+# Meet25M Base
+
+A small GPT-style causal language model trained from scratch.
+
+## Model
+
+- Architecture: GPT-style decoder-only Transformer
+- Approx size: ~25M parameters
+- Context length: 1024
+- Tokenizer: custom byte-level BPE
+- Positional encoding: RoPE
+- Normalization: RMSNorm
+- MLP: SwiGLU
+- Embeddings: tied input/output embeddings
+
+## Training Data Mix
+
+Target pretraining mix:
+
+- FineWeb-Edu
+- FineWeb general
+- Wikipedia
+- OpenWebMath
+- Project Gutenberg
+- StackOverflow / Stack Exchange style posts
+- CodeSearchNet
+
+Total target: ~250M training tokens.
+
+## Files
+
+- `model.safetensors` — safetensors checkpoint
+- `config.json` — model config
+- `tokenizer/` — tokenizer files
+- `safetensors_info.json` — checkpoint metadata
+
+## Loading
+
+This is not a standard Transformers `AutoModelForCausalLM` checkpoint.  
+Use the custom GPT class from the training script and load `model.safetensors`.

config.json

@@ -0,0 +1,12 @@
+{
+  "vocab_size": 16384,
+  "block_size": 1024,
+  "n_layer": 8,
+  "n_embd": 384,
+  "n_head": 6,
+  "dropout": 0.0,
+  "pad_id": 0,
+  "eos_id": 2,
+  "weight_format": "safetensors",
+  "tied_embeddings": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81f258ca51f4a107ac10439a25f62fbd9e25b96f692464d637a0276211769619
+size 106986416

safetensors_info.json

@@ -0,0 +1,19 @@
+{
+  "model_name": "Meet25M-Base",
+  "safetensors_file": "model.safetensors",
+  "source_checkpoint": "model.pt",
+  "num_tensors": 59,
+  "size_bytes": 106986416,
+  "config": {
+    "vocab_size": 16384,
+    "block_size": 1024,
+    "n_layer": 8,
+    "n_embd": 384,
+    "n_head": 6,
+    "dropout": 0.0,
+    "pad_id": 0,
+    "eos_id": 2,
+    "weight_format": "safetensors",
+    "tied_embeddings": true
+  }
+}

tokenizer/tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "model_max_length": 1024,
+  "pad_token": "<pad>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}