Upload bert-thetis-tiny-wikitext103/2025-10-13_20-09-33/final/README.md with huggingface_hub

bert-thetis-tiny-wikitext103/2025-10-13_20-09-33/final/README.md

@@ -0,0 +1,122 @@
+# BERT-Thetis Checkpoint
+
+**Model Variant:** bert-thetis-tiny-wikitext103  
+**Training Run:** 2025-10-13_20-09-33  
+**Checkpoint:** final  
+**Training Step:** 48,010  
+**Epoch:** 10/10
+
+## 🌊 What is BERT-Thetis?
+
+BERT-Thetis is a geometric language model that replaces traditional learned embeddings with **deterministic crystal structures**. Instead of a 50K×768 embedding table (38M parameters), we use:
+
+- **Beatrix Staircase Encodings**: Deterministic positional structure (0 parameters)
+- **Character Composition**: Learnable semantic bridge
+- **Crystal Inflation**: 5-vertex simplex generation (0 parameters)
+
+This reduces vocabulary parameters by **~95%** while maintaining competitive performance.
+
+## 📊 Performance
+
+| Metric | Value |
+|--------|-------|
+| **Validation Loss** | 5.9593 |
+| **Perplexity** | 387.32 |
+| **Best Accuracy** | 16.97% |
+| **Training Steps** | 48,010 |
+
+## 🏗️ Architecture
+
+- **Layers:** 4
+- **Dimension:** 256
+- **Attention Heads:** 4
+- **Beatrix Levels:** 16
+- **Crystal Vertices:** 5 (pentachoron)
+
+## 📦 Model Details
+
+- **Total Parameters:** ~0.0M
+- **Dataset:** WikiText-103 (~103M tokens)
+- **Batch Size:** 256
+- **Learning Rate:** 0.0005
+- **Mixed Precision:** True
+
+## 🚀 Usage
+
+```python
+import torch
+from pathlib import Path
+from geovocab2.train.model.core.bert_thetis import ThetisConfig, ThetisForMaskedLM
+
+# Path structure: weights/bert-thetis-tiny-wikitext103/2025-10-13_20-09-33/final/
+
+# Load config
+import json
+with open("config.json") as f:
+    config_dict = json.load(f)
+
+config = ThetisConfig(**config_dict)
+
+# Load model
+model = ThetisForMaskedLM(config)
+
+# Load weights (safetensors or pytorch)
+if Path("model.safetensors").exists():
+    from safetensors.torch import load_file
+    state_dict = load_file("model.safetensors")
+    model.load_state_dict(state_dict)
+else:
+    model.load_state_dict(torch.load("pytorch_model.bin"))
+
+model.eval()
+
+# Use for masked token prediction
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
+
+text = "The capital of France is [MASK]."
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model(
+    token_ids=inputs["input_ids"],
+    attention_mask=inputs["attention_mask"]
+)
+predictions = outputs.argmax(dim=-1)
+predicted_text = tokenizer.decode(predictions[0])
+print(predicted_text)
+```
+
+## 📁 Directory Structure
+
+```
+weights/
+└── bert-thetis-tiny-wikitext103/
+    └── 2025-10-13_20-09-33/
+        ├── best/              ← Best validation checkpoint
+        ├── final/             ← Final training checkpoint
+        ├── step-1000/         ← Intermediate checkpoints
+        └── epoch-1/           ← Per-epoch checkpoints
+```
+
+## 📚 Resources
+
+- **Repository:** [github.com/AbstractEyes/lattice_vocabulary](https://github.com/AbstractEyes/lattice_vocabulary)
+- **Hub:** [huggingface.co/AbstractPhil/bert-thetis-tiny-wikitext103](https://huggingface.co/AbstractPhil/bert-thetis-tiny-wikitext103)
+- **Paper:** Coming soon!
+- **Author:** AbstractPhil
+
+## 🎓 Citation
+
+```bibtex
+@misc{bert-thetis-2025,
+  author = {AbstractPhil},
+  title = {BERT-Thetis: Geometric BERT with Deterministic Crystal Embeddings},
+  year = {2025},
+  publisher = {HuggingFace},
+  url = {https://huggingface.co/AbstractPhil/bert-thetis-tiny-wikitext103}
+}
+```
+
+---
+
+**Status:** ✅ Training Complete  
+**Generated:** 2025-10-14 07:20:54 UTC