AbstractPhil commited on
Commit
7b4afe5
Β·
verified Β·
1 Parent(s): cbe88c8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +44 -3
README.md CHANGED
@@ -1,3 +1,44 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BERT-Thetis: Geometric BERT Models
2
+
3
+ This repository contains BERT-Thetis models with deterministic crystal embeddings.
4
+
5
+ ## πŸ“ Repository Structure
6
+
7
+ ```
8
+ AbstractPhil/bert-thetis-tiny-wikitext103/
9
+ β”œβ”€β”€ bert-thetis-tiny-wikitext103/
10
+ β”‚ └── YYYY-MM-DD_HH-MM-SS/ (training run timestamp)
11
+ β”‚ β”œβ”€β”€ best/ (best validation checkpoint)
12
+ β”‚ β”œβ”€β”€ final/ (final checkpoint)
13
+ β”‚ └── step-N/ (intermediate checkpoints)
14
+ ```
15
+
16
+ ## 🌊 What is BERT-Thetis?
17
+
18
+ BERT-Thetis replaces traditional learned embeddings with **deterministic crystal structures**:
19
+
20
+ - **Beatrix Staircase Encodings**: Zero-parameter positional structure
21
+ - **Character Composition**: Learnable semantic bridge
22
+ - **Crystal Inflation**: Deterministic 5-vertex simplex generation
23
+
24
+ This reduces vocabulary parameters by ~95% while maintaining performance.
25
+
26
+ ## πŸš€ Quick Start
27
+
28
+ ```python
29
+ from geovocab2.train.model.core.bert_thetis import ThetisConfig, ThetisForMaskedLM
30
+
31
+ # Load model
32
+ config = ThetisConfig.from_pretrained("AbstractPhil/bert-thetis-tiny-wikitext103")
33
+ model = ThetisForMaskedLM(config)
34
+ ```
35
+
36
+ ## πŸ“š Resources
37
+
38
+ - **Repository:** [github.com/AbstractEyes/lattice_vocabulary](https://github.com/AbstractEyes/lattice_vocabulary)
39
+ - **Author:** AbstractPhil
40
+
41
+ ---
42
+
43
+ **Latest Run:** 2025-10-13_20-09-33
44
+ **Model Variant:** bert-thetis-tiny-wikitext103