Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,39 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc-by-4.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
---
|
| 4 |
+
# PRISM Training & Evaluation Code
|
| 5 |
+
|
| 6 |
+
This repository contains the training scripts and evaluation notebooks for reproducing the experiments in "Language as a Wave Phenomenon."
|
| 7 |
+
|
| 8 |
+
## WMT14 (Machine Translation)
|
| 9 |
+
|
| 10 |
+
| Notebook | Description |
|
| 11 |
+
|----------|-------------|
|
| 12 |
+
| AIAYN_Baseline_Training.ipynb | Standard Transformer baseline (RoPE) |
|
| 13 |
+
| FNet_Train_Last.ipynb | FNet hybrid encoder training |
|
| 14 |
+
| Gated_PRISM_train_hybrid_RoPE.ipynb | PRISM model used for mechanistic interpretability analysis |
|
| 15 |
+
|
| 16 |
+
## WikiText-103 (Masked Language Modeling)
|
| 17 |
+
|
| 18 |
+
| Notebook | Description |
|
| 19 |
+
|----------|-------------|
|
| 20 |
+
| WT103_Transformer_Baseline.ipynb | Transformer baseline |
|
| 21 |
+
| FNet_Hybrid_Wikitext_Training.ipynb | FNet hybrid (6 spectral + 1 attention) |
|
| 22 |
+
| PRISM_wikitext_103_last.ipynb | PRISM with Dynamic RoSE |
|
| 23 |
+
| HSSM_Wikitext_Training.ipynb | Hybrid Spectral Sequence Model (FNet rate + PRISM phase streams) |
|
| 24 |
+
| WPT_Wikitext_103_Training.ipynb | Wave-Particle Transformer (Transformer sensory + PRISM relational) |
|
| 25 |
+
|
| 26 |
+
## Evaluation & Analysis
|
| 27 |
+
|
| 28 |
+
| Notebook | Description |
|
| 29 |
+
|----------|-------------|
|
| 30 |
+
| Eval_T4_Last.ipynb | Replicates WikiText-103 results using pretrained checkpoints |
|
| 31 |
+
| Physical_Validation.ipynb | Generates Figure 3b (iso-energetic validation on WMT14 PRISM) |
|
| 32 |
+
|
| 33 |
+
## Data
|
| 34 |
+
|
| 35 |
+
All notebooks pull pre-tokenized data from prism-lab/wikitext-103-prism-32k-seq4k and prism-lab/wmt14-de-en-* on HuggingFace.
|
| 36 |
+
|
| 37 |
+
## Note on Weight Tying
|
| 38 |
+
|
| 39 |
+
All models use tied embeddings (input embeddings = output projection weights). Checkpoint files contain duplicated weights for compatibility. Evaluation scripts redefine model classes with proper weight tying before loading.
|