Update README.md
Browse files
README.md
CHANGED
|
@@ -30,11 +30,15 @@ This repository contains the training scripts and evaluation notebooks for repro
|
|
| 30 |
| Eval_T4_Last.ipynb | Replicates WikiText-103 results using pretrained checkpoints |
|
| 31 |
| Physical_Validation.ipynb | Generates Figure 3b (iso-energetic validation on WMT14 PRISM) |
|
| 32 |
| Skewness_paper_last.ipynb | Layer-wise skewness and carrier wave threshold analysis (Section 5.4, Figure 5, Table 3) |
|
| 33 |
-
|
| 34 |
## Data
|
| 35 |
|
| 36 |
All notebooks pull pre-tokenized data from prism-lab/wikitext-103-prism-32k-seq4k and prism-lab/wmt14-de-en-* on HuggingFace.
|
| 37 |
|
| 38 |
## Note on Weight Tying
|
| 39 |
|
| 40 |
-
All models use tied embeddings (input embeddings = output projection weights). Checkpoint files contain duplicated weights for compatibility. Evaluation scripts redefine model classes with proper weight tying before loading.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
| Eval_T4_Last.ipynb | Replicates WikiText-103 results using pretrained checkpoints |
|
| 31 |
| Physical_Validation.ipynb | Generates Figure 3b (iso-energetic validation on WMT14 PRISM) |
|
| 32 |
| Skewness_paper_last.ipynb | Layer-wise skewness and carrier wave threshold analysis (Section 5.4, Figure 5, Table 3) |
|
| 33 |
+
|
| 34 |
## Data
|
| 35 |
|
| 36 |
All notebooks pull pre-tokenized data from prism-lab/wikitext-103-prism-32k-seq4k and prism-lab/wmt14-de-en-* on HuggingFace.
|
| 37 |
|
| 38 |
## Note on Weight Tying
|
| 39 |
|
| 40 |
+
All models use tied embeddings (input embeddings = output projection weights). Checkpoint files contain duplicated weights for compatibility. Evaluation scripts redefine model classes with proper weight tying before loading.
|
| 41 |
+
|
| 42 |
+
## Additional Notes
|
| 43 |
+
Phase exploration on embeddings will be replaced soon with stronger version which includes statistical test
|
| 44 |
+
Causal ablation codes and models will be added soon.
|