prism-lab commited on
Commit
c103487
·
verified ·
1 Parent(s): 4690aea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -30,11 +30,15 @@ This repository contains the training scripts and evaluation notebooks for repro
30
  | Eval_T4_Last.ipynb | Replicates WikiText-103 results using pretrained checkpoints |
31
  | Physical_Validation.ipynb | Generates Figure 3b (iso-energetic validation on WMT14 PRISM) |
32
  | Skewness_paper_last.ipynb | Layer-wise skewness and carrier wave threshold analysis (Section 5.4, Figure 5, Table 3) |
33
- Phase exploration on embeddings will be replaced soon with stronger version which includes statistical test
34
  ## Data
35
 
36
  All notebooks pull pre-tokenized data from prism-lab/wikitext-103-prism-32k-seq4k and prism-lab/wmt14-de-en-* on HuggingFace.
37
 
38
  ## Note on Weight Tying
39
 
40
- All models use tied embeddings (input embeddings = output projection weights). Checkpoint files contain duplicated weights for compatibility. Evaluation scripts redefine model classes with proper weight tying before loading.
 
 
 
 
 
30
  | Eval_T4_Last.ipynb | Replicates WikiText-103 results using pretrained checkpoints |
31
  | Physical_Validation.ipynb | Generates Figure 3b (iso-energetic validation on WMT14 PRISM) |
32
  | Skewness_paper_last.ipynb | Layer-wise skewness and carrier wave threshold analysis (Section 5.4, Figure 5, Table 3) |
33
+
34
  ## Data
35
 
36
  All notebooks pull pre-tokenized data from prism-lab/wikitext-103-prism-32k-seq4k and prism-lab/wmt14-de-en-* on HuggingFace.
37
 
38
  ## Note on Weight Tying
39
 
40
+ All models use tied embeddings (input embeddings = output projection weights). Checkpoint files contain duplicated weights for compatibility. Evaluation scripts redefine model classes with proper weight tying before loading.
41
+
42
+ ## Additional Notes
43
+ Phase exploration on embeddings will be replaced soon with stronger version which includes statistical test
44
+ Causal ablation codes and models will be added soon.