PhilipQuirke commited on
Commit
6d948c5
·
verified ·
1 Parent(s): 248a83c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -7,7 +7,7 @@ Each folder contains a transformer model that can predict addition questions, su
7
 
8
  The folder name (e.g. sub_d6_l2_h3_t20K_s173289) contains:
9
  - "add", "sub", or "mix": Shows the types of questions the model can predict.
10
- - "d5" to "d20"" How many digits the model handles e.g. a d5 sub model can predict the answer in 123450-345670=-0123230
11
  - "l1", "l2" or "l3": The number of layers in the model
12
  - "h3" or "h4": The number of attention heads in the model
13
  - "t15K" to "t85K" etc: The number of batches the model was trained on
@@ -21,9 +21,9 @@ Some folder names also contain:
21
  Each folder contains:
22
  - model.pth: The transformer model as described above
23
  - training_loss.json: Data gathered during model training. Used to plot "loss over training batches" graphs
24
- - behaviors.json: Data gathered about the behavior of the model by direct inspection.
25
- - features.json: Data gathered about hypothesised algorithm features via experimentation.
26
 
27
  The first 2 files were created by the https://github.com/PhilipQuirke/quanta_maths/blob/main/notebooks/VerifiedArithmeticTrain.ipynb notebook.
28
  The last 2 files were created by the https://github.com/PhilipQuirke/quanta_maths/blob/main/notebooks/VerifiedArithmeticAnalyse.ipynb notebook.
29
-
 
7
 
8
  The folder name (e.g. sub_d6_l2_h3_t20K_s173289) contains:
9
  - "add", "sub", or "mix": Shows the types of questions the model can predict.
10
+ - "d5" to "d20": How many digits the model handles e.g. a d5 sub model can predict the answer in 123450-345670=-0123230
11
  - "l1", "l2" or "l3": The number of layers in the model
12
  - "h3" or "h4": The number of attention heads in the model
13
  - "t15K" to "t85K" etc: The number of batches the model was trained on
 
21
  Each folder contains:
22
  - model.pth: The transformer model as described above
23
  - training_loss.json: Data gathered during model training. Used to plot "loss over training batches" graphs
24
+ - behaviors.json: Facts gathered about the behavior of the model by direct inspection. Includes attention pattern data, PCA data, answer digit impact data, etc.
25
+ - features.json: Facts gathered about hypothesised algorithm features via experimentation e.g. node P12L0H1 implements the feature A3.ST.
26
 
27
  The first 2 files were created by the https://github.com/PhilipQuirke/quanta_maths/blob/main/notebooks/VerifiedArithmeticTrain.ipynb notebook.
28
  The last 2 files were created by the https://github.com/PhilipQuirke/quanta_maths/blob/main/notebooks/VerifiedArithmeticAnalyse.ipynb notebook.
29
+ The json file are used by the algorithm testing notebook https://github.com/PhilipQuirke/quanta_maths/blob/main/notebooks/VerifiedArithmeticAlgorithm.ipynb notebook.