bnolton commited on
Commit
ebae5e1
·
verified ·
1 Parent(s): 29d467f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -35,6 +35,13 @@ LoRA is a moderate intervention model editor that seems perfect for my task.
35
  It is computationally efficient, preserves the knowledge of the base model well, and has smaller file sizes which means the latency of the model is minimally impacted.
36
  This is perfect for an AI tutor since students these days need answers immediately or they go onto to other things.
37
  They also tend to go down rabbit holes, so while this model is specifically trained for a statistics tutor, keeping the base model knowledge when the explore the rabbit holes can be important.
 
 
 
 
 
 
 
38
 
39
  ## Evaluation
40
  The metrics used to evaluate this model are the mmlu_high_school_statistics, minerva_math, and race benchmarks. The BERT benchmarks are also reported.
 
35
  It is computationally efficient, preserves the knowledge of the base model well, and has smaller file sizes which means the latency of the model is minimally impacted.
36
  This is perfect for an AI tutor since students these days need answers immediately or they go onto to other things.
37
  They also tend to go down rabbit holes, so while this model is specifically trained for a statistics tutor, keeping the base model knowledge when the explore the rabbit holes can be important.
38
+ The hyperparameters are as follows:
39
+ LoRA R: 64
40
+ LoRA Alpha: 64
41
+ LoRA Dropout: 0.05
42
+ Learning Rate: 0.00001
43
+ Epochs: 3
44
+
45
 
46
  ## Evaluation
47
  The metrics used to evaluate this model are the mmlu_high_school_statistics, minerva_math, and race benchmarks. The BERT benchmarks are also reported.