Update README.md
Browse files
README.md
CHANGED
|
@@ -35,6 +35,13 @@ LoRA is a moderate intervention model editor that seems perfect for my task.
|
|
| 35 |
It is computationally efficient, preserves the knowledge of the base model well, and has smaller file sizes which means the latency of the model is minimally impacted.
|
| 36 |
This is perfect for an AI tutor since students these days need answers immediately or they go onto to other things.
|
| 37 |
They also tend to go down rabbit holes, so while this model is specifically trained for a statistics tutor, keeping the base model knowledge when the explore the rabbit holes can be important.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## Evaluation
|
| 40 |
The metrics used to evaluate this model are the mmlu_high_school_statistics, minerva_math, and race benchmarks. The BERT benchmarks are also reported.
|
|
|
|
| 35 |
It is computationally efficient, preserves the knowledge of the base model well, and has smaller file sizes which means the latency of the model is minimally impacted.
|
| 36 |
This is perfect for an AI tutor since students these days need answers immediately or they go onto to other things.
|
| 37 |
They also tend to go down rabbit holes, so while this model is specifically trained for a statistics tutor, keeping the base model knowledge when the explore the rabbit holes can be important.
|
| 38 |
+
The hyperparameters are as follows:
|
| 39 |
+
LoRA R: 64
|
| 40 |
+
LoRA Alpha: 64
|
| 41 |
+
LoRA Dropout: 0.05
|
| 42 |
+
Learning Rate: 0.00001
|
| 43 |
+
Epochs: 3
|
| 44 |
+
|
| 45 |
|
| 46 |
## Evaluation
|
| 47 |
The metrics used to evaluate this model are the mmlu_high_school_statistics, minerva_math, and race benchmarks. The BERT benchmarks are also reported.
|