Update README.md

README.md

@@ -64,4 +64,21 @@ learning_rate = 0.0003995209593890016
 lora_alpha = 128
 lora_dropout = .1
 lora_r = 64
-```
+```
+
+I experimented with full fine tuning, however the model lost a lot of it's functionality and become a repeater. For this reason, I leveraged PEFT methods. I settled on LoRA as it was fairly simple to implement.
+
+### Evaluation 
+
+For this models evalutation I used three metrics that are common in natural language tasks. 
+
+* BERT
+* ROUGE
+* BLEU
+
+The primary evaluation is BERT score, this is a way to calcuate the similarity between two text inputs. BERT aims to to assess semantic similarity, it measure the difference between the actual forecast, and the generated forecast to see if they have similar semantic meanings, a higher BERT score is better.
+
+ROUGE (Recall-Oriented Understudy for Gisting Evaluation) This is used to see if the general gist is similar between the generated forecast and the actual human forecast. A higher ROUGE score is better.
+
+BLEU (Bilingual Evaluation Understudy) measures how many words appear in the reference generated human text. This should show if the model is picking up on the "surfer lingo" a higher BLEU score is better.
+