lr note

README.md

@@ -9,6 +9,13 @@ Special thanks to https://huggingface.co/fahadh4ilyas
 convert_v2.py
 ```
 
+Training Notes:
+```
+# dbrx trains like a much smaller model (~7B)
+# start with this as reference point and move up or down based on eval/train loss
+learning_rate = 1.5e-5  
+```
+
 Known Issues:
 
 1. [QUANT GPTQ] PENDING: You can help test quant and/or follow progress at https://github.com/AutoGPTQ/AutoGPTQ/pull/625