Update README.md

Added Training Specs

README.md

@@ -61,4 +61,49 @@ For best results, use full precision with one of the three different instruction
 - "instruction": "Create the mermaid diagram for the following story:"
 - "instruction": "Create the mermaid diagram for the following:"
 
-Exciting times ahead as we delve into the MermaidLLama revolution! 🚀
+Exciting times ahead as we delve into the MermaidLLama revolution! 🚀
+
+LoRA Rank
+Also called dimension count. Higher values = larger file, more content control. Smaller values = smaller file, less control. Use 4 or 8 for style, 128 or 256 to teach, 1024+ for fine-detail on big data. More VRAM is needed for higher ranks.
+2048
+
+LoRA Alpha
+This divided by the rank becomes the scaling of the LoRA. Higher means stronger. A good standard value is twice your Rank.
+4096
+
+Batch Size
+Global batch size. The two batch sizes together determine gradient accumulation (gradientAccum = batch / microBatch). Higher gradient accum values lead to better quality training.
+1
+
+Micro Batch Size
+Per-device batch size (NOTE: multiple devices not yet implemented). Increasing this will increase VRAM usage.
+1
+
+Cutoff Length
+Cutoff length for text input. Essentially, how long of a line of text to feed in at a time. Higher values require drastically more VRAM.
+4096
+
+Save every n steps
+If above 0, a checkpoint of the LoRA will be saved every time this many steps pass.
+1000
+
+Epochs
+Number of times every entry in the dataset should be fed into training. So 1 means feed each item in once, 5 means feed it in five times, etc.
+3
+
+Learning Rate
+In scientific notation.
+1e-6
+
+LR Scheduler
+Learning rate scheduler - defines how the learning rate changes over time. "Constant" means never change, "linear" means to go in a straight line from the learning rate down to 0, cosine follows a curve, etc.
+cosine
+
+
+Target Modules
+▼
+Selects which modules to target in training. Targeting more modules is closer to a full fine-tune at the cost of increased VRAM requirements and adapter size. NOTE: Only works for model_id='llama', other types will retain default training behavior and not use these settings.
+
+Enable q_proj
+Enable v_proj
+