Add training documentation

README.md

@@ -0,0 +1,46 @@
+# gemma-2-2b-lean-expert-optimized
+
+## Optimized Gemma Model for 94%+ Success Rate
+
+This repository contains the training configuration for an optimized Gemma-2-2B model targeting 94%+ success rate on Lean trading algorithm optimization tasks.
+
+### Training Configuration
+
+- **Base Model**: google/gemma-2-2b
+- **Dataset**: Kronu/lean-expert-optimized-2000
+- **Target Success Rate**: 94%+
+- **Expected Performance**: 96% (94-98% range)
+
+### Key Optimizations
+
+- **JSON Parsing Focus**: 1,333 examples (0% → 95% success target)
+- **Enhanced LoRA**: rank=64, alpha=128
+- **Optimized Training**: 12 epochs, 2e-4 learning rate
+- **Advanced Configuration**: Gradient checkpointing, FP16
+
+### Training Instructions
+
+To train this model using HuggingFace Jobs:
+
+1. Set up your HuggingFace token as environment variable
+2. Run the training script: `python train.py`
+3. Monitor training progress in the HuggingFace dashboard
+
+### Expected Results
+
+- **Training Time**: 25-35 minutes
+- **Cost**: $3-5
+- **Final Model**: Kronu/gemma-2-2b-lean-expert-optimized
+- **Success Rate**: 96% (94-98% range)
+
+### Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+# Load the trained model
+base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b")
+model = PeftModel.from_pretrained(base_model, "Kronu/gemma-2-2b-lean-expert-optimized")
+tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b")
+```