Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,85 @@
+---
+language: en
+license: apache-2.0
+tags:
+- llama2
+- lora
+- code
+- adapter
+datasets:
+- iamtarun/python_code_instructions_18k_alpaca
+---
+
+# LLaMA-2-7B CODE LoRA Adapter
+
+This is a LoRA adapter for LLaMA-2-7B fine-tuned on code domain data.
+
+## Model Details
+
+- **Base Model**: meta-llama/Llama-2-7b-hf
+- **Adapter Type**: LoRA (Low-Rank Adaptation)
+- **Domain**: Code
+- **Training Data**: Python code instructions
+- **Training Examples**: 2000 (1600 train, 200 val, 200 test)
+- **Epochs**: 2
+
+## LoRA Configuration
+
+- Rank (r): 16
+- Alpha: 32
+- Dropout: 0.05
+- Target Modules: q_proj, v_proj, k_proj, o_proj
+
+## Performance Metrics (100 test examples)
+
+- Loss: 0.573
+- Perplexity: 1.773
+- BLEU: 32.76
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+# Load base model
+model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Llama-2-7b-hf",
+    load_in_8bit=True,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")
+
+# Load adapter
+model = PeftModel.from_pretrained(model, "Thamirawaran/llama2-7b-code-lora")
+
+# Generate
+prompt = 'Write a Python function to sort a list'
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_length=256, temperature=0.7)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training Details
+
+- Trained with FYP_MDLE library
+- 8-bit quantization during training
+- Gradient accumulation: 16 steps
+- Learning rate: 2e-4
+- Warmup steps: 20
+
+## Citation
+
+```bibtex
+@misc{llama2-code-lora,
+  author = {Team RAISE},
+  title = {LLaMA-2-7B Code LoRA Adapter},
+  year = {2026},
+  publisher = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/Thamirawaran/llama2-7b-code-lora}}
+}
+```
+
+## License
+
+Apache 2.0