asadullahusama17/tinyllama-lora-unsloth

TinyLlama LoRA Fine-tune (Unsloth)

LoRA adapter fine-tuned on top of unsloth/tinyllama-bnb-4bit using Unsloth.

Training Configuration

• LoRA rank (r): 32
• LoRA alpha: 32
• LoRA dropout: 0.0
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Max sequence length: 4096
• Quantization: 4-bit
• Seed: 3407

Training Code

• Full training notebook and pipeline: https://github.com/MalikUsama9/tinyllama-lora-unsloth