File size: 1,751 Bytes
f2b21d7 b03deab f2b21d7 b03deab f2b21d7 b03deab f2b21d7 b03deab f2b21d7 b03deab | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
base_model: unsloth/Qwen2.5-Coder-1.5B-Instruct
library_name: peft
tags:
- lora
- sft
- unsloth
- code
- git
- commit-message-generation
license: apache-2.0
---
# commit-msg-qwen-lora
A LoRA adapter fine-tuned on top of [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct) to generate concise, conventional-style Git commit messages from staged diffs.
## Training Details
- **Base model:** Qwen2.5-Coder-1.5B-Instruct
- **Method:** QLoRA (LoRA rank 16) via Unsloth on Google Colab T4 GPU
- **Dataset:** 2,397 real (diff, commit message) pairs extracted and cleaned from personal GitHub repositories
- **Train / Val / Test split:** 1919 / 239 / 239
- **Epochs:** 3 | **Batch size:** 8 (effective) | **Learning rate:** 2e-4
- **Training loss:** 1.03 → 0.88
## Usage
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "sharad31/commit-msg-qwen-lora")
tokenizer = AutoTokenizer.from_pretrained("sharad31/commit-msg-qwen-lora")
diff = """diff --git a/src/auth.ts b/src/auth.ts
+ if (!user || !pass) throw new Error('Missing credentials');
"""
messages = [
{"role": "system", "content": "You are a precise commit message generator. Given a git diff, write a concise, conventional-style commit message."},
{"role": "user", "content": f"Diff:\n```\n{diff}\n```"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=64, temperature=0.3)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
``` |