---
base_model: unsloth/Qwen2.5-Coder-1.5B-Instruct
library_name: peft
tags:
  - lora
  - sft
  - unsloth
  - code
  - git
  - commit-message-generation
license: apache-2.0
---

# commit-msg-qwen-lora

A LoRA adapter fine-tuned on top of [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct) to generate concise, conventional-style Git commit messages from staged diffs.

## Training Details

- **Base model:** Qwen2.5-Coder-1.5B-Instruct
- **Method:** QLoRA (LoRA rank 16) via Unsloth on Google Colab T4 GPU
- **Dataset:** 2,397 real (diff, commit message) pairs extracted and cleaned from personal GitHub repositories
- **Train / Val / Test split:** 1919 / 239 / 239
- **Epochs:** 3 | **Batch size:** 8 (effective) | **Learning rate:** 2e-4
- **Training loss:** 1.03 → 0.88

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "sharad31/commit-msg-qwen-lora")
tokenizer = AutoTokenizer.from_pretrained("sharad31/commit-msg-qwen-lora")

diff = """diff --git a/src/auth.ts b/src/auth.ts
+  if (!user || !pass) throw new Error('Missing credentials');
"""

messages = [
    {"role": "system", "content": "You are a precise commit message generator. Given a git diff, write a concise, conventional-style commit message."},
    {"role": "user", "content": f"Diff:\n```\n{diff}\n```"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=64, temperature=0.3)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
```