Teaching Small LLMs to Write Git Commits
Collection
Dataset + LoRA adapters for fine-tuning small LLMs to write Conventional Commits from a git diff. Part of a local, offline commit-message tool. • 7 items • Updated
Llama-3.2-1B-Instruct — Commit Message LoRA Adapter
LoRA adapter fine-tuned on top of meta-llama/Llama-3.2-1B-Instruct
to generate Conventional Commits messages from a git diff.
Part of a project to build a local, offline prepare-commit-msg git hook.
Full write-up: https://eliotbas.com/projects/commits-fine-tuning/
Training dataset: https://huggingface.co/datasets/Elib27/commits
Training details
- Method: LoRA (r=16, alpha=32)
- Framework: Unsloth + TRL SFTTrainer
- Hardware: Google Colab A100
- Dataset: ~11k (diff, commit message) pairs, see eliotbas/commits
See the article for full evaluation results (ROUGE-L, structural checks, LLM-as-judge) across all model sizes.
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Elib27/llama-3.2-1b-commit-msg-lora
Base model
meta-llama/Llama-3.2-1B-Instruct
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/Llama-3.2-1B-Instruct") model = PeftModel.from_pretrained(base_model, "Elib27/llama-3.2-1b-commit-msg-lora")