marzoukbaig14/committed-qwen3-1.7b-lora

Committed — Qwen3-1.7B LoRA adapter

A QLoRA adapter that fine-tunes Qwen/Qwen3-1.7B to write Conventional Commits messages from a git diff: a single-file diff in, one type(scope): description subject line out.

This repo holds the LoRA adapter weights for the larger of two sizes. For local CPU inference most people want the merged, quantized GGUF instead. Use this adapter if you want to merge it yourself, train further on top of it, or run it with PEFT on GPU.

Live Demo · Gradio Space · 0.6B GGUF · 1.7B GGUF · 0.6B adapter · 1.7B adapter (this repo) · Dataset · GitHub

Committed defaults to the smaller 0.6B — it matches this 1.7B on commit-type and faithfulness at ~⅓ the size. This 1.7B is the bigger sibling, worth it for higher specificity (more consistently concrete descriptions).

Details

• Base: Qwen/Qwen3-1.7B (Apache-2.0)
• Method: QLoRA (PEFT LoRA + TRL SFTTrainer, vanilla transformers)
• Task: single-file git diff → one Conventional Commits subject line
• Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)

Usage

The trained behavior depends on the exact prompt rendering used in training (a canonical zero-shot Diff:\n{diff} format with enable_thinking=False) plus the GBNF grammar applied at decode time. Loading the adapter with a bare prompt will not reproduce the evaluated output. To match what was evaluated, run it through the project's engine.py, or use the FastAPI / Gradio Space, or the CLI (git diff | committed --model 1.7b). See github.com/marzoukbaig14/Committed.

To load the adapter on top of the base for your own use:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B")
model = PeftModel.from_pretrained(base, "marzoukbaig14/committed-qwen3-1.7b-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")

Results

This 1.7B fine-tune is the stronger of the two sizes overall, with its main edge in specificity (0.67 vs the 0.6B's 0.55); the margins on the other axes are small (graded 2.14 vs 2.09). Both base models feat-collapse (~87–96% feat); fine-tuning breaks it on both. The full four-arm comparison (0.6B/1.7B × base/fine-tune, all DeepSeek-judged — not comparable to any earlier Gemini figures), the feat-collapse analysis, and the judge validation are in the merged-model card and the eval writeup:

• Merged GGUF (with the results table) — marzoukbaig14/committed-gguf
• Eval writeup — FINDINGS_v1.md

Related

• GGUF, for local CPU serving — marzoukbaig14/committed-gguf. The merged, quantized version of this model.
• The default 0.6B (smaller, faster) — adapter — marzoukbaig14/committed-qwen3-0.6b-lora · GGUF — marzoukbaig14/committed-gguf-0.6b.
• Training dataset — marzoukbaig14/committed-train.
• Source, training & eval code — github.com/marzoukbaig14/Committed.
• Live demo — try it in the browser.

License

Apache-2.0, inherited from the Qwen3-1.7B base.

Citation

Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).