marzoukbaig14/committed-qwen3-0.6b-lora

Committed — Qwen3-0.6B LoRA adapter

A QLoRA adapter that fine-tunes Qwen/Qwen3-0.6B to write Conventional Commits messages from a git diff: a single-file diff in, one type(scope): description subject line out.

This repo holds the LoRA adapter weights. For local CPU inference most people want the merged, quantized GGUF instead — it's the default model the CLI and demo use. Use this adapter if you want to merge it yourself, train further on top of it, or run it with PEFT on GPU.

Live Demo · Gradio Space · 0.6B GGUF · 1.7B GGUF · 0.6B adapter (this repo) · 1.7B adapter · Dataset · GitHub

The 0.6B is Committed's default model — it matches the larger 1.7B on commit-type accuracy and faithfulness at ~⅓ the size, trading a little specificity. The corresponding 1.7B adapter is available as the bigger sibling.

Details

• Base: Qwen/Qwen3-0.6B (Apache-2.0)
• Method: QLoRA (PEFT LoRA + TRL SFTTrainer, vanilla transformers)
• Task: single-file git diff → one Conventional Commits subject line
• Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)

Usage

The trained behavior depends on the exact prompt rendering used in training (a canonical zero-shot Diff:\n{diff} format with enable_thinking=False) plus the GBNF grammar applied at decode time. Loading the adapter with a bare prompt will not reproduce the evaluated output. To match what was evaluated, run it through the project's engine.py, or use the FastAPI / Gradio Space, or the CLI — see github.com/marzoukbaig14/Committed.

To load the adapter on top of the base for your own use:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
model = PeftModel.from_pretrained(base, "marzoukbaig14/committed-qwen3-0.6b-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")

Results

The 0.6B fine-tune matches the 1.7B on type-correctness and faithfulness at a third the size, with the gap concentrated in specificity. Both base models feat-collapse (~87–96% feat); fine-tuning breaks it on both. The full four-arm comparison (0.6B/1.7B × base/fine-tune, all DeepSeek-judged), the feat-collapse analysis, and the judge validation are in the merged-model card and the eval writeup:

• Merged GGUF (with the results table) — marzoukbaig14/committed-gguf-0.6b
• Eval writeup — FINDINGS_v1.md

Related

• GGUF, for local CPU serving (the default model) — marzoukbaig14/committed-gguf-0.6b. The merged, quantized model most people actually want to run.
• The 1.7B adapter (bigger sibling) — marzoukbaig14/committed-qwen3-1.7b-lora.
• Training dataset — marzoukbaig14/committed-train.
• Source, training & eval code — github.com/marzoukbaig14/Committed.
• Live demo — try it in the browser.

License

Apache-2.0, inherited from the Qwen3-0.6B base.

Citation

Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).