marzoukbaig14/committed-gguf

Committed — Qwen3-1.7B (Q4_K_M GGUF) · higher-specificity option

A small model fine-tuned to write Conventional Commits messages from a git diff. It runs locally on CPU via llama.cpp, so your code never leaves your machine.

This is the larger of two sizes. Committed defaults to the smaller 0.6B GGUF — it matches this 1.7B on commit-type accuracy and faithfulness at ⅓ the size and a ~397 MB download. This 1.7B (1 GB) is the bigger sibling: worth it when you want the extra specificity (0.67 vs the 0.6B's 0.55), i.e. more consistently concrete descriptions. Most users want the 0.6B; reach for this if concreteness matters more than footprint.

This repo holds the merged, quantized GGUF used for serving. The training dataset, LoRA adapter, and source code are linked below.

Live Demo · Gradio Space · 0.6B GGUF · 1.7B GGUF (this repo) · 0.6B adapter · 1.7B adapter · Dataset · GitHub

Details

• Base: Qwen/Qwen3-1.7B (Apache-2.0)
• Method: QLoRA fine-tune (PEFT LoRA + TRL SFTTrainer, vanilla transformers), merged into the base, converted to GGUF
• Quantization: Q4_K_M (~1 GB)
• Task: single-file git diff → one Conventional Commits subject line, type(scope): description
• Decoding: GBNF grammar-constrained, so output is always a well-formed CC line
• Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)

Usage

The trained behavior depends on the exact prompt rendering used in training plus the GBNF grammar applied at decode time, so a bare llama-cpp-python prompt will not reproduce the evaluated output. Run it through the project's inference path instead.

Via the CLI (no token needed; the GGUF is public). The CLI defaults to the 0.6B — pass --model 1.7b for this model:

pip install --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu "committed @ git+https://github.com/marzoukbaig14/Committed.git"
git diff | committed --model 1.7b

Or call it through the repo's engine.py, the FastAPI endpoint, or the Gradio Space — all linked below. Full instructions and options are in the repo README.

Results

Evaluated against the un-tuned Qwen3-1.7B base on a 442-example test set, scored by an LLM judge (DeepSeek) on four axes; the judge was validated against 50 hand-rated examples. Headline numbers are reweighted to the true commit-type distribution.

The four-arm comparison — both base models, both fine-tunes, all judged by the same DeepSeek judge so the numbers are directly comparable:

Metric	0.6B base	0.6B fine-tune	1.7B base	1.7B fine-tune
Type accuracy	0.154	0.601	0.131	0.637
Type-correctness	0.296	0.726	0.296	0.778
Faithfulness	0.285	0.810	0.491	0.848
Completeness	0.353	0.729	0.543	0.776
Specificity	0.414	0.545	0.814	0.667
Conjunctive (all 4)	0.101	0.359	0.175	0.471
Graded mean (0–3)	0.777	2.094	1.447	2.139
feat-share of outputs	86.7%	9.7%	95.5%	8.4%

The finding: both base models "feat-collapse" — they label ~87–96% of all diffs feat regardless of content, scoring below a trivial always-guess-fix baseline (0.489) on type. Fine-tuning breaks the collapse on both. This 1.7B fine-tune is the stronger of the two overall, with its main edge in specificity (0.667 vs the 0.6B's 0.545) — but the margins on type, faithfulness, completeness, and the graded mean (2.139 vs 2.094) are small, which is why the 0.6B is the default and this is the higher-specificity option rather than the flagship.

Honest caveat on the judge: agreement with human raters is moderate-to-substantial on three axes (κ ≈ 0.56–0.61) but weakest on specificity (κ 0.34) — the very axis this model leads on — so that margin carries the most judge uncertainty. These DeepSeek-judged numbers are not comparable to any earlier Gemini-judged figures that may have appeared on this card previously; only the deltas within this all-DeepSeek table are valid.

Full before/after, the feat-collapse analysis, and the judge validation are in the eval writeup: FINDINGS_v1.md.

Related

• The default 0.6B GGUF (smaller, faster, recommended) — marzoukbaig14/committed-gguf-0.6b.
• This model's LoRA adapter — marzoukbaig14/committed-qwen3-1.7b-lora. The unmerged adapter weights, if you want to merge or train further.
• The 0.6B adapter — marzoukbaig14/committed-qwen3-0.6b-lora.
• Live demo (browser) — try it on the portfolio site. Paste a diff, pick a model, get a commit message.
• Live demo (Gradio) — Hugging Face Space.
• Training dataset — marzoukbaig14/committed-train, filtered from CommitChronicle, with composition tables and provenance.
• Source, training & eval code — github.com/marzoukbaig14/Committed.
• Eval writeup — FINDINGS_v1.md.

License

Apache-2.0, inherited from the Qwen3-1.7B base.

Citation

Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).