smart-commit-1.5b-v0.2

A fine-tune of Qwen/Qwen2.5-Coder-1.5B-Instruct that writes git commit messages in the repository's style, for smart-commit, a CLI that runs a small model locally (Intel NPU, GPU or CPU). Version 0.2, while the training recipe still changes; it replaces nothing: smart-commit-1.5b-v0.1 stays as it is.

A trade, not an upgrade. Trained with the same data and recipe as smart-commit-0.5b-v0.2, which kept v0.1's quality, this 1.5B copies earlier subjects half as often in young repositories and is about 15% faster on the CPU, but picks the commit type a little worse than v0.1 and writes vaguer descriptions in young repositories. With smart-commit's full prompt it recovers most of that (see below). For general use, v0.1 remains the recommended 1.5B.

What changed from v0.1:

• Trained on smart-commit's prompt without the fixed rules (72 columns, imperative mood, the Conventional Commits format): the system prompt holds only what varies by repository. The prompt is about 105 tokens shorter.
• A fifth of the examples come from young repositories (0–9 earlier subjects); release bumps whose version the diff adds are in the data.
• 20k examples instead of 40k, and no cap on the share of each type.

Files

File	What	SHA-256
model.safetensors (+ config and tokenizer)	merged weights, bf16	f47263169258b6cf176f850ebbd66ca3b6a8b82fe5e1ca503960a891539fee8a
smart-commit-1.5b-v0.2-q4_k_m.gguf	Q4_K_M for llama.cpp, 986 MB	e7b06b80497cedbdac9d7acff74a4ca9921de512e98d6e29892de5c57d424c15

Use with smart-commit

hf download LucasBRT/smart-commit-1.5b-v0.2 smart-commit-1.5b-v0.2-q4_k_m.gguf --local-dir ~/models
smart-commit model use ~/models/smart-commit-1.5b-v0.2-q4_k_m.gguf

It is not one of smart-commit's recommended models, so smart-commit sends it the full prompt, where it does best. It runs on smart-commit's llama.cpp devices (CPU and Vulkan GPUs); there is no validated OpenVINO export.

Evaluation

eval-v1: 500 commits from the test split of CommitChronicle (repositories never seen in training), 250 following Conventional Commits and 250 free-form; eval-short: 197 commits from young repositories (1–5 earlier subjects). Greedy decoding, smart-commit at 94ababd, Vulkan on an RTX 3050, with its retry and file-list fallback.

Model	type_match	type_macro_f1	first_try	fallback	word_f1	word_f1 (free)	young repos: first reply copied	young repos: ungrounded
Qwen2.5-Coder-1.5B-Instruct, upstream Q4_K_M	0.452	0.183	0.920	0.056	0.248	0.147	0.157	0.168
smart-commit-1.5b-v0.1	0.656	0.545	0.944	0.012	0.367	0.259	0.112	0.102
smart-commit-1.5b-v0.2, without the rules	0.628	0.498	0.956	0.020	0.352	0.255	0.056	0.142
smart-commit-1.5b-v0.2, with the full prompt	0.644	0.514	0.964	0.024	0.350	0.263	0.051	0.112

• type_match: same Conventional type as the real commit (of 250); type_macro_f1: the F1 of each type, averaged over the types.
• first_try: the first reply was usable; fallback: no reply was, and smart-commit wrote the message from the file list.
• word_f1: word overlap with the real subject, a rough proxy for content; ungrounded: a description with no word from the diff, likely vague.
• With n = 250 a difference under about 0.06 is noise; the drop in type_macro_f1 says the rarer types fare worse.

CPU, Ryzen 5 5500 (6 cores), the first 50 commits of eval-v1, model load counted once, without the rules: 2.16 s per message, against 2.53 s for v0.1. All the numbers and the method are in training/HYPOTHESES.md (v0.2).

Training

• Data: 20,019 examples (20,001 trained, 18 too long) from the train split of CommitChronicle (revision 5fd076e): 15,600 commits with at least 10 earlier subjects, 4,000 from young repositories, 419 release bumps whose version the diff adds; at most 150 per repository, no cap per type. Prompts built by smart-commit itself with --rules omitted, the real subject as the target (examples file sha256 23b9effc8b202e6d…).
• Method: LoRA r=16, alpha=32, dropout 0.05 on all linear layers, lr 2e-4 cosine, warmup 3%, batch 4 × 4, 1 epoch, seed 0, loss on the reply only; 4-bit NF4 base with fp16 compute (QLoRA) on a Kaggle Tesla T4, 148 minutes. torch 2.11.0, transformers 5.20.1, peft 0.21.2.
• Export: merged into the bf16 base and quantized to Q4_K_M with llama.cpp b11381 (training/export_gguf.py --merged).

Training on a GPU is not bit-for-bit reproducible; generation is (greedy). The recipe, the data selection and the scripts are in training/ (bundle b3).

Limitations

• English messages only (the training data is English).
• Worse than v0.1 on the commit type, and vaguer in young repositories, without the rules in its prompt; why the 1.5B needs them more than the 0.5B is not known yet (half the data of v0.1, or the size of the model).
• It can invent details the diff does not show, and it does not know when the staged changes are several commits.

Licence and attribution

The weights are released under Apache-2.0, like the base model. CommitChronicle (JetBrains Research) has no single licence: it gathers commits from repositories under MIT, Apache-2.0 and BSD-3-Clause, and asks that uses follow their terms, attribution included. The dataset records the repository and licence of every commit (repo and license fields), which is the provenance of the training data.