Instructions to use marzoukbaig14/committed-gguf-0.6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use marzoukbaig14/committed-gguf-0.6b with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="marzoukbaig14/committed-gguf-0.6b",
	filename="committed-0.6b-baseline-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use marzoukbaig14/committed-gguf-0.6b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Use Docker

docker model run hf.co/marzoukbaig14/committed-gguf-0.6b:Q4_K_M

LM Studio
Jan

vLLM

How to use marzoukbaig14/committed-gguf-0.6b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "marzoukbaig14/committed-gguf-0.6b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "marzoukbaig14/committed-gguf-0.6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Ollama
How to use marzoukbaig14/committed-gguf-0.6b with Ollama:
```
ollama run hf.co/marzoukbaig14/committed-gguf-0.6b:Q4_K_M
```

Unsloth Studio

How to use marzoukbaig14/committed-gguf-0.6b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for marzoukbaig14/committed-gguf-0.6b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for marzoukbaig14/committed-gguf-0.6b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for marzoukbaig14/committed-gguf-0.6b to start chatting

How to use marzoukbaig14/committed-gguf-0.6b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "marzoukbaig14/committed-gguf-0.6b:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use marzoukbaig14/committed-gguf-0.6b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use marzoukbaig14/committed-gguf-0.6b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "marzoukbaig14/committed-gguf-0.6b:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use marzoukbaig14/committed-gguf-0.6b with Docker Model Runner:
```
docker model run hf.co/marzoukbaig14/committed-gguf-0.6b:Q4_K_M
```

Lemonade

How to use marzoukbaig14/committed-gguf-0.6b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull marzoukbaig14/committed-gguf-0.6b:Q4_K_M

Run and chat with the model

lemonade run user.committed-gguf-0.6b-Q4_K_M

List all available models

lemonade list

Committed — Qwen3-0.6B (Q4_K_M GGUF) · default model

A small model fine-tuned to write Conventional Commits messages from a git diff. It runs locally on CPU via llama.cpp, so your code never leaves your machine.

This is the default, recommended model for Committed. It started as a controlled experiment — could a model a third the size of the original 1.7B do the same job? — and it essentially can: it matches the 1.7B on picking the right commit type and staying faithful to the diff, at ~⅓ the parameters, a ~397 MB download (vs ~1 GB), and faster local inference. The one honest trade is that it writes slightly vaguer messages (lower specificity), so the larger 1.7B GGUF remains available as the bigger sibling if you want maximum specificity.

This repo holds the merged, quantized GGUF used for serving. The training dataset, LoRA adapter, and source code are linked below.

Live Demo · Gradio Space · 0.6B GGUF (this repo) · 1.7B GGUF · 0.6B adapter · 1.7B adapter · Dataset · GitHub

Details

Base: Qwen/Qwen3-0.6B (Apache-2.0)
Method: QLoRA fine-tune (PEFT LoRA + TRL SFTTrainer, vanilla transformers), merged into the base, converted to GGUF
Quantization: Q4_K_M (~397 MB / 378 MiB)
Task: single-file git diff → one Conventional Commits subject line, type(scope): description
Decoding: GBNF grammar-constrained, so output is always a well-formed CC line
Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)

Usage

The trained behavior depends on the exact prompt rendering used in training plus the GBNF grammar applied at decode time, so a bare llama-cpp-python prompt will not reproduce the evaluated output. Run it through the project's inference path instead.

Easiest — the CLI, which downloads this GGUF on first run (no token needed; it's public) and wires up the prompt + grammar for you:

pip install --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu "committed @ git+https://github.com/marzoukbaig14/Committed.git"
git diff | committed

git diff | committed uses this 0.6B model by default. To use the larger 1.7B instead:

git diff | committed --model 1.7b

Or call it through the repo's engine.py, the FastAPI endpoint, or the Gradio Space — all linked below. Full instructions and options are in the repo README.

Results

Evaluated against the un-tuned Qwen3-0.6B base on a 442-example test set, scored by an LLM judge (DeepSeek) on four axes; the judge was validated against 50 hand-rated examples. Headline numbers are reweighted to the true commit-type distribution.

The four-arm comparison — both base models, both fine-tunes, all judged by the same DeepSeek judge so the numbers are directly comparable:

Metric	0.6B base	0.6B fine-tune	1.7B base	1.7B fine-tune
Type accuracy	0.154	0.601	0.131	0.637
Type-correctness	0.296	0.726	0.296	0.778
Faithfulness	0.285	0.810	0.491	0.848
Completeness	0.353	0.729	0.543	0.776
Specificity	0.414	0.545	0.814	0.667
Conjunctive (all 4)	0.101	0.359	0.175	0.471
Graded mean (0–3)	0.777	2.094	1.447	2.139
`feat`-share of outputs	86.7%	9.7%	95.5%	8.4%

The finding: both base models "feat-collapse" — they label ~87–96% of all diffs feat regardless of content, scoring below a trivial always-guess-fix baseline (0.489) on type. Fine-tuning breaks the collapse on both. The 0.6B fine-tune lands within a few points of the 1.7B on type, faithfulness, completeness, and the graded mean (2.094 vs 2.139) — the gap is concentrated in specificity (0.545 vs 0.667): the 0.6B picks the right type and states accurate facts, but names the concrete mechanism less often.

Honest caveat on the judge: agreement with human raters is moderate-to-substantial on three axes (κ ≈ 0.56–0.61) but weakest on specificity (κ 0.34) — the very axis where 0.6B trails — so that gap carries the most judge uncertainty. These DeepSeek-judged numbers are not comparable to any earlier Gemini-judged figures; only the deltas within this all-DeepSeek table are valid.

Full before/after, the feat-collapse analysis, and the judge validation are in the eval writeup: FINDINGS_v1.md.

The larger 1.7B GGUF (bigger sibling, higher specificity) — marzoukbaig14/committed-gguf.
This model's LoRA adapter — marzoukbaig14/committed-qwen3-0.6b-lora. The unmerged adapter weights, if you want to merge or train further.
Live demo (browser) — try it on the portfolio site. Paste a diff, pick a model, get a commit message.
Live demo (Gradio) — Hugging Face Space.
Training dataset — marzoukbaig14/committed-train, filtered from CommitChronicle, with composition tables and provenance.
Source, training & eval code — github.com/marzoukbaig14/Committed.
Eval writeup — FINDINGS_v1.md.

License

Apache-2.0, inherited from the Qwen3-0.6B base.

Citation

Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).

Downloads last month: 141

GGUF

Model size

0.8B params

Architecture

qwen3

Hardware compatibility

4-bit

Model tree for marzoukbaig14/committed-gguf-0.6b

Base model

Qwen/Qwen3-0.6B-Base

Finetuned

Qwen/Qwen3-0.6B

Quantized

(355)

this model

Dataset used to train marzoukbaig14/committed-gguf-0.6b

Paper for marzoukbaig14/committed-gguf-0.6b

From Commit Message Generation to History-Aware Commit Message Completion

Paper • 2308.07655 • Published Aug 15, 2023

marzoukbaig14
/

committed-gguf-0.6b

Committed — Qwen3-0.6B (Q4_K_M GGUF) · default model

Details

Usage

Results

Related

License

Citation

Model tree for marzoukbaig14/committed-gguf-0.6b

Dataset used to train marzoukbaig14/committed-gguf-0.6b

Paper for marzoukbaig14/committed-gguf-0.6b

From Commit Message Generation to History-Aware Commit Message Completion