Instructions to use evalengine/decision-4b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use evalengine/decision-4b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf evalengine/decision-4b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf evalengine/decision-4b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M

Use Docker

docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M

LM Studio
Jan
Ollama
How to use evalengine/decision-4b-gguf with Ollama:
```
ollama run hf.co/evalengine/decision-4b-gguf:Q4_K_M
```
Unsloth Desktop

How to use evalengine/decision-4b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "evalengine/decision-4b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use evalengine/decision-4b-gguf with Docker Model Runner:
```
docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M
```

Lemonade

How to use evalengine/decision-4b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull evalengine/decision-4b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.decision-4b-gguf-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use evalengine/decision-4b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default evalengine/decision-4b-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use evalengine/decision-4b-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "evalengine/decision-4b-gguf:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

decision-4b-gguf / SOURCE_REVIEW.md

johnsonchromia

Add Decision-0.8B comparison and shorten license text

76b147c verified 19 days ago

preview code

Raw History Blame Contribute Delete

4.65 kB

	# Source and license review

	Reviewed 2026-09-24 against pinned local artifacts and primary publisher pages.
	This record distinguishes source terms, packaging decisions and unresolved
	interpretation. It does not conclude that a dataset license automatically
	applies to trained weights, or that a model license grants rights over source
	text. The GGUF package includes modified model weights and metadata, not corpus text.

	## Verified source statements

	\| Material \| Observed terms \| Primary source \|
	\|---\|---\|---\|
	\| Qwen3.5-4B \| Apache-2.0; pinned license copied verbatim \| [Pinned license](https://huggingface.co/Qwen/Qwen3.5-4B/blob/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a/LICENSE) \|
	\| Tev code and authored generators \| MIT; copyright/permission notice retained \| [Pinned license](https://github.com/togethercomputer/tev1/blob/1dde7782382c9f49d627153759b8d1deab426ce0/LICENSE) \|
	\| MultiNLI \| Mixed-source corpus: card lists CC-BY-3.0, CC-BY-SA-3.0, MIT and other terms; fiction has source-specific terms \| [Dataset card](https://huggingface.co/datasets/nyu-mll/multi_nli) \|
	\| BoolQ \| CC-BY-SA-3.0 metadata recorded by the pinned Tev source audit \| [Dataset card](https://huggingface.co/datasets/google/boolq) \|
	\| Banking77 \| CC-BY-4.0 metadata recorded by the pinned Tev source audit \| [Dataset card](https://huggingface.co/datasets/legacy-datasets/banking77) \|
	\| CLINC150 \| Author repository contains CC-BY-3.0 \| [Author license](https://github.com/clinc/oos-eval/blob/master/LICENSE) \|
	\| PAWS \| Custom dataset license permits any purpose and asks for acknowledgment; it is not an Apache-2.0 license \| [Author license](https://github.com/google-research-datasets/paws/blob/master/LICENSE) \|
	\| VitaminC \| Author data license refers to applicable Wikipedia article terms, with CC-BY-SA-3.0 fallback; synthetic annotations also incorporate FEVER material \| [Author data license](https://github.com/TalSchuster/VitaminC/blob/main/DATA_LICENSE) \|
	\| HelpSteer2 \| NVIDIA card lists CC-BY-4.0 \| [Publisher card](https://huggingface.co/datasets/nvidia/HelpSteer2) \|
	\| GoEmotions \| Apache-2.0 metadata recorded at the pinned dataset revision; texts originate from Reddit \| [Publisher card](https://huggingface.co/datasets/google-research-datasets/go_emotions) \|
	\| AG News and SST-5 \| Unknown / unspecified source-license metadata in the pinned Tev audit \| [Pinned provenance](https://github.com/togethercomputer/tev1/blob/1dde7782382c9f49d627153759b8d1deab426ce0/DATA_SOURCES.md) \|

	`SOURCES.json` records the precise data revisions used. Links to branch names in
	this review are primary references checked on the review date; pinned data and
	original file hashes remain the reproducibility identity. Retain source names,
	license links and transformation descriptions when citing this work.

	## What this package does

	1. Offers new Eval Engine material under Apache-2.0 to the extent of its rights,
	retains the Qwen license and Tev MIT notice, and identifies modifications.
	2. Copies only the selected adapter tensors, rewrites its base-location metadata
	for portability, and preserves original hashes and evaluation provenance.
	3. Excludes all training/evaluation JSONL files, raw source archives, model inputs,
	response text, optimizer states and full base weights.
	4. Keeps source terms separate from the model license. No claim is made that the
	collection is uniformly permissive, wholly original, or free of third-party
	rights.

	## Remaining interpretation and review limits

	- The audit is source-level. It has not resolved every underlying Wikipedia,
	Reddit, fiction, or other third-party text right on a per-item basis.
	- Attribution/share-alike scope for transformed source text and learned weights
	are distinct questions. This record does not decide that trained parameters
	are, or are not, adaptations under every applicable term and jurisdiction.
	- AG News/SST-5 removal reduces ambiguity for the expanded training recipe; it
	does not resolve the remaining sources' terms. A baseline package still needs
	to disclose those training sources. Aggregate evaluation results are included
	without their source text.
	- No complete personal-data, memorization, or provenance audit of upstream
	pretrained Qwen weights or every source row has been performed. The short
	answer interface is not evidence that those questions are settled.
	- This is a local, reviewable experimental package. Publication is a separate
	action. The manifest records this review as documented, not blanket clearance.

	These limits should travel with the artifact. They need not be presented as a
	claim that open-weight publication is prohibited.