Instructions to use evalengine/decision-4b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use evalengine/decision-4b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf evalengine/decision-4b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf evalengine/decision-4b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf evalengine/decision-4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M

Use Docker

docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M

LM Studio
Jan
Ollama
How to use evalengine/decision-4b-gguf with Ollama:
```
ollama run hf.co/evalengine/decision-4b-gguf:Q4_K_M
```
Unsloth Desktop

How to use evalengine/decision-4b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "evalengine/decision-4b-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use evalengine/decision-4b-gguf with Docker Model Runner:
```
docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M
```

Lemonade

How to use evalengine/decision-4b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull evalengine/decision-4b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.decision-4b-gguf-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use evalengine/decision-4b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default evalengine/decision-4b-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use evalengine/decision-4b-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf evalengine/decision-4b-gguf:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "evalengine/decision-4b-gguf:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

decision-4b-gguf / SOURCES.json

johnsonchromia

Add Decision-0.8B comparison and shorten license text

76b147c verified 11 days ago

Raw History Blame Contribute Delete

6.85 kB

	{
	"schema_version": 1,
	"sources": {
	"policy_v2": {
	"repo": "togethercomputer/tev1",
	"revision": "1dde7782382c9f49d627153759b8d1deab426ce0",
	"license": "MIT repository license; authored synthetic data",
	"url": "https://github.com/togethercomputer/tev1",
	"training_decisions": 12000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"boolq": {
	"repo": "google/boolq",
	"revision": "35b264d03638db9f4ce671b711558bf7ff0f80d5",
	"config": null,
	"license_metadata": [
	"cc-by-sa-3.0"
	],
	"url": "https://huggingface.co/datasets/google/boolq",
	"training_decisions": 3000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"vitaminc": {
	"repo": "tals/vitaminc",
	"revision": "be6febb761b0b2807687e61e0b5282e459df2fa0",
	"config": "default",
	"format": "json",
	"data_files": {
	"train": "train.jsonl",
	"validation": "dev.jsonl",
	"test": "test.jsonl"
	},
	"license": "CC-BY-SA-3.0; Wikipedia-derived data terms",
	"license_url": "https://github.com/TalSchuster/VitaminC/blob/main/DATA_LICENSE",
	"training_decisions": 12000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"helpsteer2": {
	"repo": "nvidia/HelpSteer2",
	"revision": "990b2711a36180dd19d9c94b8627844866f8982a",
	"config": "default",
	"format": "json",
	"data_files": {
	"train": "train.jsonl.gz",
	"validation": "validation.jsonl.gz"
	},
	"license": "CC-BY-4.0",
	"license_url": "https://huggingface.co/datasets/nvidia/HelpSteer2",
	"training_decisions": 7968,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"research_taxonomy_v21": {
	"repo": "togethercomputer/tev1",
	"revision": "1dde7782382c9f49d627153759b8d1deab426ce0",
	"license": "MIT repository license; authored synthetic data",
	"url": "https://github.com/togethercomputer/tev1",
	"training_decisions": 3840,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"paws": {
	"repo": "google-research-datasets/paws",
	"revision": "161ece9501cf0a11f3e48bd356eaa82de46d6a09",
	"config": "labeled_final",
	"format": "parquet",
	"data_files": {
	"train": "labeled_final/train-00000-of-00001.parquet",
	"validation": "labeled_final/validation-00000-of-00001.parquet",
	"test": "labeled_final/test-00000-of-00001.parquet"
	},
	"license": "custom-any-purpose; retain Wikipedia provenance",
	"license_url": "https://github.com/google-research-datasets/paws/blob/master/LICENSE",
	"training_decisions": 8000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"routing_v2": {
	"repo": "togethercomputer/tev1",
	"revision": "1dde7782382c9f49d627153759b8d1deab426ce0",
	"license": "MIT repository license; authored synthetic data",
	"url": "https://github.com/togethercomputer/tev1",
	"training_decisions": 6000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"clinc150": {
	"repo": "clinc/clinc_oos",
	"revision": "155b9c710419136e17307b80d0a13e68cd46b4ec",
	"config": "plus",
	"format": "parquet",
	"data_files": {
	"train": "plus/train-00000-of-00001.parquet",
	"validation": "plus/validation-00000-of-00001.parquet",
	"test": "plus/test-00000-of-00001.parquet"
	},
	"license": "CC-BY-3.0",
	"license_url": "https://github.com/clinc/oos-eval/blob/master/LICENSE",
	"training_decisions": 8000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"mnli": {
	"repo": "nyu-mll/multi_nli",
	"revision": "da70db2af9d09693783c3320c4249840212ee221",
	"config": null,
	"license_metadata": [
	"cc-by-3.0",
	"cc-by-sa-3.0",
	"mit",
	"other"
	],
	"url": "https://huggingface.co/datasets/nyu-mll/multi_nli",
	"training_decisions": 5000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"go_emotions": {
	"repo": "google-research-datasets/go_emotions",
	"revision": "add492243ff905527e67aeb8b80c082af02207c3",
	"config": "simplified",
	"format": "parquet",
	"data_files": {
	"train": "simplified/train-00000-of-00001.parquet",
	"validation": "simplified/validation-00000-of-00001.parquet",
	"test": "simplified/test-00000-of-00001.parquet"
	},
	"license": "Apache-2.0 metadata; retain Reddit provenance",
	"license_url": "https://huggingface.co/datasets/google-research-datasets/go_emotions",
	"training_decisions": 4000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"banking77": {
	"repo": "legacy-datasets/banking77",
	"revision": "f54121560de48f2852f90be299010d1d6dc612ec",
	"config": null,
	"license_metadata": [
	"cc-by-4.0"
	],
	"url": "https://huggingface.co/datasets/legacy-datasets/banking77",
	"training_decisions": 3000,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	},
	"policy": {
	"repo": "togethercomputer/tev1",
	"revision": "1dde7782382c9f49d627153759b8d1deab426ce0",
	"license": "MIT repository license; authored synthetic data",
	"url": "https://github.com/togethercomputer/tev1",
	"training_decisions": 1500,
	"transformation": "Sampled and normalized into state/question/options; original labels mapped to an answer letter. Source text is not redistributed."
	}
	},
	"source_text_redistributed": false,
	"rights_review": "Source terms documented; no blanket corpus or learned-weight rights conclusion",
	"review": "SOURCE_REVIEW.md"
	}