Instructions to use evalengine/decision-4b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use evalengine/decision-4b-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf evalengine/decision-4b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf evalengine/decision-4b-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf evalengine/decision-4b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf evalengine/decision-4b-gguf:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf evalengine/decision-4b-gguf:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf evalengine/decision-4b-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf evalengine/decision-4b-gguf:Q4_K_M
Use Docker
docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use evalengine/decision-4b-gguf with Ollama:
ollama run hf.co/evalengine/decision-4b-gguf:Q4_K_M
- Unsloth Desktop
- Pi
How to use evalengine/decision-4b-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "evalengine/decision-4b-gguf:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use evalengine/decision-4b-gguf with Docker Model Runner:
docker model run hf.co/evalengine/decision-4b-gguf:Q4_K_M
- Lemonade
How to use evalengine/decision-4b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull evalengine/decision-4b-gguf:Q4_K_M
Run and chat with the model
lemonade run user.decision-4b-gguf-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use evalengine/decision-4b-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default evalengine/decision-4b-gguf:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use evalengine/decision-4b-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf evalengine/decision-4b-gguf:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "evalengine/decision-4b-gguf:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download SOURCE_REVIEW.md from evalengine/decision-4b-gguf: direct link, hf CLI and curl.
- Browser
- Download file 4.65 kB
-
https://huggingface.co/evalengine/decision-4b-gguf/resolve/main/SOURCE_REVIEW.md
- Command line
-
hf download hf://evalengine/decision-4b-gguf/SOURCE_REVIEW.md
-
curl -L -o SOURCE_REVIEW.md https://huggingface.co/evalengine/decision-4b-gguf/resolve/main/SOURCE_REVIEW.md
Source and license review
Reviewed 2026-09-24 against pinned local artifacts and primary publisher pages. This record distinguishes source terms, packaging decisions and unresolved interpretation. It does not conclude that a dataset license automatically applies to trained weights, or that a model license grants rights over source text. The GGUF package includes modified model weights and metadata, not corpus text.
Verified source statements
| Material | Observed terms | Primary source |
|---|---|---|
| Qwen3.5-4B | Apache-2.0; pinned license copied verbatim | Pinned license |
| Tev code and authored generators | MIT; copyright/permission notice retained | Pinned license |
| MultiNLI | Mixed-source corpus: card lists CC-BY-3.0, CC-BY-SA-3.0, MIT and other terms; fiction has source-specific terms | Dataset card |
| BoolQ | CC-BY-SA-3.0 metadata recorded by the pinned Tev source audit | Dataset card |
| Banking77 | CC-BY-4.0 metadata recorded by the pinned Tev source audit | Dataset card |
| CLINC150 | Author repository contains CC-BY-3.0 | Author license |
| PAWS | Custom dataset license permits any purpose and asks for acknowledgment; it is not an Apache-2.0 license | Author license |
| VitaminC | Author data license refers to applicable Wikipedia article terms, with CC-BY-SA-3.0 fallback; synthetic annotations also incorporate FEVER material | Author data license |
| HelpSteer2 | NVIDIA card lists CC-BY-4.0 | Publisher card |
| GoEmotions | Apache-2.0 metadata recorded at the pinned dataset revision; texts originate from Reddit | Publisher card |
| AG News and SST-5 | Unknown / unspecified source-license metadata in the pinned Tev audit | Pinned provenance |
SOURCES.json records the precise data revisions used. Links to branch names in
this review are primary references checked on the review date; pinned data and
original file hashes remain the reproducibility identity. Retain source names,
license links and transformation descriptions when citing this work.
What this package does
- Offers new Eval Engine material under Apache-2.0 to the extent of its rights, retains the Qwen license and Tev MIT notice, and identifies modifications.
- Copies only the selected adapter tensors, rewrites its base-location metadata for portability, and preserves original hashes and evaluation provenance.
- Excludes all training/evaluation JSONL files, raw source archives, model inputs, response text, optimizer states and full base weights.
- Keeps source terms separate from the model license. No claim is made that the collection is uniformly permissive, wholly original, or free of third-party rights.
Remaining interpretation and review limits
- The audit is source-level. It has not resolved every underlying Wikipedia, Reddit, fiction, or other third-party text right on a per-item basis.
- Attribution/share-alike scope for transformed source text and learned weights are distinct questions. This record does not decide that trained parameters are, or are not, adaptations under every applicable term and jurisdiction.
- AG News/SST-5 removal reduces ambiguity for the expanded training recipe; it does not resolve the remaining sources' terms. A baseline package still needs to disclose those training sources. Aggregate evaluation results are included without their source text.
- No complete personal-data, memorization, or provenance audit of upstream pretrained Qwen weights or every source row has been performed. The short answer interface is not evidence that those questions are settled.
- This is a local, reviewable experimental package. Publication is a separate action. The manifest records this review as documented, not blanket clearance.
These limits should travel with the artifact. They need not be presented as a claim that open-weight publication is prohibited.