Instructions to use ternary/Arbiter-8B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ternary/Arbiter-8B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use ternary/Arbiter-8B-GGUF with Ollama:
ollama run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use ternary/Arbiter-8B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ternary/Arbiter-8B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ternary/Arbiter-8B-GGUF with Docker Model Runner:
docker model run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M
- Lemonade
How to use ternary/Arbiter-8B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ternary/Arbiter-8B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Arbiter-8B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use ternary/Arbiter-8B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ternary/Arbiter-8B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ternary/Arbiter-8B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ternary/Arbiter-8B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ternary/Arbiter-8B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Arbiter-8B-GGUF (v0.5)
GGUF builds of ternary/Arbiter-8B (Arbiter v0.5) for llama.cpp, Ollama, LM Studio and other GGUF runtimes. They replace the v0.4 files, which are still available in the repository history.
Arbiter-8B is a Jev-inspired decision model for wealth and asset management, fine-tuned from Qwen/Qwen3-8B. It keeps the Qwen3 architecture, tokenizer and chat template, so any runtime that supports Qwen3 GGUF files runs it. It reads financial text (news articles, headlines, SEC Form 10-K passages, central-bank statements) and answers a typed question about a named entity, such as event type, outlook, sentiment or policy stance. It answers with the letter of the chosen option.
Files
| File | Quant | Size | Notes |
|---|---|---|---|
Arbiter-8B-Q4_K_M.gguf |
Q4_K_M | 5.0 GB | recommended for most users; within 1 point of F16 on every benchmark |
Arbiter-8B-Q5_K_M.gguf |
Q5_K_M | 5.9 GB | not benchmarked |
Arbiter-8B-Q6_K.gguf |
Q6_K | 6.7 GB | not benchmarked |
Arbiter-8B-Q8_0.gguf |
Q8_0 | 8.7 GB | near-lossless |
Arbiter-8B-F16.gguf |
F16 | 16.4 GB | full precision |
The Q4_K_M, Q5_K_M and Q6_K files were quantized with an importance matrix computed from real Arbiter decision prompts, sampled across every training question family. On the base model the importance matrix was worth 5 points at 4 bits (SEntFiN tightened 75.3% vs 70.2% without).
Accuracy
| SEntFiN (14,329) | EDT (1,933) | FOMC (496) | |
|---|---|---|---|
| Arbiter v0.5, Q4_K_M | 77.4% | 93.6% | 67.3% |
| Arbiter v0.5, Q8_0 | 76.7% | 93.6% | 66.9% |
| Arbiter v0.5, F16 | 76.8% | 93.7% | 66.3% |
| Arbiter v0.5 with its decision head (main repository) | 76.4% | 94.2% | 67.5% |
| Arbiter v0.4, Q4_K_M | 75.3% | 88.1% | 58.5% |
| Qwen3-8B, F16, not fine-tuned | 75.0% | 89.3% | 55.0% |
llama.cpp, temperature 0, one output token, original definitions. EDT and FOMC are scored on their held-out test splits; SEntFiN was never trained on. 95% confidence half-widths: SEntFiN ยฑ0.7, EDT ยฑ1.1, FOMC ยฑ4.2 points. For the comparison with Jev 1.14 and the OpenAI Decisions API, see the main repository.
Confidence. Read the letter's log-probability as the model's confidence. When the F16 build is more than 99% confident it is right 91.5โ92.6% of the time, against 59.9โ90.2% for base Qwen3-8B. The GGUF readout is more confident than it should be on SEntFiN and FOMC (expected calibration error 0.139 and 0.167 at F16; EDT 0.037). The decision head and per-question calibration in the main repository fix this (0.078, 0.067, 0.012); use transformers and the main repository when you need calibrated probabilities.
Usage
llama.cpp
llama-server -hf ternary/Arbiter-8B-GGUF:Q4_K_M -c 4096
# or with a downloaded file
llama-server -m Arbiter-8B-Q4_K_M.gguf -c 4096 -ngl 99
Ollama
ollama run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M
Prompt format
Arbiter was trained on one fixed prompt. Use it exactly, with thinking disabled, and generate a single token.
- System:
Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. Respond with only its uppercase letter, with no explanation or reasoning. - User: a JSON object with
evidence(the verbatim text),criterion(the question, followed byFocus entity: <entity>.) andoptions, each with aletterand adescriptionof the form<option id>: <definition>.
Example against llama-server's OpenAI-compatible endpoint:
import json, math, requests
SYSTEM = ("Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. "
"Respond with only its uppercase letter, with no explanation or reasoning.")
options = {
"hawkish": "points towards tighter monetary policy, such as higher rates or concern about inflation",
"neutral": "no clear lean towards tighter or looser policy",
"dovish": "points towards looser monetary policy, such as lower rates or concern about growth and employment",
}
user = json.dumps({
"evidence": "The Committee judges that inflation remains elevated.",
"criterion": "What monetary policy stance does the sentence express? Focus entity: Federal Reserve (FOMC).",
"options": [{"letter": "ABC"[i], "description": f"{k}: {v}"} for i, k in enumerate(options)],
})
reply = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
"messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}],
"max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20,
"chat_template_kwargs": {"enable_thinking": False},
}).json()["choices"][0]
letter = reply["message"]["content"].strip() # e.g. "A" -> hawkish
probs = {t["token"]: math.exp(t["logprob"]) for t in reply["logprobs"]["content"][0]["top_logprobs"]
if t["token"] in "ABC"} # confidence per option letter
The question definitions Arbiter was trained on (news event type, outlook, certainty, main subject, role, 10-K risk, legal status and filing flags, EDT corporate events, FOMC policy stance and headline sentiment, each new v0.5 question in an original and a tightened wording) are in questions.py in the main repository. Arbiter works best with those exact definitions and with at most 16 options.
Limitations
See the main repository. In short: EDT clinical-trial recall fell to 54.8% in v0.5; Arbiter under-calls neutral headlines and dovish FOMC sentences; not investment advice.
License
Apache-2.0, like the base model Qwen3-8B.
- Downloads last month
- 591
4-bit
5-bit
6-bit
8-bit
16-bit