Arbiter-8B-GGUF (v0.5)

GGUF builds of ternary/Arbiter-8B (Arbiter v0.5) for llama.cpp, Ollama, LM Studio and other GGUF runtimes. They replace the v0.4 files, which are still available in the repository history.

Arbiter-8B is a Jev-inspired decision model for wealth and asset management, fine-tuned from Qwen/Qwen3-8B. It keeps the Qwen3 architecture, tokenizer and chat template, so any runtime that supports Qwen3 GGUF files runs it. It reads financial text (news articles, headlines, SEC Form 10-K passages, central-bank statements) and answers a typed question about a named entity, such as event type, outlook, sentiment or policy stance. It answers with the letter of the chosen option.

Files

File Quant Size Notes
Arbiter-8B-Q4_K_M.gguf Q4_K_M 5.0 GB recommended for most users; within 1 point of F16 on every benchmark
Arbiter-8B-Q5_K_M.gguf Q5_K_M 5.9 GB not benchmarked
Arbiter-8B-Q6_K.gguf Q6_K 6.7 GB not benchmarked
Arbiter-8B-Q8_0.gguf Q8_0 8.7 GB near-lossless
Arbiter-8B-F16.gguf F16 16.4 GB full precision

The Q4_K_M, Q5_K_M and Q6_K files were quantized with an importance matrix computed from real Arbiter decision prompts, sampled across every training question family. On the base model the importance matrix was worth 5 points at 4 bits (SEntFiN tightened 75.3% vs 70.2% without).

Accuracy

SEntFiN (14,329) EDT (1,933) FOMC (496)
Arbiter v0.5, Q4_K_M 77.4% 93.6% 67.3%
Arbiter v0.5, Q8_0 76.7% 93.6% 66.9%
Arbiter v0.5, F16 76.8% 93.7% 66.3%
Arbiter v0.5 with its decision head (main repository) 76.4% 94.2% 67.5%
Arbiter v0.4, Q4_K_M 75.3% 88.1% 58.5%
Qwen3-8B, F16, not fine-tuned 75.0% 89.3% 55.0%

llama.cpp, temperature 0, one output token, original definitions. EDT and FOMC are scored on their held-out test splits; SEntFiN was never trained on. 95% confidence half-widths: SEntFiN ยฑ0.7, EDT ยฑ1.1, FOMC ยฑ4.2 points. For the comparison with Jev 1.14 and the OpenAI Decisions API, see the main repository.

Confidence. Read the letter's log-probability as the model's confidence. When the F16 build is more than 99% confident it is right 91.5โ€“92.6% of the time, against 59.9โ€“90.2% for base Qwen3-8B. The GGUF readout is more confident than it should be on SEntFiN and FOMC (expected calibration error 0.139 and 0.167 at F16; EDT 0.037). The decision head and per-question calibration in the main repository fix this (0.078, 0.067, 0.012); use transformers and the main repository when you need calibrated probabilities.

Usage

llama.cpp

llama-server -hf ternary/Arbiter-8B-GGUF:Q4_K_M -c 4096
# or with a downloaded file
llama-server -m Arbiter-8B-Q4_K_M.gguf -c 4096 -ngl 99

Ollama

ollama run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M

Prompt format

Arbiter was trained on one fixed prompt. Use it exactly, with thinking disabled, and generate a single token.

  • System: Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. Respond with only its uppercase letter, with no explanation or reasoning.
  • User: a JSON object with evidence (the verbatim text), criterion (the question, followed by Focus entity: <entity>.) and options, each with a letter and a description of the form <option id>: <definition>.

Example against llama-server's OpenAI-compatible endpoint:

import json, math, requests

SYSTEM = ("Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. "
          "Respond with only its uppercase letter, with no explanation or reasoning.")
options = {
    "hawkish": "points towards tighter monetary policy, such as higher rates or concern about inflation",
    "neutral": "no clear lean towards tighter or looser policy",
    "dovish": "points towards looser monetary policy, such as lower rates or concern about growth and employment",
}
user = json.dumps({
    "evidence": "The Committee judges that inflation remains elevated.",
    "criterion": "What monetary policy stance does the sentence express? Focus entity: Federal Reserve (FOMC).",
    "options": [{"letter": "ABC"[i], "description": f"{k}: {v}"} for i, k in enumerate(options)],
})
reply = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}],
    "max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20,
    "chat_template_kwargs": {"enable_thinking": False},
}).json()["choices"][0]

letter = reply["message"]["content"].strip()                      # e.g. "A" -> hawkish
probs = {t["token"]: math.exp(t["logprob"]) for t in reply["logprobs"]["content"][0]["top_logprobs"]
         if t["token"] in "ABC"}                                    # confidence per option letter

The question definitions Arbiter was trained on (news event type, outlook, certainty, main subject, role, 10-K risk, legal status and filing flags, EDT corporate events, FOMC policy stance and headline sentiment, each new v0.5 question in an original and a tightened wording) are in questions.py in the main repository. Arbiter works best with those exact definitions and with at most 16 options.

Limitations

See the main repository. In short: EDT clinical-trial recall fell to 54.8% in v0.5; Arbiter under-calls neutral headlines and dovish FOMC sentences; not investment advice.

License

Apache-2.0, like the base model Qwen3-8B.

Downloads last month
591
GGUF
Model size
8B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ternary/Arbiter-8B-GGUF

Finetuned
Qwen/Qwen3-8B
Quantized
(1)
this model