Instructions to use ternary/Arbiter-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ternary/Arbiter-8B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="ternary/Arbiter-8B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ternary/Arbiter-8B") model = AutoModelForCausalLM.from_pretrained("ternary/Arbiter-8B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Arbiter-8B is now at v0.5.
Arbiter v0.5: what changed from v0.4
Training: what changed from v0.4
Follows the definitions you give it. v0.4 often answered from its own idea of a category, even when the option definitions said otherwise. v0.5 was trained on the same questions under different definitions, where the definition changes the right answer: under one wording a partnership counts as a new contract, under another it does not. To answer correctly, the model has to read the definitions, so it now applies whatever definitions you pass.
Separates real corporate events from look-alikes. v0.4's main error on event detection was labelling non-events as events: a partnership became a "new contract", a minority stake an "acquisition". v0.5 was trained on contrast pairs (the same article, with one definition changed so that the answer flips) and on hard "none" examples (articles about partnerships, sponsorships, minority investments, financings, divestitures, patents, product launches and appointments, which look like events but are not). Precision on new contracts went from 38% to 87%.
Reads central-bank language. New question type: the policy stance of a central-bank sentence (hawkish, neutral or dovish), in a standard and a stricter wording.
Headline sentiment, two ways. New sentiment questions with an "implied" definition (what the reported facts mean for the company: a dividend is positive) and a "stated" definition (only the direction the headline itself expresses: a dividend declaration is neutral). Both come for headlines that name one company or several.
Better-calibrated confidence. Label smoothing (0.05) on hard labels keeps the model from putting all its weight on one answer, which brought calibration error on event detection from 0.099 to 0.012. Training ran 2,400 steps instead of 2,000.
Release
- Full fp16 model with the decision head and per-question-type calibration, plus the unmerged 4-bit LoRA
- GGUFs: F16, Q8_0, and Q6_K / Q5_K_M / Q4_K_M quantised with an importance matrix built from decision prompts
v0.4 β v0.5
| v0.4 | v0.5 | |
|---|---|---|
| EDT accuracy (n=1,933) | 83.5% | 94.2% |
| FOMC accuracy (n=496) | 60.1% | 67.5% |
| SEntFiN accuracy (n=14,329) | 76.0% | 76.4% |
EDT new_contract precision |
38.3% | 86.7% |
EDT none recall |
81.2% | 95.2% |
| Answers that flip when options are shuffled (FOMC) | 15.5% | 4.8% |
| Calibration error (ECE), EDT / FOMC | 0.099 / 0.127 | 0.012 / 0.067 |
| Q4_K_M GGUF, EDT / FOMC | 88.1% / 58.5% | 93.6% / 67.3% |
| EDT clinical-trial recall | 87.1% | 54.8% |
Comparison with Jev and OpenAI
Same items, same question and same option definitions for every system.
| Arbiter v0.5 | Qwen3-8B (base) | Jev 1.14 | OpenAI Decisions API | |
|---|---|---|---|---|
| EDT, event detection, accuracy (n=1,933) | 94.2% | 89.3% | 88.3% | 90.8% |
| FOMC, policy stance, accuracy (n=496) | 67.5% | 55.0% | 66.7% | 67.3% |
| SEntFiN, signals caught (n=1,000) | 92.0% | 82.8% | 91.0% | 91.7% |
| SEntFiN, signal precision (n=1,000) | 72.2% | 76.3% | 78.8% | 74.8% |
| SEntFiN, accuracy (n=1,000) | 75.7% | 75.4% | 81.6% | 78.0% |
- EDT: Arbiter leads all three (paired McNemar, z β₯ 7.5)
- FOMC: level with Jev and OpenAI (z β€ 0.4), +12.5 points over the base model
- SEntFiN: catches the most positive and negative signals; Jev is more precise
- Confidence: answers given with more than 99% confidence are right 91.5β92.6% of the time
Jev 1.14 is TypeSafe's API; the OpenAI Decisions API is gpt-6-luna. EDT and FOMC are held-out test splits. SEntFiN is a random sample of 1,000 headlines.