ARBITER-272M (BF16)

ARBITER — Attentive Relational Bidirectional Impartial Typed Encoder for Reasoning.

ARBITER-272M is a fast, ultra-compact typed decision model specialized in decision-making, routing, and structured reasoning for autonomous AI agents and enterprise workflows.

Built on the ARBITER relational architecture (Inter-Candidate Multi-Head Set-Attention & Dynamic HyperFiLM modulation), ARBITER jointly encodes context, instruction, and candidate options, allowing options to attend to each other rather than being scored in isolation.

---

Key Highlights

• Ultra-Compact & Fast: Native BF16 export of 544 MB (model.safetensors, 544,051,310 bytes), peak allocated VRAM of 759 MiB (1,562 MiB reserved), median GPU latency of 15.36 ms (RTX 5090).
• CPU Friendly: At only 272M parameters, ARBITER runs smoothly and fast on a standard CPU runtime — no GPU required (ideal for Colab CPU, local machines, or lightweight cloud instances).
• Inter-Candidate Set-Attention: Options attend to each other via a multi-head self-attention mechanism over the candidate set prior to scoring.
• Order & Permutation Robustness: 82.8% decision consistency across 3 permutation orders (500 items), measured on the evaluated subset.
• Multi-Format Native Decisions: Supports polytomous multiple-choice (choice), boolean verification (noul), and ordinal regression grading (score) in a single unified checkpoint.
• 14-Language Intent Routing: 91.33% intent accuracy across 14 languages on MASSIVE with 20 candidates per request (vs 46.69% for Laya Multilingual and 49.52% for GLiNER Multi).
• Fine-Grained Classification: 87.34% on Banking77 (77 bank intents) and 94.60% on TREC coarse.
• Autonomous Package: Self-contained weights and tokenizer; no external downloads required at inference time.

---

Quickstart

⚡ Quick Launch Options

Method	Command / Link	Description
🚀 Google Colab		1-click interactive demo in browser (runs on CPU or GPU)
🐳 Docker (GPU)	docker run --gpus all -p 8000:8000 ghcr.io/spectral-ai-lab/arbiter-serve:latest --model spectral-ai-lab/arbiter-272m	One-line production serving (vLLM / OpenAI compatible)
🐳 Docker (CPU)	docker run -p 8000:8000 ghcr.io/spectral-ai-lab/arbiter-serve:cpu --model spectral-ai-lab/arbiter-272m	Lightweight CPU container (no GPU required)
🐍 Python / PyPI	pip install "spectral-arbiter[serve]"	Official library (arbiter) & CLI (arbiter serve / python -m arbiter.serve)
🤗 Transformers	AutoModel.from_pretrained(..., trust_remote_code=True)	Direct native HF loading with remote code

Installation

pip install spectral-arbiter

1. Basic Decision Query (model.decide)

from arbiter import ArbiterModel

# Load directly from Hugging Face Hub (automatic download & caching)
model = ArbiterModel.from_pretrained("spectral-ai-lab/arbiter-272m", device="cuda:0")  # or "cpu"

# Decide which customer support service should handle a ticket
result = model.decide(
    context="My package was marked as delivered yesterday, but I never received it. The courier is unresponsive.",
    instruction="Which customer support team must handle this claim?",
    criteria={
        "delivery": "Lost parcel investigation and courier disputes",
        "billing": "Payment errors, refunds, and invoice questions",
        "technical": "Mobile application login or website bugs"
    },
    question_type="choice"
)

print(f"Selected option: {result['selected_key']}")
print(f"Confidence     : {result['confidence']:.2%}")
print(f"Probabilities  : {result['probabilities']}")
# Output:
# Selected option: delivery
# Confidence     : 86.74%

(You can also load directly via Hugging Face transformers using trust_remote_code=True:)

from transformers import AutoModel

model = AutoModel.from_pretrained("spectral-ai-lab/arbiter-272m", trust_remote_code=True)

2. Multi-Task & Agent Workflow (model.predict)

ARBITER supports multi-question decision dictionaries matching the laya and cases.jsonl benchmark format:

results = model.predict(
    state="Customer has been waiting 14 days for a replacement item under warranty.",
    questions={
        "intent": {
            "instructions": "Identify the primary intent",
            "type": "choice",
            "criteria": {"warranty_claim": "Warranty replacement", "inquiry": "General question"}
        },
        "dissatisfied": {
            "instructions": "Is the customer dissatisfied?",
            "type": "noul"
        },
        "urgency": {
            "instructions": "Rate urgency level",
            "type": "score",
            "criteria": ["Low", "Medium", "High", "Critical"]
        }
    }
)

print(results["answers"])

3. Online Serving (CLI & Docker, vLLM Compatible)

Via Python CLI

# Install serving extras
pip install "spectral-arbiter[serve]"

# Start server via CLI
arbiter serve --port 8000 --device cuda:0

# Or via Python module
python -m arbiter.serve --port 8000 --device cuda:0

Via Docker

Deploy ARBITER with zero local setup using the official container image from GitHub Container Registry:

# Run on GPU (NVIDIA CUDA 12, RTX 5090 / Blackwell ready)
docker run --gpus all -d -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  ghcr.io/spectral-ai-lab/arbiter-serve:latest \
  --model spectral-ai-lab/arbiter-272m

# Or run on CPU
docker run -d -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  ghcr.io/spectral-ai-lab/arbiter-serve:cpu \
  --model spectral-ai-lab/arbiter-272m

Supports the official vLLM Structured Decisions specification (POST /v1/systemone) and ARBITER native endpoints (POST /decide).

---

Benchmark Results

ARBITER achieves the highest micro accuracy on both evaluated protocol aggregates, with particularly strong intent routing and probability quality. Tables use a fixed set of model variants throughout; bold marks the best reported result in each row (including ties). Accuracy is in percent; ↓ means lower is better. These are local protocol comparisons, not a claim of leadership on every task.

Intent Routing & Classification

Benchmark / metric	ARBITER	Laya	Laya Multilingual	Laya Typed	GLiNER Decide	GLiNER Multi	EmbeddingGemma2
MASSIVE Intent · 14 languages	91.33%	33.95%	46.69%	34.43%	42.29%	49.52%	73.79%
MASSIVE Scenario · 14 languages	89.40%	30.10%	44.74%	28.43%	36.98%	45.26%	58.07%
Banking77 · 77 intents	87.34%	37.44%	35.94%	37.31%	65.29%	58.90%	85.55%
TREC · question classification	94.60%	88.20%	90.40%	85.40%	32.60%	36.20%	47.40%
SST-2 · sentiment	86.93%	90.02%	84.98%	89.79%	87.96%	78.78%	83.26%

Typed Decisions & Natural Language Inference

Benchmark / metric	ARBITER	Laya	Laya Multilingual	Laya Typed	GLiNER Decide	GLiNER Multi	EmbeddingGemma2
Typed Decisions	65.50%	36.20%	35.25%	76.75%	50.35%	45.95%	40.85%
XNLI · 15 languages	68.87%	54.38%	73.82%	55.64%	38.24%	38.38%	33.56%
Fast Decisions Single · micro	43.35%	52.96%	46.77%	54.92%	64.42%	59.38%	41.73%
Fast Decisions Single · macro	43.97%	53.80%	46.66%	55.66%	66.30%	60.63%	42.60%
Fast Decisions Multi · exact match	4.67% (14/300)	2.33% (7/300)	1.00% (3/300)	3.67% (11/300)	1.33% (4/300)	4.00% (12/300)	0.33% (1/300)
Fast Decisions Multi · micro F1	51.37%	51.72%	45.50%	56.41%	51.74%	48.41%	31.22%

Overall Protocol Results

Benchmark / metric	ARBITER	Laya	Laya Multilingual	Laya Typed	GLiNER Decide	GLiNER Multi	EmbeddingGemma2
Historical · micro accuracy	74.47%	48.30%	56.86%	52.66%	46.58%	47.84%	51.25%
Historical · suite macro accuracy	77.03%	47.18%	57.89%	48.33%	44.61%	47.33%	52.45%
Public · micro accuracy	69.68%	53.92%	51.18%	52.25%	62.86%	57.85%	59.87%

Historical: 21,915 decisions across 60 suites. Public: 9,252 decisions across 23 suites. Micro averages weight decisions equally; suite macro averages weight suites equally. Language variants count as separate suites. These aggregates reflect this task mixture. Column names are abbreviated: GLiNER refers to GLiNER2.5, Laya to ConvAI base, and EmbeddingGemma to the EmbeddingGemma-2 baseline pipeline.

Probability Quality

Measured on the same 21,915 historical decisions. GLiNER's reconstructed adapter probabilities are reported separately below and do not enter this comparison.

Benchmark / metric	ARBITER	Laya	Laya Multilingual	Laya Typed	GLiNER Decide	GLiNER Multi	EmbeddingGemma2
Brier score ↓	0.3395	0.7974	0.6788	0.6527	—	—	0.7106
NLL ↓	0.6761	4.3655	2.3747	1.8325	—	—	1.5450
Suite-weighted ECE · 15 bins ↓	0.0674	0.3214	0.2769	0.2166	—	—	0.2947

Brier is the mean squared error of probability vectors against one-hot targets; NLL is mean negative log probability of the target. ECE is computed per suite with 15 bins, then weighted by suite size (it differs from pooled ECE). ARBITER has 48% lower Brier and 63% lower NLL than Laya Typed in this protocol. ARBITER and Laya use their native decision probabilities; EmbeddingGemma uses cosine-similarity softmax. Application thresholds require separate validation.

GLiNER adapter probability metrics

Metric	GLiNER Decide	GLiNER Multi
Brier ↓	0.6611	0.6664
NLL ↓	1.5762	1.5555
Suite-weighted ECE ↓	0.1098	0.1379

These distributions are reconstructed by the benchmark adapter, not native full-candidate GLiNER distributions. A dash above means excluded from the comparable probability table, not that the model cannot produce confidence scores.

Inference Efficiency

Accuracy–memory trade-off: ARBITER combines 74.47% historical micro accuracy with 759 MiB peak allocated GPU memory, the lowest measured footprint among these pipeline configurations. Its 15.36 ms median latency is 28% lower than the EmbeddingGemma baseline (21.36 ms). Compared with Laya Multilingual, it delivers +17.61 accuracy points with 53% less allocated GPU memory, at higher median latency (15.36 vs 9.44 ms). These comparisons apply to the evaluated task mixture and loading formats below.

Benchmark / metric	ARBITER	Laya	Laya Multilingual	Laya Typed	GLiNER Decide	GLiNER Multi	EmbeddingGemma
Median latency · ms ↓	15.36 ms	12.07 ms	9.44 ms	11.91 ms	15.32 ms	7.97 ms	21.36 ms
Peak allocated VRAM · MiB ↓	759.0 MiB	2,415.8 MiB	1,600.4 MiB	2,438.6 MiB	3,237.4 MiB	3,823.4 MiB	862.6 MiB

Historical protocol, NVIDIA GeForce RTX 5090, single-request inference. ARBITER and EmbeddingGemma use BF16; Laya variants use FP32; GLiNER uses its default loading configuration. These are measured pipeline configurations, not equal-dtype kernel comparisons. Latency includes each adapter's execution path. These measurements predate the native batch API and do not measure batched throughput.

Protocol Notes

• MASSIVE Intent: 20 candidates (correct intent plus 19 distractors), 4,200 test decisions across 14 languages; this is not full 60-intent classification. Scenario uses 18 candidates and 4,200 decisions across the same languages.
• Banking77: 3,080 test examples with all 77 intents. TREC: 500 test examples, six coarse classes. SST-2: 872 validation examples.
• XNLI: 4,500 decisions across 15 languages. Typed Decisions: 2,000 decisions. Fast Single: 2,600 decisions. Fast Multi: 300 multi-question cases; exact match requires every decision in a case to be correct, whereas micro F1 measures the pooled binary decisions. Public micro accuracy counts those binary decisions individually, not the 300 exact-match outcomes.
• Adapters: comparisons use the repository's model-specific adapters. GLiNER's adapter classifies the state against labels without the separate instruction; results characterize these evaluated pipelines. Failed cases remain in denominators.
• Statistics: bold denotes the highest observed score, not statistical significance. Exploratory paired McNemar tests: TREC vs Laya Multilingual p=0.0019; Banking77 vs EmbeddingGemma p=0.0068. These assume independent examples and are unadjusted for multiple comparisons.

Additional reasoning and long-context evaluations

These auxiliary evaluations have a narrower comparator set than the main tables. Tasksource Long is an existing project validation set, not a certified unseen-domain test.

Evaluation Benchmark	Domain	Items	ARBITER-272M	Baseline Comparison	Note
Tasksource Long	Long context (up to 4,096 tokens)	2,373	58.45%	Laya-Typed 32.91%	Observed advantage on long-context criteria matching (+25.5 pts).
ARC-Easy	Elementary science reasoning	2,376	48.48%	Laya-Typed 48.48%	Tied with Laya-Typed (48.48%), outperforms Laya-Base (47.26%).
ARC-Challenge	Hard science reasoning	1,172	33.79%	Laya-Typed 31.57%	McNemar test p = 0.2180 (observed gain; not statistically significant).
HellaSwag	Common-sense inference	1,000	41.80%	Laya-Base 35.90%	Outperforms Laya-Base (+5.9 pts) and GLiNER (29.80%, +12.0 pts).

---

Architecture

[Input Text: Context + Instruction + <OPTION> label: desc ...]
                          │
            EmbeddingGemma-2 Text Encoder (512-dim)
                          │
             Option Tokens H ∈ ℝ^(K × 512)
                          │
    ┌─────────────────────┴─────────────────────┐
    │     Inter-Candidate Set Attention         │
    │  H_attn = H + α_attn · MHA(LN(H), LN(H)) │
    └─────────────────────┬─────────────────────┘
                          │
    ┌─────────────────────┴─────────────────────┐
    │       Context & Task HyperFiLM            │
    │  [γ_raw, β_raw] = HyperNet([h_ctx || e_t])│
    │  γ = 1 + α_film · γ_raw, β = α_film · β_raw│
    │  H' = γ ⊙ H_attn + β                      │
    └─────────────────────┬─────────────────────┘
                          │
                  Final Scoring MLP
                          │
             Logits z ∈ ℝ^K  ──► Softmax
                          │
         ┌────────────────┼────────────────┐
         │ (choice)       │ (noul)         │ (score)
         ▼                ▼                ▼
     Argmax         p("true")        ∑ i · p_i

---

Licensing & Provenance

• Source Code (arbiter/): Distributed under the Apache-2.0 License.
• Model Weights (model.safetensors): Derived from google/embeddinggemma-2 through custom architecture extension and fine-tuning. The weights are subject to the Gemma Terms of Use and Google's Additional Commercial Terms of Use.
• Environment Provenance: Verified in environment with Python 3.12, PyTorch 2.6+, and Transformers 5.19.0.

---

Authors & Contributors

• Ichrak Mokhtari (Equal contribution)
• Seif Attoui (Equal contribution)

---

Citation

@misc{arbiter2026,
  title={ARBITER: Attentive Relational Bidirectional Impartial Typed Encoder for Reasoning},
  author={Mokhtari, Ichrak and Attoui, Seif},
  year={2026},
  note={Equal contribution},
  howpublished={\url{https://huggingface.co/spectral-ai-lab/arbiter-272m}},
}