Arbiter-4B

Arbiter-4B is a structured decision model fine-tuned from TokenRhythm/NeoHorse-Jev-4B using LoRA on our collected decision datasets, followed by global temperature calibration. It returns typed probability distributions (noul, choice, score) using the NeoHorse pointer head. It is not a chat-generation model.

Training and calibration

The project data collection includes typed-decisions train, some classification tasks, HelpSteer2, and constructed decision examples. The project recipes hold out validation data and reserve the benchmark test split for final evaluation. Exact run settings, where recorded by the exporter, are retained in model_manifest.json; reference scripts under provenance/recipe_reference/ describe the project recipes rather than prove the exact mixture or hyperparameters of this checkpoint. No additional head-only training phase is claimed for this release.

The calibration implementation fits a single positive temperature on held-out validation soft labels by minimizing cross entropy. Probabilities are computed as softmax(logits / temperature). The calibration stage does not modify pointer-head tensors; a positive temperature preserves the argmax class. The temperature and available calibration records are shipped with the model.

This is a general model fine-tuned on benchmark train, not a zero-shot submission. Its scores belong in the fitted/fine-tuned category. The precise upstream base commit was not recorded and is not inferred here. Training GPU and exact hyperparameters are not claimed when absent from the exported metadata.

Format

The training exporter has already merged LoRA into the backbone. No second LoRA merge, tensor conversion, or quantization is required. The release preserves:

• backbone/: merged backbone and configuration.
• tokenizer/: tokenizer files.
• pointer_head.safetensors: independent decision pointer head.
• model_manifest.json: architecture and calibrated temperature.
• calibration.json: calibration evidence.
• vllm-fast/: the vLLM adapter and its own decision runtime, including temperature support.

Use the bundled runtime to load the complete directory. Loading only the backbone with a generic chat pipeline omits the decision head and temperature calibration.

Installation and inference

Use Python 3.11 or newer with a compatible NVIDIA GPU environment. The bundled vllm-fast/requirements.txt pins the tested runtime versions.

From the downloaded repository root:

python -m pip install -r vllm-fast/requirements.txt
CUDA_VISIBLE_DEVICES=0 python vllm-fast/server.py --bundle . --port 8080

The endpoint is POST http://127.0.0.1:8080/v1/systemone; readiness is GET /healthz.

Runtime acceleration defaults

The bundled vllm-fast server uses vLLM's compiled path by default:

• torch.compile and piecewise CUDA graphs are enabled (no --enforce-eager).
• Model weights run in bfloat16; the pointer head runs on CPU in float32.
• max_model_len=12288, max_num_batched_tokens=32768, max_num_seqs=128.
• Position pooling is enabled by default: only the <decide> and option token positions are materialized, using the bundled jev_position_pool worker patch. Use --no-pool-positions to disable it.
• Concurrent requests are micro-batched; the default 20 ms coalescing wait is used only when at least two requests are queued. A single request never waits. Set --batch-wait-ms 0 for a single-concurrency deployment.
• gpu_memory_utilization defaults to 0.85.

Cache options

• --prefix-cache: warms and reuses the shared state prefix across question branches within a request through vLLM automatic prefix caching, and enables chunked prefill. It automatically enables position pooling. Enable it when the request state is long and all questions share that prefix; leave it off for short, one-off inputs. Default: off.
• --head-cache: enables cross-request APC plus the projected PointerHead K/Q cache. It implies --prefix-cache and is intended for one request containing multiple questions, where the same state prefix is reused across question rows, or for repeated requests with a shared prefix. It adds process-local cache memory; --head-cache-max-entries (default 100000) bounds it. Default: off.

Recommended:

• single request, single question: leave both cache options off (default).
• single request, multiple questions: add --head-cache.
• long state: add --prefix-cache.
• long state with multiple questions or repeated shared prefixes: --head-cache is sufficient because it implies --prefix-cache.

Examples:

# default: position pooling on, caches off
CUDA_VISIBLE_DEVICES=0 python vllm-fast/server.py --bundle . --port 8080

# one request with multiple questions
CUDA_VISIBLE_DEVICES=0 python vllm-fast/server.py --bundle . --port 8080 --head-cache

# long shared state, single question
CUDA_VISIBLE_DEVICES=0 python vllm-fast/server.py --bundle . --port 8080 --prefix-cache

# single-concurrency latency mode: no coalescing wait
CUDA_VISIBLE_DEVICES=0 python vllm-fast/server.py --bundle . --port 8080 --batch-wait-ms 0

Reproduce the evaluation

Stop any existing server on port 8080, then run from the repository root:

CUDA_VISIBLE_DEVICES=0 python run_evaluation.py --model-dir . --output-dir ./results

The script starts the bundled vllm-fast service, evaluates the included pinned test split (400 cases / 2,000 decisions, all five questions per case), records hardware and software metadata, and stops its server. Use a fresh output directory for each model or run.

Published results are in evaluation/report.json, raw probability predictions in evaluation/predictions.jsonl, and measured hardware/configuration in evaluation/run_metadata.json. Scores are self-reported, not independently verified. KL is mean KL(gold || prediction) with natural logarithms; Brier sums squared errors against soft gold over options; ECE uses 15 equal-width bins and top-label confidence. Latency is measured at the serial HTTP client per case and includes startup warmup effects on early requests.

Limitations and license

The benchmark measures agreement with teacher-generated soft labels, not objective correctness. Fine-tuned results are not directly comparable with zero-shot results. Calibration reflects its validation distribution and may not transfer to other domains. This submission evaluates text decisions; it makes no new vision-performance claim.

Apache-2.0, following the base model and included runtime. See LICENSE, NOTICE.md and vllm-fast/jev_runtime/LICENSE and NOTICE.md. Training datasets retain their own terms; their inclusion in the project's collection is not a relicensing of their contents. Only the benchmark's Apache-2.0 test data is bundled for evaluation.

The native neohorse-decision Python package is not required. The service accepts requests without a model field; legacy NeoHorse routing aliases do not change the Arbiter-4B release name. Use the same vLLM environment as your working vllm-fast deployment; measured dependency versions are recorded with the evaluation.

Exported calibration

Temperature: 1.1942108273505827. The supplied manifest and calibration evidence are preserved unchanged.