Solar-JEV-27B-R32

The best-measured build of this release. Uncensored 27B tuned for everyday decision-making — crisp calls with honestly-calibrated confidence instead of hedge-fests — with the decision readout trained INTO the model.

What this repo is. huihui Qwen3.8-27B-abliterated + the AutoTrust r32 lm_head LoRA, merged. No solar delta. These are the same weights as the already-public SargeDev/JEV-27B-Uncensored (verified: bit-identical lm_head, md5 bb95f4fd…, and the backbone differs from the r32+solar merge only on the 400 solar-targeted tensors). This repo exists so the family has one consistent naming scheme and so the GGUF ladder for the best model is published — the r32+solar merge in SargeDev/Solar-Decision-JEV-27B is a different, weaker model.

• Full bf16 weights (51 GB), serves directly in vLLM / transformers (Qwen3_5ForCausalLM).
• Refusal rate and calibration measured below — the uncensoring claim is probed, not assumed.

What makes this one different

The decision readout is an lm_head LoRA: the 24-slot decision head is re-expressed as a LoRA on the output projection (only the 24 verbalizer rows change), so a typed decision is a single prefill with max_tokens=1, constrained to the option tokens, read back as log-probabilities. No separate head to run, no adapter to swap.

That difference is worth a lot. On the same 27,695 rows with the same harness and the same decision head + calibration:

model	decision readout	choice top-1	score top-1	score KL	noul AUROC
AutoTrust (published reference)	trained into the model	0.9041	0.8910	0.0210	0.9961
this model (r32 + lm_head)	trained into the model	0.8693	0.8135	0.0619	0.9919
solar r16 (no lm_head)	separate projection at read time	0.5054	0.5369	0.4699	0.9449

The solar r16 merge reads a decision through a separate projection over the final hidden state, and lands at 0.5054 choice top-1. The same backbone with the readout in the training loop lands at 0.8693. That is the main finding of this release: the readout has to be trained into the model.

Full ablation, including the r32+solar combined merge (which measured worse than r32 alone — 0.8162), is in the SargeDev/Solar-Decision-JEV-27B card.

Quick start (vLLM, one GB10/Spark-class box)

vllm serve SargeDev/Solar-Decision-JEV-27B \
  --served-model-name jev-judgment-r32 --trust-remote-code \
  --max-model-len 8192 --gpu-memory-utilization 0.85 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Decision gate (AutoTrust-style, test_set_30k with yuri_v1 D1 placeholders excluded)

Read this table carefully, because it contains two very different comparisons.

• The sibling merge is our other build, scored on the same 27,695 rows with the same harness and the same head + calibration. It is a sibling model, not a quality bar — but it is the honest apples-to-apples reference for what the lm_head LoRA is worth.
• AutoTrust reference is the number published by the authors of the original AutoTrust work for their model on this task set. This is the bar.

On the typed-decision kinds (multiple-choice choice, ordinal score) this model is still below the published AutoTrust reference, and the gap is large - not a rounding difference. On the binary noul kind the gap is narrower. Cells marked "(below bar)" differ from the published reference by more than 5 points.

What this means: the fine-tune produces a usable decision instrument and a clearly better everyday-chat model than its base (see the benchmark matrix below), but it is not a reproduction of AutoTrust-level typed-decision accuracy. Treat the choice and score columns as an open result, not a solved one. We are publishing the numbers as measured, including the gap.

Harness validation. These numbers were produced by our own HTTP gate client, so we checked it against a known path: re-gating the r32 merge reproduced the reference's own numbers to within 0.001 on every metric (choice top-1 0.8704 -> 0.8693, score KL 0.0623 -> 0.0619, noul AUROC 0.9920 -> 0.9919). The harness is sound, so the gaps above are real.

metric	n	Jev Judgment r32+lm_head (ours)	Jev Judgment solar r16	AutoTrust reference (published)
noul.AUROC	9969	0.9919 (below bar)	0.9449	0.9961
noul.top-1	9969	0.9486	0.8657	n/a
noul.ECE	9969	0.0122	0.0143	n/a
noul.KL	9969	0.0081	0.0448	n/a
choice.top-1	9199	0.8693 (below bar)	0.5054	0.9041
choice.ECE	9199	0.0051	0.0338	n/a
choice.KL	9199	0.0530	0.4994	n/a
score.top-1	8527	0.8135 (below bar)	0.5369	0.8910
score.ECE	8527	0.0367	0.0368	n/a
score.KL	8527	0.0619 (below bar)	0.4699	0.0210
overall.top-1	27695	0.8807	0.6448	n/a
overall.ECE	27695	0.0090 (below bar)	0.0211	0.0011
overall.KL	27695	0.0396 (below bar)	0.3267	0.0185

Ablation: where the decision readout has to live

All four rows below are scored on the SAME 27,695 rows, with the SAME harness and the SAME decision head + calibration. The only difference is where the decision distribution is produced.

model	decision readout	choice top-1	score top-1	score KL	noul AUROC	overall ECE
AutoTrust (published reference)	trained into the model (lm_head LoRA)	0.9041	0.8910	0.0210	0.9961	0.0011
r32 + lm_head LoRA (r=32, alpha=64)	trained into the model (lm_head LoRA)	0.8693	0.8135	0.0619	0.9919	0.0090
solar r16 (no lm_head LoRA)	separate projection at read time	0.5054	0.5369	0.4699	0.9449	0.0211
r32 + lm_head LoRA + solar (combined)	trained into the model (lm_head LoRA)	0.8162	0.7192	0.1332	0.9810	0.0625

Two findings, in order of how much they matter:

1. The readout has to be trained INTO the model. A separate projection over the final hidden state (the solar r16 row) reaches only 0.5054 choice top-1; the same backbone with an lm_head LoRA in the training loop reaches 0.8693 on the identical rows. A separate projection can work, but it needs the backbone to cooperate, and a backbone trained without the readout in the loop does not.

2. Adding a second, readout-free adapter on top makes the typed-decision accuracy WORSE, not better. The combined row is the r32 merge plus the solar r16 delta applied on top (bit-exact arithmetic, verified). It lands between the two parents on every metric - 0.8162 choice top-1, between r32's 0.8693 and solar's 0.5054 - rather than above r32. The two LoRA deltas are near-orthogonal (mean cosine 0.003), so they do not cancel, but solar's delta still degrades the readout r32 trained.

We publish the combined row because it is a real, reproducible result, but the r32+lm_head merge is the better model and is the one this release recommends. We are not going to present a merge as an improvement when our own numbers say it is a regression.

Benchmark matrix (lm-eval-harness 0.4.13, greedy, same prompts for every row)

model	GSM8K (8-shot CoT)	IFEval (0-shot, prompt-strict)	MMLU subset (5-shot, 8 subjects)	refusal rate
huihui Qwen3.8-27B (uncensored base)	97.7	76.0	81.2	0.0% (5/5 probes)
Jev Judgment bf16 (solar r16)	96.0	81.5	81.2	0.0% (5/5 probes)
Jev Judgment bf16 (r32+lm_head + solar)	91.0	72.5	80.8	0.0% (5/5 probes)
Jev Judgment bf16 (r32 + lm_head)	94.0	81.0	82.5	0.0% (5/5 probes)
Jev Judgment NVFP4+FP8	95.3	80.5	80.5	0.0% (5/5 probes)
GGUF Q4_K_M	95.3	79.5	82.0	0.0% (5/5 probes)
GGUF Q8_0	95.7	81.0	81.0	0.0% (5/5 probes)

Questions scored per task:

task	questions scored per model
GSM8K (8-shot CoT)	300
IFEval (0-shot, prompt-strict)	200
MMLU subset (5-shot, 8 subjects)	400

Note: GSM8K scored with flexible-extract (the model answers in its own format). MMLU subset subjects: high_school_mathematics, college_biology, professional_psychology, jurisprudence, machine_learning, astronomy, global_facts, logical_fallacies (5-shot, _generative variants). MMLU --limit applies PER SUBJECT, so the subset is 8 x per-subject-limit questions.

Reproducibility: decoding is greedy (do_sample=False, temperature 0, seed 1234), but the server batches concurrent requests, and batch composition changes floating-point reduction order. Repeat runs of the same config can therefore differ by a point or two on the small-n tasks. The error bars in the table are the harness's own stderr over the scored rows.

Lineage & credits

This release is a descendant of the earlier SargeDev/JEV-27B-Uncensored and SargeDev/JEV-27B-VL-Uncensored repos. The credit chain is unchanged and is reproduced here in full:

• AutoTrust AI — Blocks-of-Experts recipe, the System-1 decision LoRA (r16/r32) + 24-slot decision head, and the serve_decide.py decision harness, from autotrust/JEV-27B and autotrust/JEV-27B-VL (Apache-2.0). The r32 adapter used here is theirs, unmodified: the backbone LoRA with the decision head re-expressed as an lm_head LoRA, so only the 24 verbalizer rows of the output projection change.
• huihui-ai — the abliterated (uncensored) Qwen3.8-27B backbone used here (Huihui-Qwen3.8-27B-abliterated, Apache-2.0; layers 18-51 ablated, vision tower untouched).
• SargeDev — jev-distill-corpus-v3, the 740,957-row calibrated typed-decision corpus AutoTrust's JEV models were trained on (Apache-2.0), and the solar r16 adapter trained here on jev-distill-corpus-v4 (the 1,219-step JEV System-1 run, 17-domain tokenized mix).
• Qwen — Qwen3.8-27B base (Apache-2.0).
• TypeSafe AI — Jev 1.13, the original closed teacher behind the System-1 typed-decisions framing (referenced; not redistributed).

Base: huihui-ai/Qwen3.8-27B-abliterated (bf16; the text-only builds strip the vision tower and serve as a plain qwen3_5_text causal LM, the VL build keeps the tower and serves as qwen3_5 / Qwen3_5ForConditionalGeneration).

• Adapters — this release ships two, and they are NOT interchangeable:
  • solar r16 (alpha 32, 400 modules, no lm_head): trained 1,219 steps on the JEV System-1 corpus (jev-distill-corpus-v4 tokenized mix: T1 gold domains + memory_relevance + 17-domain fill), KL+RPS decision-head distillation objective (AutoTrust-style), cosine LR 2e-4, KL_W 1.0, RPS_W 0.3. Merged with peft merge_and_unload; merge fidelity verified by re-gating the bare merged model (it reproduced the adapter-mounted numbers).
  • r32 (alpha 64, 401 modules, includes lm_head): the backbone LoRA plus the 24-slot decision head re-expressed as an lm_head LoRA (only the 24 verbalizer rows change), so a typed decision is a single prefill step with max_tokens=1 constrained to the option tokens.
• Decision head + calibration.json (shipped in this repo root): the instrument-panel piece agents call for numeric confidence — the 24-slot verbalizer id list, the per-slot bias, and the per-kind temperature. This is the SAME head and the SAME calibration file (verified byte-identical, md5 11359173… and 7f0f3914…) that produced every gate number in this card, so a consumer reproducing our numbers reproduces them exactly. For everyday natural-language judgment, chat with the model directly and set enable_thinking:false in chat_template_kwargs.
• Everything measured on one NVIDIA GB10 (DGX Spark-class); quantized with llmcompressor 0.14.0 (NVFP4) and llama.cpp (GGUF ladder).