Text Generation
Transformers
Safetensors
qwen3_5_text
decision-model
system-one
typed-decisions
uncensored
abliterated
calibrated-probabilities
noul
choice
score
qwen3.8
jev
solar-jev
conversational
Instructions to use SargeDev/Solar-JEV-27B-R32 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SargeDev/Solar-JEV-27B-R32 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="SargeDev/Solar-JEV-27B-R32") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("SargeDev/Solar-JEV-27B-R32") model = AutoModelForCausalLM.from_pretrained("SargeDev/Solar-JEV-27B-R32", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use SargeDev/Solar-JEV-27B-R32 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SargeDev/Solar-JEV-27B-R32" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SargeDev/Solar-JEV-27B-R32", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/SargeDev/Solar-JEV-27B-R32
- SGLang
How to use SargeDev/Solar-JEV-27B-R32 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "SargeDev/Solar-JEV-27B-R32" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SargeDev/Solar-JEV-27B-R32", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "SargeDev/Solar-JEV-27B-R32" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SargeDev/Solar-JEV-27B-R32", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use SargeDev/Solar-JEV-27B-R32 with Docker Model Runner:
docker model run hf.co/SargeDev/Solar-JEV-27B-R32
|
Download README.md from SargeDev/Solar-JEV-27B-R32: direct link, hf CLI and curl.
- Browser
- Download file 12.6 kB
-
https://huggingface.co/SargeDev/Solar-JEV-27B-R32/resolve/main/README.md
- Command line
-
hf download hf://SargeDev/Solar-JEV-27B-R32/README.md
-
curl -L -o README.md https://huggingface.co/SargeDev/Solar-JEV-27B-R32/resolve/main/README.md
12.6 kB
| base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - decision-model | |
| - system-one | |
| - typed-decisions | |
| - uncensored | |
| - abliterated | |
| - calibrated-probabilities | |
| - noul | |
| - choice | |
| - score | |
| - qwen3.8 | |
| - jev | |
| - solar-jev | |
| datasets: | |
| - SargeDev/jev-distill-corpus-v3 | |
| license: apache-2.0 | |
| # Solar-JEV-27B-R32 | |
| **The best-measured build of this release.** Uncensored 27B tuned for everyday | |
| decision-making — crisp calls with honestly-calibrated confidence instead of | |
| hedge-fests — with the decision readout trained INTO the model. | |
| > **What this repo is.** huihui Qwen3.8-27B-abliterated + the AutoTrust `r32` | |
| > `lm_head` LoRA, merged. **No solar delta.** These are the same weights as | |
| > the already-public `SargeDev/JEV-27B-Uncensored` (verified: bit-identical | |
| > `lm_head`, md5 `bb95f4fd…`, and the backbone differs from the | |
| > r32+solar merge only on the 400 solar-targeted tensors). This repo exists so | |
| > the family has one consistent naming scheme and so the **GGUF ladder for the | |
| > best model** is published — the r32+solar merge in | |
| > `SargeDev/Solar-Decision-JEV-27B` is a *different, weaker* model. | |
| - Full bf16 weights (51 GB), serves directly in vLLM / transformers | |
| (`Qwen3_5ForCausalLM`). | |
| - Refusal rate and calibration measured below — the uncensoring claim is | |
| probed, not assumed. | |
| ## What makes this one different | |
| The decision readout is an **`lm_head` LoRA**: the 24-slot decision head is | |
| re-expressed as a LoRA on the output projection (only the 24 verbalizer rows | |
| change), so a typed decision is a **single prefill with `max_tokens=1`**, | |
| constrained to the option tokens, read back as log-probabilities. No separate | |
| head to run, no adapter to swap. | |
| That difference is worth a lot. On the same 27,695 rows with the same harness | |
| and the same decision head + calibration: | |
| | model | decision readout | choice top-1 | score top-1 | score KL | noul AUROC | | |
| |---|---|---|---|---|---| | |
| | AutoTrust (published reference) | trained into the model | 0.9041 | 0.8910 | 0.0210 | 0.9961 | | |
| | **this model (r32 + lm_head)** | trained into the model | **0.8693** | **0.8135** | **0.0619** | **0.9919** | | |
| | solar r16 (no lm_head) | separate projection at read time | 0.5054 | 0.5369 | 0.4699 | 0.9449 | | |
| The `solar r16` merge reads a decision through a *separate* projection over | |
| the final hidden state, and lands at 0.5054 choice top-1. The same backbone | |
| with the readout in the training loop lands at 0.8693. That is the main | |
| finding of this release: **the readout has to be trained into the model.** | |
| Full ablation, including the r32+solar combined merge (which measured *worse* | |
| than r32 alone — 0.8162), is in the `SargeDev/Solar-Decision-JEV-27B` card. | |
| ## Quick start (vLLM, one GB10/Spark-class box) | |
| ```bash | |
| vllm serve SargeDev/Solar-Decision-JEV-27B \ | |
| --served-model-name jev-judgment-r32 --trust-remote-code \ | |
| --max-model-len 8192 --gpu-memory-utilization 0.85 \ | |
| --default-chat-template-kwargs '{"enable_thinking": false}' | |
| ``` | |
| ## Decision gate (AutoTrust-style, test_set_30k with yuri_v1 D1 placeholders excluded) | |
| Read this table carefully, because it contains two very different comparisons. | |
| - **The sibling merge** is our other build, scored on the **same 27,695 rows** | |
| with the **same** harness and the **same** head + calibration. It is a | |
| sibling model, not a quality bar — but it is the honest apples-to-apples | |
| reference for what the `lm_head` LoRA is worth. | |
| - **AutoTrust reference** is the number published by the authors of the original | |
| AutoTrust work for *their* model on this task set. This is the bar. | |
| On the typed-decision kinds (multiple-choice `choice`, ordinal `score`) this | |
| model is **still below the published AutoTrust reference**, and the gap is | |
| large - not a rounding difference. On the binary `noul` kind the gap is | |
| narrower. Cells marked "(below bar)" differ from the published reference by | |
| more than 5 points. | |
| What this means: the fine-tune produces a *usable* decision instrument and a | |
| clearly better everyday-chat model than its base (see the benchmark matrix | |
| below), but it is **not** a reproduction of AutoTrust-level typed-decision | |
| accuracy. Treat the `choice` and `score` columns as an open result, not a | |
| solved one. We are publishing the numbers as measured, including the gap. | |
| **Harness validation.** These numbers were produced by our own HTTP gate | |
| client, so we checked it against a known path: re-gating the r32 merge | |
| reproduced the reference's own numbers to within 0.001 on every metric | |
| (choice top-1 0.8704 -> 0.8693, score KL 0.0623 -> 0.0619, noul AUROC | |
| 0.9920 -> 0.9919). The harness is sound, so the gaps above are real. | |
| | metric | n | **Jev Judgment r32+lm_head (ours)** | Jev Judgment solar r16 | AutoTrust reference (published) | | |
| |---|---|---|---|---| | |
| | noul.AUROC | 9969 | 0.9919 (below bar) | 0.9449 | 0.9961 | | |
| | noul.top-1 | 9969 | 0.9486 | 0.8657 | n/a | | |
| | noul.ECE | 9969 | 0.0122 | 0.0143 | n/a | | |
| | noul.KL | 9969 | 0.0081 | 0.0448 | n/a | | |
| | choice.top-1 | 9199 | 0.8693 (below bar) | 0.5054 | 0.9041 | | |
| | choice.ECE | 9199 | 0.0051 | 0.0338 | n/a | | |
| | choice.KL | 9199 | 0.0530 | 0.4994 | n/a | | |
| | score.top-1 | 8527 | 0.8135 (below bar) | 0.5369 | 0.8910 | | |
| | score.ECE | 8527 | 0.0367 | 0.0368 | n/a | | |
| | score.KL | 8527 | 0.0619 (below bar) | 0.4699 | 0.0210 | | |
| | overall.top-1 | 27695 | 0.8807 | 0.6448 | n/a | | |
| | overall.ECE | 27695 | 0.0090 (below bar) | 0.0211 | 0.0011 | | |
| | overall.KL | 27695 | 0.0396 (below bar) | 0.3267 | 0.0185 | | |
| ### Ablation: where the decision readout has to live | |
| All four rows below are scored on the SAME 27,695 rows, with the SAME | |
| harness and the SAME decision head + calibration. The only difference is | |
| where the decision distribution is produced. | |
| | model | decision readout | choice top-1 | score top-1 | score KL | noul AUROC | overall ECE | | |
| |---|---|---|---|---|---|---| | |
| | AutoTrust (published reference) | trained into the model (lm_head LoRA) | 0.9041 | 0.8910 | 0.0210 | 0.9961 | 0.0011 | | |
| | r32 + lm_head LoRA (r=32, alpha=64) | trained into the model (lm_head LoRA) | 0.8693 | 0.8135 | 0.0619 | 0.9919 | 0.0090 | | |
| | solar r16 (no lm_head LoRA) | separate projection at read time | 0.5054 | 0.5369 | 0.4699 | 0.9449 | 0.0211 | | |
| | r32 + lm_head LoRA + solar (combined) | trained into the model (lm_head LoRA) | 0.8162 | 0.7192 | 0.1332 | 0.9810 | 0.0625 | | |
| Two findings, in order of how much they matter: | |
| 1. **The readout has to be trained INTO the model.** A separate projection | |
| over the final hidden state (the `solar r16` row) reaches only 0.5054 | |
| choice top-1; the same backbone with an `lm_head` LoRA in the training | |
| loop reaches 0.8693 on the identical rows. A separate projection can | |
| work, but it needs the backbone to cooperate, and a backbone trained | |
| without the readout in the loop does not. | |
| 2. **Adding a second, readout-free adapter on top makes the typed-decision | |
| accuracy WORSE, not better.** The `combined` row is the r32 merge plus | |
| the solar r16 delta applied on top (bit-exact arithmetic, verified). | |
| It lands between the two parents on every metric - 0.8162 choice top-1, | |
| between r32's 0.8693 and solar's 0.5054 - rather than above r32. The | |
| two LoRA deltas are near-orthogonal (mean cosine 0.003), so they do not | |
| cancel, but solar's delta still degrades the readout r32 trained. | |
| We publish the combined row because it is a real, reproducible result, but | |
| **the r32+lm_head merge is the better model** and is the one this release | |
| recommends. We are not going to present a merge as an improvement when our | |
| own numbers say it is a regression. | |
| ## Benchmark matrix (lm-eval-harness 0.4.13, greedy, same prompts for every row) | |
| | model | GSM8K (8-shot CoT) | IFEval (0-shot, prompt-strict) | MMLU subset (5-shot, 8 subjects) | refusal rate | | |
| |---|---|---|---|---| | |
| | huihui Qwen3.8-27B (uncensored base) | 97.7 | 76.0 | 81.2 | 0.0% (5/5 probes) | | |
| | **Jev Judgment bf16** (solar r16) | 96.0 | 81.5 | 81.2 | 0.0% (5/5 probes) | | |
| | Jev Judgment bf16 (r32+lm_head + solar) | 91.0 | 72.5 | 80.8 | 0.0% (5/5 probes) | | |
| | Jev Judgment bf16 (r32 + lm_head) | 94.0 | 81.0 | 82.5 | 0.0% (5/5 probes) | | |
| | Jev Judgment NVFP4+FP8 | 95.3 | 80.5 | 80.5 | 0.0% (5/5 probes) | | |
| | GGUF Q4_K_M | 95.3 | 79.5 | 82.0 | 0.0% (5/5 probes) | | |
| | GGUF Q8_0 | 95.7 | 81.0 | 81.0 | 0.0% (5/5 probes) | | |
| Questions scored per task: | |
| | task | questions scored per model | | |
| |---|---| | |
| | GSM8K (8-shot CoT) | 300 | | |
| | IFEval (0-shot, prompt-strict) | 200 | | |
| | MMLU subset (5-shot, 8 subjects) | 400 | | |
| Note: GSM8K scored with flexible-extract (the model answers in its own format). | |
| MMLU subset subjects: high_school_mathematics, college_biology, professional_psychology, | |
| jurisprudence, machine_learning, astronomy, global_facts, logical_fallacies (5-shot, _generative variants). | |
| MMLU --limit applies PER SUBJECT, so the subset is 8 x per-subject-limit questions. | |
| Reproducibility: decoding is greedy (do_sample=False, temperature 0, seed 1234), but | |
| the server batches concurrent requests, and batch composition changes floating-point | |
| reduction order. Repeat runs of the same config can therefore differ by a point or two | |
| on the small-n tasks. The error bars in the table are the harness's own stderr over the | |
| scored rows. | |
| ## Lineage & credits | |
| This release is a descendant of the earlier `SargeDev/JEV-27B-Uncensored` and | |
| `SargeDev/JEV-27B-VL-Uncensored` repos. The credit chain is unchanged and is | |
| reproduced here in full: | |
| - **AutoTrust AI** — Blocks-of-Experts recipe, the System-1 decision LoRA | |
| (r16/r32) + 24-slot decision head, and the *serve_decide.py* decision | |
| harness, from | |
| [autotrust/JEV-27B](https://huggingface.co/autotrust/JEV-27B) and | |
| [autotrust/JEV-27B-VL](https://huggingface.co/autotrust/JEV-27B-VL) | |
| (Apache-2.0). The `r32` adapter used here is theirs, unmodified: the | |
| backbone LoRA with the decision head re-expressed as an `lm_head` LoRA, so | |
| only the 24 verbalizer rows of the output projection change. | |
| - **huihui-ai** — the *abliterated* (uncensored) Qwen3.8-27B backbone used here | |
| ([Huihui-Qwen3.8-27B-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated), | |
| Apache-2.0; layers 18-51 ablated, vision tower untouched). | |
| - **SargeDev** — [jev-distill-corpus-v3](https://huggingface.co/datasets/SargeDev/jev-distill-corpus-v3), | |
| the 740,957-row calibrated typed-decision corpus AutoTrust's JEV models were | |
| trained on (Apache-2.0), and the **solar r16** adapter trained here on | |
| [jev-distill-corpus-v4](https://huggingface.co/datasets/SargeDev/jev-distill-corpus-v4) | |
| (the 1,219-step JEV System-1 run, 17-domain tokenized mix). | |
| - **Qwen** — [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) base | |
| (Apache-2.0). | |
| - **TypeSafe AI** — Jev 1.13, the original closed teacher behind the System-1 | |
| typed-decisions framing (referenced; not redistributed). | |
| **Base**: huihui-ai/Qwen3.8-27B-abliterated (bf16; the text-only builds strip | |
| the vision tower and serve as a plain `qwen3_5_text` causal LM, the VL build | |
| keeps the tower and serves as `qwen3_5` / `Qwen3_5ForConditionalGeneration`). | |
| - **Adapters** — this release ships two, and they are NOT interchangeable: | |
| - `solar r16` (alpha 32, 400 modules, **no `lm_head`**): trained 1,219 steps | |
| on the JEV System-1 corpus (jev-distill-corpus-v4 tokenized mix: T1 gold | |
| domains + memory_relevance + 17-domain fill), KL+RPS decision-head | |
| distillation objective (AutoTrust-style), cosine LR 2e-4, KL_W 1.0, | |
| RPS_W 0.3. Merged with peft `merge_and_unload`; merge fidelity verified by | |
| re-gating the bare merged model (it reproduced the adapter-mounted | |
| numbers). | |
| - `r32` (alpha 64, 401 modules, **includes `lm_head`**): the backbone LoRA | |
| plus the 24-slot decision head re-expressed as an `lm_head` LoRA | |
| (only the 24 verbalizer rows change), so a typed decision is a single | |
| prefill step with `max_tokens=1` constrained to the option tokens. | |
| - **Decision head + calibration.json** (shipped in this repo root): the | |
| instrument-panel piece agents call for numeric confidence — the 24-slot | |
| verbalizer id list, the per-slot bias, and the per-kind temperature. This | |
| is the SAME head and the SAME calibration file (verified byte-identical, | |
| md5 `11359173…` and `7f0f3914…`) that produced every gate number in this | |
| card, so a consumer reproducing our numbers reproduces them exactly. | |
| For everyday natural-language judgment, chat with the model directly and | |
| set `enable_thinking:false` in `chat_template_kwargs`. | |
| - **Everything measured on one NVIDIA GB10 (DGX Spark-class)**; quantized with | |
| llmcompressor 0.14.0 (NVFP4) and llama.cpp (GGUF ladder). | |