Instructions to use GestaltLabs/Ornstein-9Bv2.5-NSC-ACE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GestaltLabs/Ornstein-9Bv2.5-NSC-ACE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GestaltLabs/Ornstein-9Bv2.5-NSC-ACE")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("GestaltLabs/Ornstein-9Bv2.5-NSC-ACE")
model = AutoModelForMultimodalLM.from_pretrained("GestaltLabs/Ornstein-9Bv2.5-NSC-ACE", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use GestaltLabs/Ornstein-9Bv2.5-NSC-ACE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/GestaltLabs/Ornstein-9Bv2.5-NSC-ACE

SGLang

How to use GestaltLabs/Ornstein-9Bv2.5-NSC-ACE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/Ornstein-9Bv2.5-NSC-ACE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use GestaltLabs/Ornstein-9Bv2.5-NSC-ACE with Docker Model Runner:
```
docker model run hf.co/GestaltLabs/Ornstein-9Bv2.5-NSC-ACE
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Ornstein 9B v2.5 — NSC-ACE

An NSC-ACE post-train (Neural Steering Committee for Agentic Co-Evolution) of Ornstein 3.5 9B V2.5, targeting agentic tool calling. During GRPO rollout generation, contrastive hidden-state steering directions (agentic vs. passive prompts, recomputed from the live policy) are injected at per-rollout strengths (-0.3, 0.0, 0.3, 0.6) across the last 18 transformer layers, turning each prompt's rollout group into a committee over latent modes. The reward stack is anchored by a per-row declarative validator — expected tool-call names, required arguments, and argument-value constraints checked in-process — with steered-rollout self-consistency as the structure-convergence signal. Like the rest of the line, NSC-ACE keeps the native vision tower and the multi-token-prediction (MTP) head for speculative decoding.

Status: experimental checkpoint. Released with full weights + GGUFs for community testing. See Evaluation for an honest account of what is and isn't yet measured.

Evaluation

GBS STANDARD-200 — avg@3 with sampled decoding (temperature 0.6 / top-p 0.95 / top-k 20), a single identical methodology applied to both models, both served by the same vLLM 0.24 + MTP stack (no published-number carryover). Comparison restricted to the 150 items shared by both runs — HLE, LiveCodeBench, HLCE; GPQA excluded (see caveat):

(avg@3, 150 shared items)	Overall	HLE	LiveCodeBench	HLCE
NSC-ACE (this model)	0.724	1.000	0.173	1.000
Ornstein V2.5 (base)	0.704	1.000	0.133	0.980

Honest read: a statistical tie. Item-level paired deltas: overall +0.013 (p=0.52), coding +0.030 (p=0.54), reasoning 0.000 (p=1.0). The LiveCodeBench gain (0.133 → 0.173) is not significant at n=50, and nothing regresses. Decoding methodology now matches the V2.5 card's evaluation (avg@3, identical sampling parameters), so this table can be read in the same register as that card's plot — subject to the composition caveat below.

Composition caveat: HLE and HLCE ran on small fixture sets in this run (no public loader exists for either); fixture columns saturate near 1.0 and do not discriminate. LiveCodeBench is real data (code_generation_lite) and is scored by executing each generated program against the dataset's public stdin/stdout test cases — a scoring change from the earlier assert-based fixture path. GPQA is excluded entirely: the base leg resolved to fixtures (gated-dataset access landed only after it completed), so no like-for-like GPQA pair exists. Only LiveCodeBench provides discriminative real-data signal here.

An earlier single-pass greedy paired run (delta 0.000 across all categories — no regression) is superseded by this avg@3 evaluation.

Training-side signals (right panel above; per-step means over 8 steered rollouts, quartile means Q1 → Q4):

signal	Q1 (steps 1–50)	Q4 (151–200)
Validator reward (ground-truth tool-call F1)	0.43	0.56 (Q3 peak 0.74)
Self-consistency (steered rollout agreement)	0.40	0.57
Total weighted reward	0.55	0.67

Noisy but real upward trends; the self-consistency gain is the method's structure-convergence thesis showing up in training.

Local behavioral probes (Q8_0 GGUF, llama.cpp b9940, RTX 3090): given Hermes <tools> signatures, the model selects the correct function among distractors, infers unstated argument values ("NVIDIA … in euros" → {"symbol": "NVDA", "currency": "EUR"}), emits exact <tool_call> grammar with think-first reasoning, and declines to call tools on questions where they're irrelevant.

Release line

V1 — initial reasoning fine-tune.
V1.5 — refined supervised fine-tune on quality-gated reasoning data.
V2 — reinforcement-learning post-training (DPO + GRPO verifiable-reward RL).
V2.5 — strided EBFT on a balanced AgentWorld-oracle + reasoning blend.
NSC-ACE — this release — Neural Steering Committee + validator-anchored GRPO on V2.5, targeting agentic tool calling.

Quantizations

GGUF (llama.cpp, MTP draft head embedded): DJLougen/Ornstein-9Bv2.5-NSC-ACE-GGUF — F16, Q8_0, Q6_K, Q5_K_M, Q4_K_M.

Usage

vLLM (recommended — with MTP speculative decoding)

vllm serve DJLougen/Ornstein-9Bv2.5-NSC-ACE \
  --max-model-len 8192 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Sample with the Qwen3.5 reasoning defaults: temperature=0.6, top_p=0.95, top_k=20.

llama.cpp (MTP speculative decoding, head embedded in the GGUFs)

llama-server -hf DJLougen/Ornstein-9Bv2.5-NSC-ACE-GGUF:Q8_0 \
  -ngl 99 --spec-type draft-mtp

Measured on an RTX 3090 at Q8_0: 67.6 → 100.7 tok/s on code (+49%), 90.6 → 110.9 tok/s on prose (+22%) vs. non-speculative.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "DJLougen/Ornstein-9Bv2.5-NSC-ACE"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Search for the latest CUDA release notes and summarize the key changes."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20, do_sample=True)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Tool-call grammar

Trained on Hermes-style calls with per-row tool exposure (<tools> JSON signatures in the system turn):

<tool_call>
{"name": "function_name", "arguments": {"param": "value"}}
</tool_call>

Training recipe

Method: GRPO (TRL 1.7.1) with a Neural Steering Committee — contrastive hidden-state directions from agentic vs. passive prompts, injected during rollout generation at strengths (-0.3, 0.0, 0.3, 0.6) over the last 18 layers; directions recomputed from the live policy.
Rewards (weights): validator reward 0.35 (F1 of predicted tool calls against per-row declarative validators: names + required args + eq/in/gte/lte/regex/contains value constraints) · self-consistency 0.20 · process reward 0.15 · format 0.10 · tool-count 0.10 · reasoning-depth 0.05 · diversity 0.05.
Adapter: QLoRA on NF4 4-bit base — r=64, α=128, dropout 0.05, q/k/v/o/gate/up/down (116M trainable, 1.28%), merged to bf16 for release; original MTP head and vision tower re-attached unchanged (MTP is draft-then-verify, so a frozen draft head is output-exact).
Schedule: 200 steps, 8 rollouts/prompt, max 768 completion tokens, lr 5e-6, 10% warmup, KL β 0.01 (observed KL ~0.001–0.002); 2 h 37 m on 1× A100 SXM4 80GB.
Data: NSC-ACE validator dataset — 2,000 rows converted from DJLougen/Acta (Hermes function calling); each row carries its verbatim <tools> system turn, reference <tool_call> blocks, and an auto-generated declarative validator. 200-row held-out eval split.
Base: DJLougen/Ornstein-3.5-9B-V2.5 (Qwen3.5-9B lineage).

Details

Developed by: DJLougen / GestaltLabs
Base model: DJLougen/Ornstein-3.5-9B-V2.5
Post-training: NSC-ACE (steering committee + validator-anchored GRPO, LoRA merged)
Parameters: ~9.65B
Precision: BF16
Format: ChatML / Qwen3.5 chat template (reasoning <think> enabled)
Capabilities retained: vision tower (image/video), MTP head (speculative decoding)
License: Apache 2.0

Support This Work

All training compute is self-funded on rented GPUs and a local DGX Spark. If these uploads are useful, consider buying a PhD student a coffee — it keeps the experiments running.

Support on Ko-fi

License

Apache 2.0 — inherited from the Qwen 3.5 9B base release.

Downloads last month: 13

Safetensors

Model size

10B params

Tensor type

F32

BF16

Model tree for GestaltLabs/Ornstein-9Bv2.5-NSC-ACE

Base model

DJLougen/Ornstein-3.5-9B-V2.5

Finetuned

(1)

this model

Quantizations

1 model