Instructions to use Specific-Labs/Gemma-4-31B-AntiHal with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Specific-Labs/Gemma-4-31B-AntiHal with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Specific-Labs/Gemma-4-31B-AntiHal", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Specific-Labs/Gemma-4-31B-AntiHal", trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained("Specific-Labs/Gemma-4-31B-AntiHal", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Specific-Labs/Gemma-4-31B-AntiHal with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Specific-Labs/Gemma-4-31B-AntiHal"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Specific-Labs/Gemma-4-31B-AntiHal",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Specific-Labs/Gemma-4-31B-AntiHal

SGLang

How to use Specific-Labs/Gemma-4-31B-AntiHal with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Specific-Labs/Gemma-4-31B-AntiHal" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Specific-Labs/Gemma-4-31B-AntiHal",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Specific-Labs/Gemma-4-31B-AntiHal" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Specific-Labs/Gemma-4-31B-AntiHal",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Specific-Labs/Gemma-4-31B-AntiHal with Docker Model Runner:
```
docker model run hf.co/Specific-Labs/Gemma-4-31B-AntiHal
```

Gemma-4-31B-AntiHal

A premise-challenging variant of Gemma 4 31B. Built by Specific Labs.

Gemma-4-31B-AntiHal is a variant of google/gemma-4-31b-it that examines the premise of a request before answering it — surfacing fabricated entities, invented citations, made-up facts, and requests built on incorrect assumptions, while preserving the base model's general capability.

This is not a safety-tuned or moderation model. It adds no content filtering, no refusal layer, and no harm classifier. It is designed to do one thing: challenge false premises and flawed decisions — and to raise those problems before proceeding, rather than agreeing with a confident-but-wrong user.

How it works

AntiHal uses representation steering, not fine-tuning. The behavior comes from a single, inspectable direction in the model's activation space and a schedule that controls when it is applied.

The direction. A mean-difference vector at the layer-33 residual stream separates activations on prompts where the model should push back from prompts where it should comply, unit-normalized. This isolates the internal "scrutinize the premise" direction the base model already represents.

The schedule. At inference the direction is added to the layer-33 residual stream, scaled to the layer's typical response-token norm. Rather than steering the whole response — which biases the model toward endless deliberation and hurts task completion — AntiHal uses a steer-then-release schedule: full strength for the first ~24 generated tokens, then a linear decay to zero over the next ~70. This sets the model's epistemic stance early, then returns control to the base model so it answers fluently and terminates cleanly.

The mechanism is baked into the model's generation path (modeling_antihal.py); no weights are modified.

How to use

The steering is built in — load and generate normally.

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

REPO = "Specific-Labs/Gemma-4-31B-AntiHal"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForImageTextToText.from_pretrained(
    REPO, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto").eval()

msgs = [{"role": "user", "content": "Summarize the key findings of the 2021 Zhang-Petrov paper on quantum-tunneling neural networks in Nature."}]
enc = tok(tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True), return_tensors="pt").to(model.device)
out = model.generate(**enc, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

# Toggle steering off to recover the exact base model:
# model.set_antihal(False)

trust_remote_code=True is required — the steering lives in a small, readable modeling_antihal.py that installs a forward hook during generation. Steering strength and the decay window are stored in config.json (antihal_layer, antihal_decay_full, antihal_decay_zero) and can be edited.

Evaluation

AntiHal vs. the base model, same prompts and greedy decoding:

Benchmark	Gemma-4-31B-IT	Gemma-4-31B-AntiHal
HalBench — pushback on false premises¹	26%	46%
MATH-500 — accuracy	77%	75%
LiveCodeBench v6 — pass@1	55%	52%

Roughly double the pushback on fabricated premises, for a small capability cost.

¹ Scored with Specific Labs' reason-aware pushback scorer. Preliminary results on representative samples of each benchmark; full-benchmark evaluation to follow.

Limitations

AntiHal targets premise-scrutiny; it is not a general capability upgrade and carries a small capability cost (a few points on math and coding).
Stronger premise-challenging can surface as over-questioning; behaviour on valid-but-unusual premises has not been exhaustively characterised. Use model.set_antihal(False) to fall back to the base model.
It is not a safety or moderation model and provides no guardrails beyond those of the base model.
Requires trust_remote_code=True.

License

Citation

@misc{specificlabs2026antihal,
  title  = {Gemma-4-31B-AntiHal: premise-challenging representation steering for Gemma 4 31B},
  author = {Specific Labs},
  year   = {2026},
  url    = {https://huggingface.co/Specific-Labs/Gemma-4-31B-AntiHal}
}

Downloads last month: 303

Safetensors

Model size

33B params

Tensor type

BF16