Instructions to use Worthify/Decision-1-NF4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Worthify/Decision-1-NF4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Worthify/Decision-1-NF4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Worthify/Decision-1-NF4")
model = AutoModelForCausalLM.from_pretrained("Worthify/Decision-1-NF4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Worthify/Decision-1-NF4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Worthify/Decision-1-NF4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Worthify/Decision-1-NF4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Worthify/Decision-1-NF4

SGLang

How to use Worthify/Decision-1-NF4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Worthify/Decision-1-NF4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Worthify/Decision-1-NF4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Worthify/Decision-1-NF4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Worthify/Decision-1-NF4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Worthify/Decision-1-NF4 with Docker Model Runner:
```
docker model run hf.co/Worthify/Decision-1-NF4
```

Worthify Decision-1 NF4

A pre-quantized NF4 checkpoint in the Worthify Decision-1 family. Download 7.64 GB of weight shards and load the saved NF4 tensors directly. In three short usage checks on an RTX 3090, peak PyTorch-allocated inference memory was 7.26 GiB.

We converted Decision-1 at cab55818cfb6f4163f2b2597bed31cceb6ba2030 with bitsandbytes NF4 and double quantization. This release adds no training. Decision-1 is a single-seed research preview derived from Gemma 4 12B; its original model card describes the training and BF16 evaluation.

Give the model a text state, a question, and 2–16 described options. Read the next-token scores for the supplied option letters to select an option ID. Those scores are conditional on the menu and uncalibrated. The model cannot select an omitted option.

Load the NF4 checkpoint

Use the versions in requirements.txt. We verified one visible CUDA GPU with PyTorch 2.10.0+cu128, Transformers 5.17.0, Accelerate 1.12.0, and bitsandbytes 0.50.2.

pip install torch==2.10.0 transformers==5.17.0 accelerate==1.12.0 \
  bitsandbytes==0.50.2 huggingface-hub==1.31.0 safetensors==0.8.0

Run this Python example with CUDA_VISIBLE_DEVICES=0. The code resolves an immutable repository revision before loading:

import torch
from huggingface_hub import HfApi
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Worthify/Decision-1-NF4"
REVISION = HfApi().model_info(MODEL_ID).sha

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    dtype=torch.bfloat16,
    device_map={"": "cuda:0"},
    trust_remote_code=False,
).eval()
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=REVISION)

model.config.use_cache = False
# Required FP32 final-softcap policy; forward hooks do not survive serialization.
model.lm_head.register_forward_hook(lambda _module, _inputs, output: output.float())

The saved quantization_config enables NF4 on reload. Keep the FP32 LM-head hook and disabled cache when scoring decisions. numeric-policy.json records this policy. For the decision prompt and categorical readout, see the Decision-1 source and the authored input rows.

Measured size and reload checks

Item	Measured value
Saved weight shards	7,638,435,623 bytes, or 7.64 GB
NF4 linear layers	328
Compute dtype	BF16
Final-softcap precision	FP32, cap 30
Fresh reload GPU	NVIDIA GeForce RTX 3090
Peak PyTorch-allocated inference memory	7,797,513,216 bytes, or 7.26 GiB
Fresh reload option-logit difference	0 on all three authored rows

We hashed the source shards against the pinned public checkpoint, saved tensors with native text-model names, and checked the exported file hashes. A separate process reloaded the NF4 checkpoint and reproduced all three NF4 option-logit vectors exactly. The export receipt, source weight hashes, and reload receipt retain the evidence.

The memory figure covers three short single-row inference calls. It excludes some CUDA runtime and driver overhead and does not establish a minimum GPU capacity or memory use at longer context lengths.

Observed difference from BF16

We scored the same three authored evidence examples in BF16 on an A100 and NF4 on an RTX 3090, using fresh single-row scoring, disabled caching, and the FP32 final softcap in both runs.

Example	BF16 selection	NF4 selection
Installation and inventory records support the claim	`supported`	`supported`
A vendor advisory only announces that a release is available	`insufficient`	`supported`
Installation failed and inventory records the older release	`contradicted`	`contradicted`

NF4 changed the advisory-only example to supported. The paired comparison includes the option logits; BF16 rows, NF4 export rows, and NF4 reload rows retain the complete scores.

These are three usage checks on different GPUs. They do not estimate task accuracy or reproduce Decision-1's BF16 benchmark results. Measure this quantized variant on your own held-out decisions before relying on its selections. Inputs longer than the source model's 2,048-token training prompt cap have not been validated.

Provenance and attribution

The source is Worthify's Decision-1 checkpoint at the immutable revision above, originally trained from google/gemma-4-12B-it at 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7. This repository retains its original model card, Apache-2.0 license, training-data attribution, and original BF16 numeric policy. The original policy describes the source checkpoint; the NF4 loading policy is in numeric-policy.json.

The Gemma-derived weights carry Apache-2.0 terms with applicable attribution. Source datasets retain their separate terms. Quantization adds no dataset or new training examples.

Downloads last month: -

Safetensors

Model size

12B params

Tensor type

BF16

Model tree for Worthify/Decision-1-NF4

Base model

google/gemma-4-12B

Finetuned

google/gemma-4-12B-it

Finetuned

Worthify/Decision-1

Quantized

(1)

this model