Instructions to use Dingdust/VibeThinker-3B-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dingdust/VibeThinker-3B-heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Dingdust/VibeThinker-3B-heretic")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Dingdust/VibeThinker-3B-heretic")
model = AutoModelForCausalLM.from_pretrained("Dingdust/VibeThinker-3B-heretic", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dingdust/VibeThinker-3B-heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dingdust/VibeThinker-3B-heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dingdust/VibeThinker-3B-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Dingdust/VibeThinker-3B-heretic

SGLang

How to use Dingdust/VibeThinker-3B-heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dingdust/VibeThinker-3B-heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dingdust/VibeThinker-3B-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dingdust/VibeThinker-3B-heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dingdust/VibeThinker-3B-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Dingdust/VibeThinker-3B-heretic with Docker Model Runner:
```
docker model run hf.co/Dingdust/VibeThinker-3B-heretic
```

This is a decensored version of a model, made using Heretic v1.4.0

Abliteration parameters

Parameter	Value
direction_index	23.30
attn.o_proj.max_weight	1.26
attn.o_proj.max_weight_position	21.83
attn.o_proj.min_weight	1.20
attn.o_proj.min_weight_distance	13.99
mlp.down_proj.max_weight	1.49
mlp.down_proj.max_weight_position	21.32
mlp.down_proj.min_weight	0.86
mlp.down_proj.min_weight_distance	13.56

Performance

Metric	This model	Original model (a model)
KL divergence	0.0255	0 (by definition)
Refusals	9/100	64/100

Residual Geometry

Layer	S(g,b)	S(g,b)	S(g,r)	S(g,r)	S(b,r)	S(b,r)	\|g\|	\|g*\|	\|b\|	\|b*\|	\|r\|	\|r*\|	Silh
1	0.9994	0.9994	0.1419	0.1613	0.1773	0.1966	12.07	12.06	12.14	12.14	0.44	0.44	0.0582
2	0.9994	0.9994	0.1141	0.1236	0.1484	0.1577	15.81	15.80	15.88	15.88	0.55	0.55	0.0670
3	0.9987	0.9987	0.0273	0.0306	0.0785	0.0816	16.28	16.28	16.33	16.33	0.84	0.83	0.0788
4	0.9979	0.9979	0.0460	0.0506	0.1110	0.1152	16.28	16.28	16.36	16.36	1.07	1.06	0.0868
5	0.9973	0.9974	0.1027	0.1029	0.1750	0.1743	16.83	16.83	17.00	17.01	1.25	1.23	0.0941
6	0.9975	0.9976	0.0628	0.0623	0.1332	0.1319	17.63	17.63	17.75	17.76	1.26	1.24	0.0956
7	0.9964	0.9964	-0.0058	-0.0099	0.0794	0.0748	18.16	18.18	18.22	18.23	1.55	1.54	0.0870
8	0.9941	0.9941	0.0083	0.0045	0.1172	0.1131	18.91	18.93	19.04	19.06	2.07	2.07	0.0952
9	0.9849	0.9852	-0.0096	-0.0132	0.1636	0.1582	19.40	19.44	19.66	19.68	3.40	3.37	0.1217
10	0.9854	0.9857	-0.0042	-0.0089	0.1660	0.1596	20.51	20.56	20.80	20.83	3.54	3.51	0.1232
11	0.9869	0.9871	0.0023	-0.0012	0.1637	0.1587	20.92	20.97	21.20	21.23	3.42	3.39	0.1262
12	0.9860	0.9862	-0.0215	-0.0283	0.1455	0.1377	25.71	25.78	25.98	26.02	4.33	4.31	0.1191
13	0.9838	0.9839	-0.1014	-0.1188	0.0786	0.0605	25.86	25.96	25.80	25.82	4.65	4.65	0.1191
14	0.9837	0.9839	-0.0898	-0.1108	0.0906	0.0685	26.00	26.12	26.00	26.02	4.69	4.68	0.1131
15	0.9830	0.9833	-0.0709	-0.0856	0.1133	0.0971	26.07	26.18	26.17	26.21	4.81	4.79	0.1106
16	0.9808	0.9813	0.0049	-0.0102	0.1995	0.1825	26.02	26.17	26.56	26.61	5.17	5.13	0.1142
17	0.9858	0.9862	0.0985	0.0796	0.2640	0.2438	29.96	30.13	30.91	30.97	5.21	5.15	0.1160
18	0.9857	0.9861	0.0173	0.0000	0.1854	0.1664	29.06	29.21	29.57	29.63	4.98	4.93	0.1078
19	0.9863	0.9867	0.0289	0.0055	0.1935	0.1679	31.16	31.35	31.75	31.80	5.24	5.17	0.1101
20	0.9856	0.9860	0.0036	-0.0164	0.1727	0.1508	30.69	30.85	31.16	31.20	5.27	5.21	0.1071
21	0.9810	0.9813	0.1356	0.1211	0.3253	0.3100	36.41	36.57	38.15	38.19	7.47	7.41	0.1585
22	0.9789	0.9792	0.1084	0.0922	0.3092	0.2923	38.37	38.60	40.11	40.19	8.24	8.19	0.1662
23	0.9724	0.9727	0.0068	-0.0035	0.2401	0.2288	37.52	37.71	38.65	38.74	9.02	8.99	0.1715
24	0.9685	0.9685	-0.0140	-0.0286	0.2355	0.2212	40.01	40.24	41.17	41.24	10.25	10.27	0.1857
25	0.9549	0.9544	-0.1280	-0.1385	0.1722	0.1636	42.07	42.28	42.35	42.44	12.68	12.80	0.2114
26	0.9512	0.9504	-0.1326	-0.1426	0.1798	0.1723	44.42	44.64	44.76	44.85	13.94	14.09	0.2190
27	0.9524	0.9513	-0.1822	-0.1915	0.1263	0.1202	51.97	52.19	51.51	51.60	15.97	16.20	0.2333
28	0.9331	0.9309	-0.1472	-0.1589	0.2182	0.2126	53.06	53.29	53.77	53.85	19.55	19.92	0.2326
29	0.9280	0.9251	-0.1168	-0.1313	0.2617	0.2551	61.41	61.77	63.19	63.33	23.71	24.26	0.2238
30	0.9122	0.9090	-0.1673	-0.1811	0.2514	0.2452	66.16	66.62	67.40	67.58	28.01	28.64	0.2210
31	0.9066	0.9034	-0.1648	-0.1825	0.2669	0.2567	82.24	83.01	84.17	84.45	36.02	36.82	0.2197
32	0.9124	0.9102	-0.0878	-0.1084	0.3275	0.3132	112.01	113.23	118.09	118.53	48.51	49.39	0.2145
33	0.9271	0.9252	-0.1205	-0.1459	0.2604	0.2405	146.74	148.56	150.88	151.41	56.96	58.08	0.2068
34	0.9447	0.9435	-0.0870	-0.1083	0.2446	0.2272	177.54	179.19	182.41	182.92	60.06	60.97	0.2024
35	0.9363	0.9350	-0.1146	-0.1319	0.2415	0.2282	180.56	182.04	184.84	185.34	65.32	66.30	0.2059
36	0.8726	0.8711	-0.1135	-0.1285	0.3862	0.3750	86.94	87.92	93.65	94.06	46.04	46.57	0.1981

g = mean of residual vectors for good prompts

g* = geometric median of residual vectors for good prompts

b = mean of residual vectors for bad prompts

b* = geometric median of residual vectors for bad prompts

r = refusal direction for means (i.e., b - g)

r* = refusal direction for geometric medians (i.e., b* - g*)

S(x,y) = cosine similarity of x and y

|x| = L2 norm of x

Silh = Mean silhouette coefficient of residuals for good/bad clusters

Residual Vectors Visualization

VibeThinker-3B

GitHub | Hugging Face | Technical Report

Introduction

VibeThinker-3B is a further exploration of the VibeThinker series at the 3B-parameter scale, focusing on challenging reasoning tasks with clear verification signals, such as mathematics, coding, and STEM. By systematically optimizing the Spectrum-to-Signal Principle (SSP) post-training pipeline introduced in VibeThinker-1.5B, VibeThinker-3B achieves strong performance on AIME, HMMT, IMO-AnswerBench, LiveCodeBench, and recent LeetCode contests, reaching the performance range of top-tier frontier reasoning models, including Qwen3.6 Plus, Gemini 3 Pro, GLM-5, and Kimi K2.5, on verifiable reasoning benchmarks.

Motivated by these observations, we propose the Parametric Compression-Coverage Hypothesis: different capabilities depend on model parameters in fundamentally different ways. Verifiable reasoning is closer to a highly compressible, parameter-dense capability, centered on multi-step reasoning, constraint satisfaction, self-correction, and answer verification. When the task space is sufficiently structured and feedback signals are sufficiently reliable, compact models may also carry near-frontier reasoning capabilities. In contrast, open-domain knowledge, general-purpose dialogue, and long-tail scenario understanding rely more heavily on large-scale parameters to broadly cover facts, concepts, and world knowledge.

From VibeThinker-1.5B to VibeThinker-3B, our goal is not to build a small model that replaces large-scale models, but to examine the real boundaries of small models along specific capability dimensions. With VibeThinker-3B, we aim to show that small models should not be viewed merely as a compromise for reducing deployment costs. For capability domains with clear feedback and verification mechanisms, SLMs emerge as a promising research trajectory toward frontier-level performance that is fundamentally complementary to the traditional parameter scaling paradigm.

Key Performance Data

📏 In terms of reasoning accuracy relative to model scale, VibeThinker-3B reaches 76.4 on IMO-AnswerBench, a highly challenging benchmark with 400 IMO-level problems, with only 3B parameters, and improves to 80.6 with Claim-Level Reliability Assessment (CLR), a test-time scaling strategy for answer-verifiable reasoning tasks. This demonstrates that a model within a strictly small-model regime can reach the performance range of substantially larger models, such as DeepSeek V3.2 (78.3, 671B), GLM-5 (82.5, 744B), and Kimi K2.5 (81.8, 1T).

💡 VibeThinker-3B achieves strong results across mathematics, coding, knowledge, and instruction-following benchmarks.

🔁 VibeThinker-3B achieves competitive results against first-tier reasoning models and reaches the performance range of top-tier systems on several verifiable reasoning benchmarks.

🏆 To further test the model's out-of-distribution performance, we evaluate VibeThinker-3B on recent unseen LeetCode weekly and biweekly contests (Python) from Apr. 25 to May 31, 2026. VibeThinker-3B passes 123/128 first-attempt submissions, corresponding to a 96.1% acceptance rate.

Training Pipeline

VibeThinker-3B follows the Spectrum-to-Signal Principle (SSP) introduced in VibeThinker-1.5B. The SFT stage constructs a broad spectrum of valid reasoning trajectories, while the RL stage amplifies correct reasoning signals using verifiable rewards.

The training pipeline contains the following stages:

Curriculum-based two-stage SFT
- Stage 1 focuses on broad capability coverage across math, code, STEM reasoning, general dialogue, and instruction following.
- Stage 2 shifts toward harder and longer-horizon reasoning samples.
- Diversity-Exploring Distillation is used to preserve multiple valid solution paths.
Multi-domain Reasoning RL
- VibeThinker-3B reuses MaxEnt-Guided Policy Optimization (MGPO).
- RL is applied sequentially to math, code, and STEM reasoning tasks.
- Training uses a single 64K long-context window to preserve complete long-horizon reasoning trajectories.
Offline Self-Distillation
- High-quality trajectories from Math, Code, and STEM RL checkpoints are filtered and distilled back into a unified student model.
- A learning-potential score is used to prioritize traces that are correct but not yet well modeled by the student.
Instruct RL
- The final stage improves controllability on user-facing prompts.
- Rule-based validators and rubric-based reward models are used for format-sensitive and open-ended instruction data.

Usage Guidelines

We recommend using VibeThinker-3B for competitive-style math, coding, STEM reasoning, and other tasks where the target answer can be verified. For broad open-domain knowledge tasks, larger general-purpose models may still be more suitable.

For benchmark-style evaluation, the technical report uses vLLM with:

temperature=1.0
top_p=0.95
top_k=-1

Quick Start

Required: transformers>=4.54.0

Recommended for better inference performance: vLLM==0.10.1 or SGLang>=0.4.9.post6

from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig


class VibeThinker:
    def __init__(self, model_path):
        self.model_path = model_path
        self.model = AutoModelForCausalLM.from_pretrained(
            self.model_path,
            low_cpu_mem_usage=True,
            torch_dtype="bfloat16",
            device_map="auto",
        )
        self.tokenizer = AutoTokenizer.from_pretrained(
            self.model_path,
            trust_remote_code=True,
        )

    def infer_text(self, prompt):
        messages = [{"role": "user", "content": prompt}]
        text = self.tokenizer.apply_chat_template(
            messages,
            tokenize=False,
            add_generation_prompt=True,
        )
        model_inputs = self.tokenizer([text], return_tensors="pt").to(self.model.device)

        generation_config = dict(
            max_new_tokens=102400,
            do_sample=True,
            temperature=1.0,
            top_p=0.95,
            top_k=None,
        )
        generated_ids = self.model.generate(
            **model_inputs,
            generation_config=GenerationConfig(**generation_config),
        )
        generated_ids = [
            output_ids[len(input_ids):]
            for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
        ]

        return self.tokenizer.batch_decode(
            generated_ids,
            skip_special_tokens=True,
        )[0]


if __name__ == "__main__":
    model = VibeThinker("WeiboAI/VibeThinker-3B")
    prompt = "Your Prompt"
    print(model.infer_text(prompt))

License

The model repository is licensed under the MIT License.

Citations & References

If you use VibeThinker-3B in your research or product, please cite:

@misc{xu2026vibethinker3bexploringfrontierverifiable,
      title={VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models}, 
      author={Sen Xu and Shixi Liu and Wei Wang and Jixin Min and Yingwei Dai and Zhibin Yin and Yirong Chen and Xin Zhou and Junlin Zhang},
      year={2026},
      eprint={2606.16140},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2606.16140}, 
}