arxiv:2606.16140

VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

Published on Jun 15

· Submitted by

Sen Xu on Jun 16

WeiboAI

Upvote

Authors:

Abstract

VibeThinker-3B demonstrates that compact models can achieve state-of-the-art performance on verifiable reasoning tasks through specialized training techniques, challenging conventional scaling assumptions.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

This technical report introduces VibeThinker-3B, a compact dense model with 3B parameters developed to investigate how far verifiable reasoning can be pushed within a strictly small-model regime. Building upon the Spectrum-to-Signal post-training paradigm, we systematically enhance the model through an optimized pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. Experimental evaluations demonstrate that VibeThinker-3B achieves frontier-level performance on highly demanding verifiable tasks. Specifically, it attains a score of 94.3 on AIME26 (improving to 97.1 with claim-level test-time scaling), an 80.2 Pass@1 on LiveCodeBench v6, and exhibits strong out-of-distribution generalization with a 96.1\% acceptance rate on recent unseen LeetCode contests. This effectively places it in the performance band of first-tier reasoning systems, matching or exceeding flagship models that are orders of magnitude larger, such as DeepSeek V3.2, GLM-5, and Gemini 3 Pro. Furthermore, a score of 93.4 on IFEval confirms that this extreme reasoning enhancement does not compromise strict instruction controllability. Extending our previous 1.5B work, these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios. This perspective suggests that compact models are not merely deployment-efficient substitutes, but a complementary path toward frontier-level performance in parameter-dense capability regimes.

View arXiv page View PDF Project page GitHub 577 Add to collection

Community

SenXu1123

Paper submitter about 5 hours ago

We introduce VibeThinker-3B, a 3B dense reasoning model that explores how far verifiable reasoning can be pushed within a strictly small-model regime.

The core idea behind this work is that capabilities such as mathematics, coding, and STEM reasoning may be highly compressible when the task space is structured and reliable verification signals are available. This leads to our Parametric Compression-Coverage Hypothesis: verifiable reasoning can be concentrated in compact models through search, constraint satisfaction, self-correction, and answer verification, while open-domain knowledge and general-purpose dialogue require broader parameter coverage over facts, concepts, and long-tail scenarios.

Despite its compact scale, VibeThinker-3B achieves strong results on challenging verifiable reasoning benchmarks:

94.3 on AIME 2026, improved to 97.1 with Claim-Level Reliability Assessment (CLR), a test-time scaling strategy.
80.2 Pass@1 on LiveCodeBench v6
76.4 on IMO-AnswerBench, improved to 80.6 with CLR
96.1% acceptance rate on recent unseen LeetCode contests (From Apr.25 - May 31, 2026)
93.4 on IFEval, suggesting that reasoning enhancement does not compromise instruction controllability

We hope this work provides evidence that small language models should not be viewed only as deployment-efficient substitutes for larger models, but also as a complementary research path toward frontier-level performance in parameter-dense, verifiable reasoning domains.

Code, model, and the technical report are available here: