Eliciting Reasoning with 100-500 Zebra Puzzles

Resources & Quick Links:

• Read this Post: HuggingFace (Rendered Blog) | GitHub (Markdown Source)
• GitHub Repository: tamewild/pcss
• Reproduction Notebooks (Single H100/H200): Qwen 3 4B (~6.5 mins) | Granite 4.1 3B (~23 mins) | Qwen 3.5 9B (~40 mins)
• Models: PCSS-Qwen3-4B | PCSS-Granite-4.1-3B | PCSS-Qwen3.5-9B
• Datasets: zebra_100 | instruct5

---

1. Introduction

Encountering the research on the Hyperfitting Phenomenon sparked an interest in exploring how pre-trained base models behave when micro-datasets are fine-tuned to near-zero training loss. We focused on training small pre-trained base models on 100 to 500 pure logic deduction traces (specifically 5x5 zebra puzzles) with zero mathematical data in the fine-tuning dataset. This synthetic dataset was generated by Qwen3-235B-A22B-Instruct-2507 in September 2025 using rejection sampling.

Across multiple model families and architectures:

• Qwen 3 4B Base fine-tuned on 100 zebra puzzles (~6.5 minutes on 1 GPU) reaches 85.26% on the full 5,000-problem MATH benchmark (+31.16% delta over base) and 21.67% on AIME 2025, rivaling the official post-trained model in non-thinking mode on MATH-500 while outperforming it on AIME 2025.
• Granite 4.1 3B Base fine-tuned on 500 zebra puzzles achieves 77.73% on MATH-500 and 19.44% on AIME 2025, outperforming the official IBM Granite 4.1 3B Instruct model (+11.13 and +12.77 percentage points, respectively).
• Qwen 3.5 9B Base fine-tuned on 500 zebra puzzles jumps from 17.00% to 80.50% on 5x5 zebra puzzles (Pass@3: 97.05%), while jumping from 5.42% to 22.92% on the uncontaminated ArXivMath 05/26 benchmark (Pass@3: 34.88%) and reaching 96.60% on MATH-500 and 60.67% on AIME 2025.

While this out-of-domain generalization appears primarily driven by the pure logic dataset, we chose to manage the optimization process through adaptive loss scaling. For our primary results, we utilized PCSS (Per-Example Calibrated Sigmoid Scaler): an adaptive, data-aware loss scaler derived directly from KTO.

---

2. The Mathematics of PCSS

2.1 From Preference Optimization to a Dynamic SFT Scaler

PCSS originates from KTO. But why adapt an alignment algorithm for Supervised Fine-Tuning?

The inspiration came from an experiment in the original KTO paper (Table 4). The authors demonstrated that running KTO using only desirable/positive examples (the exact same unpaired data used for SFT) yielded results highly competitive with standard SFT. While the paper conservatively notes this setup "yields similar results to SFT when $\beta \backslash beta$ is small," the results with $\beta =0.1\backslash beta=0.1$ raised the overall average benchmark score from 29.7% to 31.1%, driven by a substantial relative increase on the GSM8K mathematical reasoning benchmark (from 1.0% to 12.5%).

Why does an alignment algorithm behave this way? We can answer this by performing a gradient analysis on standard KTO when applied to purely desirable data.

Setting the gain sensitivity to 1 $({\lambda }_D=1)(\backslash lambda\_D\; =\; 1)$, the KTO loss for a desirable ground-truth sequence $y^{\ast }y^*$ is: $${\mathcal{L}}_{\text{KTO}}(\theta )=1-\sigma (\beta (r_{\theta }(x,y^{\ast })-z_0))\backslash mathcal\{L\}\_\{\backslash text\{KTO\}\}(\backslash theta)\; =\; 1\; -\; \backslash sigma(\backslash beta(r\_\backslash theta(x,\; y^*)\; -\; z\_0))$$ where $r_{\theta }(x,y)r\_\backslash theta(x,\; y)$ is the unnormalized log ratio $(\log {\pi }_{\theta }(y\mathrm{\mid }x)-\log {\pi }_{\text{ref}}(y\mathrm{\mid }x))(\backslash log\; \backslash pi\_\backslash theta(y|x)\; -\; \backslash log\; \backslash pi\_\{\backslash text\{ref\}\}(y|x))$. The loss for the ground-truth sequence is evaluated at $r_{\theta }(x,y^{\ast })r\_\backslash theta(x,\; y^*)$, and $z_{0}z\_0$ is the KL divergence estimate. Crucially, KTO explicitly does not backpropagate through $z_{0}z\_0$.

Taking the gradient with respect to $\theta \backslash theta$: $${\mathrm{\nabla }}_{\theta }{\mathcal{L}}_{\text{KTO}}(\theta )=-{\sigma }^{\mathrm{\prime }}(\beta (r_{\theta }(x,y^{\ast })-z_0))\cdot \beta \cdot {\mathrm{\nabla }}_{\theta }{r}_{\theta }(x,y^{\ast })\backslash nabla\_\backslash theta\; \backslash mathcal\{L\}\_\{\backslash text\{KTO\}\}(\backslash theta)\; =\; -\backslash sigma\text{'}(\backslash beta(r\_\backslash theta(x,\; y^*)\; -\; z\_0))\; \backslash cdot\; \backslash beta\; \backslash cdot\; \backslash nabla\_\backslash theta\; r\_\backslash theta(x,\; y^*)$$ Because the reference model is frozen, ${\mathrm{\nabla }}_{\theta }{r}_{\theta }(x,y^{\ast })={\mathrm{\nabla }}_{\theta }\log {\pi }_{\theta }(y^{\ast }\mathrm{\mid }x)\backslash nabla\_\backslash theta\; r\_\backslash theta(x,\; y^*)\; =\; \backslash nabla\_\backslash theta\; \backslash log\; \backslash pi\_\backslash theta(y^*|x)$. Since the standard SFT loss $L_{\text{sft}}(\theta )L\_\{\backslash text\{sft\}\}(\backslash theta)$ is the negative mean log-probability over $NN$ tokens, $\log {\pi }_{\theta }(y^{\ast }\mathrm{\mid }x)=-N\cdot L_{\text{sft}}(\theta )\backslash log\; \backslash pi\_\backslash theta(y^*|x)\; =\; -N\; \backslash cdot\; L\_\{\backslash text\{sft\}\}(\backslash theta)$. Therefore, ${\mathrm{\nabla }}_{\theta }\log {\pi }_{\theta }(y^{\ast }\mathrm{\mid }x)=-N\cdot {\mathrm{\nabla }}_{\theta }{L}_{\text{sft}}(\theta )\backslash nabla\_\backslash theta\; \backslash log\; \backslash pi\_\backslash theta(y^*|x)\; =\; -N\; \backslash cdot\; \backslash nabla\_\backslash theta\; L\_\{\backslash text\{sft\}\}(\backslash theta)$.

Substituting this back in gives the exact gradient update for standard KTO on positive data: $${\mathrm{\nabla }}_{\theta }{\mathcal{L}}_{\text{KTO}}(\theta )=[\beta \cdot {\sigma }^{\mathrm{\prime }}(\beta (r_{\theta }(x,y^{\ast })-z_0))\cdot N]{\mathrm{\nabla }}_{\theta }{L}_{\text{sft}}(\theta )\backslash nabla\_\backslash theta\; \backslash mathcal\{L\}\_\{\backslash text\{KTO\}\}(\backslash theta)\; =\; \backslash Big[\; \backslash beta\; \backslash cdot\; \backslash sigma\text{'}(\backslash beta(r\_\backslash theta(x,\; y^*)\; -\; z\_0))\; \backslash cdot\; N\; \backslash Big]\; \backslash nabla\_\backslash theta\; L\_\{\backslash text\{sft\}\}(\backslash theta)$$

When you apply KTO to purely desirable examples, it is no longer performing preference alignment against negative data. Instead, it mathematically reduces to the standard SFT gradient multiplied by a dynamic scaler.

However, this same equation highlights why standard KTO would struggle on logic chains spanning up to 11,000+ tokens:

1. Length-Induced Gradient Inflation: The scaler is multiplied by the sequence length $NN$. For long traces, the gradient magnitude increases sharply by a factor of hundreds.
2. Length Bias & Gradient Saturation: Because $r_{\theta }(x,y^{\ast })r\_\backslash theta(x,\; y^*)$ is an unnormalized sum, its magnitude also scales linearly with $NN$, quickly saturating the sigmoid. The derivative ${\sigma }^{\mathrm{\prime }}\backslash sigma\text{'}$ drops toward zero, which would cause the optimizer to prematurely stop learning on long sequences.

To convert this mechanism into the stable PCSS scaler, we make two fundamental mathematical transformations:

Transformation 1: Sequence-Length Normalization

PCSS explicitly normalizes the implied reward by sequence length using standard mean cross-entropy losses (where $N=\mathrm{\mid }{y}^{\ast }\mathrm{\mid }N\; =\; |y^*|$ ):

• Current model SFT loss:
• Frozen base model reference loss:

The normalized reward becomes the difference: $r_{\text{norm}}=L_{\text{ref}}-L_{\text{sft}}(\theta )r\_\{\backslash text\{norm\}\}\; =\; L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{sft\}\}(\backslash theta)$.

Transformation 2: Dropping the KL Anchor $(z_0=0)(z\_0\; =\; 0)$

The canonical KTO value function includes $z_{0}z\_0$ as a baseline expectation $(z_0={\mathbb{E}}_{y\sim {\pi }_{\theta }(\cdot \mathrm{\mid }x)}[r_{\theta }(x,y)])(z\_0\; =\; \backslash mathbb\{E\}\_\{y\; \backslash sim\; \backslash pi\_\backslash theta(\backslash cdot|x)\}[r\_\backslash theta(x,\; y)])$.

While KTO's batch-dependent $z_{0}z\_0$ estimate breaks down at $bsz=1bsz=1$, mathematical analysis indicates that if we want the scaler to natively decay upon mastery in SFT, dropping the dynamic KL anchor is a straightforward architectural adjustment, making $z_0^{\text{norm}}=0z\_0^\{\backslash text\{norm\}\}\; =\; 0$ a natural design choice.

During fine-tuning on a ground-truth logic trace $(y^{\ast })(y^*)$, the policy's probability mass concentrates entirely on the correct answer $({\pi }_{\theta }(y^{\ast }\mathrm{\mid }x)\to 1)(\backslash pi\_\backslash theta(y^*|x)\; \backslash to\; 1)$. If we plug this mastered state back into the formal sequence-normalized definition of $z_{0}z\_0$: $$z_0^{\text{norm}}=\sum _y{\pi }_{\theta }(y\mathrm{\mid }x)[\frac{1}{\mathrm{\mid }y\mathrm{\mid }}\log {\pi }_{\theta }(y\mathrm{\mid }x)-\frac{1}{\mathrm{\mid }y\mathrm{\mid }}\log {\pi }_{\text{ref}}(y\mathrm{\mid }x)]\approx L_{\text{ref}}z\_0^\{\backslash text\{norm\}\}\; =\; \backslash sum\_y\; \backslash pi\_\backslash theta(y|x)\; \backslash Big[\; \backslash frac\{1\}\{|y|\}\; \backslash log\; \backslash pi\_\backslash theta(y|x)\; -\; \backslash frac\{1\}\{|y|\}\; \backslash log\; \backslash pi\_\{\backslash text\{ref\}\}(y|x)\; \backslash Big]\; \backslash approx\; L\_\{\backslash text\{ref\}\}$$

Upon convergence $(L_{\text{sft}}(\theta )\to 0)(L\_\{\backslash text\{sft\}\}(\backslash theta)\; \backslash to\; 0)$, the reward is $r_{\text{norm}}=L_{\text{ref}}-L_{\text{sft}}(\theta )\approx L_{\text{ref}}r\_\{\backslash text\{norm\}\}\; =\; L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{sft\}\}(\backslash theta)\; \backslash approx\; L\_\{\backslash text\{ref\}\}$. If we kept the baseline anchor $(z_0^{\text{norm}})(z\_0^\{\backslash text\{norm\}\})$, the advantage would evaluate to: $$\text{Advantage}=r_{\text{norm}}-z_0^{\text{norm}}\approx L_{\text{ref}}-L_{\text{ref}}=0\backslash text\{Advantage\}\; =\; r\_\{\backslash text\{norm\}\}\; -\; z\_0^\{\backslash text\{norm\}\}\; \backslash approx\; L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{ref\}\}\; =\; 0$$

Because the derivative of the sigmoid $({\sigma }^{\mathrm{\prime }})(\backslash sigma\text{'})$ reaches its peak value at an input of exactly $00$, keeping the baseline anchor $(z_0^{\text{norm}})(z\_0^\{\backslash text\{norm\}\})$ would cause the scaler to trend back toward its maximum value at the end of training, failing to suppress residual gradients and increasing the risk of late-stage overfitting.

By explicitly setting $z_0^{\text{norm}}=0z\_0^\{\backslash text\{norm\}\}\; =\; 0$, the advantage becomes an absolute measure of progress against the base model $(L_{\text{ref}}-L_{\text{sft}}(\theta ))(L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{sft\}\}(\backslash theta))$. Upon mastery $(L_{\text{sft}}(\theta )\to 0)(L\_\{\backslash text\{sft\}\}(\backslash theta)\; \backslash to\; 0)$, the advantage becomes strictly positive $(L_{\text{ref}})(L\_\{\backslash text\{ref\}\})$. The scaler structurally decays toward zero as mastery increases, naturally suppressing residual gradients to help mitigate late-stage overfitting.

The Resulting Gradient

The PCSS loss for a chosen example becomes: $${\mathcal{L}}_{\text{chosen}}(\theta )=1-\sigma (\beta \cdot r_{\text{norm}})\backslash mathcal\{L\}\_\{\backslash text\{chosen\}\}(\backslash theta)\; =\; 1\; -\; \backslash sigma(\backslash beta\; \backslash cdot\; r\_\{\backslash text\{norm\}\})$$ $${\mathrm{\nabla }}_{\theta }{\mathcal{L}}_{\text{chosen}}(\theta )=[\beta \cdot {\sigma }^{\mathrm{\prime }}(\beta \cdot r_{\text{norm}})]{\mathrm{\nabla }}_{\theta }{L}_{\text{sft}}(\theta )\backslash nabla\_\backslash theta\; \backslash mathcal\{L\}\_\{\backslash text\{chosen\}\}(\backslash theta)\; =\; \backslash Big[\; \backslash beta\; \backslash cdot\; \backslash sigma\text{'}(\backslash beta\; \backslash cdot\; r\_\{\backslash text\{norm\}\})\; \backslash Big]\; \backslash nabla\_\backslash theta\; L\_\{\backslash text\{sft\}\}(\backslash theta)$$

Because this gradient mathematically reduces to the standard SFT gradient scaled by a dynamic multiplier, we can implement it directly by scaling the standard SFT loss output by the detached multiplier: $${\mathcal{L}}_{\text{PCSS}}(\theta )=L_{\text{sft}}(\theta )\cdot \text{detach}(\beta \cdot {\sigma }^{\mathrm{\prime }}(\beta \cdot r_{\text{norm}}))\backslash mathcal\{L\}\_\{\backslash text\{PCSS\}\}(\backslash theta)\; =\; L\_\{\backslash text\{sft\}\}(\backslash theta)\; \backslash cdot\; \backslash text\{detach\}(\backslash beta\; \backslash cdot\; \backslash sigma\text{'}(\backslash beta\; \backslash cdot\; r\_\{\backslash text\{norm\}\}))$$

---

2.2 Standard Deviation Normalization & Decoupled Scaling

To standardize the dynamic multiplier across model families and decouple its magnitude from its shape, PCSS applies two scaling adjustments:

1. Scaled Advantage: $a=\frac{L_{\text{ref}}-L_{\text{sft}}(\theta )}{s_{\text{global}}}a\; =\; \backslash frac\{L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{sft\}\}(\backslash theta)\}\{s\_\{\backslash text\{global\}\}\}$, where $s_{\text{global}}={\text{Std}}_{\mathcal{D}}(L_{\text{ref}})s\_\{\backslash text\{global\}\}\; =\; \backslash text\{Std\}\_\{\backslash mathcal\{D\}\}(L\_\{\backslash text\{ref\}\})$ is the pre-computed standard deviation of reference losses across the training split.
2. Decoupled Peak Scaling: $\text{Scale}(a)=\text{peak\_scale}\cdot (4.0\cdot {\sigma }^{\mathrm{\prime }}(\beta \cdot a))\backslash text\{Scale\}(a)\; =\; \backslash text\{peak\backslash \_scale\}\; \backslash cdot\; (4.0\; \backslash cdot\; \backslash sigma\text{'}(\backslash beta\; \backslash cdot\; a))$

Since $\max (4.0\cdot {\sigma }^{\mathrm{\prime }})=1.0\backslash max(4.0\; \backslash cdot\; \backslash sigma\text{'})\; =\; 1.0$, peak_scale directly defines the maximum multiplier applied to the SFT gradient.

Applying these scaling adjustments, the programmatic PCSS loss becomes: $${\mathcal{L}}_{\text{PCSS}}(\theta )=L_{\text{sft}}(\theta )\cdot \text{detach}(\text{Scale}(a))\backslash mathcal\{L\}\_\{\backslash text\{PCSS\}\}(\backslash theta)\; =\; L\_\{\backslash text\{sft\}\}(\backslash theta)\; \backslash cdot\; \backslash text\{detach\}(\backslash text\{Scale\}(a))$$

---

2.3 The Integrated Objective

While PCSS is implemented by scaling the standard SFT loss with a detached multiplier, this mechanism is not an arbitrary gradient heuristic. Since the gradient of our implementation with respect to $\theta \backslash theta$ is $\text{Scale}(a)\cdot {\mathrm{\nabla }}_{\theta }{L}_{\text{sft}}(\theta )\backslash text\{Scale\}(a)\; \backslash cdot\; \backslash nabla\_\backslash theta\; L\_\{\backslash text\{sft\}\}(\backslash theta)$, we can find the antiderivative of $\text{Scale}(a)\backslash text\{Scale\}(a)$ with respect to $L_{\text{sft}}L\_\{\backslash text\{sft\}\}$ to recover the exact integrated objective being optimized (choosing the constant of integration to match the $1-\sigma (\dots )1\; -\; \backslash sigma(\backslash dots)$ form of the original KTO objective):

$${\mathcal{L}}_{\text{integrated}}(\theta )=\text{peak\_scale}\cdot \frac{4.0\cdot s_{\text{global}}}{\beta }[1-\sigma (\beta \frac{L_{\text{ref}}-L_{\text{sft}}(\theta )}{s_{\text{global}}}\left)\right]\backslash mathcal\{L\}\_\{\backslash text\{integrated\}\}(\backslash theta)\; =\; \backslash text\{peak\backslash \_scale\}\; \backslash cdot\; \backslash frac\{4.0\; \backslash cdot\; s\_\{\backslash text\{global\}\}\}\{\backslash beta\}\; \backslash Bigg[\; 1\; -\; \backslash sigma\backslash Bigg(\backslash beta\; \backslash frac\{L\_\{\backslash text\{ref\}\}\; -\; L\_\{\backslash text\{sft\}\}(\backslash theta)\}\{s\_\{\backslash text\{global\}\}\}\backslash Bigg)\; \backslash Bigg]$$

By anchoring the constant of integration to KTO's baseline, this explicit form demonstrates that PCSS preserves the exact functional shape of the original objective (the core $1-\sigma (\dots )1\; -\; \backslash sigma(\backslash dots)$ ). More importantly, it reveals how standard deviation normalization and decoupled peak scaling scale the overall magnitude of the loss surface by a factor of $\text{peak\_scale}\cdot \frac{4.0\cdot s_{\text{global}}}{\beta }\backslash text\{peak\backslash \_scale\}\; \backslash cdot\; \backslash frac\{4.0\; \backslash cdot\; s\_\{\backslash text\{global\}\}\}\{\backslash beta\}$.

---

2.4 Optimizer Dynamics & Hyperparameters

To accommodate training with an effective batch size of 1, we apply the constant token half-life scaling heuristic from recent small-batch training research to mitigate the rapid decay of AdamW's second moment estimate. Scaling a baseline ${\beta }_2=0.999\backslash beta\_2=0.999$ from batch size 16 down to batch size 1 yields: $${\beta }_2^{(1\mathrm{/}16)}\approx 0.99994\backslash beta\_2^\{(1/16)\}\; \backslash approx\; 0.99994$$

Across all PCSS experiments, we fix:

• Effective Batch Size: 1
• Optimizer: 8-bit AdamW
• Epochs: 5
• Scheduler: Constant learning rate with 1-epoch linear warmup
• Weight Decay: 0.01
• ${\beta }_2\backslash beta\_2$ Parameter: 0.99994
• PCSS Parameters: $\beta =0.65\backslash beta\; =\; 0.65$, $\text{peak\_scale}=5.0\backslash text\{peak\backslash \_scale\}\; =\; 5.0$
• MiSS Parameter: $r=512r=512$

---

2.5 Preliminary Multi-Token Prediction (MTP) Extension

For modern foundation architectures that incorporate native Multi-Token Prediction (such as dense Qwen 3.5 models, which iteratively unroll an MTP decoder layer across speculative steps), PCSS can be extended to jointly govern both the primary next-token prediction and subsequent look-ahead steps.

For the Qwen 3.5 9B experiments in this post, we utilized a preliminary formulation with a single speculative step $(k=1)(k=1)$, computing independent scalers for each prediction step: $$a_0=\frac{L_{\text{ref}}^{(0)}-L_{\text{sft}}^{(0)}(\theta )}{s_{\text{global}}^{(0)}},a_1=\frac{L_{\text{ref}}^{(1)}-L_{\text{sft}}^{(1)}(\theta )}{s_{\text{global}}^{(1)}}a\_0\; =\; \backslash frac\{L\_\{\backslash text\{ref\}\}^\{(0)\}\; -\; L\_\{\backslash text\{sft\}\}^\{(0)\}(\backslash theta)\}\{s\_\{\backslash text\{global\}\}^\{(0)\}\},\; \backslash quad\; a\_1\; =\; \backslash frac\{L\_\{\backslash text\{ref\}\}^\{(1)\}\; -\; L\_\{\backslash text\{sft\}\}^\{(1)\}(\backslash theta)\}\{s\_\{\backslash text\{global\}\}^\{(1)\}\}$$ $${\mathcal{L}}_{\text{MTP-PCSS}}(\theta )=\frac{1}{2}[L_{\text{sft}}^{(0)}(\theta )\cdot \text{detach}(\text{Scale}(a_0))+L_{\text{sft}}^{(1)}(\theta )\cdot \text{detach}(\text{Scale}(a_1))]\backslash mathcal\{L\}\_\{\backslash text\{MTP-PCSS\}\}(\backslash theta)\; =\; \backslash frac\{1\}\{2\}\; \backslash left[\; L\_\{\backslash text\{sft\}\}^\{(0)\}(\backslash theta)\; \backslash cdot\; \backslash text\{detach\}(\backslash text\{Scale\}(a\_0))\; +\; L\_\{\backslash text\{sft\}\}^\{(1)\}(\backslash theta)\; \backslash cdot\; \backslash text\{detach\}(\backslash text\{Scale\}(a\_1))\; \backslash right]$$

While treating the single speculative step independently performed well for Qwen 3.5 9B, it remains an initial exploration. Scaling to deeper speculative horizons $(k>1)(k\; >\; 1)$ introduces optimization challenges for independent scalers.

We plan to address these limitations and propose a more generalized approach in a future post.

---

3. Empirical Results

All training runs were conducted on vast.ai instances with a single H200 NVL GPU. Unless otherwise noted, models were evaluated on their zero-shot Chain-of-Thought (CoT) capabilities with 16,384 max completion tokens via a pinned version of vLLM (0.19.1).

Evaluation Notes:

• Exploratory Runs: Results represent exploratory single-seed training runs; score differences between configurations should not be taken as definitive conclusions.
• Multiple Generations: To account for natural sampling stochasticity as well as vLLM's runtime batching non-determinism (which affects even greedy decoding), most benchmarks were evaluated by drawing multiple generations per problem (e.g., $3\times 3\backslash times$ on GPQA Diamond, and $6\text{–}10\times 6\backslash text\{–\}10\backslash times$ on AIME). Single generations were used only when the evaluation set was large enough to provide natural stability (e.g., full MATH, MMLU Redux, and MATH-500 for certain ablations).
• Pass@3: Reported only for benchmarks with at least $n\ge 2kn\; \backslash ge\; 2k$ $(n\ge 6)(n\; \backslash ge\; 6)$ generations per question.

---

3.1 Qwen 3 4B Base Models & Ablations

Our fine-tuned models were evaluated using greedy decoding with 16,384 max completion tokens. Scores for the official post-trained and base models are self-reported figures taken directly from the Qwen 3 Technical Report.

Headline Performance (100 Zebra Puzzles)

100-Sample Ablation Matrix (Qwen 3 4B Base)

The configurations evaluated below share the following default hyperparameters:

• Shared Settings: 5 epochs, 1-epoch linear warmup, 8-bit AdamW optimizer.
• PEFT Defaults: LoRA uses $r=512r=512$, $\alpha =512\backslash alpha=512$, learning rate $9\times {10}^{-5}9\backslash times\; 10^\{-5\}$. MiSS uses $r=512r=512$.
• Optimizer $({\beta }_2)(\backslash beta\_2)$: $0.999940.99994$ for $bsz=1bsz=1$, and $0.9990.999$ for $bsz=16bsz=16$.

(Note: Training times were recorded on individual vast.ai H200 instances across TRL and custom trainer setups; minor variations within 6–8 minutes reflect framework callbacks and host instance differences. For $bsz=16bsz=16$, a micro-batch size of 16 required gradient checkpointing to fit in VRAM, whereas reaching the same effective batch size via gradient accumulation would roughly equalize speed).

Ablation Takeaway: Across all tested configurations—including standard batch size 16 with cosine decay—MATH-500 performance consistently approached or exceeded 80%, suggesting that reasoning generalization from pure logic traces is robust across different training setups rather than dependent on a single hyperparameter configuration.

---

3.2 Granite 4.1 3B Base (500 Zebra Puzzles)

Evaluated with greedy decoding:

Granite Delimiters & Template Ablation: During exploratory evaluations of Granite 4.1 3B Base, we observed that performance appeared sensitive to the string <extra_id_1>. We compared two fine-tuning configurations: one using the generic chat template from our Qwen runs, and another adapted to use <extra_id_1>User and <extra_id_1>Assistant. Despite <extra_id_1> being processed as standard text tokens rather than a special control token, the run formatted with these delimiters demonstrated slightly better loss convergence and higher downstream accuracy. This is the primary checkpoint reported in the table above.

---

3.3 Qwen 3.5 9B

Chat Template Pre-training Note: Qwen 3.5 9B Base was released with its chat template control tokens (e.g., <|im_start|>, <|im_end|>) explicitly pre-trained "to allow efficient LoRA-style PEFT with the official chat template." Applying this template allowed us to evaluate the untrained base, our fine-tuned checkpoint, and the official post-trained model directly in non-thinking mode for strict comparability.

Evaluated with sampled decoding ( $T=1.0,\text{top\_p}=0.95,\text{top\_k}=20T=1.0,\; \backslash text\{top\backslash \_p\}=0.95,\; \backslash text\{top\backslash \_k\}=20$, with presence_penalty=1.5 for the official post-trained model, as recommended by Qwen):

---

3.4 Token Length & Repetition Analysis (Qwen 3.5 9B)

To better understand the structural generation dynamics accompanying these performance gains—specifically regarding inference length and repetition failure modes—we measured completion lengths and looping rates.

MATH-500 Completion Token Lengths

Token counts were calculated using the Qwen 3.5 tokenizer on MATH-500 generation outputs:

Despite training exclusively on long-form logic traces (with many spanning up to 11,000+ tokens), the PCSS fine-tuned Qwen 3.5 9B model exhibits a shorter median completion length on MATH-500 than both its untrained base model and the official post-trained model.

Greedy GPQA Diamond Looping Rates

To evaluate repetition failure modes without stochastic sampling variance, models were evaluated under greedy decoding on GPQA Diamond. Repetition loops were detected using Liquid AI's antidoom inner-repetition detection heuristic:

Model-Dependent Generation Behaviors

The structural shifts and repetition reductions observed here should not be viewed as universal features of this training process. Exploratory runs indicate that different base model families internalize and generalize from these logic traces in substantially different ways—some models contract their output, while others expand it. Furthermore, although longer sequences inherently provide more opportunities for repetition, a model's underlying susceptibility to looping can shift in ways that are not strictly tied to generation length. Because this analysis is not exhaustive, even the specific dynamics of Qwen 3.5 9B may vary across different reasoning tasks. While these initial results suggest its performance leap does not strictly rely on pure inference scaling and it reduces repetitive loops on GPQA Diamond, generation behaviors remain complex and highly model-dependent.

---

4. Conclusion

Our exploratory runs highlight two distinct takeaways for post-training base models:

1. Impact of Pure Logic Micro-Datasets: In these exploratory runs, the primary driver of performance appears to be the data itself. Fine-tuning on as few as 100 pure logic deduction traces (zebra puzzles) can elicit substantial out-of-domain reasoning generalization, enabling small base models to match or outperform multi-stage instruct models on competitive mathematics benchmarks.
2. Data-Aware Optimization (PCSS): While standard SFT is capable of eliciting this generalization, shifting from static time-based learning rate decay to data-aware loss calibration via PCSS provides a principled optimization mechanism.

Ultimately, fine-tuning on these micro-datasets—whether through standard SFT or an adaptive loss scaler like PCSS—can provide a highly efficient way to elicit a base model's latent reasoning capabilities. Since training on these logic traces with PCSS appears to rapidly approximate the reasoning performance of a fully post-trained model, it may offer practical utility beyond standard alignment. For researchers evaluating pre-trained models, this methodology could potentially serve as an efficient evaluation proxy—providing a fast, low-cost way to gauge downstream reasoning performance without the need to run extensive post-training pipelines.