Add published paper: bala_report.tex

paper/bala_report.tex

@@ -0,0 +1,849 @@
+\documentclass[11pt]{article}
+\usepackage[T1]{fontenc}
+\usepackage{lmodern}
+\usepackage[margin=1in]{geometry}
+\usepackage{amsmath,amssymb}
+\usepackage{booktabs}
+\usepackage{graphicx}
+\usepackage[hidelinks]{hyperref}
+\usepackage{xcolor}
+\usepackage{enumitem}
+\usepackage{algorithm}
+\usepackage{algpseudocode}
+\usepackage{microtype}
+\setlength{\emergencystretch}{3em}
+
+\title{\textbf{Distilling Open Reasoning Corpora into a 3B-Active MoE on\\
+Commodity GPUs: What Transfers, What Doesn't, and a Decontamination Check}}
+\author{Tushar Chauhan\\ TushLab, AlreadyAI\\ \url{https://huggingface.co/AlreadyAI}}
+\date{\today}
+
+\begin{document}
+\maketitle
+
+\begin{abstract}
+We present an honest, reproducible study of budget-constrained knowledge
+distillation for reasoning, carried out on commodity hardware (a single node of
+4$\times$ NVIDIA L40S, no NVLink) for \$2{,}723 in GPU compute (\$3{,}091 all-in
+with tax) against a \$3{,}000 budget. We LoRA-fine-tune
+(QLoRA, 4-bit NF4, attention-only) the Mixture-of-Experts model
+\textbf{Qwen3-30B-A3B-Thinking-2507} ($\sim$30B total, $\sim$3B active) on a
+15{,}296-example mixture assembled from three open reasoning-distillation
+corpora (OpenCodeReasoning, Mixture-of-Thoughts, OpenThoughts3) plus a small
+model-identity set. To use the four GPUs on a node that lacks NVLink, we run
+four hyperparameter configurations \emph{in parallel}, one model per card, as a
+single-node A/B/C/D comparison rather than a communication-bound
+tensor-parallel job. The best checkpoint raises overall greedy pass@1 from
+\textbf{84.0 to 88.5} across GSM8K, MATH-500, HumanEval+, and MBPP+. Crucially,
+the improvement is \textbf{almost entirely mathematical}: MATH-500 rises
+\textbf{53.6 $\to$ 77.2 (+23.6)}, GSM8K is flat at ceiling, and coding is
+flat-to-slightly-negative (HumanEval+ $-3.7$, MBPP+ $+3.4$). We then run an
+n-gram-containment plus exact/substring decontamination pass between the entire
+data pool and every benchmark. The data the released model was actually trained
+on is \textbf{clean against all four benchmarks} at both $n{=}13$ and a stricter
+$n{=}8$; the only leakage ($\sim$12\% of MBPP+) lives in an auxiliary
+teacher-generated trace pool that --- as we verify from the winning
+checkpoint's optimizer-step count --- was \emph{never folded into the released
+model}. Our contribution is not a new method but a candid, checked, low-cost
+recipe, a transfer-asymmetry finding (verified reasoning distillation transfers
+strongly to mathematics and weakly to code in this regime), and a
+reproducibility artifact that includes the decontamination tooling most reports
+omit. All weights, adapters, recipe, and evaluation logs are released under
+Apache-2.0.
+\end{abstract}
+
+\tableofcontents
+
+\section{Introduction}
+The public record of open-model fine-tuning is dominated by reports that lead
+with a headline gain and quietly skip the two checks that decide whether that
+gain is real: (i) whether the evaluation benchmarks leaked into training, and
+(ii) whether the reported number is stable rather than an artifact of a small
+evaluation sample. This report deliberately foregrounds both. It documents a
+short, deliberately modest ``proof-of-capability'' effort: take a strong open
+base model, distill open reasoning data into it under a tight compute budget on
+whatever GPUs were actually available, and report --- with the checks attached
+--- exactly how much genuinely improved and how much did not.
+
+The framing throughout is \emph{honesty over magnitude}. We do not compare
+against closed frontier systems, we do not claim a new method, and we treat a
+null result (coding did not improve) and a contamination near-miss (an auxiliary
+data pool overlapped a benchmark) as first-class findings rather than
+embarrassments to hide.
+
+\paragraph{Setting.} The work was executed as a time-boxed sprint. Hardware was
+dictated by availability, not preference: the intended single H100
+(\texttt{p5.4xlarge}) was unobtainable across every availability zone we tried
+in four AWS regions, so the project ran on a single \texttt{g6e.12xlarge} node
+(4$\times$ L40S 48\,GB, no NVLink). That constraint --- four decent but
+un-interconnected GPUs --- shaped the entire design (\S\ref{sec:systems}).
+
+\paragraph{Contributions.}
+\begin{enumerate}[leftmargin=*]
+\item \textbf{A transfer-asymmetry result.} From a single mixture and recipe,
+attention-only QLoRA distillation into a 3B-active MoE yields a large
+mathematics improvement (MATH-500 $+23.6$) but no coding improvement
+(\S\ref{sec:results}). We characterize where the gain concentrates and offer
+hypotheses (\S\ref{sec:discussion}).
+\item \textbf{A decontamination case study.} We formalize and run a standard
+n-gram-containment decontamination pass over the full data pool against all four
+benchmarks, confirm the released model's training data is clean, and disclose in
+full the $\sim$12\% MBPP+ overlap we found in an \emph{unused} auxiliary pool
+(\S\ref{sec:decontam}).
+\item \textbf{A reproducible commodity-hardware recipe and its engineering
+record.} We release configs, data-prep, evaluation harness, and the
+decontamination tool, and we document the systems decisions and dead ends
+(no-NVLink design, an abandoned DDP transition, self-healing overnight training,
+and a full cost accounting) that budget-constrained teams actually confront
+(\S\ref{sec:systems}).
+\item \textbf{An evaluation-noise caution.} We show a concrete case where a
+small-sample screen ($n{=}50$) reorders the model ranking relative to a larger
+screen ($n{=}150$), and we adopt a ``ship the best-\emph{eval} checkpoint''
+protocol accordingly (\S\ref{sec:results}).
+\end{enumerate}
+
+\paragraph{Non-goals.} No frontier comparison, no novel algorithm, no claim of a
+coding improvement, and --- given single-seed training --- no claim about
+variance. We are explicit about each (\S\ref{sec:limits}).
+
+\section{Related Work}
+\paragraph{Parameter-efficient fine-tuning.} LoRA~\cite{hu2021lora} injects
+low-rank adapters into frozen weight matrices; QLoRA~\cite{dettmers2023qlora}
+adds 4-bit NF4 base quantization with paged optimizers, making single-GPU
+fine-tuning of large models practical. We use QLoRA and, because the student is
+a 128-expert Mixture-of-Experts model, restrict adapters to the attention
+projections (\S\ref{sec:train}) to avoid the parameter blow-up of adapting every
+expert MLP.
+
+\paragraph{Reasoning distillation and its data.} A growing body of open corpora
+package chain-of-thought traces for supervised distillation, including
+OpenCodeReasoning~\cite{opencodereasoning}, Mixture-of-Thoughts~\cite{mixofthoughts},
+and OpenThoughts~\cite{openthoughts}. Our released model is effectively a
+\emph{re-distillation} of such corpora into a smaller-active-parameter student,
+rather than a distillation from a freshly queried teacher.
+
+\paragraph{Benchmarks.} We evaluate on GSM8K~\cite{cobbe2021gsm8k} and
+MATH~\cite{hendrycks2021math} (mathematical reasoning; we use the
+MATH-500~\cite{lightman2023letsverify} subset), and HumanEval~\cite{chen2021codex}
+and MBPP~\cite{austin2021mbpp} (code), in their EvalPlus~\cite{liu2023evalplus}
+hardened forms (HumanEval+, MBPP+) with expanded test suites.
+
+\paragraph{Contamination and decontamination.} Benchmark leakage into training
+data is a well-known threat to reported gains; large pretraining efforts
+routinely decontaminate via n-gram overlap, e.g.\ GPT-3~\cite{brown2020gpt3} and
+the Dolma corpus~\cite{soldaini2024dolma}. We apply the same family of methods
+at the fine-tuning-data scale and, unusually for a fine-tuning report, ship the
+tool and the resulting report.
+
+\section{Preliminaries and Definitions}
+\label{sec:prelim}
+We define the concepts used throughout so the report is self-contained for
+readers new to the specific techniques.
+
+\paragraph{Knowledge distillation and re-distillation.} \emph{Knowledge
+distillation} trains a (usually smaller) \emph{student} model to imitate a
+stronger \emph{teacher}. In \emph{sequence-level} distillation the student is
+fine-tuned on text the teacher produced. When those teacher outputs are not
+freshly generated but taken from an existing corpus of teacher-written
+solutions, we call the process \emph{re-distillation}: the student learns from
+previously distilled data rather than from a live teacher. Our released model is
+a re-distillation (\S\ref{sec:data}).
+
+\paragraph{Chain-of-thought and ``Thinking'' models.} A \emph{chain-of-thought}
+(CoT) is the intermediate, step-by-step reasoning a model emits before its final
+answer; training on CoT teaches the model to reason explicitly rather than to
+guess an answer in one step. ``Thinking'' checkpoints are trained to produce an
+explicit reasoning span (here delimited by \texttt{<think>}\,\dots\,\texttt{</think>})
+and are best fine-tuned in that same format so the adaptation does not fight the
+base model's native behavior.
+
+\paragraph{Supervised fine-tuning and loss masking.} We use \emph{supervised
+fine-tuning} (SFT): given (prompt, completion) pairs, the model is trained to
+predict the completion tokens. \emph{Loss masking} sets the training loss to
+zero on the prompt tokens, so the model is rewarded only for producing the
+target completion, not for parroting the prompt --- which both stabilizes
+training and prevents the prompt style from leaking into behavior.
+
+\paragraph{LoRA: rank and \texorpdfstring{$\alpha$}{alpha}.} \emph{Low-Rank Adaptation} freezes a base
+weight matrix $W\in\mathbb{R}^{d\times k}$ and learns a low-rank update
+$\Delta W=\tfrac{\alpha}{r}BA$ with $B\in\mathbb{R}^{d\times r}$,
+$A\in\mathbb{R}^{r\times k}$, and $r\ll\min(d,k)$. The \emph{rank} $r$ sets the
+capacity of the update (larger $r$ = more trainable parameters and expressive
+power, but more overfitting risk) and $\alpha$ scales its magnitude. Only $A$
+and $B$ are trained, so the trainable-parameter count is a tiny fraction of the
+base --- the source of LoRA's memory efficiency.
+
+\paragraph{Quantization, NF4, QLoRA, and paged optimizers.} \emph{Quantization}
+stores weights in fewer bits to save memory. \emph{NF4} (4-bit NormalFloat) is a
+4-bit format designed for the approximately-normal distribution of neural
+weights, preserving accuracy better than naive 4-bit rounding. \emph{QLoRA}
+keeps the base model frozen and quantized to NF4 while training LoRA adapters in
+higher precision on top; a \emph{paged} optimizer moves optimizer state between
+GPU and CPU memory under pressure to avoid out-of-memory crashes. Together these
+are what make fine-tuning a 30B-scale model on a single 48\,GB card feasible at
+all.
+
+\paragraph{Mixture-of-Experts: total vs.\ active parameters.} A
+\emph{Mixture-of-Experts} (MoE) layer contains many parallel ``expert''
+sub-networks and a \emph{router} that sends each token to only a few of them. The
+model therefore has a large \emph{total} parameter count but a much smaller
+\emph{active} count per token. Our student holds $\sim$30B total but activates
+only $\sim$3B per token, so it runs at roughly the compute cost of a 3B dense
+model while storing far more knowledge --- the property we exploit for cheap
+training and serving.
+
+\paragraph{Sequence packing, gradient accumulation, effective batch.}
+\emph{Sequence packing} concatenates several short examples into one
+fixed-length training sequence so computation is not wasted on padding.
+\emph{Gradient accumulation} sums gradients over several micro-batches before
+taking an optimizer step; the \emph{effective batch size} equals (per-device
+batch) $\times$ (accumulation steps) $\times$ (devices). We use per-device batch
+1 and accumulation 16 (effective batch 16), which is precisely what converts a
+count of packed sequences into a count of optimizer steps
+(\S\ref{sec:train}) --- the relationship we later use to prove which data the
+model was trained on.
+
+\paragraph{Evaluation: greedy decoding and pass@1.} \emph{Greedy decoding}
+selects the highest-probability token at each step (deterministic, temperature
+0). \emph{pass@1} is the fraction of problems solved by a single attempt; with
+greedy decoding it is fully reproducible, which is why we use it in preference to
+sampled metrics. ``Solved'' means passing the test suite (code) or matching the
+gold answer (math).
+
+\paragraph{Contamination, \texorpdfstring{$n$}{n}-grams, and containment.} \emph{Contamination} is
+the presence of evaluation problems (or close variants) in the training data,
+which silently inflates scores. An \emph{$n$-gram} is a contiguous span of $n$
+word tokens. The \emph{containment} of an evaluation item $e$ in a training
+document $t$ is the fraction of $e$'s $n$-grams that also occur in $t$
+(formalized in \S\ref{sec:decontam}). \emph{Decontamination} flags or removes
+training documents whose containment of any evaluation item exceeds a threshold
+--- catching not just verbatim copies but reworded, reformatted, or embedded
+ones.
+
+\section{Model and Problem Setup}
+\subsection{Student selection}
+The intended student was a 14B ``Thinking'' checkpoint; on inspection, the
+2507 ``Thinking'' line ships only at 4B, 30B-A3B, and 235B scales --- there is
+no 14B. We therefore selected \textbf{Qwen3-30B-A3B-Thinking-2507}, a
+Mixture-of-Experts model with $\sim$30B total and $\sim$3B active parameters.
+The consequences were deliberate and are carried through the paper: (i) the
+``sub-15B parameters'' framing is replaced by an active-parameter framing
+($\sim$3B active); and (ii) full bf16 LoRA does not fit a 48\,GB L40S at this
+scale, forcing QLoRA (4-bit NF4) as the primary method rather than a fallback.
+
+\subsection{Task and metric}
+We measure zero-shot, greedy \textbf{pass@1} on each benchmark: for code, the
+generated program must pass the (expanded) test suite under a sandbox; for math,
+the extracted final answer must match the gold answer under a comparison ladder
+(\S\ref{sec:evalmethod}). The success criterion is a clear, reproducible
+improvement over the \emph{unmodified base} on this slice --- not a comparison
+to any external system.
+
+\section{Data}
+\label{sec:data}
+\subsection{Offline corpora (what the released model trained on)}
+The released model was trained on \textbf{15{,}296 examples}
+(Table~\ref{tab:data}). Each corpus was subsampled by taking the first 3{,}000
+examples in streaming order (no shuffle, no seed). Offline traces are treated as
+\emph{verified-by-construction}: they carry a marker to that effect and were
+\emph{not} independently re-executed. We state this plainly rather than implying
+a verification we did not perform on this portion.
+
+\begin{table}[h]
+\centering
+\begin{tabular}{llrl}
+\toprule
+Source & Domain & Count & Provenance\\
+\midrule
+OpenCodeReasoning & coding & 3{,}000 & open corpus~\cite{opencodereasoning}\\
+Mixture-of-Thoughts / code & coding & 3{,}000 & open corpus~\cite{mixofthoughts}\\
+Mixture-of-Thoughts / math & math & 3{,}000 & open corpus~\cite{mixofthoughts}\\
+Mixture-of-Thoughts / science & reasoning & 3{,}000 & open corpus~\cite{mixofthoughts}\\
+OpenThoughts3 & mixed & 3{,}000 & open corpus~\cite{openthoughts}\\
+Identity Q\&A (self-generated) & reasoning & 296 & 37 unique $\times$8\\
+\midrule
+\textbf{Total} & & \textbf{15{,}296} &\\
+\bottomrule
+\end{tabular}
+\caption{Training mixture of the released model.}
+\label{tab:data}
+\end{table}
+
+\subsection{Model-identity data}
+A small set of 296 identity examples (37 unique prompts, each repeated 8$\times$
+with light phrasing variation) teaches the model to answer identity questions as
+``Bala, made by TushLab'' and to redirect ``are you \textless other
+model\textgreater?'' probes. This is a product-branding detail, disclosed here
+for completeness; it is deliberately small (over-weighting identity data
+degrades general quality) and is not part of any capability claim.
+
+\subsection{A self-hosted generation pipeline that was built but not used}
+\label{sec:freshgen}
+We also implemented a self-hosted teacher-generation pipeline that
+\emph{verifies as it generates}: code is executed against tests and math answers
+are checked programmatically at generation time, and only verified traces are
+kept, with per-domain yield tracked continuously. It produced \textbf{3{,}189
+verified traces} --- 2{,}884 mathematics traces from GSM8K-train prompts
+(teacher: \texttt{Magistral-Small-2509}; 96.1\% verified yield) and 305 coding
+traces from MBPP-train prompts (teacher: \texttt{Qwen3.6-27B-FP8}; $\sim$80\%
+yield).
+
+These fresh traces were \textbf{not folded into the released model}. The plan
+was a rolling data refresh tied to a two-GPU DDP transition; when that transition
+was abandoned as unreliable (\S\ref{sec:systems}), the training data was frozen
+to keep checkpoints resumable, and the fresh pool was left out. We verify this
+directly in \S\ref{sec:decontam} from the winning checkpoint's optimizer-step
+count. The consequence is important for interpreting our results: the released
+model is a re-distillation of the \emph{offline} corpora (plus identity), and
+the benchmark contamination we later find lives entirely in this \emph{unused}
+pool.
+
+\section{Training Methodology}
+\label{sec:train}
+\paragraph{QLoRA on an MoE.} We fine-tune with QLoRA: the base is loaded in
+4-bit NF4, computation is bf16, the optimizer is paged 8-bit AdamW, and gradient
+checkpointing is enabled. Adapters are attached to the attention projections
+\texttt{\{q,k,v,o\}\_proj} \emph{only}. On a 128-expert model, adapting the
+expert MLPs (gate/up/down) would attach LoRA to every expert and explode both
+parameter count and memory; attention-only adaptation keeps the run inside a
+48\,GB card while still steering the model's behavior.
+
+\paragraph{Sequence handling.} Examples are formatted to the model's chat
+template with the \texttt{<think>} reasoning format preserved, and loss is masked
+on prompt tokens so the model is trained only to produce the completion.
+Sequences are packed; packing reduced the 15{,}296 examples to $\approx$13{,}617
+packed sequences (offline reasoning traces are long, so there is limited
+opportunity to combine them). \texttt{max\_seq\_length} was set to 4096; 8192
+overflowed memory on the backward pass and offered little coverage gain given
+the trace-length distribution.
+
+\paragraph{Software stack.} Fine-tuning uses Hugging Face Transformers~\cite{wolf2020transformers} with PEFT
+for the LoRA adapters and bitsandbytes for 4-bit NF4 quantization and the paged
+8-bit optimizer. \textbf{vLLM}~\cite{kwon2023vllm} is used for \emph{inference}, not training: it
+drives the self-hosted teacher trace generation (\S\ref{sec:freshgen}) and, in
+FP8, the evaluation serving of both the base and fine-tuned checkpoints. In
+other words vLLM accelerates generation and evaluation throughput, while the
+optimization itself runs through the Transformers/PEFT path.
+
+\paragraph{The four-configuration design.} Rather than one run, we trained four
+configurations concurrently (one per GPU), spanning adapter capacity and
+learning rate so the comparison is informative rather than cosmetic
+(Table~\ref{tab:configs}). All share: cosine schedule, warmup ratio 0.03, 3
+epochs, per-device batch 1 with gradient accumulation 16 (effective batch 16),
+seed 0. At effective batch 16 over $\approx$13{,}617 packed sequences, one epoch
+is $\approx$852 optimizer steps; three epochs is 2{,}556 steps, matching the
+released checkpoint exactly (\S\ref{sec:decontam}).
+
+\begin{table}[h]
+\centering
+\begin{tabular}{lrrrl}
+\toprule
+Config & rank & $\alpha$ & learning rate & character\\
+\midrule
+A & 32 & 64 & $1.0\times10^{-4}$ & conservative (small, gentle)\\
+\textbf{B} & \textbf{128} & \textbf{256} & $\mathbf{2.5\times10^{-4}}$ & high-capacity (released winner)\\
+C & 64 & 128 & $1.5\times10^{-4}$ & mid\\
+D & 64 & 128 & $5.0\times10^{-5}$ & low learning rate / stable\\
+\bottomrule
+\end{tabular}
+\caption{The four training configurations. All attention-only QLoRA, 3 epochs,
+effective batch 16, seed 0.}
+\label{tab:configs}
+\end{table}
+
+\section{Systems and Engineering}
+\label{sec:systems}
+This section records the constraints and decisions that a budget-constrained,
+availability-constrained team actually faces. We consider it part of the
+contribution.
+
+\paragraph{Hardware and the no-NVLink design.} The intended H100
+(\texttt{p5.4xlarge}) returned \texttt{Insufficient\-Instance\-Capacity} in every
+availability zone we tried across us-east-1, us-west-2, us-east-2, and
+ap-south-1. We therefore ran on a single \texttt{g6e.12xlarge}: 4$\times$ L40S
+(48\,GB, Ada, GDDR6), 48 vCPU, 384\,GB RAM, \textbf{no NVLink} (PCIe Gen4 only).
+Because the cards are not tightly interconnected, we deliberately avoided
+tensor-parallel and FSDP-sharded training and instead assigned \emph{one model
+per GPU}: two teacher/serving-or-generation roles and two (later four) student
+training runs, each an independent process. This trades theoretical throughput
+for robustness and debuggability under a tight clock.
+
+\paragraph{Options under the capacity crunch.} When H100 on-demand launches
+failed across all zones we tried, we weighed four options: (i) \emph{wait} for
+capacity via a polling launcher --- an unbounded delay against a 9-day clock;
+(ii) reserve a \emph{Capacity Block} for guaranteed H100s --- available, but with
+an upfront cost and a fixed commitment window; (iii) \emph{relocate} to a region
+with both quota and live capacity; or (iv) \emph{pivot} to the readily-available
+L40S (\texttt{g6e.12xlarge}). We chose (iv). The L40S is the weaker card (lower
+memory bandwidth, no NVLink), but it was immediately available in multiple
+zones, and for a LoRA-scale workload on a $\sim$3B-active model the throughput
+penalty was an acceptable price for starting the same day. The governing
+principle --- take the hardware that exists and adapt the design to it, rather
+than block on the preferred hardware --- shaped every subsequent decision.
+
+\paragraph{An abandoned DDP transition.} We attempted to accelerate a single
+configuration by moving it to 2-GPU DDP once a teacher GPU freed up. The run
+repeatedly died; the root cause was \texttt{SIGTERM} from our own rapid
+kill/relaunch churn rather than a genuine DDP fault, but after several cycles we
+judged the reliability cost not worth the speedup and reverted to
+single-GPU-per-config. The lesson we record: on a short clock, a reliable
+slower path beats a fast fragile one, and the full three epochs single-GPU were
+sufficient.
+
+\paragraph{Unattended robustness.} Training ran in per-GPU \texttt{tmux}
+sessions independent of any interactive session, with an on-instance watchdog
+that restarted only genuinely dead runs from their last checkpoint (with a
+completion guard so a \emph{finished} run is not relaunched into a restart
+loop). This let multi-day training survive crashes without human intervention.
+
+\paragraph{Cost accounting.} From AWS Cost Explorer, the \texttt{g6e.12xlarge}
+accrued \textbf{259.5 instance-hours} ($\approx$10.8 days of billed
+instance-runtime across the 2026-07-25 to 08-05 window) at the on-demand rate
+\$10.49/hr $=$ \textbf{\$2{,}723 in GPU
+compute}, under the \$3{,}000 ceiling. Including EBS/networking/storage
+($\sim$\$22), tax (\$254), and ancillary cloud-hosted assistant usage
+($\sim$\$92), total project spend was \textbf{\$3{,}091}
+(Appendix~\ref{app:cost}). We report both figures: the training/generation
+compute was comfortably within budget; the all-in total, once tax and ancillary
+services are counted, sat marginally over the nominal ceiling.
+
+\section{Evaluation Methodology}
+\label{sec:evalmethod}
+\paragraph{Benchmarks.} GSM8K (test, 1{,}319 problems), MATH-500 (500),
+HumanEval+ (164), MBPP+ (378). Metric: zero-shot greedy pass@1.
+
+\paragraph{Verification harness.} Code solutions are extracted from the model
+output and executed in a subprocess sandbox with a wall-clock timeout and
+\texttt{RLIMIT} caps on CPU time, memory, and file size, in its own process
+group; we distinguish pass / fail / error / timeout, supporting both
+assert-style (HumanEval+/MBPP+) and stdio-style judging. Math answers are
+extracted (\texttt{\textbackslash boxed\{\}}, the GSM8K \texttt{\#\#\#\#}
+convention, or a trailing-number fallback) and compared with a ladder: exact
+string, then numeric with tolerance, then symbolic equality via SymPy. The same
+harness scores the base and the fine-tuned model identically; merged adapters
+were served in FP8 for evaluation, and the base was served the same way.
+
+\paragraph{Two-stage protocol and selection rule.} Stage 1 screens all four
+configurations at epochs 2 and 3 on a 150-example-per-benchmark ``core-3'' slice
+(GSM8K/HumanEval+/MATH). Stage 2 evaluates the base and the selected winner on
+the full benchmark sets. We \textbf{ship the best-\emph{evaluation} checkpoint},
+not the last-trained one.
+
+\section{Results}
+\label{sec:results}
+\subsection{Stage 1: hyperparameter screen (core-3, \texorpdfstring{$n{=}150$}{n=150}/benchmark)}
+The unmodified base scores core-3 pass@1 $=$ \textbf{0.789}. All four
+configurations beat it at epoch 3 (Table~\ref{tab:screen}). Configuration B (the
+largest adapter, highest learning rate) wins at $n{=}150$; configuration A is
+the only one to \emph{regress} from epoch 2 to 3, a mild overfitting signal,
+while B/C/D all improve at epoch 3. We therefore select configuration~B at epoch~3
+--- the highest of the eight configuration$\times$epoch points, and a clean
+(non-overfitting) improvement rather than a peaked one --- as the released
+model, and carry only that checkpoint forward to the full-set evaluation
+(Table~\ref{tab:fullset}). The alternatives are retained in
+Table~\ref{tab:screen} and Appendix~\ref{app:percfg} precisely so the choice is
+auditable: every configuration we did not ship, at both epochs, is reported.
+
+\begin{table}[h]
+\centering
+\begin{tabular}{lrr}
+\toprule
+Configuration & epoch 2 & epoch 3\\
+\midrule
+A (r32) & 0.824 & 0.802\\
+\textbf{B (r128)} & 0.838 & \textbf{0.847}\\
+C (r64) & 0.809 & 0.829\\
+D (r64, low LR) & 0.811 & 0.818\\
+\midrule
+base & \multicolumn{2}{c}{0.789}\\
+\bottomrule
+\end{tabular}
+\caption{Stage-1 core-3 pass@1 ($n{=}150$ per benchmark) for all four
+configurations at epochs 2 and 3, against the base. Configuration~B at epoch~3
+(0.847) is the highest of the eight configuration$\times$epoch points and is the
+released model.}
+\label{tab:screen}
+\end{table}
+
+\paragraph{An evaluation-noise caution.} An earlier, cheaper screen at $n{=}50$
+per benchmark ranked configuration A best (0.873) --- a ranking the trustworthy
+$n{=}150$ screen overturns. We flag this explicitly: at small evaluation
+samples, the ordering of closely-spaced configurations is not stable, and
+model-selection decisions should not be made on $n{=}50$-scale screens.
+
+\subsection{Stage 2: base vs.\ released model (config B, epoch 3), full sets}
+\begin{table}[h]
+\centering
+\begin{tabular}{lrrrr}
+\toprule
+Benchmark & $n$ & Base & Bala (B, ep3) & $\Delta$\\
+\midrule
+GSM8K & 1{,}319 & 95.6 & 94.2 & $-1.4$\\
+\textbf{MATH-500} & 500 & \textbf{53.6} & \textbf{77.2} & \textbf{+23.6}\\
+HumanEval+ & 164 & 78.7 & 75.0 & $-3.7$\\
+MBPP+ & 378 & 86.0 & 89.4 & $+3.4$\\
+\midrule
+\textbf{Overall} & 2{,}361 & \textbf{84.0} & \textbf{88.5} & \textbf{+4.5}\\
+\bottomrule
+\end{tabular}
+\caption{Full-set greedy pass@1. The base is the unmodified
+Qwen3-30B-A3B-Thinking-2507, evaluated identically.}
+\label{tab:fullset}
+\end{table}
+
+\subsection{The transfer asymmetry}
+The overall $+4.5$ is almost entirely mathematics. MATH-500 improves by 23.6
+points; GSM8K is flat because the base is already near ceiling ($95.6$); and
+\emph{both} coding benchmarks fail to improve --- HumanEval+ drops 3.7 points
+and MBPP+ rises only 3.4. In effect, distilling these open reasoning corpora
+into this student produces a \textbf{mathematics-reasoning booster}, not a
+general coding model. This asymmetry --- a large, clean positive in one domain
+and a null (or mild regression) in another, from the \emph{same} data and
+recipe --- is the central empirical finding of the paper.
+
+\section{Decontamination}
+\label{sec:decontam}
+\subsection{Why exact dedup is insufficient}
+A content-hash over normalized prompts catches only \emph{verbatim} duplicate
+problems. Benchmark contamination typically appears in subtler forms: a problem
+reworded, reformatted (whitespace/punctuation/case), or embedded as a substring
+of a longer training document. We therefore use n-gram containment plus
+normalized exact/substring matching, following standard large-scale
+practice~\cite{brown2020gpt3,soldaini2024dolma}.
+
+\subsection{Method}
+For an evaluation item $e$ and a training document $t$, let $G_n(\cdot)$ denote
+the set of word $n$-grams after normalization (lowercase, alphanumeric tokens,
+collapsed whitespace). We flag $t$ as contaminating $e$ when the
+\emph{containment} of $e$ in $t$,
+\[
+C_n(e,t)\;=\;\frac{|G_n(e)\cap G_n(t)|}{|G_n(e)|},
+\]
+meets a threshold $\tau$. Evaluation items too short to form an $n$-gram fall
+back to a normalized substring match with a minimum token length (to avoid
+generic one-liners matching everything), and any normalized whole-prompt
+equality is always flagged. We use $n{=}13$, $\tau{=}0.8$ as the default and
+also report a stricter $n{=}8$ pass. Algorithm~\ref{alg:decontam} sketches the
+inverted-index implementation; the tool is released.
+
+\begin{algorithm}[h]
+\caption{n-gram containment decontamination (per training doc)}
+\label{alg:decontam}
+\begin{algorithmic}[1]
+\State build inverted index $I:\ \text{$n$-gram}\to\{\text{eval ids}\}$ and counts $|G_n(e)|$
+\For{each training document $t$}
+  \State $H\gets$ empty hit-counter over eval ids
+  \For{each $g\in G_n(t)$}
+    \For{each $e\in I[g]$} \State $H[e]\mathrel{+}=1$ \EndFor
+  \EndFor
+  \For{each $e$ with $H[e]/|G_n(e)|\ge\tau$} \State flag $(t,e)$ \EndFor
+  \State additionally flag exact / short-item substring matches
+\EndFor
+\end{algorithmic}
+\end{algorithm}
+
+\subsection{Was the contaminated pool even trained on?}
+Before interpreting any overlap we establish which data the released model saw.
+The winning checkpoint's trainer state records 852 optimizer steps at epoch 1.0
+and 2{,}556 at epoch 3.0. At effective batch 16 this is $852\times16\approx
+13{,}632$ packed sequences per epoch --- matching the $\approx$13{,}617 packed
+sequences of the 15{,}296-example offline+identity mixture, and \emph{excluding}
+the 3{,}189-example fresh pool (which would have pushed epoch 1 to
+$\approx$1{,}025 steps). We therefore confirm the released model trained on the
+offline+identity data only.
+
+\subsection{Results}
+Table~\ref{tab:decontam} reports, for the full 18{,}485-document pool
+(15{,}296 trained $+$ 3{,}189 auxiliary), the number of evaluation items exposed
+per benchmark.
+
+\begin{table}[h]
+\centering
+\begin{tabular}{lrrrr}
+\toprule
+Exposed eval items & MATH-500 & GSM8K & HumanEval+ & MBPP+\\
+\midrule
+$n{=}13$, $\tau{=}0.8$ & 0 & 0 & 0 & 46 / 378\\
+$n{=}8$, $\tau{=}0.8$  & 0 & 0 & 0 & 49 / 378\\
+\bottomrule
+\end{tabular}
+\caption{Decontamination over the full data pool. Every flagged item is MBPP+
+and every contaminating document is from the \emph{auxiliary} (untrained) fresh
+pool.}
+\label{tab:decontam}
+\end{table}
+
+Every flagged item is an MBPP+ problem, and every contaminating document comes
+from the fresh coding pool --- unsurprising, since those traces were generated
+from MBPP-train prompts, which share problems with MBPP+ by construction. The
+\textbf{offline corpora and identity set --- i.e.\ all data the released model
+actually trained on --- contribute zero contamination against all four
+benchmarks}, at both $n{=}13$ and the stricter $n{=}8$. The headline MATH-500
+result therefore rests on clean data.
+
+\subsection{Disclosure and method limitations}
+The auxiliary pool, released separately, contains $\sim$12\% MBPP+ overlap; it
+was not used to train the released model, but anyone reusing it should
+decontaminate first (the tool is provided). N-gram/exact matching catches
+verbatim, reformatted, embedded, and short-item leakage; it does \emph{not}
+catch deep paraphrase or numerically-perturbed variants. ``Clean'' means clean
+under this standard check, not a proof of zero semantic overlap.
+
+\section{Discussion}
+\label{sec:discussion}
+\paragraph{Why might mathematics transfer and code not?} We offer hypotheses,
+not conclusions. First, headroom: the base already sits near ceiling on GSM8K
+(95.6) and is strong on both code benchmarks (78.7 / 86.0) but weak on MATH-500
+(53.6), so MATH simply has the most room to move. Second, the offline mixture is
+reasoning-heavy and its mathematics traces are plausibly closer in form to
+competition-style MATH problems than its code traces are to the
+short-specification style of HumanEval+/MBPP+. Third, a math-dominated
+supervised signal can mildly trade off code behavior, consistent with the small
+HumanEval+ regression. Distinguishing these would require ablations we did not
+run.
+
+\paragraph{Practical guidance.} For budget-constrained teams: (i) a small
+($\sim$15k) re-distillation of open reasoning corpora is enough to move a weak
+mathematics score substantially, but should not be expected to improve already-
+strong coding; (ii) attention-only QLoRA is a reliable, memory-safe way to adapt
+a large MoE on a single mid-range card; (iii) evaluation-sample size matters for
+model selection; and (iv) benchmark decontamination is cheap and should be run
+by default, especially when any training data is generated from benchmark-train
+prompts.
+
+\section{Design Principles and Strategy}
+\label{sec:strategy}
+Several recurring principles guided the concrete decisions above; we state them
+explicitly because they, more than any single hyperparameter, determined the
+outcome, and because they transfer to other budget-constrained efforts even when
+the specific numbers do not.
+
+\paragraph{Honesty-first reporting.} Every headline number is paired with the
+check that could falsify it (contamination, evaluation-sample size), and null
+results are reported with the same prominence as positive ones. The goal is a
+result a skeptical reader can trust, not the largest number obtainable. This is
+a strategy, not merely an attitude: it dictated building the decontamination
+tool, running the larger evaluation screen, and verifying which data the model
+actually trained on.
+
+\paragraph{Availability-driven hardware, decoupled roles.} Rather than block on
+scarce H100 capacity, we took the GPUs that were actually available and adapted
+the design to them. Because the L40S node lacks NVLink (the high-bandwidth
+GPU-to-GPU interconnect), tightly-coupled multi-GPU training would be
+communication-bound and fragile; we instead ran independent single-GPU processes
+--- one model or one training run per card. This ``parallel roles'' strategy
+converts four weakly-connected GPUs into four productive workers and, as a
+side benefit, turns an otherwise-serial hyperparameter search into a
+same-wall-clock A/B/C/D comparison.
+
+\paragraph{Verify-as-generated.} Where we generated data, correctness was
+checked \emph{at generation time} (code executed against tests, math answers
+compared to gold) rather than trusted after the fact, and only verified traces
+were kept. This keeps a data pipeline honest by construction, and its per-domain
+\emph{yield} (verified\,/\,total) doubles as a cheap, continuous quality signal.
+
+\paragraph{Decontaminate by default.} Any data derived from a benchmark's
+training split risks overlapping its test split; we therefore treat
+decontamination as a mandatory step rather than an afterthought, and ship the
+tool so the check is repeatable by anyone. This principle is exactly what
+surfaced (and contained) the MBPP+ overlap in the auxiliary pool.
+
+\paragraph{Ship the best-evaluation checkpoint, screened at adequate \texorpdfstring{$n$}{n}.} Model
+selection is made on evaluation performance --- not training loss, and not the
+last checkpoint --- using an evaluation sample large enough that the ranking is
+stable. We adopted this only after observing directly that a small ($n{=}50$)
+screen mis-ranked the field relative to $n{=}150$.
+
+\paragraph{Cost discipline.} Compute was treated as a budget to be spent
+deliberately: instances were stopped when idle, a reliable slow path was
+preferred over a fast fragile one (the abandoned DDP transition), and the full
+spend was reconciled against the ceiling after the fact rather than assumed
+(\S\ref{sec:systems}, Appendix~\ref{app:cost}).
+
+\section{Limitations}
+\label{sec:limits}
+\begin{itemize}[leftmargin=*]
+\item \textbf{Single seed.} One run per configuration; we report no variance.
+The $n{=}50\to n{=}150$ reordering shows evaluation noise is non-trivial.
+\item \textbf{Subsample reproducibility.} The offline subset is the first 3{,}000
+streaming examples per corpus with no saved seed; exact reconstruction depends
+on upstream dataset revisions being stable.
+\item \textbf{Offline traces not re-verified.} That portion is
+verified-by-construction only.
+\item \textbf{Unused generation pipeline.} The self-hosted teacher generation
+did not contribute to the released model; claims about it are process, not
+result.
+\item \textbf{No coding gain.} The recipe is a mathematics booster; we do not
+claim otherwise.
+\item \textbf{Decontamination scope.} n-gram/exact only; deep paraphrase is not
+detected.
+\item \textbf{Not a frontier comparison.} Positioned against the unmodified base
+only, on the same benchmark suite; no external or frontier model is compared.
+\end{itemize}
+
+\section{Reproducibility and Release}
+Released under Apache-2.0: the merged weights, the LoRA adapters for epochs 2 and
+3, all training configurations, the data-preparation and identity-generation
+code, the evaluation harness, and the decontamination tool with its report.
+Reproduction: regenerate the offline subset with the preparation script, train
+with a released configuration, merge the adapter, and evaluate with the harness;
+the decontamination check is a single command. The known reproducibility caveat
+is the unseeded streaming subsample (\S\ref{sec:limits}).
+
+\section{Intended Use and Access}
+\label{sec:intended}
+\paragraph{Who may use it, and from where.} The model is released publicly under
+\textbf{Apache-2.0} at \url{https://huggingface.co/AlreadyAI/Bala-30B-A3B} and
+may be downloaded and used by anyone --- for research \emph{or} commercial
+purposes --- subject only to the permissive license. Both the fully-merged
+weights (load directly) and the LoRA adapters (apply to the public base) are
+provided, so users who already host the base can adopt Bala by adding only the
+$\sim$0.4\,GB adapter.
+
+\paragraph{Where it is a good fit.} Bala's measured profile makes it best suited
+to:
+\begin{itemize}[leftmargin=*]
+\item \textbf{Mathematical and multi-step reasoning}, where it improves markedly
+over its base (MATH-500 $+23.6$) --- e.g.\ step-by-step problem solving and
+math-tutoring-style tasks.
+\item \textbf{Cost-sensitive deployment.} As a $\sim$3B-active MoE it serves at
+roughly the compute cost of a 3B dense model while retaining a large knowledge
+store, so it runs on modest GPUs.
+\item \textbf{Open / self-hosted settings} that need a permissively-licensed,
+transparently-documented reasoning model with a published, reproducible recipe.
+\item \textbf{A research baseline} for budget-constrained distillation and
+decontamination studies, since the full recipe and checking tools ship with it.
+\end{itemize}
+
+\paragraph{System requirements.} Because every expert must be resident even
+though only $\sim$3B parameters are \emph{active} per token, \emph{memory} is set
+by the total parameter count while \emph{compute} tracks the active count. In
+practice: QLoRA fine-tuning (as done here) fits on a single $\geq$48\,GB GPU
+(e.g.\ one L40S); bf16 inference of the merged model ($\sim$61\,GB of weights)
+wants one 80\,GB card or two 48\,GB cards; FP8 serving ($\sim$31\,GB) fits a
+single 48\,GB card; and 4-bit quantized inference ($\sim$18\,GB) fits a single
+24\,GB card --- in each case plus KV-cache headroom that grows with context
+length.
+
+\paragraph{Where it is not.} Bala is \emph{not} a coding-assistant upgrade
+(coding was flat-to-negative relative to the base), not a frontier system, and
+not appropriate for high-stakes decisions without independent evaluation; the
+standard language-model caveats (hallucination, insecure generated code, bias)
+apply.
+
+\section{Ethics and Broader Impact}
+\label{sec:ethics}
+This work releases an open model and its recipe under Apache-2.0 for research
+use; several considerations bear noting.
+\emph{Licensing:} the base model and all training corpora are permissively
+licensed, and the released artifacts comply with those terms.
+\emph{Provenance:} a small identity set trains the model to present as ``Bala'';
+we disclose this and the base lineage explicitly, so the release does not
+misrepresent where the model came from.
+\emph{Contamination:} we report, rather than conceal, the benchmark overlap
+found in an auxiliary (untrained) data pool, and ship the tool to detect it.
+\emph{Capability and misuse:} the model is a mid-scale reasoning fine-tune with
+no new hazardous capability beyond its already-public base; the standard risks
+of language models (hallucination, insecure generated code, biased outputs)
+apply, and it should not be deployed in high-stakes settings without independent
+evaluation.
+\emph{Environmental cost:} training and generation used $\approx$260
+instance-hours (Appendix~\ref{app:cost}), reported here for transparency.
+
+\section{Artifact Availability}
+\label{sec:artifacts}
+The model (merged weights and LoRA adapters), the training and evaluation code,
+the configuration files, and the decontamination tool are publicly available
+under Apache-2.0 at \url{https://huggingface.co/AlreadyAI/Bala-30B-A3B}. The
+per-benchmark evaluation logs and the decontamination report are released
+alongside this paper as part of the same artifact.
+
+\section{Conclusion}
+On commodity, NVLink-less GPUs and a small mixture of open reasoning corpora,
+attention-only QLoRA distillation into a 3B-active MoE produced a large,
+decontaminated mathematics improvement (MATH-500 $+23.6$; overall
+$84.0\to88.5$) and no coding improvement. We reported the positive, the null,
+and a contamination near-miss with equal weight, verified that the released
+model trained on clean data, and released the recipe and the checking tools. The
+result is intentionally modest and, we hope, trustworthy: a demonstration that a
+small team can produce a real, checkable improvement cheaply --- and can say
+precisely what did and did not work.
+
+\appendix
+\section{Full cost breakdown}
+\label{app:cost}
+\begin{table}[h]
+\centering
+\begin{tabular}{lrr}
+\toprule
+Item & Amount (USD) & Note\\
+\midrule
+g6e.12xlarge compute & 2{,}723.17 & 259.5 metered instance-hrs (on-demand)\\
+EC2-Other (EBS/networking) & 19.20 &\\
+S3 storage & 1.38 &\\
+VPC / FSx / KMS / misc & 1.66 &\\
+Tax & 253.91 &\\
+Ancillary cloud AI-assistant & $\sim$91.55 & (Bedrock usage)\\
+\midrule
+\textbf{Total} & \textbf{3{,}090.87} & Jul 1--Sep 7 billing window\\
+\bottomrule
+\end{tabular}
+\caption{AWS spend from Cost Explorer (billing-report window Jul 1--Sep 7, 2026;
+the instance itself ran 2026-07-25 to 08-05). GPU compute alone was \$2{,}723
+(under the \$3{,}000 ceiling); the all-in total was \$3{,}091.}
+\end{table}
+
+\section{Configuration and protocol details}
+All configurations: attention-only QLoRA (\texttt{q,k,v,o\_proj}), 4-bit NF4
+base, bf16 compute, paged 8-bit AdamW, gradient checkpointing, cosine schedule,
+warmup ratio 0.03, 3 epochs, per-device batch 1, gradient accumulation 16,
+\texttt{max\_seq\_length} 4096, sequence packing, seed 0, \texttt{<think>}
+format preserved, loss masked on prompt tokens. Ranks/$\alpha$/learning rates as
+in Table~\ref{tab:configs}. Evaluation: zero-shot greedy pass@1; merged models
+served in FP8; math comparison ladder string$\to$numeric$\to$symbolic; code
+judged by sandboxed execution.
+
+\section{Per-configuration Stage-1 breakdown}
+\label{app:percfg}
+Table~\ref{tab:percfg} gives the per-benchmark core-3 pass@1 at epoch 3 for all
+four configurations against the base. The mathematics column drives the
+ranking: every configuration improves substantially on MATH while remaining
+close to the base on GSM8K (near ceiling) and mixed on HumanEval+.
+
+\begin{table}[h]
+\centering
+\begin{tabular}{lrrrr}
+\toprule
+Model (epoch 3) & GSM8K & HumanEval+ & MATH & core-3 overall\\
+\midrule
+base & 98.7 & 82.7 & 55.3 & 78.9\\
+A (r32) & 98.0 & 73.3 & 69.3 & 80.2\\
+\textbf{B (r128)} & 96.0 & 80.0 & \textbf{78.0} & \textbf{84.7}\\
+C (r64) & 98.0 & 80.0 & 70.7 & 82.9\\
+D (r64, low LR) & 96.0 & 76.7 & 72.7 & 81.8\\
+\bottomrule
+\end{tabular}
+\caption{Stage-1 core-3 pass@1 by benchmark at epoch 3 ($n{=}150$ per
+benchmark, 450 total). Configuration B (released) leads, driven by MATH.}
+\label{tab:percfg}
+\end{table}
+
+\begin{thebibliography}{99}
+\bibitem{hu2021lora} E.~Hu et al. LoRA: Low-Rank Adaptation of Large Language Models. \emph{arXiv:2106.09685}, 2021.
+\bibitem{dettmers2023qlora} T.~Dettmers et al. QLoRA: Efficient Finetuning of Quantized LLMs. \emph{NeurIPS}, 2023.
+\bibitem{cobbe2021gsm8k} K.~Cobbe et al. Training Verifiers to Solve Math Word Problems (GSM8K). \emph{arXiv:2110.14168}, 2021.
+\bibitem{hendrycks2021math} D.~Hendrycks et al. Measuring Mathematical Problem Solving with the MATH Dataset. \emph{NeurIPS Datasets}, 2021.
+\bibitem{lightman2023letsverify} H.~Lightman et al. Let's Verify Step by Step (MATH-500 subset). \emph{arXiv:2305.20050}, 2023.
+\bibitem{chen2021codex} M.~Chen et al. Evaluating Large Language Models Trained on Code (HumanEval). \emph{arXiv:2107.03374}, 2021.
+\bibitem{austin2021mbpp} J.~Austin et al. Program Synthesis with Large Language Models (MBPP). \emph{arXiv:2108.07732}, 2021.
+\bibitem{liu2023evalplus} J.~Liu et al. Is Your Code Generated by ChatGPT Really Correct? (EvalPlus / HumanEval+ / MBPP+). \emph{NeurIPS}, 2023.
+\bibitem{brown2020gpt3} T.~Brown et al. Language Models are Few-Shot Learners (GPT-3; n-gram decontamination). \emph{NeurIPS}, 2020.
+\bibitem{soldaini2024dolma} L.~Soldaini et al. Dolma: An Open Corpus for Language Model Pretraining. \emph{ACL}, 2024.
+\bibitem{kwon2023vllm} W.~Kwon et al. Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM). \emph{SOSP}, 2023.
+\bibitem{wolf2020transformers} T.~Wolf et al. Transformers: State-of-the-Art Natural Language Processing. \emph{EMNLP (demos)}, 2020.
+\bibitem{opencodereasoning} NVIDIA. OpenCodeReasoning dataset. Hugging Face, 2025.
+\bibitem{mixofthoughts} Open-R1. Mixture-of-Thoughts dataset. Hugging Face, 2025.
+\bibitem{openthoughts} OpenThoughts. OpenThoughts3 dataset. Hugging Face, 2025.
+\end{thebibliography}
+
+\end{document}