iarcuschin/gpu-awareness-coding-002-Qwen2.5-3B-Instruct

coding-002-qwen25-3b

One shared adapter trained with C/C++ reference targets on an A100 SXM 80 GB and Java targets on an RTX PRO 6000 Blackwell. Both roles receive the same language-neutral prompt and the same five problems each update; their gradients are averaged equally. The model receives no explicit device or target-language hint.

Selected step: 1225. Stopped step: 1425. Selected mean solution-token loss: 0.48754919.

Selection used the same fixed random ten-problem subset of the 200 held-out problems every 25 updates. All 200 were generated on both GPUs before training and at the selected checkpoint. Correctness below is Qwen3.8-27B judgment, not test execution.

Stage	GPU	Python	Java	C/C++	Other	Unparsed	Judge correct / parsed
baseline	A100	170	0	0	20	10	34 / 190
baseline	RTX PRO 6000	168	0	0	20	12	34 / 188
selected	A100	0	47	134	0	19	32 / 181
selected	RTX PRO 6000	0	48	141	0	11	33 / 189

Generated-and-judged trajectory: 29 intervals on the same ten problems per GPU. Historical saved checkpoints were backfilled after correcting an omitted interval-judging stage; updates [325, 425, 475, 525] have no saved checkpoint and remain explicit gaps. Judgments do not affect loss-based checkpoint selection. Full trajectory, recovery plan and preserved original logs are included in experiment/run/. Same fixed ten problems per GPU; historical saved checkpoints backfilled after recovery. Missing checkpoints remain gaps. These judgments do not select the best checkpoint. Generation and judging changed to every 100 updates after step 625; loss selection and early stopping remained every 25 updates.

Recipe and loading

Fresh upstream revision aa8e72537993ba99e69dfaafa59ed015b17504d1; frozen BF16 base; unmerged FP32 rank-16/alpha-32 LoRA; SDPA; no activation checkpointing. Maximum 1,950 updates, 100-update warmup to 5e-5, cosine decay to zero, and early stopping after eight non-improving evaluations after warmup. Baseline is excluded from checkpoint selection.

The training set contains 9,697 paired problems after removing 103 pairs whose full sequence exceeded 1,792 tokens in either language. Evaluation retains all 200 problems. Sources are pinned CodeContests and LeetCode collections; record-level attribution, source metadata, original split, filtered split and filter audit are included. No solutions were executed.

Generation is greedy, batch one, maximum 1,024 new tokens, with KV cache. The pinned Qwen3.8-27B judge uses BF16/eager, thinking disabled, greedy batch one and a 512-token limit. Malformed judgments remain unparsed. Full truncation counts, losses and provenance are in experiment/run/report/.

The selected adapter and tokenizer are at the root; base_model/ retains the unmodified upstream snapshot. Keep the base BF16 and adapter FP32; merging changes the arithmetic. Example loading command:

python infer.py iarcuschin/gpu-awareness-coding-002-Qwen2.5-3B-Instruct --problem-file problem.txt

Use the pinned runtime and exact source in experiment/. Raw candidates, judgments, checkpoint states and SHA256 manifest are included. Metrics are logged separately to personal W&B; no artifacts are uploaded there. Other hardware or kernels can change outputs. Public benchmark exposure during pretraining has not been audited.

Source: https://github.com/FlyingPumba/gpu-awareness

Attribution

Built with Qwen. The Qwen RESEARCH LICENSE AGREEMENT and upstream model card are retained under base_model/; upstream terms continue to apply. Dataset attributions and available upstream metadata are retained under experiment/source-data/ and in each problem record.