GoodStartLabs/opus-4b-cube-py-step120-2026-04-30

opus-4b-cube-py-step120

LoRA adapter for Qwen/Qwen3.5-4B, RL-trained on Opus Magnum-style motion-planning puzzles. The agent writes Python that places parts (arm, track, bonder, calcification glyph) and programs the arm to deliver atoms; the verifier scores solved=True when the program produces the target output.

Snapshot from training step 120 of the 4B-cube run.

Load it

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.5-4B"
adapter = "GoodStartLabs/opus-4b-cube-py-step120-2026-04-30"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

Or with vLLM / SGLang via --lora-modules.

Inference settings the model was trained for

These match the rollouts the model saw during training; deviating will degrade performance.

setting	value
Renderer	qwen3_5_disable_thinking (no <think> blocks; cookbook variant)
Coord mode	cube — 3-tuple (x, y, z) coordinates with x+y+z=0
Representation	python (model emits a Python program calling place_arm, place_track, place_calcification, place_bonder, arm.grab/drop/rotate_*/track_*, then run())
Sampling temperature	1.0
Max tokens / turn	8192
Max trajectory tokens	40000
Max turns / episode	6 (multi-turn with verifier feedback between turns)

Training recipe (RL)

param	value
Algorithm	GRPO (importance-sampling loss)
LoRA rank	32
Learning rate	1e-5
Optimizer	Adam (β₁=0.9, β₂=0.95)
Group size	8
Groups per batch	16 (→ 128 rollouts/batch)
num_substeps	1
kl_penalty_coef	0
temperature	1.0
remove_constant_reward_groups	False
Reward	+1.0 solve, +0.1 valid-but-wrong submission (stacks), -0.1 overflow, -0.05 no-code on last turn

Task / data

• Distances trained on: 1, 2, 3, 4 (hex distance from input to output cell)
• Task types: move, transmute (bond was excluded — see history below)
• Each batch samples 16 distinct puzzles × 8 rollouts = 32 trajectories on each of 4 puzzles… (well, groups_per_batch=16 distinct puzzles × group_size=8 rollouts = 128 rollouts/batch)
• Puzzles drawn deterministically by seed across the run

Training history

This adapter is the latest checkpoint of a multi-phase run (resumed across phases due to a tinker incident and a bond-task ablation). Effective progression:

phase	batches	task_types	notes
1	0–60	move, transmute, bond	initial training, full task set
2	60–100	move, transmute (no bond)	bond removed (was 0% solve rate, pure compute waste)
3	100–120	move, transmute (no bond)	extension to step 150 (in flight at upload time)

Behavioral notes

• The model has converged on a stereotyped recipe: place_arm → grab → rotate_cw/ccw → drop → reset. About half of solved trajectories also use place_track(...); tracks are productive when used.
• Transmute tasks add place_calcification(...) before grab, off the rotation path.
• Move tasks rarely use tracks at d1/d2; tracks become valuable at d3/d4.
• One-shot solves (verifier returns solved=True on turn 1) account for the majority of successful trajectories on easy puzzles. On d3/d4 the model often needs 2-3 turns of verifier feedback (mostly correcting overlap-with-input or arm-pivot errors).

Approximate solve rate at this checkpoint (4B-cube run, late iters)

Trajectory-level (fraction of rollouts where verifier returned solved=True):

distance	rate
d1	~94%
d2	~99%
d3	~71%
d4	~76%

Move tasks ~91%, transmute tasks ~72%. The d3/d4 deficit is mostly in transmute, where the model occasionally gets stuck in track-overlap-with-arm-pivot loops.

Provenance

• Base: Qwen/Qwen3.5-4B
• Trained on Tinker; LoRA exported via tinker_cookbook.weights.{download, build_lora_adapter} and pushed via publish_to_hf_hub.
• Source repo (training harness, prompts, verifier wrapper): the opus-lite-rl codebase.
• Snapshot date: 2026-04-30